NIST’s AI Standards “Zero Drafts” Pilot Project to Accelerate Standardization, Broaden Input

NIST’s AI Standards Zero Drafts project is piloting a process to broaden participation in and accelerate the creation of standards to meet the AI community’s needs and unleash AI innovation.

NIST is collecting input on topics with a science-backed body of work and using it to develop “zero drafts”—preliminary, stakeholder-driven drafts of standards that are as thorough as possible. These drafts will ultimately be submitted into the private sector-led standardization system as proposals for further development into voluntary consensus standards.

The Need

Voluntary consensus standards are typically produced by standards developing organizations (SDOs) through rigorous, sometimes lengthy consensus-building processes among interested experts. As NIST has gathered input on standardization needs for AI, stakeholders have repeatedly emphasized two problems, which are especially challenging to address simultaneously:

• AI standards need to be developed expeditiously to address urgent needs, prevent fragmentation of governance frameworks, and keep up with the lightning-fast pace of advances—while still maintaining the rigor of the process and the quality of the resulting standards.
• AI standards demand a wide range of expertise and perspectives, many of which are not typically involved in standardization. Stakeholders seek to address many needs via AI standards. This makes it important for the standards to draw on multi-disciplinary perspectives, including from many kinds of organizations and stakeholders who develop, use, research, or are affected by AI systems.

NIST’s Solution: Convening the Community to Develop Zero Drafts

NIST seeks to expand participation in AI standards development and help SDOs achieve consensus more quickly by releasing thorough, stakeholder-driven proposals for new SDO projects.  NIST is collecting inputs from stakeholders and crafting them into detailed drafts meant to accompany new project proposals, following the process below:

1. NIST proposes topics and solicits stakeholder input on which topics to prioritize, how to scope them, and what content to include or build on for each priority topic and scope.
2. For each topic selected by NIST, based on this feedback:
   1. NIST releases a concept paper outlining a proposed direction for a standard.
   2. After receiving broad stakeholder input, NIST proposes an initial draft standard.
   3. NIST iterates on drafts based on further rounds of input; and
   4. The resulting document is submitted to an SDO via established processes as a proposal for formal standardization.

Zero drafts will be thorough, high-quality documents reflecting stakeholder inputs, but they ultimately will be NIST proposals. Assuming they are taken up by SDOs, they will likely still change during the formal standardization process.

Initial Topics and Proposed Scopes 

The two zero drafts in the initial pilot are as follows:

• Documentation about system and data characteristics for transparency among AI actors (initial public draft; early outline)
  Scope: This zero draft provides guidance on model and dataset documentation for public consumption, including lists of fields that should or could be included in a documentation artifact.
• AI testing, evaluation, verification, and validation (TEVV) (draft outline)
  Scope: This zero draft will provide a general framework for organizations to approach TEVV based on clear concepts and conceptual relationships, organization-defined requirements and objectives, and methodological constraints. The draft will not focus on specific TEVV methods

NIST selected these topics from a larger list based on stakeholder input and adjusted to minimize overlap with existing standards projects. Other topics and potential scopes on that list, which may be revisited if the Zero Drafts project is extended past the pilot, were:

Maps of concepts and terminology regarding AI system designs, architectures, processes, and actors

Potential Scopes:

1. Clarification of the “AI stack”—the layers of technology and resources used to build AI applications, including the roles, responsibilities, and processes involved in each layer across the AI lifecycle.
2. Reference architectures or design patterns for AI systems to establish shared understanding of AI system components and their relationships. 

Technical measures for reducing risks posed by synthetic content

Potential Scopes:

1. A taxonomy of approaches and terms to refer to these approaches (e.g., digital content transparency, provenance data tracking, signed metadata, watermarking).
2. Methods and metrics for evaluating and reporting the effectiveness of such measures.