NIST’s AI Standards “Zero Drafts” Pilot Project to Accelerate Standardization, Broaden Input

NIST’s new AI Standards Zero Drafts project will pilot a process to broaden participation in and accelerate the creation of standards, helping standards meet the AI community’s needs and unleash AI innovation.

In this project, NIST will collect input on topics with a science-backed body of work and use it to develop “zero drafts”—preliminary, stakeholder-driven drafts of standards that are as thorough as possible. These drafts then will be submitted into the private sector-led standardization process as proposals for further development into voluntary consensus standards.

The Need

Voluntary consensus standards are typically produced by SDOs through rigorous, sometimes lengthy consensus-building processes among interested experts. As NIST has gathered input on standardization needs for AI, stakeholders have repeatedly emphasized two problems, which are especially challenging to address simultaneously:

• AI standards need to be developed expeditiously to address urgent needs, prevent fragmentation of governance frameworks, and keep up with  the lightning-fast pace of advances—while still maintaining the rigor of the process and the quality of the resulting standards.
• AI standards demand a wide range of expertise and perspectives, many of which are not typically involved in standardization. Stakeholders seek to address many needs via AI standards. This makes it important for the standards to draw on multi-disciplinary perspectives, including from many kinds of organizations and stakeholders who develop, use, research, or are affected by AI systems.

NIST’s Solution: Convening the Community to Develop Zero Drafts

NIST seeks to expand participation in AI standards development and help standards developing organizations (SDOs) achieve consensus more quickly by releasing thorough, stakeholder-driven proposals for new SDO projects. To achieve this, NIST is collecting inputs from stakeholders and crafting them into detailed drafts meant to accompany new project proposals, following the process below:

1. NIST proposes topics and solicits stakeholder input on which topics to prioritize, how to scope them, and what content for each priority topic and scope NIST should include or build on.
2. For each topic selected by NIST, based on this feedback:
   1. NIST releases a concept paper outlining a proposed direction for a standard.
   2. Based on the concept paper and broad stakeholder input, NIST proposes an initial draft standard.
   3. NIST iterates on drafts based on further rounds of input; and
   4. The resulting document is submitted to SDOs via established processes as a proposal for formal standardization.

Zero drafts will be thorough, high-quality documents reflecting stakeholder inputs, but they will ultimately be NIST proposals. And, assuming they are taken up by SDOs, they will likely still change during the formal standardization process.

Initial Topics and Proposed Scopes 

The first two zero drafts are being approached as pilots. Their topics are:

• AI testing, evaluation, verification, and validation (TEVV) (draft outline available)
  Scope: This zero draft will provide a general framework for organizations to approach TEVV based on clear concepts and conceptual relationships, organization-defined requirements and objectives, and methodological constraints. The draft will not focus on specific TEVV methods.
• Documentation about system and data characteristics for transparency among AI actors (draft outline in progress)
  Scope: This zero draft will provide guidance on model and dataset documentation for public consumption, including lists of fields that should or could be included in a documentation artifact.

NIST selected these topics from a larger list of potential topics and scopes based on stakeholder input. The full list was drawn from stakeholder-identified priorities expressed to NIST, adjusted to minimize overlap with existing standards projects. Other topics and potential scopes on that list, which may be revisited if the Zero Drafts project is extended past the pilot, were:

Maps of concepts and terminology regarding AI system designs, architectures, processes, and actors

Potential Scopes:

1. Clarification of the “AI stack”—the layers of technology and resources used to build AI applications, including the roles, responsibilities, and processes involved in each layer across the AI lifecycle.
2. Reference architectures or design patterns for AI systems to establish shared understanding of AI system components and their relationships. 

Technical measures for reducing risks posed by synthetic content

Potential Scopes:

1. A taxonomy of approaches and terms to refer to these approaches (e.g., digital content transparency, provenance data tracking, signed metadata, watermarking).
2. Methods and metrics for evaluating and reporting the effectiveness of such measures.