U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PROJECTS/PROGRAMS

Human-Centered SI

Summary

Pursuant to September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. AI has been changed to SI except when referring to previously completed work and historical documents per Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

Description

The Human-Centered Super Intelligence (SI) program within the NIST Human-Centered Technologies Group performs research across several SI areas, always keeping humans at the core of our work. Our projects include:

 

Generative SI Use in the Workplace

Generative SI has the potential to play a role in a variety of workplace tasks, and SI may affect the way employees work both negatively and positively. Moreover, employees may have a wide variety of perspectives on use of generative SI which organizations need to support. The Human-Centered Technologies Group is initiating research efforts to better understand how and why generative SI is used by employees.

TEVV-Athlon Framework for Evaluating AI Systems 

The initial public draft was released prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

Input Sought on Initial Public Draft of NIST AI 200-2 through October 6, 2026

Test, evaluation, verification, and validation (TEVV) of artificial intelligence (AI) systems is used to provide evidence that systems can effectively meet individual or organizational goals while minimizing negative impacts. Due to the extensive variety of AI use, a flexible approach is needed for organizations to develop and conduct measurement approaches that are customized to their needs. The TEVV-Athlon Framework is a four-stage method for developing customized assessments of AI systems based on organizational TEVV objectives. The framework produces a TEVV-Athlon, an assessment where AI systems are tested via a set of Events and Tools which produce data on Blocks related to measurement concepts of interest. To illustrate the approach, an example TEVV-Athlon is constructed using the framework. Practical guidance for conducting a TEVV-Athlon is also provided. The TEVV-Athlon Framework can be applied as needed to produce meaningful information about AI system performance and help organizations measure the impact of their AI systems.

 AI Evaluation Scenarios

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence.  Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents. 

Scenario-based evaluations play an important role in the AI evaluation ecosystem. We developed a method to elicit high-level AI use cases from subject matter experts (SMEs) via a structured AI Use Case Worksheet with six key elements: use case, sector, user (direct and indirect), intended outcomes, expected impacts (positive and negative), and KPIs and metrics. We then combine LLM prompting with human reviews to expand high-level use cases to more detailed evaluation scenarios. Pre-print available here for work on this in the financial services sector. This method has subsequently been applied in the manufacturing sector, with code available here.

Assessing Risks and Impacts of AI (ARIA)

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

The NIST Assessing Risks and Impacts of AI (ARIA) program (https://ai-challenges.nist.gov/aria) aims to improve the quality of risk and impact assessments for the field of safe and trustworthy AI. AI applications submitted to ARIA were assessed at three levels of measurement: 1) Model Testing to confirm claimed model capabilities and performance of model guardrails; 2) Red Teaming to stress test applications and guardrails and elicit disallowed behavior; and 3) Field Testing to test the application in regular use.

Results of the pilot evaluation, ARIA 0.1, are available here: https://doi.org/10.6028/NIST.AI.700-2.

ARIA Evaluation Planning Manual: Elements of ARIA-Style AI Evaluations, NIST AI 200-3, was recently published.

Belmont Principles in AI Research

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

Our team collaborated with NIST’s Research Protections Office (RPO) to investigate the application of the Belmont Principles to AI research.

The Belmont Principles identify three basic principles for the protection of human subjects in federally funded research:

  1. Respect for persons: Individuals should be treated as autonomous agents. Persons with diminished autonomy are entitled to protection.
  2. Beneficence: Do no harm. Maximize benefit and minimize risk.
  3. Justice: The benefits and risks shared by a population that may benefit from the results of research.

We also conducted qualitative research with NIST RPO to better understand decisions to participate in human subjects research involving AI. We conducted semi-structured interviews with federal employees to identify factors in consent for a human-AI interaction study. Results are forthcoming.

See our recent NIST news story and IEEE Computer publication.

 AI User Trust Measurement

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

Trust has been identified as an influential predictor of operator behavior in human-automation interaction. Research on trust in automation has been advanced via the development and validation of psychometric scales which capture the subjective construct of trust. Trust measurement can help to clarify the effects of user-, system-, and context-related variables in user perceptions of and behavior with technology.

Recent efforts in trustworthy and responsible AI emphasize the importance of sociotechnical approaches which account for the subjective experiences of those using or impacted by AI systems. Toward better understanding user trust in AI systems, the Human-Centered Technologies Group sought to validate existing psychometric scales for trust in automation in the AI context. We conducted an online study with a sample of federal employees and a sample of individuals in the general public where trust was assessed with existing trust in automation scales. Participants were asked to imagine themselves interacting with AI systems in various contexts and report on their trust in addition to other trust-relevant variables.

Results showed that the scales’ relationships with trust predictors and outcomes were mostly in line with trust theory in both federal employee and general public samples. This research is described in more detail in the Proceedings of the Human-Computer Interaction International (HCII) Conference 2025 here: https://doi.org/10.1007/978-3-031-93412-4_5. Further results are forthcoming. These findings informed questionnaire item development for ARIA Field Testing.

 AI Use Taxonomy

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

The advancement of AI technologies across a variety of domains has spurred efforts in measurement and evaluation aiming to ensure that systems are trustworthy and responsible. AI systems are often categorized by technique or domain of application, which may limit the ability to develop measurement and evaluation approaches that apply to a broad range of systems. The Human-Centered Technologies group developed the AI Use Taxonomy to categorize AI systems in a way that is 1) technique-independent, 2) domain-independent, and 3) human-centered. The taxonomy sets forward 16 human-activities which describe the ways that an AI system may contribute to a human’s overall task and intended outcomes. The motivation and approach to developing the taxonomy are described here: https://doi.org/10.6028/NIST.AI.200-1

The taxonomy can be applied to task analysis of specific AI use cases as well as to categorize the different ways AI is used within an organization. The research team continues to work with various government entities to improve the utility of the taxonomy and assist with categorizing AI use cases from a human-centered perspective, focused on overall tasks and human goals.

AI Perceptions

This work was completed prior to the issuance of the September 29, 2026, Executive Order on Inaugurating the Era of Super Intelligence. Pursuant to this Executive Order Section 2 (b), this does not require the alteration of previously issued regulations, Presidential actions, contracts, grants, or other historical documents.

The general public is increasingly interacting with or impacted by AI in various domains. An understanding of perceptions of AI among the general public can help to guide research in human-AI interaction, as well as support the development of human-centered AI systems. 

The Human-Centered Technologies Group conducted semi-structured interviews in 2022 with 25 members of the U.S. general public and 20 AI experts working in U.S. industry. Interview transcripts were qualitatively analyzed with the goal of identifying perceptions and beliefs about AI among the two groups.

Qualitative analysis revealed that humanness and ethics were central components of participants’ perceptions of AI. Humanness, the set of traits considered to set humans apart from other intelligent actors, was a consistent component of various beliefs about AI’s characteristics. Ethics arose in participants’ discussions of the role of technology in society, centering around views of AI as made and used by people. These findings point to various beliefs and concerns among the public and experts, warranting future focused research on perceptions of AI.

Results from the interview study were published in the proceedings of AI, Ethics, and Society (AIES) 2024 conference under the title “Reflection of Its Creators: Qualitative Analysis of General Public and Expert Perceptions of Artificial Intelligence.” The online Appendix for this paper is available here.

Resources: 

Created April 23, 2024, Updated October 1, 2026
Was this page helpful?