U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

Assessing Risks and Impacts of AI (ARIA): Pilot Evaluation Report

Published

Author(s)

Razvan Amironesei, Afzal Godil, Craig Greenberg, Kristen Greene, Johnston Patrick Hall, Theodore Jensen, Jonathan Fiscus, Noah Schulman

Abstract

This document describes the procedure used for a pilot of NIST's Assessing Risks and Impacts of AI (ARIA) evaluation: ARIA 0.1. Five organizations participated, submitting a total of 7 AI applications to be evaluated. In this document, we first describe the design of the three evaluation scenarios (TV Spoilers, Meal Planner, Pathfinder) and the three testing levels (model testing, red teaming, field testing). We then discuss the methods used for assessment via dialogue annotation and tester questionnaires. Finally, we describe our approach to measuring validity of AI applications using measurement trees.
Citation
NIST Trustworthy and Responsible AI - 700-2
Report Number
700-2
Pub Type
NIST Pubs

Download Paper

https://doi.org/10.6028/NIST.AI.700-2
Local Download
AI measurement and evaluation

Citation

Amironesei, R. , Godil, A. , Greenberg, C. , Greene, K. , Hall, J. , Jensen, T. , Fiscus, J. and Schulman, N. (2025), Assessing Risks and Impacts of AI (ARIA): Pilot Evaluation Report, NIST Trustworthy and Responsible AI, National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.AI.700-2, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=960511 (Accessed April 22, 2026)

Issues

If you have any questions about this publication or are having problems accessing it, please contact [email protected].

Created November 13, 2025, Updated November 14, 2025
Was this page helpful?