Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
TRUC is a new evaluation conference series that will build the infrastructure required for evaluating aspects of trustworthiness in AI systems. TRUC is distinct from, but related to, the NIST AI Risk Management Framework effort. The purpose of the framework is to provide guidance on managing the risks associated with the use of artificial intelligence. The goal of TRUC is to identify risk/harms of systems developed to perform selected tasks before such systems are deployed and to define (and eventually create) data sets and evaluation infrastructure that will allow system builders to detect the extent to which their system exhibits those harms. Experience suggests that such infrastructure will support the development of better---more trustworthy---systems for those tasks.
workshop objectives
This initial workshop in the series will be a planning workshop consisting of three separate tracks, each track focused on a different task: an automated employment decision track, a track focused on automatic language translation within the context of international border crossings, and a track focused on the management of consumer health (mis)information. Within each track attendees will participate in a roundtable discussion to identify the most salient potential harm(s) an automated system for the task is likely to engender, and to define an evaluation protocol, including necessary data resources, that measures the extent to which a given system exhibits harmful behavior. Workshops in subsequent years will implement the protocols providing community evaluations for the target tasks. (TRUC is being patterned after the Text REtrieval Conference (TREC) series that have successfully supported research on information retrieval through community evaluations.)
Agenda
The meeting will run from 9:00am to 5:00pm EDT with a lunch break from 12:00-1:45pm each day.
- Tuesday, October 04 - Employment track
- Wednesday, October 05 - Translation track
- Thursday, October 06 - Misinformation track