U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Artificial intelligence

AI measurement and evaluation

Search Publications

Search Title, Abstract, Conference, Citation, Keyword or Author
Published Date
Displaying 26 - 36 of 36

Overview of TREC 2023

May 1, 2024
Author(s)
Ian Soboroff
TREC 2023 is the thirty-second edition of the Text REtrieval Conference (TREC). The main goal of TREC is to create the evaluation infrastructure required for large-scale testing of information retrieval (IR) technology. This includes research on best

AI-Based Environment Segmentation Using a Context-Aware Channel Sounder

April 26, 2024
Author(s)
Anuraag Bodi, Samuel Berweger, Raied Caromi, Jihoon Bang, Jelena Senic, Camillo Gentile
We describe how the data acquired from the camera and Lidar systems of our context-aware radio-frequency (RF) channel sounder is used to reconstruct a 3D mesh of the surrounding environment, segmented and classified into discrete objects. First, the images

Context-Aware Channel Sounder for AI-Assisted Radio-Frequency Channel Modeling

April 26, 2024
Author(s)
Camillo Gentile, Jelena Senic, Anuraag Bodi, Samuel Berweger, Raied Caromi, Nada Golmie
We describe a context-aware channel sounder that consists of three separate systems: a radio-frequency system to extract multipaths scattered from the surrounding environment in the 3D geometrical domain, a Lidar system to generate a point cloud of the

2024 NIST Generative AI (GenAI): Data Creation Specification for Text-to-Text (T2T) Generators

April 1, 2024
Author(s)
Yooyoung Lee, George Awad, Asad Butt, Lukas Diduch, Kay Peterson, Seungmin Seo, Ian Soboroff, Hariharan Iyer
Generator (G) teams will be tested on their system ability to generate content that is indistinguishable from human-generated content. For the pilot study, the evaluation will help determine strengths and weaknesses in their approaches including insights

2024 NIST Generative AI (GenAI): Evaluation Plan for Text-to-Text (T2T) Discriminators

April 1, 2024
Author(s)
Yooyoung Lee, George Awad, Asad Butt, Lukas Diduch, Kay Peterson, Seungmin Seo, Ian Soboroff, Hariharan Iyer
Generator (G) teams will be tested on their system's ability to generate content that is indistinguishable from human-generated content. For the pilot study, the evaluation will help determine strengths and weaknesses in their approaches including insights

The BETTER Cross-Language Information Retrieval Datasets

July 27, 2023
Author(s)
Ian Soboroff
The IARPA BETTER (Better Extraction from Text Through Enhanced Retrieval) program held three evaluations of information retrieval (IR) and information extraction (IE). For both tasks, the only training data available was in English, but systems had to

Cluster Association for 3D Environment Based on 60 GHz Indoor Channel Measurements

May 31, 2023
Author(s)
Raied Caromi, Jian Wang, Anuraag Bodi, Camillo Gentile
In this paper, we present a ray tracing (RT) assisted multipath cluster association method. This work is based on an indoor channel measurement at 60 GHz, where a light detection and ranging (LiDAR) sensor was co-located with channel sounder and time

Device Modeling Bias in ReRAM-Based Neural Network Simulations

January 20, 2023
Author(s)
Imtiaz Hossen, Matthew Daniels, Martin Lueker-Boden, Andrew Dienstfrey, Gina Adam, Osama Yousuf
The study of resistive-RAM (ReRAM) devices for energy efficient machine learning accelerators requires fast and robust simulation frameworks that incorporate realistic models of the device population. Jump table modeling has emerged as a phenomenological

OpenMFC 2022 Evaluation Program

January 3, 2023
Author(s)
Haiying Guan, Baptiste Chocot, Ilia Ghorbanian Bajgiran, Lukas Diduch, Yooyoung Lee, Christopher Tu
Was this page helpful?