Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

TrojAI

Trojans in Artificial Intelligence


Using current machine learning methods, an artificial intelligence (AI) is trained on data, learns relationships in that data, and then is deployed to the world to operate on new data. For example, an AI can be trained on images of traffic signs, learn what stop signs and speed limit signs look like, and then be deployed as part of an autonomous car. The problem is that an adversary that can disrupt the training pipeline can insert Trojan behaviors into the AI. For example, an AI learning to distinguish traffic signs can be given just a few additional examples of stop signs with yellow squares on them, each labeled “speed limit sign.” If the AI were deployed in a self-driving car, an adversary could cause the car to run through the stop sign just by putting a sticky note on it.

The goal of the TrojAI program is to combat such Trojan attacks by inspecting AIs for Trojans. This page is a list of resources for doing research on detecting Trojan attacks, including a leaderboard for Trojan detectors, code to create AIs with/without Trojans at scale, etc.

Neural Network Calculator is an interactive visualization of neural networks that operates on datasets and neural network coefficients as opposed to simple numbers. The GitHub deployment and repositories of Neural Network Calculator can be found at:

 

Created December 9, 2019, Updated February 17, 2021