Baseline Pruning-Based Approach to Trojan Detection in Neural Networks

Published

May 7, 2021

Author(s)

Peter Bajcsy, Michael Majurski

Abstract

This paper addresses the problem of detecting trojans in neural networks (NNs) by analyzing how NN accuracy responds to systematic pruning. This study leverages the NN models generated for the TrojAI challenges. Our pruning-based approach (1) detects any deviations from the reference NN models, (2) measures the accuracy of a set of systematically pruned NN models using multiple pruning configurations, and (3) classifies each NN model as clean or poisoned by learning a mapping between accuracy measurements and reference clean or poisoned NN model labels. This work outlines a theoretical and experimental framework for finding the optimal mapping over a large search space of pruning parameters. Based on our experiments using Rounds 1 - 4 TrojAI Challenge datasets, the approach achieves average classification accuracy between 68.51 % and 91.06 %. Reference model graphs and source code are available from GitHub.

Proceedings Title

Proceedings of the International Conference on Learning Representations (ICLR) 2021, Security and Safety in Machine Learning Systems Workshop

Conference Dates

May 3-7, 2021

Conference Location

virtual, MD, US

Conference Title

Security and Safety in Machine Learning Systems Worksho

Pub Type

Conferences

Download Paper

Local Download

Keywords

artificial intelligence, trojan attacks, AI model pruning

Citation

Bajcsy, P. and Majurski, M. (2021), Baseline Pruning-Based Approach to Trojan Detection in Neural Networks, Proceedings of the International Conference on Learning Representations (ICLR) 2021, Security and Safety in Machine Learning Systems Workshop, virtual, MD, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931789 (Accessed November 24, 2025)