Interactive Simulations of Backdoors in Neural Networks

Published

May 21, 2024

Author(s)

Peter Bajcsy, Maxime Bros, Matthew Coudron

Abstract

This work addresses the problem of planting and defending cryptographic-based backdoors in artificial intelligence models. The motivation comes from our lack of understanding and the implications of using cryptographic techniques for planting undetectable backdoors under theoretical assumptions in the large AI model systems deployed in practice. Our approach is based on designing a web-based simulation playground that enables planting and defending cryptographic backdoors in neural networks (NN). Simulations of planting and activating backdoors are enabled for two scenarios (extension of NN model architecture to support digital signature verification and minimal source code modification). Simulations of backdoor defense against backdoors are available based on proximity analysis and provide a playground for a game of planting and defending against backdoors. The simulations are available at \hrefhttps://pages.nist.gov/nn-calculator}https://pages.nist.gov/nn-calculator}

Citation

Cornell University arXiv

Pub Weblink

https://arxiv.org/abs/2405.13217

Pub Type

Websites

Download Paper

Local Download

Keywords

AI model backdoors, web-based interactive simulations, foundational AI

Citation

Bajcsy, P. , Bros, M. and Coudron, M. (2024), Interactive Simulations of Backdoors in Neural Networks, Cornell University arXiv, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=957883, https://arxiv.org/abs/2405.13217 (Accessed March 7, 2026)