AI Research - Security and Resilience

The trustworthiness of AI technologies depends in part on how secure they are. The NIST AI Risk Management Framework (AI RMF) identifies “Secure and Resilient” as one of the primary characteristics of AI trustworthiness. The NIST Information Technology Laboratory (ITL) conducts research and develops guidelines that improve the security and resilience of AI applications and mitigate or manage their  vulnerabilities. 

Some cybersecurity risks related to AI systems are common (or identical) to cybersecurity risks across  software development and deployment. Overlapping risks include security concerns related to the confidentiality, integrity, and availability of the system and its training and output data – along with the general security of the underlying software and hardware for AI systems. ITL develops  a wide array of cybersecurity standards, guidelines, best practices, and other resources which complement its portfolio of AI activities.

The security and resilience of AI technologies is an area of active research, and challenges and potential solutions are changing very rapidly. For example, existing frameworks and guidance are unable to comprehensively address security concerns related to evasion, model extraction, membership inference, availability, or other machine learning attacks. They also do not account for the complex attack surface of AI systems or other security abuses enabled by AI systems.

AI technologies also have the potential to transform cybersecurity. They offer the prospect of giving defenders new tools that can address security vulnerabilities and even as they can enhance the capabilities of those seeking to target organizations and individuals through information technology (IT) and operational technology (OT) attacks.

Examples of AI security efforts underway by ITL include:

Workshop on Securing AI Data Center: Architecture, Security Posture, and Emerging Standards

Data centers are the computing infrastructure that powers the training and inference of AI and have become a critical element of national security, economic strength, and technological dominance. To address the urgent need for robust technical standards outlined in America’s AI Action Plan, NIST and the High Performance Computing Modernization Program hosted a virtual workshop on this topic on July 22-23, 2026. Learn More. 

Control Overlays for Securing AI Systems (COSAiS)

NIST ITL is developing a series of Control Overlays for Securing AI Systems (COSAiS). This implementation-focused series of guidelines will address use cases involving different types of AI systems and specific AI system components (e.g., training and test data, model weights and configuration settings). The overlays focus on protecting the confidentiality, integrity, and availability of information and users for each of the following proposed use cases:

• Adapting and Using Generative AI – Assistant/Large Language Model (LLM)
• Using and Fine-Tuning Predictive AI
• Using AI Agent Systems (AI Agents) – Single Agent
• Using AI Agent Systems (AI Agents) – Multi-Agent
• Security Controls for AI Developers

The overlays use the NIST Special Publication (SP) 800-53 controls and will also leverage NIST SP 800-218A, Draft NIST AI 800-1, and NIST AI 100-2e2025.

Platform development and demonstration capability

NIST ITL researchers are developing a platform, Dioptra, that is intended to be a shared resource to further AI research across NIST and the community. The platform serves as a testbed to research and develop metrics and best practices to assess vulnerabilities of AI models and the effectiveness of defenses against AI.  

Secure Software Design 

AI systems are built and operate on software. Security concerns of any data or information system apply to AI systems. In addition to the security concerns of traditional software, it is important to govern, map, measure, and manage AI-specific risks.  NIST ITL worked with interagency collaborators – and sought feedback from the broader community through open and transparent processes – to develop a companion to the NIST Secure Software Development Framework (SSDF). NIST held a virtual workshop on Secure Development Practices for AI Models and released a draft Generative AI companion resource that incorporates secure development practices for generative AI and dual-use foundation models. That document was finalized after receiving public comments. 

Taxonomy and Terminology: Attacks and Mitigations

In March 2025, NIST ITL finalized a report on  Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2e2025).  The report develops a taxonomy of concepts and defines terminology in the field of adversarial machine learning. Created with input from industry and academia, the finalized report includes updates to reflect recent technological developments and a new index of attacks and mitigations to improve the document’s usability. 

AI in Critical Infrastructure 

NIST ITL is supporting Critical Infrastructure (CI) sectors by launching the development of the AI RMF Trustworthy AI in Critical Infrastructure Profile. This profile will guide CI operators towards specific risk management practices to consider when engaging AI-enabled capabilities. It will then help them communicate their trustworthiness requirements in an actionable way to teams, developers, and other stakeholders across the AI and CI lifecycles and supply chains.