Reflections from the Second NIST Cyber AI Profile Workshop

Thank you to everyone who participated in the Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile) Workshop in January! The input we received on the Preliminary Draft during this workshop has been invaluable and is informing the development of the next draft of the NIST Cyber AI Profile. We are working toward publishing a full workshop summary soon that captures themes and highlights from the event. In the interim, we would like to share a preview of what we heard…

What We Heard During the Workshop

During workshop discussions, participants expressed support for the development of the Cyber AI Profile and appreciated that it is filling gaps in much needed guidance. Participants emphasized the need for both enterprise risk management and implementation level resources, especially to benefit smaller organizations or organizations overwhelmed with addressing strategic adoption, integration, and use of AI (while continuing operational cybersecurity risk management activities). Below are some key themes we heard from participants during the workshop:

• Agentic AI: Agentic AI may require special considerations with respect to all three Focus Areas outlined in the Profile (and more agentic AI examples need to be included).
• Longevity of Profile and Innovation: Participants requested that the Profile not include guidelines that are too specific. Technology is changing quickly, and overly specific content will keep the Profile from remaining usable and helpful as innovation occurs.
• Need for Consistent AI Taxonomy: Participants stated that the Cyber AI Profile is on track to provide a consistent, industry-agnostic AI taxonomy that will enable organizations across industries to communicate clearly about a wide range of AI topics.
• Use Cases: Participants said that it would be helpful to include use cases (e.g., operational technology (OT) cybersecurity) and illustrative examples in the Profile.
• Ideas for Enhancing the Usability of the Profile: Participants suggested creating a more flexible format for filtering the profile (i.e., workbook) as well as providing a machine-readable format and increasing the use of hyperlinks.
• AI Governance and Accountability: AI governance remains top of mind and current approaches are varied. Participants shared insights regarding evolving practices related to roles and responsibilities, including multidisciplinary AI governance bodies and new roles like chief AI officers. Questions remain open regarding how to manage the notion of accountability relative to AI cybersecurity outcomes.
• Guidelines on Testing and Evaluation: Participants voiced a common challenge in testing and managing AI systems and tools. Performance metrics, certifications, and benchmarking were discussed as means to support AI system testing and evaluation.
• Cybersecurity’s Role in Trustworthiness of AI Decisions: Participants highlighted how cybersecurity can help with challenges in adopting AI. Cybersecurity measures can help address sources of concern, such as insider threats posed by AI, “shadow AI,” and lack of visibility into how AI systems are trained.
• Transparency, Integrity, and Accountability: Participants mentioned that considerations in the Profile should highlight the need to enhance transparency and accountability to support cybersecurity objectives. Cybersecurity measures such as cryptographic signing, certification systems, and the inclusion of AI Bills of Materials (AIBOM) that address needs related to confidentiality, integrity, and availability were suggested as means to ensure the integrity of AI systems across the supply chain.
• Continued Need for Human-in-the-Loop for Cybersecurity: Participants expressed that human-in-the-loop (HITL) processes and training remain critical at this time in AI adoption to ensure effective implementation and oversight of AI systems used for cybersecurity and noted that HITL remains the current standard for AI accountability.

These takeaways are instrumental in both creating the next draft of the Profile and identifying the community’s areas of priority as AI continues to evolve.

Additional Workshop Highlights

NIST also shared updates regarding the AI and cybersecurity projects below. These projects at NIST work together and inform one another's efforts in advancing AI technologies, managing AI-related risks, and ensuring the safe and secure use of AI:

• AI Risk Management Framework (AI RMF)
• Center for AI Standards and Innovation (CAISI)
• Adversarial Machine Learning
• Dioptra
• Secure Software Development Framework (SSDF) AI Profile
• PETs Test Bed
• Agent Identities
• NCCoE Chatbot
• AI Accelerators
• COSAiS

Next Steps

The Cyber AI Profile team is reviewing and analyzing the 1,400+ comments received during the workshop and the public comment period and determining next steps in the public engagement process. Stay tuned for more information regarding future community of interest working session and workshop dates, as well as the next draft of the Cyber AI Profile (the Initial Public Draft)! You can find our latest roadmap here.

As always, you are welcome to email us at CyberAIProfile [at] nist.gov (CyberAIProfile[at]nist[dot]gov). If you have not already done so, please consider joining our COI to be notified about opportunities for community engagement and receive updates on the Cyber AI Profile.