Building Intelligence for First Responders: From Foundation Models to Interoperable Indoor Data

When a disaster strikes outdoors, first responders are able to capture data from a variety of sources to adequately prepare their response. Satellite imagery, GIS, GPS, and drones all provide the ability to gather information about the scene. However, when a disaster happens indoors, none of those capabilities carry over. The available data becomes 2D, typically a blueprint if it exists, and does not include standpipes, shutoffs, alarm panels, or AEDs. These missing elements are necessary for a safe and timely response by first responders. Without indoor spatial intelligence, responders must make high stakes and high-risk decisions with incomplete information. Fixing the issue requires data that cannot be produced at scale and currently relies on manual documentation. NIST’s Communications Technology Laboratory (CTL) Public Safety Communications Research (PSCR) Innovation Lab is tackling indoor mapping to address this issue.

The Gap and the Mandate
Indoor mapping is difficult for three specific reasons:

• No Global Navigation Satellite System (GNSS) Signal: Building materials attenuate satellite signals, limiting access to internal mapping, and resulting in limited information during pre-incident planning stages.
• No Aerial Access: Drones and satellites provide fast and broad mapping capabilities from above, but interiors must be surveyed room by room, with heavy occlusion from walls, furniture, and debris, resulting in slower, incomplete, and more dangerous reconnaissance.
• Low Resolution 2D Records: Current records (when they exist) are schematic, can be outdated, and carry no reliable geometry for ceiling height, vertical connections, or clutter, meaning first responders cannot judge traversability, hose or ladder reach, or crawl space availability from the data provided.

NFPA 1660 and NFPA 950, safety rulebooks published by the National Fire Protection Association (NFPA), require fire departments and first responders to have detailed, interoperable pre-incident data available; but departments currently have no way to produce it at scale, and this data must be produced manually. Compliance continues to stall as coverage remains limited and selective.

Applicable Changes
PSCR is addressing this issue by testing an automated pipeline, called INSIGHT (Indoor Scene Intelligence from Geometric-Semantic Hierarchy Transfer), that turns building scans into structured data that can support pre-incident planning, incident command, navigation, and scene documentation. While current work focuses strictly on the pre-incident phase of indoor disaster response, future phases of the work include live tracking and command applications. Implementing automation lowers the overall cost of an existing requirement and produces faster results. The pipeline remains compliant with the NIST AI Risk Management Framework (RMF) by keeping a human reviewer in the loop to correct a generated graph produced by the automation, which is exponentially less work and timelier than labeling and producing the data from scratch.

Safety fixtures, such as exit signs, alarm panels, and AEDs, are small, visually varied, and rare in existing training data. Purpose built 3D models score near zero in these classes. PSCR measured this directly in the 2025 cross dataset analysis. Foundation models recognize these features without task specific training. Registered depth places them in 3D, so a building gets labeled without being manually annotated. Geometry supports navigation while semantics support intelligence and planning.

Building Trustworthy Results
Without proper verification, results could endanger first responders by providing inaccurate scene understanding. The NIST AI RMF requires explainable and accountable automation use in high risk and high consequence settings and is directly applicable to the work PSCR is doing in this space. Independent detection paths within the pipeline cross check each other and disagreement between the paths is flagged for human review. The available building code provides data and boundaries for any implausible results while a human reviewer stays in the loop to verify and ensure accuracy. These boundaries help strike the balance in providing quick results that don’t sacrifice the accuracy of the location data.

Practical Application
Pre-incident planning is an immediate application as the pipeline runs on data collected ahead of time, and the output structure can be adapted given the mission at hand, controlling what crews see and supporting additional analytics and measurements. The resulting information reaches responders in a matter of seconds over existing broadband infrastructure, providing a level of detail suited to knowing what a space contains and how to navigate it. The pipeline is built on a public dataset and open-sources tools, creating a reproducible approach not tied to proprietary systems and readily available for application. An additional feature allowing role filtering at query time serves dispatch, incident command, and discipline specific views, allowing responders to select which view is most relevant.

NIST Contributions
One major contributor to this research is PSCR funded Point Cloud City, one of the few indoor 3D datasets with public safety features rather than general scene categories. The 2025 cross dataset analysis used data from Point Cloud City to measure where existing 3D methods fail, finding near zero performance on small safety fixtures (i.e. exit signs, AEDs, alarm panels). PSCR’s INSIGHT responds to that measured gap by producing labeled 3D data and scene graphs without requiring manual documentation and annotation of the target building. The output is reproducible, following ISO 19164, IndoorGML, and IFC naming rather than a proprietary format in addition to providing the source code and dataset documentation through publication. The NIST AI RMF shapes the design with a second detection path that can be inspected at every stage and human review before using the outputs. PSCR plans to extend work in this space to include live tracking and command applications, providing additional resources to first responders, and creating more accurate and timely data applications to minimize risk and increase safety for first responders.