NIST has released Validation in Forensic Science: Guiding Principles for the Collection and Use of Validation Data (NIST IR 8589), which builds on ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories and the ISO 21043 Forensic Sciences series. It lays out five principles for how validation studies should be designed, documented, and used. To help laboratories put these principles into practice, NIST has developed VAST (Validation Assessment Tool). VAST is a free application designed to help non-statisticians interact with available performance data and provide summaries relevant to interpreting a result given the conditions of a particular case.
VAST takes validation data and produces performance characterizations with uncertainty, using a distribution-free approach suitable for small sample sizes. Users point and click to look for trends showing which factors affect performance, then filter and split to focus analysis on data from circumstances most relevant to a case at hand. Users can quickly export a report for documentation and review.
Performance measures include assessments of repeatability and reproducibility, along with discrimination measures including Receiver Operating Characteristic (ROC) curves, Area Under the Curve (AUC) values, and observation rate ratios.
VAST is currently in beta testing with a public release planned for later this year.