Statistical Engineering Division has had an ongoing collaboration with NIST's Precision Engineering Division (now Semiconductor and Dimensional Metrology Division) and Office of Law Enforcement Standards (now in the Office of Special Programs) in the field of firearms and tool mark identification. These efforts have provided a more scientific basis for identification of firearms, helped ballistics analysts improve their procedures, and explored topographic approaches to firearms identification that improves on the older image-based approaches.
The collaboration began with statistical support for NIST's reference materials for the firearms community, SRM 2460 Standard Bullet and SRM 2461 Standard Cartridge Case. These reference materials have since been used in two exercises (the National Ballistic Imaging Comparisons I & II) that are assisting quality control efforts in firearms identification laboratories.
This collaboration intensified with the NIST Ballistic Imaging Database Evaluation (NBIDE). One of the fruits of NBIDE was an experiment following a design by NIST statistician Jim Filliben, with the resulting analysis of the results provided important insights on the feasibility of a ballistic database using then current technology. One of its most far-reaching conclusions was the superiority of 3D topography measurements to the gray-scale picture images for firearms identification.
Meanwhile, the 2009 National Academy of Sciences report on the state of forensic science has brought new emphasis on the scientific basis of forensic disciplines, in particular the estimation of error rates for procedures. Thus, there has been an intensification of research efforts related to more objective, algorithmic firearms identification methods and their associated error rates. Modeling the distributions and likelihoods of associated similarity or correlations metrics is a promising approach that still remains problematic.
John Song of the NIST Surface Metrology Group and collaborators has developed a Congruent Matching Cell (CMC) method for matching breech face topographies.It involves dividing the surface of a breech face into cells and finding the resulting CMC,which is the count of how many cells "match" in terms of high point-by-point correlation while remaining "congruent" (spatially aligned). While the method has shown great promise and very good results in a pilot study, its performance properties are still to be well characterized.
For the data so far, a simple binomial model for the number of matching cells fits the CMC distribution of the known non-match (KNM) casing pairs; however, the CMC distribution of the known match (KM) casing pairs is too complex and over dispersed for a traditional binomial.A natural generalization is to use the Beta Binomial distribution instead of the binomial. One justification is that in a Bayesian framework, the Beta distribution for p (=probability that a cell pair matches), is the conjugate distribution of the Binomial. The resulting posterior distribution is the beta binomial.
Dan Ott of the Surface Metrology Group has proposed a discretized and truncated normal distribution for a mixture of binomial distribution originating from a normal distribution of parameter p ; while such a model has some advantages, its properties and ultimate validity are still being studied and tested.
So far, the beta binomial provides a much better (though still not great) fit for the over dispersion in the empirical CMC distribution.There is continuing theoretical work on how (if at all) it captures the correlations that might occur because of the cells being on the same breech face surface. Ott's normal-binomial is also an improvement over the traditional binomial. There is an understanding that the current pilot study data set can probably not be tightly fit by any standard distribution with only 2-3 parameters.
Another possibility is to consider families of distributions that are more general in the freedom to vary p between cells as well as between casings, such as the Lexian Distribution. Also, there are other possible avenues on how to incorporate possible correlations between cells,such as the Lazarsfeld-Bahadur expansion. Both of these were suggested by N. F. Zhang. Another possibility is empirical likelihood, which is in the process of being tried in this and other forensic fields as well by H.Iyer and S.Lund.
These modeling issues have to the fore because of the NAS report's mention of the need for error rates for forensic sciences procedures.In addition, NIST's own long history in uncertainty analysis has propelled work in error rates and ways to address uncertainty in error rates, which is still a matter of some controversy and continuing study. There is current work delving into how similar or different these tool mark analyses are from current DNA analyses, and how different, if at all, should be the resulting analysis.This work encompasses exploration of likelihood ratios including those of score-based methodologies.The uncertainty analysis includes but is not limited to various types of bootstrap analysis.