Six Impossible Things Before Breakfast: Assigning Key Comparison Reference Values in Chemical Measurement

CCQM, the CIPM Consultative Committee for Quantity of Material, coordinates comparisons between National Measurement Institutes and Designated Institutes in the field of chemical and biological measurement. These comparisons cover a range of different problems, from the preparation and analysis of primary gas standards to the measurement of trace organic substances in complex food, biological and environmental matrices. Like many other CIPM comparisons, the basic principle involves circulation of a material or test item, collation of results from laboratories, assignment of a Key Comparison Reference value and assessment of laboratory performance in terms of their deviation from that KCRV. In the longer terms, this performance is relevant to the declaration of Calibration and Measurement Capabilities listed in the CMC database maintained by BIPM.

Some of these studies, particularly those in the gas metrology area, are very similar to intercomparisons of primary standards in other fields, and are based on a long history of measurements of similar substances in familiar and sometimes very simple matrices. These studies typically generate data in which the majority of labs agree very well and in which the uncertainties reported by the participants fully account for the observed dispersion. In most other cases, however, the range of materials to determine and the number of different matrices in which they must be measured is wide; unexpected problems arise and may disproportionately affect a significant fraction of participant results. Although agreement remains very good compared to routine measurements, there is often evidence of unexpected deviation for a particular lab or of some underestimation of uncertainty. These issues can make the assignment of a key Comparison reference value (KCRV) surprisingly contentious.

Against this background, CCQM has historically used a wide variety of treatments for KC and Pilot Comparison data, and is currently seeking to harmonise its approach to KCRV assignment, estimation of KCRV uncertainty and estimation of degrees of equivalence and their respective uncertainties. In 2008, a set of general principles were proposed [1]. Since then, a CCQM ad-hoc working group has been attempting to implement these principles in guidance aimed at KC and Pilot study coordinators intending to provide a KCRV based on a consensus of laboratory results.

This presentation will review the proposed CCQM principles and some typical data sets, and describe a possible framework for putting the principles into practice. This framework relies on identifying the nature and scale of deviations from ideal behaviour and tailoring the data treatment to suit the actual problems presented by the data. It will be shown that even this adaptive approach presents considerable difficulties, some arising from the small size of the data set and some from the often competing desires for formal rigour, statistically valid treatment, simplicity and appropriate differentiation of measurement capabilities among laboratories. Finally, some options for assessing Key Comparison data in the absence of a reliable KCRV will be suggested, perhaps offering alternative ways of evaluating laboratory agreement when an independent, traceable KCRV is not available.

M Cox (2008) Data Evaluation Principles for CCQM Key Comparisons. CCQM reference CCQM-08-08.