Measurement systems such as those based on microarrays routinely produce thousands of parallel data sets. This project seeks to develop an approach to experimental characterization of the technical variation associated with such measurement systems. An approach is proposed that is based on observation of the technical variation through analysis of corresponding parallel data sets. Empirical Bayes methods are applied to summarization of these parallel data sets.
An archetype among biological experiments is gene-by-gene determination of the expression difference between two groups of biological units. For each gene, the experiment yields a data set consisting of one or more expression values for each unit. Gene to gene, these data sets can be called parallel although the expression differences may be unrelated. A gene-expression microarray gives expression values for thousands of genes, in other words, a response with thousands of dimensions. Thus, measurement systems based on gene-expression microarrays can give thousands of parallel data sets, one for each dimension. Other measurement systems can give even more parallel data sets. Such systems are the topic of this project.
Instead of something in the subject matter, measurement systems themselves can be the object of experimentation. Experiments are performed on measurement systems for assessment of the technical variation, the variation associated with repeated measurements on the same material. Steps to reduce the technical variation may then follow. This project presents a class of such experiments suitable for large-scale parallel measurement systems.
Measurement systems produce measurements according to the levels of the measurand in the materials measured. It follows that test materials define a measurement system experiment. The general experimental concept is interpretation of test material measurements in terms of what is known a priori about the measurand levels in the test materials. The class of experiments considered here involves biological units with differences that reflect only the unit-to-unit biological variation that is unavoidable despite the similar handling of the units. From each biological unit, two test materials with differences in measurand levels are extracted. These differences are intended to be in a useful range for a large proportion of the system dimensions. From each pair of test materials, additional test materials are obtained by mixing. What is known a priori is that for test materials from a particular biological unit have measurands that are related because of the test material mixing and that analogous test materials from different units have similar measurands because of the similar unit handling.
Technical variation, which is the target of a measurement system experiment, has structure that is linked to changes in measurement conditions. These conditions can be held constant (repeatability conditions), or certain conditions can be allowed to change (reproducibility conditions). Measurements made under constant conditions constitute a batch. A set of measurements may be made in several batches that are delineated by changes in specific measurement conditions. Technical variation observed within a batch is usually considered random variation. Variations observed among batches beyond what is observed under constant conditions are batch effects. The most common instances of batch effects are laboratory effects and operator effects. Characterization of a batch effect through a measurement system experiment requires that the batch delineation be suitably imposed on the measurement of the test materials.
For each dimension, the measurement system experiments considered here produce a data set consisting of univariate measurements made on the test materials and made under different measurement conditions. Modeling of such a data set leads to characteristics of the technical variation for a particular dimension. The modeling includes testing the hypothesis that a particular change in measurement conditions produces no batch effect. The modeling also includes comparison of the sizes of the batch effects with the size of the variation in the biological units.
Characterization of the technical variation for large-scale measurement systems requires summarization over the dimensions, that is, summarization of the parallel modeling of the parallel data sets. Bradley Efron's book Large-Scale Inference presents an empirical Bayes approach to such summarization. This approach is applied to large-scale measurement system experiments in this paper. Efron's book focuses on the comparison of two groups of biological units. The parallel modeling in measurement system experiments requires some adaptations.
Submitted the following paper for publication and presented it at the 2010 Joint Statistical Meetings:
Submitted the following paper for publication:
Status: Rejected by three journals.
These comments suggest that this paper be rewritten for a statistics audience with emphasis on what is new in the design and analysis.