Running the R-code Program

PT Test Evaluation

1. Open and review the data file containing your verification data (for example, LL 107, HH 109, or SH 102). This is a ".csv" comma delineated ASCII file that can be opened in a spreadsheet.
   

2. Search the data file for your mean value and check that this row contains your individual test values.

• Make your test data the first row in the file after the header information (cut and then insert cut cells in row 4).

• Make sure to move your data (not copy) to the new position so that your data does not appear twice in the file.

• Save the NIST PT analysis file to the "NIST PT Folder" (c:\documents and settings\NIST PT Folder).
  

1. Open the R software on your computer.

• From the file menu "open script" for the NIST PT analysis program.

• The "path" for where your files are located is defaulted to be in a folder on your desktop named "NIST PT Folder". You can edit the script of the NIST program and change the "path" if needed.

• Change the "id" in the NIST PT analysis script to reflect the name of the verification data file that you want to evaluate (without the .csv extension).

1. To run the program, click in the window containing the script (to select it), open the "Edit" menu option and select "Run all". The program may take a minute or two to run if there are a large number of laboratories in the data file.
   

1. The NIST PT analysis program will produce one ASCII file with numerical results as well as two .jpeg files containing plots of the results (see example on right). These file will be saved in the "NIST PT Folder".
   

• The ASCII file gives the mean energy and standard deviation for the machine under test, and the individual absorbed energy values for the test. It also provides the consensus value (J) of the machines tested, the standard deviation between the machines, and the percentiles for the distribution of the machine averages. The Z-score for the machine under test is also given (ratio of the standard deviation for the machine to the standard deviation among machines).
  

• The first plot displays percent differences from the consensus value for all machines selected for the test. The statistics shown at the top of the graph are based on all machines except for the machine under test. The blue horizontal lines on the plot represent twice the standard deviation among machines (on a percentage scale). The red horizontal lines depict three times the standard deviation among machines. The percent difference for the machine under test is shown in red on the far left side of the plot.

• A second plot displays the standard deviation versus the average of the verification test data for the SRM of interest. The plot only includes machines within two standard deviations among machines (the horizontal blue lines on the first plot) so that we can compare the machine under test to the best performing machines. If the machine under test is outside the two standard deviation limits, it will not be displayed on the second plot. The plot also displays a histogram of the standard deviations (right) with a blue line on the histogram indicating the 95.0 percentile of the distribution of standard deviations. A histogram of averages is shown at the top of the plot with blue lines indicating the 2.5 and 97.5 percentiles of the distribution of machine averages. The machine under test is excluded from the calculation of percentiles and appears as a red solid dot on the main plot.

1. Repeat steps 3 and 4 for each SRM ID of interest.

2. In the example given:
   

• The text file (and first plot) show the consensus value is 15.52 J, and that 115 test results were used to generate statistics for this particular proficiency test. The standard deviation among machines is 0.88 J. The machine under test is within the blue limits (two standard deviations among machines), so it is compared with other machines that performed within these limits in the second plot. If the difference for the machine had been greater than two standard deviations, this would indicate that the machine performance should be questioned, in the context of how the machine is used and what constitutes an acceptable level of performance for that application. A z-score of greater than 2 would also indicate that the machine performance should be questioned.
  

• The second plot indicates that the standard deviation for the machine under test is typical for this SRM (about 0.75 J) and that the mean energy is within the bounds displayed on the top of the histogram. The mean value for the machine under test is somewhat low compared to the performance of the other machines, but the mean is within the bounds (2.5% to 97.5%). So, because the variation of the machine is reasonable, the mean energy is within bounds, and the z-score is less than 2, the performance of this machine is not questioned.
  

Technical questions regarding R software and the NIST PT analysis program can be addressed to Jolene Splett ( jsplett [at] boulder.nist.gov (jsplett[at]boulder[dot]nist[dot]gov)).