Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Algorithm Testing

Test data points on a cylinder with simulated form error.

The Dimensional Metrology Group (683.01) provides algorithm testing for least-squares fitting of elemental geometries used in coordinate metrology. Specifically, the geometries for which testing is available include lines (2d and 3d data), planes, circles (2d and 3d data), spheres, cylinders, and cones. Testing is modeled after the ASME B89.4.10-2000 standard, Methods for Performance Evaluation of Coordinate Measuring System Software.

The testing process works by NIST supplying data sets to the customer to be run through the software under test. The data sets mimic actual CMM measurements and are designed with imposed, simulated form and measurement errors described in the Standard above. The fit results are then compared with NIST reference fits, and a performance evaluation summarizing the differences is then generated.

Each algorithm calibration includes a NIST calibration report that provides the measured error and uncertainty for each elemental geometry.

Data Set Downloads


There are also 240 data sets along with their reference fits available from NIST at no cost for aid in development or for unofficial, self-testing needs.

Uncertainties


The Algorithm Testing System computes in a precision much greater than the double precision normally used in scientific computing. In fact, 60 digits of working precision are used to compute the reference fits. Because of this, the uncertainty in the least-squares reference fits is limited only by the accuracy of the input. For ASME B89.4.10 default test data sets, which are theoretically exact (i.e., data values are assumed to have infinite trailing zeros) the expanded uncertainty U (k = 2) is much less than 10-14 m for distances and 10-15 radians for angles.


Other Special Tests in the Dimensional Metrology Group >>

Created March 25, 2009, Updated November 15, 2019