NIST logo

Publication Citation: Theory and Algorithms for Weighted Total Least-Squares Fitting of Lines, Planes, and Parallel Planes to Support Tolerancing Standards

NIST Authors in Bold

Author(s): Craig M. Shakarji; Vijay Srinivasan;
Title: Theory and Algorithms for Weighted Total Least-Squares Fitting of Lines, Planes, and Parallel Planes to Support Tolerancing Standards
Published: August 16, 2013
Abstract: We present the theory and algorithms for fitting a line, a plane, two parallel planes (corresponding to a slot or a slab) or many parallel planes in a total (orthogonal) least-squares sense to coordinate data that is weighted. Each of these problems is reduced to a simple 3×3 matrix eigenvalue/eigenvector problem or an equivalent singular value decomposition problem, which can be solved using reliable and readily available commercial software. These methods were numerically verified by comparing them with brute-force minimization searches. We demonstrate the need for such weighted total least-squares fitting in coordinate metrology to support new and emerging tolerancing standards, for instance, ISO 14405-1:2010. The widespread practice of unweighted fitting works well enough when point sampling is controlled and can be made uniform (e.g., using a discrete point contact Coordinate Measuring Machine). However, we show by example that nonuniformly sampled points (arising from many new measurement technologies) coupled with unweighted least-squares fitting can lead to erroneous results. When needed, the algorithms presented also solve the unweighted cases simply by assigning the value one to each weight. We additionally prove convergence from the discrete to continuous cases of least-squares fitting as the point sampling becomes dense.
Citation: Journal of Computing and Information Science in Engineering
Volume: 13
Issue: 3
Pages: pp. 031008-1 - 031008-11
Keywords: coordinate metrology; computational metrology; least-squares; line; line fitting; plane; plane fitting; point sampling; sampling; standards; surface fitting; total least-squares; weighted least-squares fitting; weighted fitting; parallel plane fitting; tolerance
Research Areas: Length, Metrology and Standards for Manufacturing Systems and Data, Software, Manufacturing
DOI: http://dx.doi.org/10.1115/1.4024854  (Note: May link to a non-U.S. Government webpage)
PDF version: PDF Document Click here to retrieve PDF version of paper (295KB)