Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Fitting Weighted Total Least-Squares Planes and Parallel Planes to Support Tolerancing Standards

Published

Author(s)

Craig M. Shakarji, Vijay Srinivasan

Abstract

We present elegant algorithms for fitting a plane, two parallel planes (corresponding to a slot or a slab) or many parallel planes in a total (orthogonal) least-squares sense to coordinate data that is weighted. Each of these problems is reduced to a simple 3×3 matrix eigenvalue/eigenvector problem or an equivalent singular value decomposition problem, which can be solved using reliable and readily available commercial software. These methods were numerically verified by comparing them with brute-force minimization searches. We demonstrate the need for such weighted total least-squares fitting in coordinate metrology to support new and emerging tolerancing standards, for instance, ISO 14405-1:2010. The widespread practice of unweighted fitting works well enough when point sampling is controlled and can be made uniform (e.g., using a discrete point contact Coordinate Measuring Machine). However, we demonstrate that nonuniformly sampled points (arising from many new measurement technologies) coupled with unweighted least-squares fitting can lead to erroneous results. When needed, the algorithms presented also solve the unweighted cases simply by assigning the value one to each weight. We additionally prove convergence from the discrete to continuous cases of least-squares fitting as the point sampling becomes dense.
Proceedings Title
Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference
Volume
2
Conference Dates
August 12-15, 2012
Conference Location
Chicago, IL
Conference Title
The ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference

Keywords

coordinate metrology, computational metrology, least-squares, plane, plane fitting, point sampling, sampling, surface fitting, total least-squares, weighted least-squares fitting, weighted fitting, parallel plane fitting, tolerance

Citation

Shakarji, C. and Srinivasan, V. (2012), Fitting Weighted Total Least-Squares Planes and Parallel Planes to Support Tolerancing Standards, Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Chicago, IL, [online], https://doi.org/10.1115/DETC2012-70398 (Accessed May 28, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created October 19, 2012, Updated November 10, 2018