NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.
Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.
An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
We investigate the influence of finite resolution on measurement uncertainty from a perspective of the Guide to the Expression of Uncertainty in Measurement (GUM). Finite resolution in the presence of Gaussian noise yields a distribution of results that strongly depends on the relative location of the true value to the resolution increment. We show that although there is no simple expression relating the standard deviation of the distribution to the uncertainty at a specified level of confidence, there is an analytic relation between the mean value and the standard deviation. We further investigate the conflict between the GUM and ISO 14253-2 regarding the method of evaluating the standard uncertainty due to finite resolution and show that, on average, the GUM method is superior, but still approximate.
Proceedings Title
Proceedings of ASPE Spring Topical Meeting on Uncertainty Analysis in Measurement and Design
Conference Dates
June 30-July 1, 2004
Conference Location
State College, PA
Conference Title
ASPE Spring Topical Meeting on Uncertainty Analysis in Measurement and Design
Pub Type
Conferences
Keywords
A/D, analog to digital conversion, error, resolution, uncertainty
Phillips, S.
, Tolman, B.
and Estler, W.
(2004),
Uncertainty Due to Finite Resolution Measurements, Proceedings of ASPE Spring Topical Meeting on Uncertainty Analysis in Measurement and Design, State College, PA
(Accessed October 14, 2025)