Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Screening for factors affecting application performance in profiling measurements

Published

Author(s)

David W. Flater

Abstract

This report provides an example application of screening techniques in experimental computer science, including validation and selection of metrics and measures, the screening experiment itself, and supporting statistical methods. Together with related references, it is intended to encourage other computer scientists to make more use of established statistical methods in performance evaluations. The described screening experiment sought to identify which of three factors (environment, frame pointers, and profiler options) affected the performance of the application under test. The results showed that they all had a measurable impact, but the shapes and ranges of the distributions of measurement results would likely make their impacts immaterial in experiments using smaller samples. The validation phase of this experiment yielded evidence that the ill-behaved distributions are attributable to actual performance variations rather than measurement error.
Citation
Technical Note (NIST TN) - 1855
Report Number
1855

Keywords

software, performance, measurement, profiling

Citation

Flater, D. (2014), Screening for factors affecting application performance in profiling measurements, Technical Note (NIST TN), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.TN.1855 (Accessed October 9, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created October 30, 2014, Updated November 10, 2018