Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

The ghost in the machine: Don't let it haunt your software performance measurements



Vreda Pieterse, David Flater


This paper describes pitfalls, issues, and methodology for measuring software performance. Ideally, measurement should be performed and reported in such a way that others will be able to reproduce the results in order to confirm their validity. We aim to motivate scientists to apply the necessary rigor to the design and execution of their software performance measurements to achieve reliable results. Repeatability of experiments, comparability of reported results, and verifiability of claims that are based on such results can be achieved only when measurements and reporting procedures can be trusted. In short, this paper urges the reader to measure the right performance and to measure the performance right.
Technical Note (NIST TN) - 1830
Report Number


Pieterse, V. and Flater, D. (2014), The ghost in the machine: Don't let it haunt your software performance measurements, Technical Note (NIST TN), National Institute of Standards and Technology, Gaithersburg, MD, [online], (Accessed May 23, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created April 27, 2014, Updated October 12, 2021