Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance of Test Supermartingale Confidence Intervals for the Success Probability of Bernoulli Trials



Peter Wills, Emanuel Knill, Kevin Coakley, Yanbao Zhang


Given a composite null hypothesis H0, test supermartingales are non-negative supermartingales with respect to H0 with an initial value of 1. Large values of test supermartingales provide evidence against H0. As a result, test supermartingales are an effective tool for rejecting H0, particularly when the p-values obtained are very small and serve as certificates against the null hypothesis. Examples include the rejection of local realism as an explanation of Bell test experiments in the foundations of physics and the certification of entanglement in quantum information science. Test supermartingales have the advantage of being adaptable during an experiment and allowing for arbitrary stopping rules. By inversion of acceptance regions, they can also be used to determine confidence sets. We used an example to compare the performance of test supermartingales for computing p-values and confidence intervals to Chernoff-Hoeffding bounds and the "exact" p-value. The example is the problem of inferring the probability of success in a sequence of Bernoulli trials. There is a cost in using a technique that has no restriction on stopping rules, and, for a particular test supermartingale, our study quantifies this cost.
Journal of Research (NIST JRES) -


asymptotics, Bernoulli trials, Chernoff-Hoeffding bounds, confidence intervals, hypothesis tests, large deviations, p-values, test supermartingales


Wills, P. , Knill, E. , Coakley, K. and Zhang, Y. (2020), Performance of Test Supermartingale Confidence Intervals for the Success Probability of Bernoulli Trials, Journal of Research (NIST JRES), National Institute of Standards and Technology, Gaithersburg, MD, [online],, (Accessed July 19, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created February 4, 2020, Updated October 12, 2021