Performance of Test Supermartingale Confidence Intervals for the Success Probability of Bernoulli Trials

Published

February 5, 2020

Author(s)

Peter Wills, Emanuel Knill, Kevin Coakley, Yanbao Zhang

Abstract

Given a composite null hypothesis H0, test supermartingales are non-negative supermartingales with respect to H0 with an initial value of 1. Large values of test supermartingales provide evidence against H0. As a result, test supermartingales are an effective tool for rejecting H0, particularly when the p-values obtained are very small and serve as certificates against the null hypothesis. Examples include the rejection of local realism as an explanation of Bell test experiments in the foundations of physics and the certification of entanglement in quantum information science. Test supermartingales have the advantage of being adaptable during an experiment and allowing for arbitrary stopping rules. By inversion of acceptance regions, they can also be used to determine confidence sets. We used an example to compare the performance of test supermartingales for computing p-values and confidence intervals to Chernoff-Hoeffding bounds and the "exact" p-value. The example is the problem of inferring the probability of success in a sequence of Bernoulli trials. There is a cost in using a technique that has no restriction on stopping rules, and, for a particular test supermartingale, our study quantifies this cost.

Citation

Journal of Research (NIST JRES) -

Volume

125

NIST Pub Series

Journal of Research (NIST JRES)

Pub Type

NIST Pubs

Download Paper

https://doi.org/10.6028/jres.125.003

Local Download

Keywords

asymptotics, Bernoulli trials, Chernoff-Hoeffding bounds, confidence intervals, hypothesis tests, large deviations, p-values, test supermartingales

Citation

Wills, P. , Knill, E. , Coakley, K. and Zhang, Y. (2020), Performance of Test Supermartingale Confidence Intervals for the Success Probability of Bernoulli Trials, Journal of Research (NIST JRES), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/jres.125.003, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=923940 (Accessed October 9, 2025)