Impact of dependency on the distribution of p-value

We study the impact of dependence assumptions on the distribution of p-values and quantiles for repeated testing on dependent data. This leads us to considering the general problem of the quality of a binomial approximation to the distribution of a sum of dependent indicator variables. Whenever possible we use classical and adhoc versions of Stein’s method to provide tight bounds on classical probability distances. In many cases, however, the relevant expressions are intractable and we resort to empirical analysis by extensive simulations. We apply our findings to a realistic real-life scenario.