The Power of Effect Size Stabilization

Determining an appropriate sample size in psychological experiments is a common challenge, requiring a balance between maximizing the chance of detecting a true effect (minimizing false negatives) and minimizing the risk of observing an effect where none exists (minimizing false positives). A recent study proposes the use of effect size stabilization, a form of optional stopping, to define sample size without increasing the risk of false positives. In effect size stabilization, researchers monitor the effect size of their samples throughout the sampling process and stop sampling when the effect no longer varies beyond predefined thresholds. This study aims to improve our understanding of effect size stabilization properties. Simulations involving effect size stabilization are presented, with parametric modulation of the true effect in the population and the strictness of the stabilization rule. Results indicate that optional stopping based on effect size stabilization consistently yields unbiased samples over the long run, as previously demonstrated. However, simulations also reveal that effect size stabilization does not guarantee the detection of a true effect in the population. Consequently, researchers adopting effect size stabilization put themselves at risk of increasing type-2 error probability. Instead of using effect size stabilization procedures for testing, researchers should use them for their intended purpose: Reaching accurate parameter estimates.