Statistical Power in Meta-Analysis: Distinguishing Retrospective and Prospective Contexts

Background: Statistical power is a central concept in confirmatory research. It informs the minimal sample size to detect a hypothetical effect size for a given alpha level and statistical test. In systematic review with meta-analysis, high statistical power is often assumed to be guaranteed. This assumption overlooks two key issues: whether the meta-analysis is conducted in a retrospective or prospective context, and the impact of between-study heterogeneity. Previous work has examined statistical properties of meta-analytic power, but little attention has been paid to its context and interpretation.
Objectives: This poster discusses how to conceptualize and interpret statistical power for the meta-analytic test of the mean effect size, arguing that its meaning depends on whether the meta-analysis is conducted within a retrospective or prospective context.
Methods: We distinguish retrospective and prospective meta-analytic contexts and examine their implications for the interpretation of statistical power. Using simulations under fixed-effect and random-effects models, we examine how the number of studies and between-study heterogeneity affect statistical power.
Results: In retrospective contexts, statistical power does not serve a planning function but reflects an informational property of the available body of evidence, informing the smallest detectable effect size or whether the literature had sufficient power to detect a prespecified effect size. In prospective contexts, statistical power plays a planning role analogous to that in primary research, informing the number of studies required to detect a minimally meaningful effect size. Simulations show that meta-analytic power cannot be reduced to the sum of primary sample sizes, that heterogeneity substantially reduces power under random-effects models, and that increasing the number of studies does not guarantee adequate power when heterogeneity is large.
Conclusion: Statistical power in meta-analysis is a context- and purpose-specific concept. Distinguishing retrospective and prospective contexts clarifies its interpretation, helps prevent misinterpretation of null findings, and supports more principled planning of cumulative evidence.