The Statistical Tests Used in the Literature on Conditioned Place Preference Induced by Nicotine in Mice Are Underpowered

Introduction: In recent decades, concerns have been raised about the validity and reliability of many research results. Inadequate research practices, including insufficient statistical power, have contributed to the replication crisis. The statistical tests of numerous studies in the neurobehavioral sciences are often underpowered, with inflated effect sizes and high rates of false discoveries. To explore these issues in experimental psychopharmacology, this study focused on the rewarding effects of nicotine using the conditioned place preference (CPP) task in mice. Methods: We assessed, across a set of selected articles, whether the statistical power of the included tests, calculated using external small, medium, and large hypothetical effect sizes (Cohen’s f and d), reached the conventional .80 threshold, and how these powers related to the observed effect sizes. We also examined the association between observed effect sizes and sample sizes and estimated Positive Predictive Value (PPV) and False Discovery Rate (FDR) for pre-study probabilities of H1 being true ranging from .001 to .99. Results: Across 61 articles (1995–2024) comprising 129 statistical tests, all calculated powers fall below the conventional 0.80 threshold for small and medium hypothetical effect sizes, and only 7.1 to 41.7 percent reach this level for large effect sizes. None of the studies report a complete and justified power analysis or any other formal sample size justification. Effect sizes are negatively related to both power and sample size, with smaller, underpowered studies tending to report inflated effects. As a result, many studies, especially those with low or medium levels of pre-study probabilities, show reduced PPV and elevated FDR. Discussion: This study highlights the lack of statistical power in CPP nicotine research in mice, suggesting that much of the published evidence may be unreliable, inflated, or difficult to replicate. We encourage researchers, while providing a few avenues for reflection, to identify the smallest effect sizes that meaningfully capture nicotine CPP and, on this basis, to define realistic hypothetical effects for sample size planning.