Are Studies Published in Open Access Journals of Higher Statistical Quality Compared to Studies Published in Subscription Journals

Scientists increasingly publish in Open Access (OA) journals, which offer free access to scientific literature and are considered more transparent and rigorous in their editorial and peer-review processes. While Open Science promotes methodological and statistical quality, no study has examined whether this principle is reflected in OA versus subscription-based journals in psychological sciences. This study addresses this gap by comparing the statistical quality of publications assessing the efficacy of psychotherapies for depression in adults.

We retrieved 467 effects from 357 articles in the Metapsy database (Miguel et al., 2022), including 199 effects from 167 OA journal articles and 268 effects from 190 subscription journal articles. We compared observed effect sizes, sample sizes, and statistical power across publication types.

Effects in OA journals exhibited (i) smaller effect sizes, (ii) larger sample sizes, and (iii) higher statistical power compared to subscription journals. Median statistical power was below the recommended 80% threshold for small (d = 0.2) and medium (d = 0.5) effect sizes in both publication types. However, for large effect sizes, OA journal articles had a median statistical power above 80%, while subscription journal articles remained below this threshold.

Our findings suggest that OA journals publish studies with higher statistical quality, likely due to their greater transparency and rigorous standards.