Statistical power; False Report Probability; True Report Probablity; Winner's curse; Type M error; Type S error; Alcohol; Deprivation effect; Relapse; Treatment
Abstract :
[en] While adequate power and sample sizes are indispensable for the detection (by the statistical
test) of a meaningful effect size (ES), many published studies in psychology do not describe
sample-size calculation (SSC), which weakens the study methodological quality. Omission of
SSC is often associated with a lack of prospective power, which exaggerates observed ES and
increases the False Report Probability (FRP), thereby jeopardizing results reproducibility. Our
purpose is to investigate in which extent this practice concerns the literature assessing the
efficacy of the psychological and pharmacological treatments of the alcohol-deprivation
effect (operationalized in an animal model). We will firstly select articles published from 1993
to 2020 using the database PubMed and check whether they mention a SSC. We will then
classify the articles mentioning a SSC according to five components of a complete description
of SSC (ex. power analysis with or without details). We will also check whether the
hypothetical ES (used to determine sample sizes) is justified and the observed ES interpreted
(discussion). Thereupon, in order to assess a possible ES overestimation in the selected
literature, we will examine the relationship between the observed ES and the sample sizes.
We will compute the “power-to-detect” of each relevant statistical test using small, medium
and large ES (classifications). Finally, we will compute the FRP and the True Discovery Rate
using a risk alpha of 1, 2.5 or 5%, the median “power-to-detect” and a representative range
of pre-study odds (from 0.01 to 0.99), according to Ioannidis (2005) and Szucs & Ioannidis
(2017).
Disciplines :
Animal psychology, ethology & psychobiology Social & behavioral sciences, psychology: Multidisciplinary, general & others
Author, co-author :
Léonard, François ; Université de Liège - ULiège > Département de Psychologie > Méta-recherche et éthique de la méthodologie quantitative
Tirelli, Ezio ; Université de Liège - ULiège > Département de Psychologie > Méta-recherche et éthique de la méthodologie quantitative
Language :
English
Title :
Are the studies (tests) assessing the efficacy of the treatments of the alcoholdeprivation effect underpowered?