Article (Scientific journals)
Exploring scoring methods for research studies: Accuracy and variability of visual and automated sleep scoring
Berthomier, Christian; Muto, Vincenzo; Schmidt, Christina et al.
2020In Journal of Sleep Research
Peer reviewed
 

Files


Full Text
Berthomier_Muto2020Exploring scoring methods for research studies Accuracy and variability of visual and automated sleep scoring.pdf
Author preprint (558.64 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
automatic scoring; visual scoring; scoring variability; large datasets
Abstract :
[en] Sleep studies face new challenges in terms of data, objectives and metrics. This requires reappraising the adequacy of existing analysis methods, including scoring methods. Visual and automatic sleep scoring of healthy individuals were compared in terms of reliability (i.e., accuracy and stability) to find a scoring method capable of giving access to the actual data variability without adding exogenous variability. A first dataset (DS1, four recordings) scored by six experts plus an autoscoring al-gorithm was used to characterize inter-scoring variability. A second dataset (DS2, 88 recordings) scored a few weeks later was used to explore intra-expert variabil-ity. Percentage agreements and Conger's kappa were derived from epoch-by-epoch comparisons on pairwise and consensus scorings. On DS1 the number of epochs of agreement decreased when the number of experts increased, ranging from 86% (pairwise) to 69% (all experts). Adding autoscoring to visual scorings changed the kappa value from 0.81 to 0.79. Agreement between expert consensus and autoscor-ing was 93%. On DS2 the hypothesis of intra-expert variability was supported by a systematic decrease in kappa scores between autoscoring used as reference and each single expert between datasets (.75–.70). Although visual scoring induces inter- and intra-expert variability, autoscoring methods can cope with intra-scorer variabil-ity, making them a sensible option to reduce exogenous variability and give access to the endogenous variability in the data.
Disciplines :
Human health sciences: Multidisciplinary, general & others
Electrical & electronics engineering
Neurosciences & behavior
Electrical & electronics engineering
Human health sciences: Multidisciplinary, general & others
Neurosciences & behavior
Author, co-author :
Berthomier, Christian ;  PHYSIP, Paris, France
Muto, Vincenzo   ;  Université de Liège - ULiège > CRC In vivo Imaging-Sleep and chronobiology
Schmidt, Christina  ;  Université de Liège - ULiège > CRC In vivo Imaging-Sleep and chronobiology
Vandewalle, Gilles  ;  Université de Liège - ULiège > CRC In vivo Imaging-Sleep and chronobiology
Jaspar, Mathieu ;  Université de Liège - ULiège > Département de Psychologie > Ergonomie et intervention au travail
Devillers, Jonathan
Gaggioni, Giulia ;  Université de Liège - ULiège > GIGA
Chellappa, Sarah Laxhmi 
Meyer, Christelle 
Phillips, Christophe  ;  Université de Liège - ULiège > CRC In vivo Im.-Neuroimaging, data acquisition & processing
Salmon, Eric  ;  Université de Liège - ULiège > Département des sciences cliniques > Neuroimagerie des troubles de la mémoire et revalid. cogn.
Berthomier, Pierre;  PHYSIP, Paris, France
Prado, Jacques;  PHYSIP, Paris, France
Benoit, Odile;  PHYSIP, Paris, France
Bouet, Romain;  University of Lyon 1, Lyon, France > Lyon Neuroscience Research Center, INSERM U1028, CNRS UMR 5292,
Brandewinder, Marie;  PHYSIP, Paris, France
Mattout, Jérémie;  University of Lyon 1, Lyon, France > Lyon Neuroscience Research Center, INSERM U1028, CNRS UMR 5292,
Maquet, Pierre  ;  Université de Liège - ULiège > Département des sciences cliniques > Neurologie
More authors (8 more) Less
 These authors have contributed equally to this work.
Language :
English
Title :
Exploring scoring methods for research studies: Accuracy and variability of visual and automated sleep scoring
Publication date :
2020
Journal title :
Journal of Sleep Research
Peer reviewed :
Peer reviewed
Available on ORBi :
since 19 February 2020

Statistics


Number of views
167 (23 by ULiège)
Number of downloads
294 (16 by ULiège)

Scopus citations®
 
23
Scopus citations®
without self-citations
13
OpenCitations
 
11

Bibliography


Similar publications



Contact ORBi