Variability of visual and automated sleep stage scoring in the elderly

Visual scoring of sleep recordings is characterized by inter-scorer variability. This methodological issue can be amplified in older individual recordings because sleep changes markedly in aging. Here, we aimed to investigate sleep scoring variability in aged participants through a visual-automatic sleep scoring comparison.
Sleep recordings of 20 subjects (10 women, 61±5 years) were included. Automatic sleep scoring (AS) was performed by Aseega algorithm, previously validated on young healthy participants. Visual scoring (VS) was performed by two experts (VS1, VS2) from different centers according to AASM rules. Epoch-by-epoch agreements (concordance and Conger’s kappa coefficient, ) were computed. Generalized linear mixed models assessed potential scorer effects on sleep parameters (time spent in N1/N2/N3/REM, tN1/tN2/tN3/tREM; wake after sleep onset, WASO; total sleep time, TST; sleep efficiency, SE).
Overall agreement between the 3 scorings was  = 0.60 (moderate). Pairwise agreements were as follows: VS1 vs. VS2, 76% (=0.67); AS vs. VS1, 67% (0.54), AS vs. VS2, 74% (0.60). Agreement between AS and consensual VS was 78% (0.60). GLMMs showed disparate pairs of agreeing scorers depending on the sleep parameter considered. For tN1, AS showed differences with both VS (p < .0001) who did not differ between themselves. Differences were found between both VS for tN2 and tN3 (p < .0001) and WASO (p = .006), while AS showed no significant difference with VS2. All three scorers differed for TST (p = .05) and SE (p = .04). No differences across scorers were found for tREM.
Agreement between scorers, whether between VS or AS and VS proved lower than what is usually reported in the literature for the general population. This is likely due to the fact that with ageing, sleep undergoes a series of changes with, at the macrostructural level, lower sleep stability and, at the microstructural level, lower EEG voltage dynamics. This certainly renders sleep more difficult to score, which might lead to increased inter-rater variability and rises methodological questions relative to sleep scoring in the aged population.