Article (Scientific journals)
Agreement between an isolated rater and a group of raters
Vanbelle, Sophie; Albert, Adelin
2009In Statistica Neerlandica, 63 (1), p. 82-100
Peer Reviewed verified by ORBi
 

Files


Full Text
stan_412.pdf
Publisher postprint (572.13 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
kappa coefficient; nominal scale; ordinal scale; expert group
Abstract :
[en] The agreement between two raters judging items on a categorical scale is traditionally assessed by Cohen’s kappa coefficient. We introduce a new coefficient for quantifying the degree of agreement between an isolated rater and a group of raters on a nominal or ordinal scale. The group of raters is regarded as a whole, a reference or gold-standard group with its own heterogeneity. The coefficient, defined on a population-based model, requires a specific definition of the concept of perfect agreement. It has the same properties as Cohen’s kappa coefficient and reduces to the latter when there is only one rater in the group. The new approach overcomes the problem of consensus within the group of raters and generalizes Schouten’s index. The method is illustrated on published syphilis data and on data collected from a study assessing the ability of medical students in diagnostic reasoning when compared with expert knowledge.
Disciplines :
Mathematics
Author, co-author :
Vanbelle, Sophie ;  Université de Liège - ULiège > Département des sciences de la santé publique > Département des sciences de la santé publique
Albert, Adelin  ;  Université de Liège - ULiège > Département des sciences de la santé publique > Informatique médicale et biostatistique - Département de mathématique
Language :
English
Title :
Agreement between an isolated rater and a group of raters
Publication date :
January 2009
Journal title :
Statistica Neerlandica
ISSN :
0039-0402
Publisher :
Blackwell Publishing, Oxford, United Kingdom
Volume :
63
Issue :
1
Pages :
82-100
Peer reviewed :
Peer Reviewed verified by ORBi
Commentary :
The definitive version is available at www.blackwell-synergy.com
Available on ORBi :
since 10 December 2008

Statistics


Number of views
84 (13 by ULiège)
Number of downloads
378 (5 by ULiège)

Scopus citations®
 
25
Scopus citations®
without self-citations
23
OpenCitations
 
22

Bibliography


Similar publications



Contact ORBi