[en] A frequent criticism formulated against the use of weighted kappa coeffcients is
that the weights are arbitrarily defined. We show that using linear weights for a K-ordinal scale is equivalent to deriving a kappa coeffcient from K-1 embedded 2 x 2 tables.
Disciplines :
Mathematics
Author, co-author :
Vanbelle, Sophie ; Université de Liège - ULiège > Département des sciences de la santé publique > Département des sciences de la santé publique
Albert, Adelin ; Université de Liège - ULiège > Département des sciences de la santé publique > Informatique médicale et biostatistique - Département de mathématique
Language :
English
Title :
A note on the linearly weighted kappa coefficient for ordinal scales
Agresti A. Categorical Data Analysis. 2nd ed. (2002), John Wiley and Sons, New York
Brenner H., and Kliebsch U. Dependence of weighed kappa coefficients on the number of categories. Epidemiology 7 (1996) 199-202
Cicchetti D., and Allison T. A new procedure for assessing reliability of scoring eeg sleep recordings. American Journal EEG Technology 11 (1971) 101-109
Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 (1960) 37-46
Cohen J. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 70 (1968) 213-220
Fleiss J.L., and Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measure of reliability. Educational and Psychological Measurement 33 (1973) 613-619
Gilmour E., Ellerbrock T., Koulos J., Chiasson M., Williamson J., Kubn L., and Wright T.J. Measuring cervical ectopy: Direct visual assessment versus computerized planimetry. American Journal of Obstetrics and Gynecology 176 (1997) 108-111
Graham P., and Jackson R. The analysis of ordinal agreement data: Beyond weighted kappa. Journal of Clinical Epidemiology 46 (1993) 1055-1062
Ludbrook J. Statistical techniques for comparing measures and methods of measurement: A critical review. Clinical and Experimental Pharmacology and Physiology 29 (2002) 527-536
Schuster C. A note on the interpretation of weighted kappa and its relation to other rater agreement statistics for metric scales. Educational and Psychological Measurement 64 (2004) 243-253
