[en] Cohen’s kappa coefficient is traditionally used to quantify the degree of agreement between two raters on a nominal scale. Correlated kappas occur in many settings (e.g., repeated agreement by raters on the same individuals, concordance between diagnostic tests and a gold standard) and often need to be compared. While different techniques are now available to model correlated κ coefficients, they are generally not easy to implement in practice. The present paper describes a simple alternative method based on the bootstrap for comparing correlated kappa coefficients. The method is illustrated by examples and its type I error studied using simulations. The method is also compared with the generalized estimating equations of the second order and the weighted least-squares methods.
Disciplines :
Mathematics
Author, co-author :
Vanbelle, Sophie ; Université de Liège - ULiège > Département des sciences de la santé publique > Département des sciences de la santé publique
Albert, Adelin ; Université de Liège - ULiège > Département des sciences de la santé publique > Informatique médicale et biostatistique - Département de mathématique
Language :
English
Title :
A bootstrap method for comparing correlated kappa coefficients
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
Cohen, J., 1960, A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
Fleiss, J.L., 1981, Statistical methods for rates and proportions (2nd edn) (New York: Wiley).
McKenzie, D.P. et al., 1996, Comparing correlated kappas by resampling: is one level of agreement significantly different from another? Journal of psychiatric research, 30, 483-492.
Williamson, J.M. et al., 2000, Modeling kappa for measuring dependent categorical agreement data, Biostatistics, 1, 191-202.
Lipsitz, S.R. et al., 2001, A simple method for estimating a regression model for κ between a pair of raters, Journal of the Royal Statistical Society A, 164, 449-465.
Barnhart, H.X. and Williamson J.M., 2002, Weighted least-squares approach for comparing correlated kappa, Biometrics, 58, 1012-1019.
Efron, B. and Tibshirani, R.J., 1993, An introduction to the bootstrap (New York: Chapman and Hall).
Kraemer, H.C., 1979, Ramification of a population model for κ as a coefficient of reliability, Psychometrika, 44, 461-472.
Cohen, J., 1968, Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit, Psychological bulletin, 70, 213-220.
Similar publications
Sorry the service is unavailable at the moment. Please try again later.
This website uses cookies to improve user experience. Read more
Save & Close
Accept all
Decline all
Show detailsHide details
Cookie declaration
About cookies
Strictly necessary
Performance
Strictly necessary cookies allow core website functionality such as user login and account management. The website cannot be used properly without strictly necessary cookies.
This cookie is used by Cookie-Script.com service to remember visitor cookie consent preferences. It is necessary for Cookie-Script.com cookie banner to work properly.
Performance cookies are used to see how visitors use the website, eg. analytics cookies. Those cookies cannot be used to directly identify a certain visitor.
Used to store the attribution information, the referrer initially used to visit the website
Cookies are small text files that are placed on your computer by websites that you visit. Websites use cookies to help users navigate efficiently and perform certain functions. Cookies that are required for the website to operate properly are allowed to be set without your permission. All other cookies need to be approved before they can be set in the browser.
You can change your consent to cookie usage at any time on our Privacy Policy page.