Efficient standard error formulas of ability estimators with dichotomous item response models

[en] This paper focuses on the computation of asymptotic standard errors (ASE) of ability estimators with dichotomous item response models. A general framework is considered and ability estimators are defined from a very restricted set of assumptions and formulas. This approach encompasses most standard methods such as maximum likelihood, weighted likelihood, maximum a posteriori and robust estimators. A general formula for the ASE is derived from the theory of M-estimation. Well-known results are found back as particular cases for the maximum and robust estimators, while new ASE proposals for the weighted likelihood and maximum a posteriori estimators are presented. These new formulas are compared to traditional ones by means of a simulation study under Rasch modeling.

Disciplines :

Education & instruction

Author, co-author :

Magis, David ; Université de Liège - ULiège > Département Education et formation > Psychométrie et édumétrie

Language :

English

Title :

Efficient standard error formulas of ability estimators with dichotomous item response models

Publication date :

2016

Journal title :

Psychometrika

ISSN :

0033-3123

eISSN :

1860-0980

Publisher :

Psychonomic Society, Research Triangle Park, United States - Virginia

Volume :

Pages :

184-200

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBi :

since 29 January 2015

Statistics

Number of views

80 (9 by ULiège)

Number of downloads

2 (2 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

Baker, F. B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (chapters) (pp. 17–20). Reading, MA: Addison-Wesley.
Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258–276.
Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a micro computer environment. Applied Psychological Measurement, 6, 431–444.
Carroll, R. J., & Pederson, S. (1993). On robustness in the logistic regression model. Journal of the Royal Statistical Society: Series B, 55, 693–706.
Doebler, A. (2012). The problem of bias in person parameter estimation in adaptive testing. Applied Psychological Measurement, 36, 255–270. doi:10.1177/0146621612443304.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New York: Erlbaum.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer.
Huber, P. J. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 35, 73–101. doi:10.1214/aoms/1177703732.
Huber, P.J. (1967). The behavior of maximum likelihood estimates under non-standard conditions. In Proceeding of the 5th Berkeley Symposium, (vol. 1, pp. 221–233).
Huber, P. J. (1981). Robust statistics. New York: Wiley.
Koralov, L., & Sinai, Y. G. (2007). Theory of probability and random processes. New York: Springer.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
Lord, F. M. (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability. Psychometrika, 48, 233–245. doi:10.1007/BF02294018.
Lord, F. M. (1986). Maximum likelihood and Bayesian parameter estimation in item response theory. Journal of Educational Measurement, 23, 157–162. doi:10.1111/j.1745-3984.1986.tb00241.x.
Magis, D. (2014a). On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models. British Journal of Mathematical and Statistical Psychology, 67, 430–450. doi:10.1111/bmsp.12027.
Magis, D. (2014b). Accuracy of asymptotic standard errors of the maximum and weighted likelihood estimators of proficiency levels with short tests. Applied Psychological Measurement, 38, 105–121. doi:10.1177/0146621613496890.
Magis, D., & Raîche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48, 1–31.
Mislevy, R. J. (1986). Bayes modal estimation in item response theory. Psychometrika, 51, 177–195. doi:10.1007/BF02293979.
Mislevy, R. J., & Bock, R. D. (1982). Biweight estimates of latent ability. Educational and Psychological Measurement, 42, 725–737. doi:10.1177/001316448204200302.
Mosteller, F., & Tukey, J. (1977). Exploratory data analysis and regression. Reading, MA: Addison-Wesley.
Nydick, S.W. (2013). catIrt: An R package for simulating IRT-based computerized adaptive tests. R package version 0.4-1.
Ogasawara, H. (2013a). Asymptotic properties of the Bayes and pseudo Bayes estimators of ability in item response theory. Journal of Multivariate Analysis, 114, 359–377. doi:10.1016/j.jmva.2012.08.013.
Ogasawara, H. (2013b). Asymptotic cumulants of the ability estimators using fallible item parameters. Journal of Multivariate Analysis, 119, 144–162. doi:10.1016/j.jmva.2013.04.008.
Partchev, I. (2012). irtoys: Simple interface to the estimation and plotting of IRT models. R package version 0.1.6.
Patton, J. M., Cheng, Y., Yuan, K.-H., & Diao, Q. (2013). The influence of item calibration error on variable-length computerized adaptive testing. Applied Psychological Measurement, 37, 24–40. doi:10.1177/0146621612461727.
R Core Team (2014). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
Rao, M. M. (1984). Probability theory with applications. New York: Academic Press.
Reif, M. (2014). PP: Estimation of person parameters for the 1, 2, 3, 4-PL model and the GPCM. R package version 0.5.3.
Schuster, C., & Yuan, K.-H. (2011). Robust estimation of latent ability in item response models. Journal of Educational and Behavioral Statistics, 36, 720–735. doi:10.3102/1076998610396890.
Sijisma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.
Stefanski, L. A., & Boos, D. D. (2002). The calculus of M-estimation. The American Statistician, 56, 29–38. doi:10.1198/000313002753631330.
Wainer, H. (2000). Computerized adaptive testing: A primer (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Wainer, H., & Wright, B. D. (1980). Robust estimation of ability in the Rasch model. Psychometrika, 45, 373–391. doi:10.1007/BF02293910.
Warm, T. A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427–450. doi:10.1007/BF02294627.
Warm, T. A. (2007). Warm (Maximum) likelihood estimates of Rasch measures. Rasch Measurement Transactions, 21, 1094.
Wu, M. L., Adams, R. J., & Wilson, M. R. (1997). ConQuest: Multi-aspect test software [Computer program]. Camberwell, Australia: Australian Council for Educational Research.
Yuan, K.-H., & Jennrich, R. I. (1998). Asymptotics of estimating equations under natural conditions. Journal of Multivariate Analysis, 65, 245–260. doi:10.1006/jmva.1997.1731.
Zeileis, A. (2006). Object-oriented computation of sandwich estimators. Journal of Statistical Software, 16(9), 1–16.