Random generation of response patterns under computerized adaptive testing with the R package catR

Magis, David; Raîche, Gilles

doi:10.18637/jss.v048.i08

Request a copy

Article (Scientific journals)

Random generation of response patterns under computerized adaptive testing with the R package catR

Magis, David; Raîche, Gilles

2012 • In Journal of Statistical Software, 48, p. 1-31

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/2268/108235

DOI
10.18637/jss.v048.i08

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

v48i08.pdf

Publisher postprint (506.29 kB)

Request a copy

All documents in ORBi are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Computerized adaptive testing; Item selection; Ability estimation; R package

Abstract :

[en] This paper outlines a computerized adaptive testing (CAT) framework and presents an R package for the simulation of response patterns under CAT procedures. This package, called catR, requires a bank of items, previously calibrated according to the four-parameter logistic (4PL) model or any simpler logistic model. The package proposes several methods to select the early test items, several methods for next item selection, di erent estimators of ability (maximum likelihood, Bayes modal, expected a posteriori, weighted likelihood), and three stopping rules (based on the test length, the precision of ability estimates or the classi cation of the examinee). After a short description of the di erent steps of a CAT process, the commands and options of the catR package are presented and practically illustrated.

Disciplines :

Mathematics

Author, co-author :

Magis, David ; Université de Liège - ULiège > Département de mathématique > Statistique mathématique

Raîche, Gilles

Language :

English

Title :

Random generation of response patterns under computerized adaptive testing with the R package catR

Alternative titles :

[fr] Génération aléatoire de patrons de réponses en testing adaptatif informatisé avec la librairie catR

Publication date :

2012

Journal title :

Journal of Statistical Software

ISSN :

1548-7660

Publisher :

University of California at Los Angeles, United States - California

Volume :

Pages :

1-31

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBi :

since 16 January 2012

Statistics

Number of views

106 (6 by ULiège)

Number of downloads

3 (2 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

101

Bibliography

Andersen EB (1970). Asymptotic Properties of Conditional Maximum Likelihood Estimators. Journal of the Royal Statistical Society B, 32, 283-301.
Andersen EB (1972). The Numerical Solution of a Set of Conditional Estimation Equations. Journal of the Royal Statistical Society B, 34, 42-54.
Assessment Systems Corporation (2012). CATSim: Comprehensive Simulation of Computerized Adaptive Testing. St. Paul, MN. URL http://www.assess.com/.
Barton MA, Lord FM (1981). An Upper Asymptote for the Three-Parameter Logistic Model. Technical Report RR-81-20, Educational Testing Service, Princeton, NJ.
Birnbaum A (1968). Some Latent Trait Models and Their Use in Inferring an Examinee's Ability. In FM Lord, MR Novick (eds.), Statistical Theories of Mental Test Scores, pp. 395-479. Addison-Wesley, Reading.
Birnbaum A (1969). Statistical Theory for Logistic Mental Test Models with a Prior Distribution of Ability. Journal of Mathematical Psychology, 6, 258-276.
Bock RD (1972). Estimating Item Parameters and Latent Ability when Responses Are Scored in Two or More Nominal Categories. Psychometrika, 37, 29-51.
Bock RD, Aitkin M (1981). Marginal Maximum Likelihood Estimation of Item Parameters. An Application of the EM Algorithm. Psychometrika, 37, 29-51.
Bock RD, Mislevy RJ (1982). Adaptive EAP Estimation of Ability in a Microcomputer Environment. Applied Psychological Measurement, 6, 431-444.
Chang H, Ying Z (1999). a-Stratified Multistage Computerized Adaptive Testing. Applied Psychological Measurement, 23, 211-222.
Chang SW, Ansley TN (2003). A Comparative Study of Item Exposure Control Methods in Computerized Adaptive Testing. Journal of Educational Measurement, 40, 71-103.
Cheng Y, Chang HH (2009). The Maximum Priority Index Method for Severely Constrained Item Selection in Computerized Adaptive Testing. British Journal of Mathematical and Statistical Psychology, 62, 369-383.
Choi SW (2009). Firestar: Computerized Adaptive Testing Simulation Program for Polytomous Item Response Theory Models. Applied Psychological Measurement, 33, 644-645.
Choi SW, Swartz RJ (2009). Comparison of CAT Item Selection Criteria for Polytomous Items. Applied Pscyhological Measurement, 32, 419-440.
Davey T, Parshall CG (1995). New Algorithms for Item Selection and Exposure Control with Computerized Adaptive Testing. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.
Drasgow F, Olson-Buchanan JB (1999). Innovations in Computerized Assessment. Lawrence Erlbaum Associates, Hillsdale.
Georgiadou EG, Triantafillou E, Economides AA (2007). A Review of Item Exposure Control Strategies for Computerized Adaptive Testing Developed from 1983 to 2005. Journal of Technology, Learning, and Assessment, 5, 1-38.
Green BF (2011). A Comment on Early Student Blunders on Computer-Based Adaptive Tests. Applied Psychological Measurement, 35, 165-174.
Haley DC (1952). Estimation of the Dosage Mortality Relationship when the Dose Is Subject to Error. Technical Report no 15, Applied Mathematics and Statistics Laboratory, Stanford University, Palo Alto, CA.
Hetter RD, Sympson JB (1997). Item Exposure Control in CAT-ASVAB. In WA Sands, BK Waters, JR McBride (eds.), Computerized Adaptive Testing: From Inquiry to Operation, pp. 141-144. American Psychological Association, Washington, DC.
Hoijtink H, Boomsma A (1995). On Person Parameter Estimation in the Dichotomous Rasch Model. In GH Fischer, IW Molenaar (eds.), Rasch Models. Foundations, Recent Developments, and Applications, pp. 53-68. Springer-Verlag.
Irvine SH, Kyllonen PC (2002). Item Generation for Test Development. Lawrence Erlbaum Associates, Mahwah.
Jeffreys H (1939). Theory of Probability. Oxford University Press, Oxford.
Jeffreys H (1946). An Invariant Form for the Prior Probability in Estimation Problems. Proceedings of the Royal Society of London A, 186, 453-461.
Kingsbury GG, Zara AR (1989). Procedures for Selecting Items for Computerized Adaptive Testing. Applied Measurement in Education, 2, 359-375.
Kingsbury GG, Zara AR (1991). A Comparison of Procedures for Content-Sensitive Item Selection in Computerized Adaptive Tests. Applied Measurement in Education, 4, 241-261.
Kosinski M, Rust J (2011a). The Development of Concerto: An Open Source Online Adaptive Testing Platform. Paper presented at the International Association for Computerized and Adaptive Testing, Pacific Grove, CA.
Kosinski M, Rust J (2011b). The Development of Concerto: An Open Source Online Adaptive Testing Platform. Paper presented at the International Meeting of the Psychometric Society, Hong Kong.
Laurier M, Froio L, Pearo C, Fournier M (1998). Test de classement d'anglais langue seconde au collégial. Technical report, College de Maisonneuve, Montreal, QC.
Leung CK, Chang HH, Hau KT (2003). Computerized Adaptive Testing: A Comparison of Three Content Balancing Methods. Journal of Technology, Learning, and Assessment, 2, 1-13.
Loken E, Rulison KL (2010). Estimation of a Four-Parameter Item Response Theory Model. British Journal of Mathematical and Statistical Psychology, 63, 509-525.
Lord FM (1980). Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum Associates, Hillsdale.
Lord FM (1983). Unbiased Estimators of Ability Parameters, of their Variance, and of their Parallel-Forms Reliability. Psychometrika, 48, 233-245.
Lord FM (1984). Maximum Likelihood and Bayesian Parameter Estimation in Item Response Theory. Technical Report No RR-84-30-ONR, Educational Testing Service, Princeton, NJ.
Magis D, Raiche G (2011). CatR: An R Package for Computerized Adaptive Testing. Applied Psychological Measurement, 35, 576-577.
McBride JR, Martin JT (1983). Reliability and Validity of Adaptive Ability Tests in a Military Setting. In DJ Weiss (ed.), New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing, pp. 223-236. Academic Press, New York.
Meijer RR, Nering ML (1999). Computerized Adaptive Testing: Overview and Introduction. Applied Psychological Measurement, 23, 187-194.
Parshall CG, Davey T, Nering ML (1998). Test Development Exposure Control for Adaptive Testing. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
Parshall CG, Spray JA, Kalohn JC, Davey T (2002). Practical Considerations in ComputerBased Testing. Springer-Verlag.
Raiche G (2002). Le dépistage du sous-classement aux tests de classement en anglais, langue seconde, au collégial [The Detection of Under-Classification to the Collegial English as a Second Language Placement Tests]. Technical report, College de l'Outaouais, Gatineau, QC.
Raiche G, Blais JG (2006). SIMCAT 1.0 - A SAS Computer Program for Simulating Computer Adaptive Testing. Applied Psychological Measurement, 30, 60-61.
R Development Core Team (2012). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http: //www.R-project.org/
Riley BB, Dennis ML, Conrad KJ (2010). A Comparison of Content-Balancing Procedures for Estimating Multiple Clinical Domains in Computerized Adaptive Testing: Relative Precision, Validity, and Detection of Persons With Misfitting Responses. Applied Psychological Measurement, 34, 410-423.
Rulison KL, Loken E (2009). I've Fallen and I Can't Get Up: Can High-Ability Students Recover from Early Mistakes in CAT? Applied Psychological Measurement, 33, 83-101.
Stocking ML, Lewis C (1995). A New Method of Controlling Item Exposure in Computerized Adaptive Testing. Research Report No 95-25, Educational Testing Service, Princeton, NJ.
Stocking ML, Lewis C (1998). Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing. Journal of Educational and Behavioral Statistics, 23, 57-75.
Swaminathan H, Gifford JA (1985). Bayesian Estimation in the Two-Parameter Logistic Model. Psychometrika, 50, 349-364.
Swaminathan H, Gifford JA (1986). Bayesian Estimation in the Three-Parameter Logistic Model. Psychometrika, 51, 589-601.
Urry VW (1970). A Monte Carlo Investigation of Logistic Test Models. Ph.D. thesis, Purdue University, West Lafayette, IN.
van der Linden WJ (1998). Bayesian Item Selection Criteria for Adaptive Testing. Psychometrika, 63, 201-216.
van der Linden WJ, Glas CAW (2000). Computerized Adaptive Testing. Theory and Practice. Kluwer, Boston.
van der Linden WJ, Pashley PJ (2000). Item Selection and Ability Estimation in Adaptive Testing. In WJ van der Linden, CAW Glas (eds.), Computerized Adaptive Testing. Theory and Practice, pp. 1-25. Kluwer, Boston.
Veerkamp WJJ, Berger MPF (1997). Some New Item Selection Criteria for Adaptive Testing. Journal of Educational and Behavioral Statistics, 22, 203-226.
Wainer H (2000). Computerized Adaptive Testing: A Primer. 2nd edition. Lawrence Erlbaum Associates, Mahwah.
Wainer H (2010). Fourteen Conversations about Three Things. Journal of Educational and Behavioral Statistics, 35, 5-25.
Warm TA (1989). Weighted Likelihood Estimation of Ability in Item Response Models. Psychometrika, 54, 427-450.
Weiss D (1983). New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing. Academic Press, New York.