Reference : Empirical comparison of scoring rules at early stages of CAT
Scientific congresses and symposiums : Unpublished conference/Abstract
Social & behavioral sciences, psychology : Education & instruction
Empirical comparison of scoring rules at early stages of CAT
Magis, David mailto [Université de Liège > Département Education et formation > Psychométrie et édumétrie >]
Conference of the International Association for Computerized Adaptive Testing
14-16 septembre 2015
University of Cambridge
United Kingdom
[en] Computerized adaptive testing ; Early scoring rule
[en] Usual scoring rules in CATs include maximum likelihood (ML), weighted likelihood (WL) and Bayesian approaches. However, at early stages of adaptive testing, only a few item responses are available so the amount of information is very limited and in addition constant patterns (i.e. only correct or only incorrect responses) are often observed, yielding ML scoring intractable. Specific scoring rules (such as fixed- or variable stepsize adjustments) were developed for that purpose. However recent research highlighted that both Bayesian and WL scoring rules may provide finite values even with small sets of items.
The purpose of this presentation is twofold: (a) to make a quick review of available scoring rules at early stages of CAT, and (b) to present empirical results from a simulation study that compares those scoring rules. More precisely, three scoring scenarios will be investigated: stepsize adjustment followed by ML, Bayes or WL followed by ML, and constant scoring rule throughout the CAT. These methods will be compared by means of simulated item banks and under various CAT scenarios for next item selection and stopping rules. Empirical results will be presented and practical guidelines for early stage scoring will be outlined.

File(s) associated to this reference

Additional material(s):

File Commentary Size Access
Private access
IACAT 2015 - David Magis.pdf287.67 kBRequest copy

Bookmark and Share SFX Query

All documents in ORBi are protected by a user license.