Computerized adaptive testing: the capitalization on chance problem
Entity
UAM. Departamento de Psicología Social y MetodologíaPublisher
Cambridge University PressDate
2013-01-10Citation
10.5209/rev_SJOP.2012.v15.n1.37348
Spanish Journal of Psychology 15.1 (2012): 424-441
ISSN
1138-7416 (print); 1988-2904 (online)DOI
10.5209/rev_SJOP.2012.v15.n1.37348Funded by
This research was partly supported by two grants from the Spanish Ministerio de Educación y Ciencia (projects PSI2008-01685 and PSI2009-10341) and by the UAM-IIC Chair Psychometric Models and Applications.Project
Gobierno de España. PSI2008-01685; Gobierno de España. PSI2009-10341Editor's Version
http://dx.doi.org/10.5209/rev_SJOP.2012.v15.n1.37348Subjects
Computerized adaptive testing; Capitalization on chance; Ítem parameter estimation; PsicologíaRights
© Copyright 2012 by The Spanish Journal of PsychologyAbstract
This paper describes several simulation studies that examine the effects of capitalization on chance in the selection of items and the ability estimation in CAT, employing the 3-parameter logistic model. In order to generate different estimation errors for the item parameters, the calibration sample size was manipulated (! = 500, 1000 and 2000 subjects) as was the ratio of item bank size to test length (banks of 197 and 788 items, test lengths of 20 and 40 items), both in a CAT and in a random test. Results show that capitalization on chance is particularly serious in CAT, as revealed by the large positive bias found in the small sample calibration conditions. For broad ranges of θ, the overestimation of the precision (asymptotic Se) reaches levels of 40%, something that does not occur with the RMSE (θ). The problem is greater as the item bank size to test length ratio increases. Potential solutions were tested in a second study, where two exposure control methods were incorporated into the item selection algorithm. Some alternative solutions are discussed.
Files in this item
Google Scholar:Olea Díaz, Julio
-
Barrada, Juan Ramón
-
Abad García, Francisco José
-
Ponsoda, Vicente
This item appears in the following Collection(s)
Related items
Showing items related by title, author, creator and subject.