Determining the number of aAttributes in cognitive diagnosis modeling
Entity
UAM. Departamento de Psicología Social y MetodologíaDate
2021-02-15Citation
10.3389/fpsyg.2021.614470
Frontiers In Psychology 12 (2021): 614470
ISSN
1664-1078 (online)DOI
10.3389/fpsyg.2021.614470Funded by
This research was partially supported by Ministerio de Ciencia, Innovación y Universidades, Spain (Grant PSI2017-85022-P), European Social Fund, and Cátedra de Modelos y Aplicaciones Psicométricas (Instituto de Ingeniería del Conocimiento and Autonomous University of Madrid)Project
Gobierno de España. PSI2017-85022-PEditor's Version
https://doi.org/10.3389/fpsyg.2021.614470Subjects
cognitive diagnostic models; dimensionality assessment; machine learning; model comparison; parallel analysis; Q-matrix validation; PsicologíaRights
© 2021 The AuthorsAbstract
Cognitive diagnosis models (CDMs) allow classifying respondents into a set of discrete attribute profiles. The internal structure of the test is determined in a Q-matrix, whose correct specification is necessary to achieve an accurate attribute profile classification. Several empirical Q-matrix estimation and validation methods have been proposed with the aim of providing well-specified Q-matrices. However, these methods require the number of attributes to be set in advance. No systematic studies about CDMs dimensionality assessment have been conducted, which contrasts with the vast existing literature for the factor analysis framework. To address this gap, the present study evaluates the performance of several dimensionality assessment methods from the factor analysis literature in determining the number of attributes in the context of CDMs. The explored methods were parallel analysis, minimum average partial, very simple structure, DETECT, empirical Kaiser criterion, exploratory graph analysis, and a machine learning factor forest model. Additionally, a model comparison approach was considered, which consists in comparing the model-fit of empirically estimated Q-matrices. The performance of these methods was assessed by means of a comprehensive simulation study that included different generating number of attributes, item qualities, sample sizes, ratios of the number of items to attribute, correlations among the attributes, attributes thresholds, and generating CDM. Results showed that parallel analysis (with Pearson correlations and mean eigenvalue criterion), factor forest model, and model comparison (with AIC) are suitable alternatives to determine the number of attributes in CDM applications, with an overall percentage of correct estimates above 76% of the conditions. The accuracy increased to 97% when these three methods agreed on the number of attributes. In short, the present study supports the use of three methods in assessing the dimensionality of CDMs. This will allow to test the assumption of correct dimensionality present in the Q-matrix estimation and validation methods, as well as to gather evidence of validity to support the use of the scores obtained with these models. The findings of this study are illustrated using real data from an intelligence test to provide guidelines for assessing the dimensionality of CDM data in applied settings
Files in this item
Google Scholar:Najera Álvarez, Pablo
-
Abad García, Francisco José
-
Sorrel Luján, Miguel Ángel
This item appears in the following Collection(s)
Related items
Showing items related by title, author, creator and subject.