Cognitive Ability-Demand Gap Analysis With Latent Response Models

Abstract

A better understanding of human cognitive ability-demand gap (ADG) is critical in designing assistive technology solution that is accurate and adaptive over a wide range of human-agent interaction. The main goal is to design systems that can adapt with the user's abilities and needs over a range of cognitive tasks. It will also enable the system to provide feedback consistent with the situation. However, the latent structure and relationship between human ability to respond to cognitive task (demand on human by the agent) remains unknown. Robust modeling of cognitive ADG will be a paradigm shift from the current trends in assistive technology design. The key idea is to estimate the gap, based on human-agent cognitive task interaction. In particular, latent response model was adopted to quantify the gap. First, we used one parameter Rasch model and extended Rasch model (rating scale model, partial credit model) with dichotomous and polytomous responses, respectively. Residues between expected and observed ability scores were considered as gap parameter in case of dichotomous response. In extended Rasch modeling, response latitudes are considered as an indicator of the gap. Additionally, we performed model fitting, standard error measurement, kernel density estimation, and differential item functioning to test the suitability of Rasch model. Empirical analyses on a number of data set show that proposed analytical method can model the cognitive ADG from dichotomous and polytomous responses. In dichotomous case, the model better fits for mixed responses (combination of easy, medium, and hard) data set rather than monotonic (e.g., only easy) data. Results show that Rasch model can be reliably used to estimate cognitive gap with different cognitive task types.