The Issue of Fuzzy Concepts in Test Construction and Possible Remedies

Abstract

With the first issue of 2015, I want to continue the series ofeditorials aimed at highlighting specific topics relevant dur-ing test construction. More importantly, I focus on issuesthat repeatedly lead to paper rejections. In this issue, I wantto take up the cudgels for a mixed methods approach in testconstruction. The quantitative aspects of testing a newlyconstructed assessment tool range prominently within thisjournal (Alonso-Arbiol & van de Vijver, 2010). However,some issues arising during the test development processcannot be dealt with using quantitative methods alone.Along with the coauthors of this editorial – experts in theapplication of a mixed methods approach in test construc-tion – I want to explore these issues.Many test constructions show a lack of awareness con-cerning the comprehensibility of items, specifically in per-sonality tests. However, this lack of awareness can haveimplications for psychometric quality. Frequently, it is sim-ply assumed that respondents’ understanding of a test itemmatches the meaning implied by the test developer. It is,however, rarely ever tested whether all respondents of thetargeted population actually understand the test items cor-rectly and in a similar way. As pointed out in the last edito-rial (Ziegler, 2014), item content should be precisely tunedto the needs of every potential respondent in a populationtargeted by a newly developed test. More specifically, per-son variables such as age, gender, and education (Ramm-stedt & Kemper, 2011) have to be taken into accountwhen constructing test items in order to assure that eachpotential respondent fully understands the meaning andmay respond accordingly. If these person variables areneglected in the process of test construction, the psycho-metric quality of a test may be substantially affected byfuzzy concepts. The goal of this editorial is to raise aware-ness for the detrimental effects fuzzy concepts can have intest development and possible remedies.