Evaluating a Brief Big Five Personality Test in a Diverse Chinese Sample: The Role of Midpoint Designs and Reversely-Worded Items

Abstract

Response styles present a threat to the validity of data from Likert scales. This study evaluates a brief personality test administered by interviewers to a diverse Chinese sample, focusing on two factors with direct impacts on response styles, the midpoint designs and the inclusion of reversely-worded items. We randomly assign respondents into three midpoint designs: masked midpoint, explicit midpoint and removed midpoint. Cronbach’s alpha tends to be low and is not drastically affected by midpoint designs. Score reliability estimates from the removed and explicit midpoint designs are above or close to .60 along the whole score range for openness, but not the other subscales. The masked midpoint design presents some advantages in criterion relationship validity evidence based on correlations with demographic, behavioral, subjective and cognitive variables. Dropping reversely-worded items increases the estimates of Cronbach’s alpha, but shows no positive effects on the criterion relationship validity. In conclusion, for interviewer-administered brief personality scales applied to Chinese respondents, masked midpoint design that aims to reduce a possible overuse of midpoint presents some advantages in the overall psychometric properties, but the inclusion of reversely-worded items shows mixed effects.