(Retest-)Reliable and valid despite low alphas? An example from a typical performance situational judgment test of emotional management

Abstract

Situational judgments tests (SJTs) offer many advantages over traditional trait questionnaires but often show low internal consistency, presumably due to heterogeneity of the underlying constructs (Lievens et al., 2008; Whetzel & McDaniel, 2009). Some have suggested test-retest-reliability (TRT) as better-suited reliability indicator for SJTs. Here, we analyzed the TRT of the typical-performance emotional management test (TEMT, Freudenthaler & Neubauer, 2005)—an emotional management SJT. We additionally report validity evidence with respect to a maximum-performance emotional management SJT (STEM, MacCann & Roberts, 2008) and the widely-used Trait Emotional Intelligence Questionnaire (TEIQue, Petrides & Furnham, 2003). For a 2 week-interval, we found satisfying TRT for both scales of the TEMT (inter- and intrapersonal emotional management; 0.76 and 0.83, respectively). Intrapersonal ability correlated moderately to highly with all four TEIQue main factors and the total score, whereas interpersonal ability showed low to medium relations. Correlations with the STEM were smaller. Our results support that SJTs can show good reliabilities—when assessed via TRT—and emphasize the importance of distinguishing between typical- and maximum-performance tests of emotional abilities.