A psychometric analysis of signal detection measures from ratings versus repeated and nonrepeated forced choices.

Abstract

An analysis of the covariance and mean structure of signal detection measures for assessing recognition performance was conducted using data from ratings and repeated k-alternative forced choices (k-AFC). Measures were parameters of the unequal variance signal detection (UVSDT) and dual process signal detection (DPSDT) model and functions thereof, as well as area measures computed from the empirical receiver operating characteristic (ROC) curve. General sensitivity measures computed from UVSDT model parameters revealed reliabilities of about .70 based on 120 test trials. Doubling the number of test trials did not result in a substantial increase of reliability. Halving the number of test trials reduced the reliabilities to about .60. General sensitivity measures based on estimated parameters of the SDT models were slightly more favorable to measures based on the empirical ROC curve. General sensitivity measures resulting from different tasks exhibited similar reliabilities yet differed in size, with the measures from repeated k-AFC tasks being lower than those from the rating tasks. Considering the first selection of the k-AFC tasks only, assuming equal variance of the old and new familiarity distribution, resulted in sensitivity measures of similar size and reliability as those resulting from the rating tasks. Measures d' (familiarity-based sensitivity) and ρ (recollection probability) of the DPSDT model revealed reliabilities that were, in general, inacceptable low. This was particularly pronounced for the measures from the k-AFC tasks. The joint analysis of d' and ρ of the DPSDT model revealed that both measure the same latent construct. (PsycInfo Database Record (c) 2022 APA, all rights reserved).