Signal Detection Theory Model Research Articles

Inverse modeling of human contrast response

Mathematical singularities found in the Signal Detection Theory (SDT) based analysis of the 2-Alternative-Forced-Choice (2AFC) method [Katkov, M., Tsodyks, M., & Sagi, D. (2006a). Analysis of two-alternative force-choice Signal Detection Theory model. Journal of Mathematical Psychology, 50, 411–420; Katkov, M., Tsodyks, M., & Sagi, D. (2006b). Singularities in the inverse modeling of 2AFC contrast discrimination data. Vision Research, 46, 256–266; Katkov, M., Tsodyks, M., & Sagi, D. (2007). Singularities explained: Response to Klein. Vision Research, doi:10.1016/j.visres.2006.10.030] imply that contrast discrimination data obtained with the 2AFC method cannot always be used to reliably estimate the parameters of the underlying model (internal response and noise functions) with a reasonable number of trials. Here we bypass this problem with the Identification Task (IT) where observers identify one of N contrasts. We have found that identification data varies significantly between experimental sessions. Stable estimates using individual session data showed Contrast Response Functions (CRF) with high gain in the low contrast regime and low gain in the high contrast regime. Noise Amplitudes (NA) followed a decreasing function of contrast at low contrast levels, and were practically constant above some contrast level. The transition between these two regimes corresponded approximately to the position of the dipper in the Threshold versus Contrast (TvC) curves that were computed using the estimated parameters and independently measured using 2AFC.

Singularities explained: Response to Klein

Klein [Klein, A. S. (2006). Separating transducer nonlinearities and multiplicative noise in contrast discrimination. Vision Research, 46, 4279–4293] questions the existence of intrinsic singularities in two-alternative force-choice (2AFC) Signal Detection Theory (SDT) models, suggesting that the singularities found in Katkov et al. [Katkov, M., Tsodyks, M., & Sagi, D. (2006a). Singularities in the inverse modeling of 2AFC contrast discrimination data. Vision Research, 46, 259–266; Katkov, M., Tsodyks, M., & Sagi, D. (2006b). Analysis of two-alternative force-choice Signal Detection Theory model. Journal of Mathematical Psychology, 50, 411–420] are due to discarding higher order terms in the Taylor expansion of d′ and/or limited to steep psychometric functions. Here we provide some simple intuitive examples that illustrate the results described in Katkov et al. (2006a, 2006b). We show, for the constant noise model, that singularities exist when exact values of d′ are computed and that the singularities are not limited to steep psychometric functions. In these cases the disambiguation of the different models requires millions of trials.

A Measurement of the ability to assess imagery vividness based on a model of working memory and signal detection theory

本研究では，Aプライムが，主観的なイメージ鮮明度査定能力の指標として妥当か否かを検討した．視覚イメージ鮮明度質問紙(VVIQ)の，16項目のうち半数の項目に対しては閉眼・描画動作条件で，また，残りの半数の項目に対しては，開眼・眼球運動条件でイメージ形成が行われた．前者の条件では鮮明度が増加し，後者の条件では鮮明度が減少した．前者における評定値1(実際の知覚と同じくらい鮮明)もしくは2(実際ほどではないが，かなり鮮明)の選択をHit，後者における同様な選択をFalse Alarmとみなして， Aプライムが計算された．Aプライムは，VVIQの合計評定値よりも記憶成績をよく予測した．この結果は，鮮明度評定値の合計という慣習的な指標よりも，新しい指標Aプライムのほうが，イメージ能力をより鋭敏に測定できることを示唆している．さらに，Aプライムは，視空間ワーキングメモリでイメージの生成に携わっている内的書記の機能を測ると想定される，心的回転テストの得点と相関がなかった．この結果は，イメージの査定と生成が相互に独立であることを示唆している．

The mirror effect and mixture signal detection theory.

The mirror effect for word frequency refers to the finding that low-frequency words have higher hit rates and lower false alarm rates than high-frequency words. This result is typically interpreted in terms of conventional signal detection theory (SDT), in which case it indicates that the order of the underlying old item distributions mirrors the order of the new item distributions. However, when viewed in terms of a mixture version of SDT, the order of hits and false alarms does not necessarily imply the same order in the underlying distributions because of possible effects of mixing. A reversal in underlying distributions did not appear for fits of mixture SDT models to data from 4 experiments.

Metamnemonic control over the discriminability of memory evidence: A signal detection analysis of warning effects in the associative list paradigm

According to signal detection theory (SDT), retrieval warnings may decrease false memory in the associative list paradigm either by inducing a conservative criterion shift or by decreasing the amount of evidence that critical theme words were studied. Fitting a SDT model to 12 existing datasets revealed suggestive evidence that warnings impact critical theme evidence, and two new experiments confirmed this conclusion. We argue that this pattern of results is consistent with warned participants relying less on relational and more on item-specific forms of information at retrieval as compared to unwarned participants. We conclude that warnings enhance metamnemonic awareness, thus allowing participants to select a retrieval strategy that capitalizes on discriminative forms of evidence.

A More Parsimonious Approach to Estimating Signal Detection Theory Measures

Signal Detection Theory (SDT) has proven to be an effective tool for analyzing perceptual and decision-making performance in many different areas. The main contribution of SDT is its capability to separate the discrimination process from the decision process by distinguishing between measures of sensitivity and measures of response bias. However, the usefulness of the traditional SDT measures of sensitivity, d', and response bias, c, is questionable in applied settings where observers do not make decisions based on an underlying psychophysical continuum. The purpose of our research was to derive alternative measures of sensitivity and response bias, overcoming the limitations of the traditional SDT model. Results from a Monte Carlo simulation showed support for our more parsimonious model.

A comparison between objective and subjective image quality measurements for a full field digital mammography system

This paper presents pre-sampling modulation transfer function (MTF), normalized noise power spectrum (NNPS) and detective quantum efficiency (DQE) results for an amorphous selenium (a-Se) full field digital mammography system. MTF was calculated from the image of an angled 0.5 mm thick Cu edge, acquired without additional beam filtration. NNPS data were acquired at detector air-kerma levels ranging from 9.1 µGy to 331 µGy, using a standard mammography x-ray spectrum of 28 kV, Mo/Mo target/filter combination and 4 cm of PMMA additional filtration. Prior to NNPS estimation, the image statistics were assessed using a variance image. This method was able to easily identify a detector artefact and should prove useful in routine quality assurance (QA) measurements. Detector DQE, calculated from the NNPS and MTF data, dropped to 0.3 for low detector air-kerma settings but reached an approximately constant value of 0.6 above 50 µGy at the detector. Subjective image quality data were also obtained at these detector air-kerma settings using the CDMAM contrast-detail (c-d) test object. The c-d data reflected the trend seen in DQE, with threshold contrast increasing at low detector air-kerma values. The c-d data were then compared against predictions made using two established models, the Rose model and a standard signal detection theory model. Using DQE(0), the Rose model gave results within approximately 15% on average for all the detector air-kerma values studied and for detail diameters down to 0.2 mm. Similar agreement was also found between the measured c-d data and the signal detection theory results, which were calculated using an ideal human visual response function and a system magnification of unity. The use of full spatial frequency DQE improved the agreement between the calculated and observer results for detail sizes below 0.13 mm.

Analysis of a two-alternative force-choice signal detection theory model

A basic problem in psychophysics is recovering the mean internal response and noise amplitude from sensory discrimination data. Since these components cannot be estimated independently, several indirect methods were suggested to resolve this issue. Here we analyze the two-alternative force-choice method (2AFC), using a signal detection theory approach, and show analytically that the 2AFC data are not always suitable for a reliable estimation of the mean internal responses and noise amplitudes. Specifically, we show that there is a subspace of internal parameters that are highly sensitive to sampling errors (singularities), which results in a large range of estimated parameters with a finite number of experimental trials. Four types of singular models were identified, including the models where the noise amplitude is independent of the stimulus intensity, a situation often encountered in visual contrast discrimination. Finally, we consider two ways to avoid singularities: (1) inserting external noise to the stimuli, and (2) using one-interval forced-choice scaling methods (such as the Thurstonian scaling method for successive intervals).

Sensory suppression during shifts of attention between surfaces in transparent motion

During transparent motion, attention to changes in the direction of one illusory surface will impede recognition of a similar event affecting the other surface if both are close together in time. This is a form of object-based attentional blink (AB). Here, we show that this AB is related to a smaller N200 response to the change in direction and that the response is even smaller for trials on which the subject makes mistakes compared to those with correct responses consistent with signal detection theory models. The variation of N200 associated with the AB can be modeled by an attenuation of current sources estimated in visual extrastriate cortex. These results suggest that the AB in the transparent motion paradigm is due to the suppression of sensory signals in early visual areas.

Animal behavioral psychoacoustics: Issues related to methodology and interpretation

A brief survey of the literature in animal behavioral psychophysics shows that researchers use numerous methods to obtain information on sound detection, discrimination, and identification. Behavioral methodology in animal psychoacoustics includes both classical and operant conditioning, go/no-go or multiple alternative forced-choice tasks, and various methods of estimating detection and discrimination thresholds. Recent emphasis on comparing data across subjects, species, and media (e.g., hearing in air versus water), as well as investigations into the effects of age, noise, and ototoxins on auditory perception highlight the need for methodological standardization. In this paper we will discuss several important issues related to behavioral audiometrics, focusing primarily on marine mammals. These issues include variability among species, individual subjects, and laboratories; experimental concerns such as time constraints; adaptive versus non-adaptive psychophysical methodology and threshold reliability; signal detection theory versus threshold models of audition and the search for unbiased estimates of auditory performance; and measurement and interpretation of subject response bias. Standards for animal psychoacoustic methodology should be sensitive to each of these factors.

A Model of Rater Behavior in Essay Grading Based on Signal Detection Theory

An approach to essay grading based on signal detection theory (SDT) is presented. SDT offers a basis for understanding rater behavior with respect to the scoring of construct responses, in that it provides a theory of psychological processes underlying the raters' behavior. The approach also provides measures of the precision of the raters and the accuracy of classifications. An application of latent class SDT to essay grading is detailed, and similarities to and differences from item response theory (IRT) are noted. The validity and utility of classifications obtained from the SDT model and scores obtained from IRT models are compared. Validity coefficients were found to be about equal in magnitude across SDT and IRT models. Results from a simulation study of a 5‐class SDT model with eight raters are also presented.

Dual-Process Models of Associative Recognition in Young and Older Adults: Evidence From Receiver Operating Characteristics.

In 3 experiments, young and older adults studied lists of unrelated word pairs and were given confidence-rated item and associative recognition tests. Several different models of recognition were fit to the confidence-rating data using techniques described by S. Macho (2002, 2004). Concordant with previous findings, item recognition data were best fit by an unequal-variance signal detection theory model for both young and older adults. For both age groups, associative recognition performance was best explained by models incorporating both recollection and familiarity components. Examination of parameter estimates supported the conclusion that recollection is reduced in old age, but inferences about age differences in familiarity were highly model dependent. Implications for dual-process models of memory in old age are discussed.

A detection theory account of change detection

Previous studies have suggested that visual short-term memory (VSTM) has a storage limit of approximately four items. However, the type of high-threshold (HT) model used to derive this estimate is based on a number of assumptions that have been criticized in other experimental paradigms (e.g., visual search). Here we report findings from nine experiments in which VSTM for color, spatial frequency, and orientation was modeled using a signal detection theory (SDT) approach. In Experiments 1-6, two arrays composed of multiple stimulus elements were presented for 100 ms with a 1500 ms ISI. Observers were asked to report in a yes/no fashion whether there was any difference between the first and second arrays, and to rate their confidence in their response on a 1-4 scale. In Experiments 1-3, only one stimulus element difference could occur (T = 1) while set size was varied. In Experiments 4-6, set size was fixed while the number of stimuli that might change was varied (T = 1, 2, 3, and 4). Three general models were tested against the receiver operating characteristics generated by the six experiments. In addition to the HT model, two SDT models were tried: one assuming summation of signals prior to a decision, the other using a max rule. In Experiments 7-9, observers were asked to directly report the relevant feature attribute of a stimulus presented 1500 ms previously, from an array of varying set size. Overall, the results suggest that observers encode stimuli independently and in parallel, and that performance is limited by internal noise, which is a function of set size.

The effect of background color on asymmetries in color search.

Many previous studies have shown that background color affects the discriminability and appearance of color stimuli. However, research on visual search has not typically considered the role that the background may play. Rosenholtz (2001a) has suggested that color search asymmetries result from the relationship between the stimuli and the background. Here we test the hypothesis that background color should have an effect on asymmetries in visual search based on color, using searches for color stimuli on different colored backgrounds. Observers searched for a single known target stimulus among homogeneous distractor stimuli. The target stimulus differed from the distractors only in chromaticity, but targets and distractors both differed from the backgrounds in luminance so that they were easily visible regardless of chromaticity. Target/distractor pairs differed primarily in saturation (Experiments 1, 2, & 3) or in hue (Experiment 4). Each member of each pair of colors served as target and distractor color on both achromatic and red backgrounds. When the stimuli were presented on an achromatic background, response times were shorter when the more saturated member of each pair of colors served as the target color. When the same stimuli were presented on a red background, the asymmetry was either reversed or abolished. When target and distractors differed in hue, there was little asymmetry on the achromatic background but a sizable asymmetry for some color pairs on the red background. On both backgrounds, the magnitude of the asymmetry varied with the difference between the stimulus colors and the background color. Results confirm that asymmetries in color search are dependent on the relationship between the stimulus colors and the background color. Two candidate models are suggested that show promise in predicting these experimental results: Rosenholtz' saliency model (1999, 2001a) and a modification to signal detection theory models in which the observation noise is proportional to the difference between target/distractor color and background color.

Signal detection theory applied to three visual search tasks — identification, yes/no detection and localization

Adding distracters to a display impairs performance on visual tasks (i.e. the set-size effect). While keeping the display characteristics constant, we investigated this effect in three tasks: 2 target identification, yes-no detection with 2 targets, and 8-alternative localization. A Signal Detection Theory (SDT) model, tailored for each task, accounts for the set-size effects observed in identification and localization tasks, and slightly under-predicts the set-size effect in a detection task. Given that sensitivity varies as a function of spatial frequency (SF), we measured performance in each of these three tasks in neutral and peripheral precue conditions for each of six spatial frequencies (0.5-12 cpd). For all spatial frequencies tested, performance on the three tasks decreased as set size increased in the neutral precue condition, and the peripheral precue reduced the effect. Larger set-size effects were observed at low SFs in the identification and localization tasks. This effect can be described using the SDT model, but was not predicted by it. For each of these tasks we also established the extent to which covert attention modulates performance across a range of set sizes. A peripheral precue substantially diminished the set-size effect and improved performance, even at set size 1. These results provide support for distracter exclusion, and suggest that signal enhancement may also be a mechanism by which covert attention can impose its effect.

"Nonparametric" A' and other modern misconceptions about signal detection theory.

Many modern descriptions of signal detection theory (SDT) are, at best, distorted caricatures of the Gaussian equal-variance model of SDT (G-SDT). The distortions have sometimes led to important, but unwarranted, conclusions about the nature of cognitive processes. Some researchers reject using d' and beta because of concerns about the validity of explicit underlying assumptions (that are shared with most inferential statistics), instead using either the supposedly "nonparametric" measures of A' and B" or measures known to confound ability and bias. The origins, development, and underlying assumptions of SDT are summarized, then contrasted with modern distortions and misconceptions. The nature and interpretation of common descriptive statistics for sensitivity and bias are described along with important pragmatic considerations about use. A deeper understanding of SDT provides researchers with tools that better evaluate both their own findings and the validity of conclusions drawn by others who have utilized SDT measures and analyses.

Comparing integration rules in visual search.

Search performance for a target tilted in a known direction among vertical distractors is well explained by signal detection theory models. Typically these models use a maximum-of-outputs rule (Max rule) to predict search performance. The Max rule bases its decision on the largest response from a set of independent noisy detectors. When the target is tilted in either direction from the reference orientation and the task is to identify the sign of tilt, the loss of performance with set size is much greater than predicted by the Max rule. Here we varied the target tilt and measured psychometric functions for identifying the direction of tilt from vertical. Measurements were made at different set sizes in the presence of various levels of orientation jitter. The orientation jitter was set at multiples of the estimated internal noise, which was invariant across set sizes and measurement techniques. We then compared the data to the predictions of two models: a Summation model that integrates both signal and noise from local detectors and a Signed-Max model that first picks the maxima on both sides of vertical and then chooses the value with the highest absolute deviation from the reference. Although the function relating thresholds to set size had a slope consistent with both the Signed-Max and the Summation models, the shape of individual psychometric functions was in the most crucial conditions better predicted by the Signed-Max model, which chooses the largest tilt while keeping track of the direction of tilt.

Model for stochastic-resonance-type behavior in sensory perception.

Recently it was found that noise could help improve human detection of sensory stimuli via stochastic-resonance-type behavior. Specifically, the ability of an individual to detect a weak tactile stimulus could be enhanced by adding a certain amount of noise. Here we propose, from the perspective of classical signal detection theory, a simple and general model to elucidate the mechanism underlying this phenomenon. We demonstrate that noise-mediated enhancements and decrements in human sensation can be well reproduced by our model. The predicted upper bound of the performance improvement by adding noise is also consistent with the experimental data. We suggest additional experiments to further test the model.

Detecting and locating landmine fields from vehicle- and air-borne measured IR Images

Air- and vehicle-borne sensor-based technique is a potentially attractive approach for fast detecting landmines and locating landmine fields towards humanitarian demining. For images measured from airborne and vehicle-borne cameras, landmines may be indicated by direct or indirect signs, e.g., spatial difference from their surroundings due to digging or, due to thermal and material signatures. The background in images usually consists of various types of noise and clutter, e.g., thermal noise, sand, gravel road and vegetation, thus making the detection even more difficult. This paper is focused on the following aspects: (1) Finding a robust detector that is suitable for detecting/locating landmine candidates and man-made landmarks by using infrared images measured from vehicle- or air-borne sensors; (2) Interpreting the detector using the 2D isotropic bandpass filter, matched filter, detection theory and thermodynamic-based landmine models; (3) Extending the detector to a multiscale version where landmine detectability is enhanced by automatically selecting a proper scale and localization is improved by inter-scale position tracing. We propose a special type of isotropic feature detector that exploits the characteristic difference between landmines and their surroundings in the spatial-frequency domain under the multiscale framework. Experiments were performed on several infrared images measured from vehicle-borne sensors as well as airborne sensors on a helicopter over the test bed scenarios. The performance of the detector was also evaluated in terms of detectability, localization, and automatic scale selection of the detector. These results and evaluations have shown the effectiveness of the method and its potential in landmine field detection.

Disambiguating Conscious and Unconscious Influences: Do Exclusion Paradigms Demonstrate Unconscious Perception?

Does exclusion failure--responding with previously presented words despite instructions to avoid doing so--demonstrate unconscious influences? This article examines exclusion-based evidence for unconscious perception. I propose an alternative signal detection theory (SDT) framework that can account for exclusion failure and ostensibly convergent qualitative differences without positing additional unconscious perceptual mechanisms. In the proposed SDT model, exclusion failure is a criterion artifact, similar to classic SDT-based critiques of subjective threshold approaches. However, it is suggested that exclusion approaches do demonstrate that response strategies are applied only to above-criterion stimuli and thereby illustrate important qualitative differences between two conscious processes: phenomenal awareness itself and higher-order (i.e., metacognitive) decision processes.