Extension Of Signal Detection Theory Research Articles

Empirically Identifying and Computationally Modeling the Brain-Behavior Relationship for Human Scene Categorization.

Humans effortlessly make quick and accurate perceptual decisions about the nature of their immediate visual environment, such as the category of the scene they face. Previous research has revealed a rich set of cortical representations potentially underlying this feat. However, it remains unknown which of these representations are suitably formatted for decision-making. Here, we approached this question empirically and computationally, using neuroimaging and computational modeling. For the empirical part, we collected EEG data and RTs from human participants during a scene categorization task (natural vs. man-made). We then related EEG data to behavior to behavior using a multivariate extension of signal detection theory. We observed a correlation between neural data and behavior specifically between ∼100 msec and ∼200 msec after stimulus onset, suggesting that the neural scene representations in this time period are suitably formatted for decision-making. For the computational part, we evaluated a recurrent convolutional neural network (RCNN) as a model of brain and behavior. Unifying our previous observations in an image-computable model, the RCNN predicted well the neural representations, the behavioral scene categorization data, as well as the relationship between them. Our results identify and computationally characterize the neural and behavioral correlates of scene categorization in humans.

Stereotypical Processing of Emotional Faces: Perceptual and Decisional Components.

People tend to associate anger with male faces and happiness or surprise with female faces. This angry-men-happy-women bias has been ascribed to either top-down (e.g., well-learned stereotypes) or bottom-up (e.g., shared morphological cues) processes. The dissociation between these two theoretical alternatives has proved challenging. The current effort addresses this challenge by harnessing two complementary metatheoretical approaches to dimensional interaction: Garner's logic of inferring informational structure and General Recognition Theory—a multidimensional extension of signal detection theory. Conjoint application of these two rigorous methodologies afforded us to: (a) uncover the internal representations that generate the angry-men-happy-women phenomenon, (b) disentangle varieties of perceptual (bottom-up) and decisional (top-down) sources of interaction, and (c) relate operational and theoretical meanings of dimensional independence. The results show that the dimensional interaction between emotion and gender is generated by varieties of perceptual and decisional biases. These outcomes document the involvement of both bottom-up (e.g., shared morphological structures) and top-down (stereotypes) factors in social perception.

Glow in the dark: Using a heat-sensitive camera for blind individuals with prosthetic vision

To date, retinal implants are the only available treatment for blind individuals with retinal degenerations such as retinitis pigmentosa. Argus II is the only visual implant with FDA approval, with more than 300 users worldwide. Argus II stimulation is based on a grayscale image coming from a head-mounted visible-light camera. Normally, the 11°×19° field of view of the Argus II user is full of objects that may elicit similar phosphenes. The prosthesis cannot meaningfully convey so much visual information, and the percept is reduced to an ambiguous impression of light. This study is aimed at investigating the efficacy of simplifying the video input in real-time using a heat-sensitive camera. Data were acquired from four Argus II users in 5 stationary tasks with either hot objects or human targets as stimuli. All tasks were of m-alternative forced choice design where precisely one of the m≥2 response alternatives was defined to be “correct” by the experimenter. To compare performance with heat-sensitive and normal cameras across all tasks, regardless of m, we used an extension of signal detection theory to latent variables, estimating person ability and item difficulty in d' units. Results demonstrate that subject performance was significantly better across all tasks with the thermal camera compared to the regular Argus II camera. The future addition of thermal imaging to devices with very poor spatial resolution may have significant real-life benefits for orientation, personal safety, and social interactions, thereby improving quality of life.

Prosthetic Visual Performance Using a Disparity-Based Distance-Filtering System.

PurposeAt present, Argus II is the only retinal prosthesis approved by the US Food and Drug Administration that induces visual percepts in people who are blind from end-stage outer retinal degenerations such as retinitis pigmentosa. It has been shown to work well in sparse, high-contrast settings, but in daily practice visual performance with the device is likely to be hampered by the cognitive load presented by a cluttered real-world environment. In this study, we investigated the effect of a stereo-disparity–based distance-filtering system on four experienced Argus II users for a range of tasks: object localization, depth discrimination, orientation and size discrimination, and people detection and direction of motion.MethodsFunctional vision was assessed in a semicontrolled setup using unfiltered (normal camera) and distance-filtered (stereo camera) imagery. All tasks were forced choice designs and an extension of signal detection theory to latent (unobservable) variables was used to analyze the data, allowing estimation of person ability (person measures) and task difficulty (item measures) on the same axis.ResultsAll subjects performed better with the distance filter compared with the unfiltered image (P < 0.001 on all tasks except localization).ConclusionsOur results show that depth filtering using a disparity-based algorithm has significant benefits for people with Argus II implants.Translational RelevanceThe improvement in functional vision with the distance filter found in this study may have an important impact on vision rehabilitation and quality of life for people with visual prostheses and ultra low vision.

Estimating measures of latent variables from m-alternative forced choice responses.

Signal Detection Theory is the standard method used in psychophysics to estimate person ability in m-alternative forced choice tasks where stimuli are typically generated with known physical properties (e.g., size, frequency, contrast, etc …) and lie at known locations on a physical measurement axis. In contrast, variants of Item Response Theory are preferred in fields such as medical research and educational testing where the axis locations of items on questionnaires or multiple choice tests are not defined by any observable physical property and are instead defined by a latent (or unobservable) variable. We provide an extension of Signal Detection Theory to latent variables that employs the same strategy used in Item Response Theory and demonstrate the practical utility of our method by applying it to a set of clinically relevant face perception tasks with visually impaired individuals as subjects. A key advantage of our approach is that Signal Detection Theory explicitly models the m-alternative forced choice task while Item Response Theory does not. We show that Item Response Theory is inconsistent with key assumptions of the m-alternative forced choice task and is not a valid model for this paradigm. However, the simplest Item Response Theory model–the dichotomous Rasch model–is found to be a special case of SDT and provides a good approximation as long as the number of response alternatives m is small and remains fixed for all items.

The psychological interaction of spam email features

This study explored distinct perceptual and decisional contributions to spam email mental construal. Participants classified spam emails according to pairings of three stimulus features – presence or absence of awkward prose, abnormal message structure, and implausible premise. We examined dimensional interactions within general recognition theory (GRT; a multidimensional extension of signal detection theory). Classification accuracy was highest for categories containing either two non-normal dimension levels (e.g. awkward prose and implausible premise) or two normal dimension levels (e.g. normal prose and plausible premise). Modelling indicated both perceptual and decisional contributions to classification responding. In most cases, perceptual discriminability was higher along one dimension when stimuli contained a non-normal level of the paired dimension (e.g. prose discriminability was higher with abnormal structure). Similarly, decision criteria along one dimension were biased in favour of the non-normal response when stimuli contained a non-normal level of the paired dimension. Potential applications for training are discussed.Practitioner summary: We applied general recognition theory (i.e. multivariate signal detection theory) to spam email classification at low or high levels of three stimulus dimensions: premise plausibility, prose quality, and email structure. Relevant to training, this approach helped identify perceptual and decisional biases that could be leveraged to individualise training.

Testing Separability and Independence of Perceptual Dimensions with General Recognition Theory: A Tutorial and New R Package (grtools).

Determining whether perceptual properties are processed independently is an important goal in perceptual science, and tools to test independence should be widely available to experimental researchers. The best analytical tools to test for perceptual independence are provided by General Recognition Theory (GRT), a multidimensional extension of signal detection theory. Unfortunately, there is currently a lack of software implementing GRT analyses that is ready-to-use by experimental psychologists and neuroscientists with little training in computational modeling. This paper presents grtools, an R package developed with the explicit aim of providing experimentalists with the ability to perform full GRT analyses using only a couple of command lines. We describe the software and provide a practical tutorial on how to perform each of the analyses available in grtools. We also provide advice to researchers on best practices for experimental design and interpretation of results when applying GRT and grtools

On the Internal Representation of Numerical Magnitude and Physical Size

A nascent idea in the numerical cognition literature--the analogical hypothesis (Pinel, Piazza, Bihan, & Dehaene, 2004)--assumes a common noisy code for the representation of symbolic (e.g., numerals) and nonsymbolic (e.g., numerosity, physical size, luminance) magnitudes. The present work subjected this assumption to various tests from the perspective of General Recognition Theory (GRT; Ashby &Townsend, 1986)--a multidimensional extension of Signal Detection Theory (Green & Swets, 1966). The GRT was applied to the dimensions of numerical magnitude and physical size with the following goals: (a) characterizing the internal representation of these dimensions in the psychological space, and (b) assessing various types of (in)dependence and separability governing the perception of these dimensions. The results revealed various violations of independence and separability with Stroop incongruent, but not with Stroop congruent stimuli. The outcome suggests that there are deep differences in architecture between Stroop congruent and incongruent stimuli that reach well beyond the semantic relationship involved.

Sensory Integration Across Modalities: How Kinaesthesia Integrates with Vision in Visual Orientation Discrimination

Stimuli in one modality can affect the appearance and discriminability of stimuli in another, but how they do so is not well understood. Here we propose a theory of the integration of sensory information across modalities. This is based on criterion setting theory (CST; Treisman and Williams, 1984), an extension of signal detection theory which models the setting and adjustment of decision criteria. The theory of sensory integration based on CST (CST-SI) offers an account of cross-modal effects on sensory decision-making; here we consider its application to orientation anisotropy. In this case, CST-SI postulates that the postural senses are concerned with the relations between momentary body posture and the cardinal dimensions of space, vertical and horizontal, and that they also contribute to stabilizing perception of the cardinal orientations in vision through actions on the corresponding visual decision criteria, but that they have little effect on perception of diagonal orientations. Predictions from CST-SI are tested by experimentally separating the contributions that different information sources make to stabilizing the visual criteria. It is shown that reducing relevant kinaesthetic input may increase the variance for discrimination of the visual cardinal axes but not the obliques. Predictions that shift in the location of the psychometric function would be induced by varying the distribution of the test stimuli, and that this effect would be greater for oblique than cardinal axes were confirmed. In addition, peripheral visual stimuli were shown to affect the discrimination of cardinal but not oblique orientations at the focus of vision. These results support the present account of anisotropies.

A Criterion Setting Theory of Discrimination Learning that Accounts for Anisotropies and Context Effects

We can discriminate departures from the vertical or horizontal more accurately than from other orientations. This may reflect perceptual learning, but the mechanisms behind such learning are not well understood. Here we derive a theory of discrimination learning based on criterion setting theory (CST; Treisman and Williams, 1984), an extension of signal detection theory in which judgment of the current stimulus is partly determined by previous discriminations and context. The CST-based theory of discrimination learning (CST-DL) describes mechanisms which use information from previous acts of discrimination to improve current decision making. CST-DL distinguishes between types of decision criteria and provides an account of anisotropies and context effects affecting discrimination. Predictions from this model are tested in experiments on anisotropies in orientation and depth perception. The results obtained support CST-DL. They also support the conclusion that the account of the retention of sensory information in delayed discrimination provided by CST is superior to the traditional belief that information retention relies on a fixed memory trace or representation of the stimulus.

A Latent Class Extension of Signal Detection Theory, with Applications

A latent class extension of signal detection theory is presented and applications are illustrated. The approach is useful for situations where observers attempt to detect latent categorical events or where the goal of the analysis is to select or classify cases. Signal detection theory is shown to offer a simple summary of the observers' performance in terms of detection and response criteria. Implications of the view via signal detection for the training of raters are noted, as are approaches to validating the parameters and classifications. An extension of the signal detection model to more than two latent classes, with a simple restriction on the detection parameters, is introduced. Sample programs to fit the models using software for latent class analysis or software for second generation structural equation modeling are provided.

Signal detection theory with finite mixture distributions: theoretical developments with applications to recognition memory.

An extension of signal detection theory (SDT) that incorporates mixtures of the underlying distributions is presented. The mixtures can be motivated by the idea that a presentation of a signal shifts the location of an underlying distribution only if the observer is attending to the signal; otherwise, the distribution is not shifted or is only partially shifted. Thus, trials with a signal presentation consist of a mixture of 2 (or more) latent classes of trials. Mixture SDT provides a general theoretical framework that offers a new perspective on a number of findings. For example, mixture SDT offers an alternative to the unequal variance signal detection model; it can also account for nonlinear normal receiver operating characteristic curves, as found in recent research.

Perceptual interactions of facial dimensions in speeded classification and identification.

The representation underlying the identification and classification of semirealistic line drawings taken from a computer model of the face was investigated by using a speeded classification task and an identification task. These data were analyzed by using a multidimensional extension of signal detection theory, within which varieties of perceptual interactions between dimensions within and across stimuli can be characterized. The dimensions of interest here were eye separation, nose length, and mouth width. The response time and accuracy data from the speeded classification task suggest that processing of a given feature did depend on whether other features were present or absent, but given that other features were present, the results strongly support separability (a macrolevel, across-stimulus form of invariance) for all pairs of facial dimensions used. This separability was confirmed by the subsequent identification task. Owing to its greater resolution, the identification task can reveal interactions that might exist at more microlevels of processing. In fact, the identification data did indicate the presence of perceptual dependence between facial dimensions within a stimulus when the dimensions that were varied were close in spatial proximity (i.e., eye separation and nose length). Within the theoretical framework, perceptual dependence can be interpreted as correlated noise between otherwise separate channels (and hence, is logically distinct from separability). This dependence was greatly reduced for dimensions that were more distant (eyes and mouth). The relation between these results and the configural effects that have been observed with faces as stimuli in other studies is discussed.

The effect of shared responsibility and competition in perceptual games: A test of cognitive game-theoretic extension of signal-detection theory.

The effect of shared responsibility and competition in perceptual games: A test of cognitive game-theoretic extension of signal-detection theory.

Implications of marginal and conditional detection parameters for the separabilities and independence of perceptual dimensions

Ashby and Townsend developed a theory and related methodology for assessing various types of perceptual independence within recognition accuracy experiments. A special case of considerable import was the General Gaussian Recognition model based on multivariate normal distributions with arbitrary covariance matrices across stimuli. The General Recognition model can also be viewed as a multidimensional extension of signal detection theory and thereby yields two sets of signal detection analyses, macro- and microanalyses, introduced ealier by Townsend and his colleagues. The present work develops logical relationships among these two sets of signal detection parameters and the various concepts of “perceptual independence,” including perceptual and decisional separability, marginal response invariance, sampling independence, and perceptual independence. These relationships lead to new tests of the varieties of independence and permit a qualitative construction of the underlying perceptual space.

Brain events underlying detection and recognition of weak sensory signals.

Through the use of a quantitative extension of signal detection theory, the brain events associated with the detection and recognition of weak acoustic signals were examined by recording brain event-related potentials. The early N100 componenet of the event-related potential varied only with detection, whereas the late P300 component varied with both detection and recognition. P300 amplitude accurately predicted recognition performance on a trial-by-trial basis. The results suggest that detection and recognition are partially concurrent processes in perception and demonstrate that the electrocortical events occurring during the perception of sensory stimuli are closely associated with both detection and recognition of these stimuli by the nervous system.

Detection in a four-observation two-channel task

Two models are proposed for a four-observation two-channel forced-choice paradigm of signal detection. The first model, an extension of signal detection theory, fails, while the second model, an extension of the classical decision threshold model, succeeds.