Low Target Prevalence Research Articles

Safety in numbers: Target prevalence affects the detection of vehicles during simulated driving

The "low-prevalence effect" refers to the fact that observers often fail to detect rare targets (<5 % prevalence) during visual search tasks. Previous research has demonstrated robust prevalence effects in real-world tasks that employ static images, such as airport luggage screening. No published research has examined prevalence effects in dynamic tasks, such as driving. We conducted a driving simulator experiment to investigate whether target prevalence effects influence the detection of other vehicles while driving. The target vehicles were motorcycles and buses, with prevalence being manipulated both within and between subjects: Half of the subjects experienced a high prevalence of motorcycles with a low prevalence of buses, and half experienced a high prevalence of buses with a low prevalence of motorcycles. Consistent with our hypotheses, drivers detected high-prevalence targets faster than low-prevalence targets for both vehicle types. Overall, our results support the notion that increasing the prevalence of visual search targets makes them more salient, and consequently easier to detect.

Prevalence effects in newly trained airport checkpoint screeners: trained observers miss rare targets, too.

Many socially important search tasks are characterized by low target prevalence, meaning that targets are rarely encountered. For example, transportation security officers (TSOs) at airport checkpoints encounter very few actual threats in carry-on bags. In laboratory-based visual search experiments, low prevalence reduces the probability of detecting targets (Wolfe, Horowitz, & Kenner, 2005). In the lab, this "prevalence effect" is caused by changes in decision and response criteria (Wolfe & Van Wert, 2010) and can be mitigated by presenting a burst of high-prevalence search with feedback (Wolfe et al., 2007). The goal of this study was to see if these effects could be replicated in the field with TSOs. A total of 125 newly trained TSOs participated in one of two experiments as part of their final evaluation following training. They searched for threats in simulated bags across five blocks. The first three blocks were low prevalence (target prevalence ≤ .05) with no feedback; the fourth block was high prevalence (.50) with full feedback; and the final block was, again, low prevalence. We found that newly trained TSOs were better at detecting targets at high compared to low prevalence, replicating the prevalence effect. Furthermore, performance was better (and response criterion was more "liberal") in the low-prevalence block that took place after the high-prevalence block than in the initial three low-prevalence blocks, suggesting that a burst of high-prevalence trials may help alleviate the prevalence effect in the field.

The Ultra-Rare-Item Effect

Accuracy is paramount in radiology and security screening, yet many factors undermine success. Target prevalence is a particularly worrisome factor, as targets are rarely present (e.g., the cancer rate in mammography is ~0.5%), and low target prevalence has been linked to increased search errors. More troubling is the fact that specific target types can have extraordinarily low frequency rates (e.g., architectural distortions in mammography—a specific marker of potential cancer—appear in fewer than 0.05% of cases). By assessing search performance across millions of trials from the Airport Scanner smartphone application, we demonstrated that the detection of ultra-rare items was disturbingly poor. A logarithmic relationship between target detection and target frequency (adjusted R2 = .92) revealed that ultra-rare items had catastrophically low detection rates relative to targets with higher frequencies. Extraordinarily low search performance for these extraordinarily rare targets—what we term the ultra-rare-item effect—is troubling given that radiological and security-screening searches are primarily ultra-rare-item searches.

Evidence for a Positive Relationship between Working-Memory Capacity and Detection of Low-Prevalence Targets in Visual Search

Our ability to detect a target in visual search relates to the prevalence of the target, whereby rare targets are missed more than common targets. The current study sought to identify operator characteristics that could account for the higher miss rates associated with rare targets. The results found that working-memory capacity, which is strongly related to attentional control and inhibition of irrelevant information, was significantly correlated with the ability to detect low-prevalence targets. High-capacity observers also exhibited lengthened target-absent responses with rare targets, suggesting that the high-capacity observers were more persistent in their searches than others.

False feedback increases detection of low-prevalence targets in visual search

Many critical search tasks, such as airport and medical screening, involve searching for targets that are rarely present. These low-prevalence targets are associated with extremely high miss rates Wolfe, Horowitz, & Kenner (Nature, 435, 439-440, 2005). The inflated miss rates are caused by a criterion shift, likely due to observers attempting to equate the numbers of misses and false alarms. This equalizing strategy results in a neutral criterion at 50% target prevalence, but leads to a higher proportion of misses for low-prevalence targets. In the present study, we manipulated participants' perceived number of misses through explicit false feedback. As predicted, the participants in the false-feedback condition committed a higher number of false alarms due to a shifted criterion. Importantly, the participants in this condition were also more successful in detecting targets. These results highlight the importance of perceived prevalence in target search tasks.

Prevalence of Abnormalities Influences Cytologists' Error Rates in Screening for Cervical Cancer

Medical screening tasks are often difficult, visual searches with low target prevalence (low rates of disease). Under laboratory conditions, when targets are rare, nonexpert searchers show decreases in false-positive results and increases in false-negative results compared with results when targets are common. This prevalence effect is not due to vigilance failures or target unfamiliarity. To determine whether prevalence effects could be a source of elevated false-negative errors in medical experts. We studied 2 groups of cytologists involved in cervical cancer screening (Boston, Massachusetts, and South Wales, UK). Cytologists evaluated photomicrographs of cells at low (2% or 5%) or higher (50%) rates of abnormality prevalence. Two versions of the experiment were performed. The Boston, Massachusetts, group made decisions of normal or abnormal findings using a 4-point rating scale. Additionally, the group from South Wales localized apparent abnormalities. In both groups, there is evidence for prevalence effects. False-negative errors were 17% (higher prevalence), rising to 30% (low prevalence) in the Boston, Massachusetts, group. The error rate was 27% (higher prevalence), rising to 42% (low prevalence) in the South Wales group. (Comparisons between the 2 groups are not meaningful because the stimulus sets were different.) These results provide the first evidence, to our knowledge, that experts are not immune to the effects of prevalence even with stimuli from their domain of expertise. Prevalence is a factor to consider in screening for disease by human observers and has significant implications for cytology-based cervical cancer screening in the post-human papillomavirus vaccine era, when prevalence rates of high-grade lesions in the population are expected to decline.

Spatial and temporal separation fails to counteract the effects of low prevalence in visual search

Recent research has shown that, in visual search, participants can miss 30–40% of targets when they only appear rarely (i.e., on 1–2% of trials). Low target prevalence alters the behaviour of the searcher. It can lead participants to quit their search prematurely (Wolfe, Horowitz, & Kenner, 2005), to shift their decision criteria (Wolfe et al., 2007), and/or to make motor or response errors (Fleck & Mitroff, 2007). In this paper we examine whether the low prevalence (LP) effect can be ameliorated if we split the search set in two, spreading the task out over space and/or time. Observers searched for the letter “T” among “L”s. In Experiment 1, the left or right half of the display was presented to the participants before the second half. In Experiment 2, items were spatially intermixed but half of the items were presented first, followed by the second half. Experiment 3 followed the methods of Experiment 2 but allowed observers to correct perceived errors. All three experiments produced robust LP effects with higher errors at 2% prevalence than at 50% prevalence. Dividing up the display had no beneficial effect on errors. The opportunity to correct errors reduced but did not eliminate the LP effect. Low prevalence continues to elevate errors even when observers are forced to slow down and permitted to correct errors.

High or low target prevalence increases the dual-target cost in visual search.

Previous studies have demonstrated a dual-target cost in visual search. In the current study, the relationship between search for one and search for two targets was investigated to examine the effects of target prevalence and practice. Color-shape conjunction stimuli were used with response time, accuracy and signal detection measures. Performance was lower in dual-target search compared with the combined performance for two independent single-target searches. The cost in response time slope disappeared with practice, but the cost in accuracy remained. Sensitivity was lower and the decision criterion more conservative in dual-target search than in single-target searches, suggesting that the representation of the target was less effective in dual-target search than in single-target search. Manipulation of target prevalence induced a bias in favor of the more likely correct response: target-present responses were likely under high target prevalence and target-absent responses were likely under low target prevalence. The prevalence effect was greater in dual-target search than single-target search, causing the dual-target cost to be larger under target prevalences that differed from 50%. These findings are important for applied tasks in which targets appear rarely and can differ from each other. For example, the low target prevalence in X-ray security searches may magnify the dual-target cost implicated in previous research with X-ray images (see Menneer, Cave, & Donnelly, 2009). Such a result would increase the need for security personnel to consider alternatives to dual-target search, such as specialization in detecting one target type or training to encourage independent searches for each target.

Even in correctable search, some types of rare targets are frequently missed.

Socially important visual search tasks, such as airport baggage screening and tumor detection, place observers in situations where the targets are rare and the consequences of failed detection are substantial. Recent laboratory studies have demonstrated that low target prevalence yields substantially higher miss errors than do high-prevalence conditions, in which the same targets appear frequently (Wolfe, Horowitz, & Kenner, 2005; Wolfe et al., 2007). Under some circumstances, this "prevalence effect" can be eliminated simply by allowing observers to correct their last response (Fleck & Mitroff, 2007). However, in three experiments involving search of realistic X-ray luggage images, we found that the prevalence effect is eliminated neither by giving observers the choice to correct a previous response nor by requiring observers to confirm their responses. This prevalence effect, obtained when no trial-by-trial feedback was given, was smaller than the effect obtained when observers searched through the same stimuli but were given trial-by-trial feedback about accuracy. We suggest that low prevalence puts pressure on observers in any search task, and that the diverse symptoms of that pressure manifest themselves differently in different situations. In some relatively simple search tasks, misses may result from motor or response errors. In other, more complex tasks, shifts in decision criteria appear to be an important contributor.

Why do we miss rare targets? Exploring the boundaries of the low prevalence effect.

Observers tend to miss a disproportionate number of targets in visual search tasks with rare targets. This 'prevalence effect' may have practical significance since many screening tasks (e.g., airport security, medical screening) are low prevalence searches. It may also shed light on the rules used to terminate search when a target is not found. Here, we use perceptually simple stimuli to explore the sources of this effect. Experiment 1 shows a prevalence effect in inefficient spatial configuration search. Experiment 2 demonstrates this effect occurs even in a highly efficient feature search. However, the two prevalence effects differ. In spatial configuration search, misses seem to result from ending the search prematurely, while in feature search, they seem due to response errors. In Experiment 3, a minimum delay before response eliminated the prevalence effect for feature but not spatial configuration search. In Experiment 4, a target was present on each trial in either two (2AFC) or four (4AFC) orientations. With only two response alternatives, low prevalence produced elevated errors. Providing four response alternatives eliminated this effect. Low target prevalence puts searchers under pressure that tends to increase miss errors. We conclude that the specific source of those errors depends on the nature of the search.

Rare Targets Are Rarely Missed in Correctable Search

Failing to find a tumor in an x-ray scan or a gun in an airport baggage screening can have dire consequences, making it fundamentally important to elucidate the mechanisms that hinder performance in such visual searches. Recent laboratory work has indicated that low target prevalence can lead to disturbingly high miss rates in visual search. Here, however, we demonstrate that misses in low-prevalence searches can be readily abated. When targets are rarely present, observers adapt by responding more quickly, and miss rates are high. Critically, though, these misses are often due to response-execution errors, not perceptual or identification errors: Observers know a target was present, but just respond too quickly. When provided an opportunity to correct their last response, observers can catch their mistakes. Thus, low target prevalence may not be a generalizable cause of high miss rates in visual search.

Low target prevalence is a stubborn source of errors in visual search tasks.

In visual search tasks, observers look for targets in displays containing distractors. Likelihood that targets will be missed varies with target prevalence, the frequency with which targets are presented across trials. Miss error rates are much higher at low target prevalence (1%-2%) than at high prevalence (50%). Unfortunately, low prevalence is characteristic of important search tasks such as airport security and medical screening where miss errors are dangerous. A series of experiments show this prevalence effect is very robust. In signal detection terms, the prevalence effect can be explained as a criterion shift and not a change in sensitivity. Several efforts to induce observers to adopt a better criterion fail. However, a regime of brief retraining periods with high prevalence and full feedback allows observers to hold a good criterion during periods of low prevalence with no feedback.