Non-target Images Research Articles

Memory deficit in patients with cerebral small vessel disease: evidence from eye tracking technology.

Cerebral small vessel disease is the one of the most prevalent causes of vascular cognitive impairment. We aimed to find objective and process-based indicators related to memory function to assist in the detection of memory impairment in patients with cerebral small vessel disease. Thirty-nine cerebral small vessel disease patients and 22 healthy controls were invited to complete neurological examinations, neuropsychological assessments, and eye tracking tasks. Eye tracking indicators were recorded and analyzed in combination with imaging features. The cerebral small vessel disease patients scored lower on traditional memory task and performed worse on eye tracking memory task performance compared to the healthy controls. The cerebral small vessel disease patients exhibited longer visit duration and more visit count within areas of interest and targets and decreased percentage value of total visit duration on target images to total visit duration on areas of interest during decoding stage among all levels. Our results demonstrated the cerebral small vessel disease patients performed worse in memory scale and eye tracking memory task, potentially due to their heightened attentional allocation to nontarget images during the retrieval stage. The eye tracking memory task could provide process-based indicators to be a beneficial complement to memory assessment and new insights into mechanism of memory impairment in cerebral small vessel disease patients.

RETRACTED ARTICLE: Dual Siamese Anchor Points Adaptive Tracker with Transformer for RGBT Tracking

Due to environmental conditions, such as rainy days, foggy days, and dim lighting, objects in visible light images are not prominently displayed, leading to an easy loss of targets during tracking. In recent years, many RGB visible light trackers have achieved significant success in addressing visual tracking challenges. However, these trackers perform poorly when tracking targets under special conditions, such as occlusions and low-light scenarios. In contrast, objects in thermal infrared images are more distinct in poor lighting conditions. Given this characteristic, researchers have shown increased interest in the development of trackers that combine thermal infrared and visible light imagery. However, some mainstream RGBT (red–green–blue and thermal) algorithms, such as MANET and ADNET, are based on the anchor-based theory, requiring consideration of anchor box sizes and introducing a substantial number of hyperparameters. This can lead to suboptimal performance when tracking dynamically changing targets. Moreover, these models rely on convolutional neural networks for feature extraction, which have limitations in capturing global features. In this paper, we introduce a novel training network model called DAPAT, which combines the anchor-free concept with Transformer theory. DAPAT differs from previous models in several ways. Specifically, we have designed a straightforward model to extract precise global features from template and search images. We have also incorporated two enhancement modules into the model to improve template and search images of different sizes while suppressing the impact of non-target images. We employ a dual-stream feature fusion network to reduce the loss of image feature information due to feature correlation operations. Finally, we compare the performance of the tracking model proposed in this paper with some advanced RGBT trackers on three data sets (RGBT234, RGBT210, and GTOT). The test results demonstrate that our tracker exhibits improvements in robustness and success rate, among other performance aspects.

Human and machine recognition of dynamic and static facial expressions: prototypicality, ambiguity, and complexity.

A growing body of research suggests that movement aids facial expression recognition. However, less is known about the conditions under which the dynamic advantage occurs. The aim of this research was to test emotion recognition in static and dynamic facial expressions, thereby exploring the role of three featural parameters (prototypicality, ambiguity, and complexity) in human and machine analysis. In two studies, facial expression videos and corresponding images depicting the peak of the target and non-target emotion were presented to human observers and the machine classifier (FACET). Results revealed higher recognition rates for dynamic stimuli compared to non-target images. Such benefit disappeared in the context of target-emotion images which were similarly well (or even better) recognised than videos, and more prototypical, less ambiguous, and more complex in appearance than non-target images. While prototypicality and ambiguity exerted more predictive power in machine performance, complexity was more indicative of human emotion recognition. Interestingly, recognition performance by the machine was found to be superior to humans for both target and non-target images. Together, the findings point towards a compensatory role of dynamic information, particularly when static-based stimuli lack relevant features of the target emotion. Implications for research using automatic facial expression analysis (AFEA) are discussed.

Analysis and Classification of Event-Related Potentials During Image Observation.

In the field of cognitive neuroscience, researchers have conducted extensive studies on object categorization using Event-Related Potential (ERP) analysis, specifically by analyzing electroencephalographic (EEG) response signals triggered by visual stimuli. The most common approach for visual ERP analysis is to use a low presentation rate of images and an active task where participants actively discriminate between target and non-target images. However, researchers are also interested in understanding how the human brain processes visual information in real-world scenarios. To simulate real-life object recognition, this study proposes an analysis pipeline of visual ERPs evoked by images presented in a Rapid Serial Visual Presentation (RSVP) paradigm. Such an approach allows for the investigation of recurrent patterns of visual ERP signals across specific categories and subjects. The pipeline includes segmentation of the EEGs in epochs, and the use of the resulting features as inputs for Support Vector Machine (SVM) classification. Results demonstrate common ERP patterns across the selected categories and the ability to obtain discriminant information from single visual stimuli presented in the RSVP paradigm.

Characteristics of Event-related Potentials during a Target Selection Task with Fingerspelling Images

We have been developing a cognitive assessment system based on an electroencephalography (EEG)-based brain-machine interface (BMI) that uses event-related potentials (ERPs) as a virtual “Mind Switch.” ERPs reflect temporal changes in attention and are also known as potential biomarkers for assessing cognitive function. We recorded EEG data from 10 healthy subjects who performed target-selection tasks based upon the visual discrimination of fingerspelling images with or without motor responses (key release). We found differential responses between target and non-target images, which were quantitatively evaluated by the decoding accuracy of the target. We also found that ERPs were enhanced when accompanied by key releases, which might indicate that a more attentive state was involved. These results suggest that fingerspelling images are a feasible tool for cognitive assessment. Future studies should be conducted to demonstrate the applicability of our system in preventing frailty in older people.

A deep-learning toolkit for visualization and interpretation of segmented medical images

SummaryGeneralizability of deep-learning (DL) model performance is not well understood and uses anecdotal assumptions for increasing training data to improve segmentation of medical images. We report statistical methods for visual interpretation of DL models trained using ImageNet initialization with natural-world (TII) and supervised learning with medical images (LMI) for binary segmentation of skin cancer, prostate tumors, and kidneys. An algorithm for computation of Dice scores from union and intersections of individual output masks was developed for synergistic segmentation by TII and LMI models. Stress testing with non-Gaussian distributions of infrequent clinical labels and images showed that sparsity of natural-world and domain medical images can counterintuitively reduce type I and type II errors of DL models. A toolkit of 30 TII and LMI models, code, and visual outputs of 59,967 images is shared to identify the target and non-target medical image pixels and clinical labels to explain the performance of DL models.

The orientation-difference cue in figure-ground separation: border ownership and timing

Figure-ground separation, a crucial component of visual processing, can be driven by differences in contrast, orientation, and other local cues. We previously showed (VSS 2020) that the orientation cue has distinctive characteristics: figure-ground separation depends not only on the magnitude of the orientation difference, but also on whether the oriented component is present in the figure vs. the ground. Here we show that this dependence is invariant with respect to convexity vs. concavity of the figure, and determine how it evolves over time. In a 2-AFC task, subjects (N=3) distinguished a target image containing five randomly-positioned figures rendered by one synthetic texture superimposed on a background defined by a second synthetic texture, from a non-target image consisting of a uniform texture whose image statistics matched the spatial average of the target’s statistics. Target and non-target images consisted of 64 x 64 arrays of 10-min black and white checks, and the figures covered 25% of the target’s area. Texture construction enabled control of the spatial frequency content of figure and ground in one or two orientations. For figures consisting of circular discs, figure-ground separation was facilitated when the figure texture was oriented, compared to when the ground was oriented. This difference was linked to the degree of anisotropy, as determined from the textures’ spectra (r=0.82, p<0.001). When figure boundaries were changed from convex to concave, results were similar, indicating that figure content and border ownership, but not border shape, determined performance. In preliminary results (N=1), we find that this asymmetry develops slowly; it is largely absent for a presentation time of 125 ms but close to maximal at 250 ms. In sum, figure-ground separation uses orientation differences in a way that depends on figure-ground assignment, suggesting a recurrent process that evolves over time.

Two brains, one target: Design of a multi-level information fusion model based on dual-subject RSVP

Background. Rapid serial visual presentation (RSVP) based brain-computer interface (BCI) is widely used to categorize the target and non-target images. The available information limits the prediction accuracy of single-trial using single-subject electroencephalography (EEG) signals.New Method. Hyperscanning is a new manner to record two or more subjects’ signals simultaneously. So we designed a multi-level information fusion model for target image detection based on dual-subject RSVP, namely HyperscanNet. The two modules of this model fuse the data and features of the two subjects at the data and feature layers. A chunked long and short-term memory artificial neural network (LSTM) was used in the time dimension to extract features at different periods separately, completing fine-grained underlying feature extraction. While the feature layer is fused, some plain operations are used to complete the fusion of the data layer to ensure that important information is not missed.Results. Experimental results show that the F1-score (the harmonic mean of precision and recall) of this method with best group of channels and segment length is 82.76%.Comparison with existing methods. This method improves the F1-score by at least 5% compared to single-subject target detection.Conclusions. Target detection can be accomplished by the two subjects’ collaboration to achieve a higher and more stable F1-score than a single subject.

A Cross-Session Dataset for Collaborative Brain-Computer Interfaces Based on Rapid Serial Visual Presentation.

Brain-computer interfaces (BCIs) based on rapid serial visual presentation (RSVP) have been widely used to categorize target and non-target images. However, it is still a challenge to detect single-trial event related potentials (ERPs) from electroencephalography (EEG) signals. Besides, the variability of EEG signal over time may cause difficulties of calibration in long-term system use. Recently, collaborative BCIs have been proposed to improve the overall BCI performance by fusing brain activities acquired from multiple subjects. For both individual and collaborative BCIs, feature extraction and classification algorithms that can be transferred across sessions can significantly facilitate system calibration. Although open datasets are highly efficient for developing algorithms, currently there is still a lack of datasets for a collaborative RSVP-based BCI. This paper presents a cross-session EEG dataset of a collaborative RSVP-based BCI system from 14 subjects, who were divided into seven groups. In collaborative BCI experiments, two subjects did the same target image detection tasks synchronously. All subjects participated in the same experiment twice with an average interval of ∼23 days. The results in data evaluation indicate that adequate signal processing algorithms can greatly enhance the cross-session BCI performance in both individual and collaborative conditions. Besides, compared with individual BCIs, the collaborative methods that fuse information from multiple subjects obtain significantly improved BCI performance. This dataset can be used for developing more efficient algorithms to enhance performance and practicality of a collaborative RSVP-based BCI system.

Prediction of Human Inhibition Brain Function with Inter-Subject and Intra-Subject Variability.

The stop signal task has been used to quantify the human inhibitory control. The inter-subject and intra-subject variability was investigated under the inhibition of human response with a realistic environmental scenario. In present study, we used a battleground scenario where a sniper-scope picture was the background, a target picture was a go signal, and a nontarget picture was a stop signal. The task instructions were to respond on the target image and inhibit the response if a nontarget image appeared. This scenario produced a threatening situation and endorsed the evaluation of how subject’s response inhibition manifests in a real situation. In this study, 32 channels of electroencephalography (EEG) signals were collected from 20 participants during successful stop (response inhibition) and failed stop (response) trials. These EEG signals were used to predict two possible outcomes: successful stop or failed stop. The inter-subject variability (between-subjects) and intra-subject variability (within-subjects) affect the performance of participants in the classification system. The EEG signals of successful stop versus failed stop trials were classified using quadratic discriminant analysis (QDA) and linear discriminant analysis (LDA) (i.e., parametric) and K-nearest neighbor classifier (KNNC) and Parzen density-based (PARZEN) (i.e., nonparametric) under inter- and intra-subject variability. The EEG activities were found to increase during response inhibition in the frontal cortex (F3 and F4), presupplementary motor area (C3 and C4), parietal lobe (P3 and P4), and occipital (O1 and O2) lobe. Therefore, power spectral density (PSD) of EEG signals (1-50Hz) in F3, F4, C3, C4, P3, P4, O1, and O2 electrodes were measured in successful stop and failed stop trials. The PSD of the EEG signals was used as the feature input for the classifiers. Our proposed method shows an intra-subject classification accuracy of 97.61% for subject 15 with QDA classifier in C3 (left motor cortex) and an overall inter-subject classification accuracy of 71.66% ± 9.81% with the KNNC classifier in F3 (left frontal lobe). These results display how inter-subject and intra-subject variability affects the performance of the classification system. These findings can be used effectively to improve the psychopathology of attention deficit hyperactivity disorder (ADHD), obsessive-compulsive disorder (OCD), schizophrenia, and suicidality.

A Benchmark Dataset for RSVP-Based Brain-Computer Interfaces.

This paper reports on a benchmark dataset acquired with a brain–computer interface (BCI) system based on the rapid serial visual presentation (RSVP) paradigm. The dataset consists of 64-channel electroencephalogram (EEG) data from 64 healthy subjects (sub1,…, sub64) while they performed a target image detection task. For each subject, the data contained two groups (“A” and “B”). Each group contained two blocks, and each block included 40 trials that corresponded to 40 stimulus sequences. Each sequence contained 100 images presented at 10 Hz (10 images per second). The stimulus images were street-view images of two categories: target images with human and non-target images without human. Target images were presented randomly in the stimulus sequence with a probability of 1∼4%. During the stimulus presentation, subjects were asked to search for the target images and ignore the non-target images in a subjective manner. To keep all original information, the dataset was the raw continuous data without any processing. On one hand, the dataset can be used as a benchmark dataset to compare the algorithms for target identification in RSVP-based BCIs. On the other hand, the dataset can be used to design new system diagrams and evaluate their BCI performance without collecting any new data through offline simulation. Furthermore, the dataset also provides high-quality data for characterizing and modeling event-related potentials (ERPs) and steady-state visual evoked potentials (SSVEPs) in RSVP-based BCIs. The dataset is freely available from http://bci.med.tsinghua.edu.cn/download.html.

Rapid identification of the face in infants

Our visual system can rapidly process stimuli relevant to our current behavioral goal within various irrelevant stimuli in natural scenes. This ability to detect and identify target stimuli during nontarget stimuli has been mainly studied in adults, so that the development of this high-level visual function has been unknown among infants, although it has been shown that 15-month-olds’ temporal thresholds of face visibility are close to those of adults. However, we demonstrate here that infants younger than 15 months can identify a target face among nontarget but meaningful scene images. In the current study, we investigated infants’ ability to detect and identify a face in a rapid serial visual presentation. Experiment 1 examined whether 5- to 8-month-olds could discriminate the difference in the presentation duration of visual streams (100 vs. 11 ms). Results showed that 7- and 8-month-olds successfully discriminated between the presentation durations. In Experiment 2, we examined whether 5- to 8-month-olds could detect the face presented for 100 ms and found that 7- and 8-month-olds could detect the face embedded in rapid serial visual streams. To further clarify the face processing at this age of infants, we tested whether infants could identify upright and inverted faces in rapid visual streams in Experiments 3a and 3b. The results showed that 7- and 8-month-olds identified upright faces, but not inverted faces, during the visual stream, which reflected face inversion effects. Overall, we suggest that the temporal speed of face processing at 7 and 8 months of age would be comparable to that of adults.

EventFinder: a program for screening remotely captured images.

Camera traps are becoming ubiquitous tools for ecologists. While easily deployed, they require human time to organize, review, and classify images including sequences of images of the same individual, and non-target images triggered by environmental conditions. For such cases, we developed an automated computer program, named EventFinder, to reduce operator time by pre-processing and classifying images using background subtraction techniques and color histogram comparisons. We tested the accuracy of the program against images previously classified by a human operator. The automated classification, on average, reduced the data requiring human input by 90.8% with an accuracy of 96.1%, and produced a false positive rate of only 3.4%. Thus, EventFinder provides an efficient method for reducing the time for human operators to review and classify images making camera trap projects, which compile a large number of images, less costly to process. Our testing process used medium to large animals, but will also work with smaller animals, provided their images occupy a sufficient area of the frame. While our discussion focuses on camera trap image reduction, we also discuss how EventFinder might be used in conjunction with other software developments for managing camera trap data.

Field Testing of a New Feral Hog Feeder to Minimize Bait Exposure to Non-target Wildlife

Author(s): Poche, Richard M.; Franckowiak, Gregory A.; Poche, David; Tseveenjav, Batchimeg; Connor, Joseph P.; Germer, David | Abstract: The feral hog is an invasive species that inflicts billions of dollars in agricultural damage every year in the U.S. Hog-specific baits have shown promise in reducing feral hog abundance but require species-specific feeders to exclude domestic animals and non-target wildlife. Four feeder types were tested during 2016 and 2017 field studies, including commercial feeders, prototype feeders, and a new species-specific feral hog feeder. Commercial feeders with 2.3 kg, 4.5 kg, and 6.8 kg weighted doors were used in 2016 and raccoons were observed on camera opening doors 10 times out of 164 camera observed visitations (6.1%). No other non-target species were observed entering the feeders. The following year, new feeders with 7.7 kg double-sided guillotine doors were used in a field study and no raccoons opened doors during 153 camera-observed visitations. Out of over 1,600 non-target camera images recorded, only one mouse was observed inside the feeders. Results of this study suggest the new hog feeder may provide reduced-risk to non-target species and a promising tool for controlling hog populations.

Detection of Underwater Targets Using a Subspace-Based Method With Learning

This paper introduces a new subspace-based detection method for multichannel (high frequency and broadband) synthetic aperture sonar (SAS) imagery. An image-dependent dictionary learning method is applied to form the appropriate dictionary matrices for representing target and nontarget image snippets. The hypothesis testing is done by forming a test statistic that relies on the residual error power ratio in representing an unknown image snippet using the target and nontarget dictionary matrices. To avoid the computational bottleneck in most dictionary learning methods, a new recursive method is introduced which does not require any matrix inversion or singular value decomposition (SVD). The proposed detection method was then implemented and benchmarked against a matched subspace detection method for detecting mine-like objects. Results are then presented on two sonar imagery data sets collected in two geographically disparate locations.

Semi-automated camera trap image processing for the detection of ungulate fence crossing events.

Remote cameras are an increasingly important tool for ecological research. While remote camera traps collect field data with minimal human attention, the images they collect require post-processing and characterization before it can be ecologically and statistically analyzed, requiring the input of substantial time and money from researchers. The need for post-processing is due, in part, to a high incidence of non-target images. We developed a stand-alone semi-automated computer program to aid in image processing, categorization, and data reduction by employing background subtraction and histogram rules. Unlike previous work that uses video as input, our program uses still camera trap images. The program was developed for an ungulate fence crossing project and tested against an image dataset which had been previously processed by a human operator. Our program placed images into categories representing the confidence of a particular sequence of images containing a fence crossing event. This resulted in a reduction of 54.8% of images that required further human operator characterization while retaining 72.6% of the known fence crossing events. This program can provide researchers using remote camera data the ability to reduce the time and cost required for image post-processing and characterization. Further, we discuss how this procedure might be generalized to situations not specifically related to animal use of linear features.

More Than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People With Aphasia Using Multiple-Choice Image Displays.

Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic comprehension may influence performance. In this study we explore the influence of physical image characteristics of multiple-choice image displays on visual attention allocation by PWA. Eye fixations of 41 PWA were recorded while they viewed 40 multiple-choice image sets presented with and without verbal stimuli. Within each display, 3 images (majority images) were the same and 1 (singleton image) differed in terms of 1 image characteristic. The mean proportion of fixation duration (PFD) allocated across majority images was compared against the PFD allocated to singleton images. PWA allocated significantly greater PFD to the singleton than to the majority images in both nonverbal and verbal conditions. Those with greater severity of comprehension deficits allocated greater PFD to nontarget singleton images in the verbal condition. When using tasks that rely on multiple-choice displays and verbal stimuli, one cannot assume that verbal stimuli will override the effect of visual-stimulus characteristics.

Misaligned and Polarity-Reversed Faces Determine Face-specific Capacity Limits.

Previous research using flanker paradigms suggests that peripheral distracter faces are automatically processed when participants have to classify a single central familiar target face. These distracter interference effects disappear when the central task contains additional anonymous (non-target) faces that load the search for the face target, but not when the central task contains additional non-face stimuli, suggesting there are face-specific capacity limits in visual processing. Here we tested whether manipulating the format of non-target faces in the search task affected face-specific capacity limits. Experiment 1 replicated earlier findings that a distracter face is processed even in high load conditions when participants looked for a target name of a famous person among additional names (non-targets) in a central search array. Two further experiments show that when targets and non-targets were faces (instead of names), however, distracter interference was eliminated under high load—adding non-target faces to the search array exhausted processing capacity for peripheral faces. The novel finding was that replacing non-target faces with images that consisted of two horizontally misaligned face-parts reduced distracter processing. Similar results were found when the polarity of a non-target face image was reversed. These results indicate that face-specific capacity limits are not determined by the configural properties of face processing, but by face parts.

Words jump-start vision: a label advantage in object recognition

People use language to shape each other's behavior in highly flexible ways. Effects of language are often assumed to be high-level in that, whereas language clearly influences reasoning, decision making, and memory, it does not influence low-level visual processes. Here, we test the prediction that words are able to provide top-down guidance at the very earliest stages of visual processing by acting as powerful categorical cues. We investigated whether visual processing of images of familiar animals and artifacts was enhanced after hearing their name (e.g., dog) compared with hearing an equally familiar and unambiguous nonverbal sound (e.g., a dog bark) in 14 English monolingual speakers. Because the relationship between words and their referents is categorical, we expected words to deploy more effective categorical templates, allowing for more rapid visual recognition. By recording EEGs, we were able to determine whether this label advantage stemmed from changes to early visual processing or later semantic decision processes. The results showed that hearing a word affected early visual processes and that this modulation was specific to the named category. An analysis of ERPs showed that the P1 was larger when people were cued by labels compared with equally informative nonverbal cues-an enhancement occurring within 100 ms of image onset, which also predicted behavioral responses occurring almost 500 ms later. Hearing labels modulated the P1 such that it distinguished between target and nontarget images, showing that words rapidly guide early visual processing.

The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.