Visual Task Performance Research Articles

A Comparative Study of Swin Transformer and CNN Models for Crowd Counting

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

Dopaminergic modulation of visual attention in the prefrontal cortex

An essential cognitive ability that enables creatures to interpret pertinent information from their surroundings only is visual attention. The prefrontal cortex (PFC) is involved in modulating visual attention, and this review focuses on the dopaminergic aspects of this process. The PFC is crucial for top-down attention management, improving visual responses, and coordinating neural activity. This is especially true of the frontal eye field (FEF). Dopamine is an important neuromodulator that directly influences the PFC's visual signal processing by adjusting sensory input and the variability of neuronal response. The substantia nigra (SN) and ventral tegmental area (VTA) send dopaminergic pathways to the prefrontal cortex, where D1 and D2 receptors have different functions in attention regulation. Dopamine receptor modification has been shown in studies involving rodents and primates to have a major effect on visual attention and cognitive task performance. The exact mechanisms underlying dopamine's involvement in executive control are still unknown despite a great deal of research, which calls for more study, especially in rodent models. The present review highlights the significance of dopamine in the domain of high-level cognitive regulation and advocates for more investigation to clarify its mechanisms in visual attention.

Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles

Drone vision is widely used in change detection, disaster response, and military reconnaissance due to its wide field of view and flexibility. However, under haze and thin cloud conditions, image quality is usually degraded due to atmospheric scattering. This results in issues like color distortion, reduced contrast, and lower clarity, which negatively impact the performance of subsequent advanced visual tasks. To improve the quality of unmanned aerial vehicle (UAV) images, we propose a dehazing method based on calibration of the atmospheric scattering model. We designed two specialized neural network structures to estimate the two unknown parameters in the atmospheric scattering model: the atmospheric light intensity A and medium transmission t. However, calculation errors always occur in both processes for estimating the two unknown parameters. The error accumulation for atmospheric light and medium transmission will cause the deviation in color fidelity and brightness. Therefore, we designed an encoder-decoder structure for irradiance guidance, which not only eliminates error accumulation but also enhances the detail in the restored image, achieving higher-quality dehazing results. Quantitative and qualitative evaluations indicate that our dehazing method outperforms existing techniques, effectively eliminating haze from drone images and significantly enhancing image clarity and quality in hazy conditions. Specifically, the compared experiment on the R100 dataset demonstrates that the proposed method improved the peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM) metrics by 6.9 dB and 0.08 over the second-best method, respectively. On the N100 dataset, the method improved the PSNR and SSIM metrics by 8.7 dB and 0.05 over the second-best method, respectively.

Individual Differences in the Impact of Distracting Environmental Sounds on the Performance of a Continuous Visual Task in Older Adults

Background/Objectives: Vulnerability to sound distraction is commonly reported in older adults with dementia and tends to be associated with adverse impacts on daily activity. However, study outcome heterogeneity is increasingly evident, with preserved resistance to distraction also evident. Contributory factors may include individual differences in distractibility in older adulthood per se, and failure to consider the influence of how difficult a person found the test. Methods: We therefore measured distractibility in a group of older adults by comparing the performance of a primary visual task (Swansea Test of Attentional Control), which includes an adaptive algorithm to take into account how difficult a person finds the test under both no-sound and sound conditions. Results: Analysis revealed no significant difference in group mean performance between no-sound versus sound conditions [t (33) = 0.181, p = 0.858; Cohen’s effect size d = −0.028], but individual differences in performance both within and between sound and no-sound conditions were evident, indicating that for older adults, distracting sounds can be neutral, detrimental, or advantageous with respect to visual task performance. It was not possible to determine individual thresholds for whether sound versus no-sound conditions affected a person’s actual behaviour. Conclusions: Nevertheless, our findings indicate how variable such effects may be in older adults, which in turn may help to explain outcome heterogeneity in studies including people living with dementia. Furthermore, such within-group heterogeneity highlights the importance of considering a person’s individual performance in order to better understand their behaviour and initiate interventions as required.

Time courses of brain plasticity underpinning visual motion perceptual learning

Visual perceptual learning (VPL) refers to a long-term improvement of visual task performance through training or experience, reflecting brain plasticity even in adults. In human subjects, VPL has been mostly studied using functional magnetic resonance imaging (fMRI). However, due to the low temporal resolution of fMRI, how VPL affects the time course of visual information processing is largely unknown. To address this issue, we trained human subjects to perform a visual motion direction discrimination task. Their behavioral performance and magnetoencephalography (MEG) signals responding to the motion stimuli were measured before, immediately after, and two weeks after training. Training induced a long-lasting behavioral improvement for the trained direction. Based on the MEG signals from occipital sensors, we found that, for the trained motion direction, VPL increased the motion direction decoding accuracy, reduced the motion direction decoding latency, enhanced the direction-selective channel response, and narrowed the tuning profile. Following the MEG source reconstruction, we showed that VPL enhanced the cortical response in early visual cortex (EVC) and strengthened the feedforward connection from EVC to V3A. These VPL-induced neural changes co-occurred in 160–230 ms after stimulus onset. Complementary to previous fMRI findings on VPL, this study provides a comprehensive description on the neural mechanisms of visual motion perceptual learning from a temporal perspective and reveals how VPL shapes the time course of visual motion processing in the adult human brain.

LGIT: local–global interaction transformer for low-light image denoising

Transformer-based methods effectively capture global dependencies in images, demonstrating outstanding performance in multiple visual tasks. However, existing Transformers cannot effectively denoise large noisy images captured under low-light conditions owing to (1) the global self-attention mechanism causing high computational complexity in the spatial dimension owing to a quadratic increase in computation with the number of tokens; (2) the channel-wise self-attention computation unable to optimise the spatial correlations in images. We propose a local–global interaction Transformer (LGIT) that employs an adaptive strategy to select relevant patches for global interaction, achieving low computational complexity in global self-attention computation. A top-N patch cross-attention model (TPCA) is designed based on superpixel segmentation guidance. TPCA selects top-N patches most similar to the target image patch and applies cross attention to aggregate information from them into the target patch, effectively enhancing the utilisation of the image’s nonlocal self-similarity. A mixed-scale dual-gated feedforward network (MDGFF) is introduced for the effective extraction of multiscale local correlations. TPCA and MDGFF were combined to construct a hierarchical encoder-decoder network, LGIT, to compute self-attention within and across patches at different scales. Extensive experiments using real-world image-denoising datasets demonstrated that LGIT outperformed state-of-the-art (SOTA) convolutional neural network (CNN) and Transformer-based methods in qualitative and quantitative results.

Introducing ART: A new method for testing auditory memory with circular reproduction tasks.

Theories of visual working memory have seen significant progress through the use of continuous reproduction tasks. However, these tasks have mainly focused on studying visual features, with limited examples existing in the auditory domain. Therefore, it is unknown to what extent newly developed memory models reflect domain-general limitations or are specific to the visual domain. To address this gap, we developed a novel methodology: the Auditory Reproduction Task (ART). This task utilizes Shepard tones, which create an infinite rising or falling tone illusion by dissecting pitch chroma and height, to create a 1-360° auditory circular space. In Experiment 1, we validated the perceptual circularity and uniformity of this auditory stimulus space. In Experiment 2, we demonstrated that auditory working memory shows similar set size effects to visual working memory-report error increased at a set size of 2 relative to 1, caused by swap errors. In Experiment 3, we tested the validity of ART by correlating reproduction errors with commonly used auditory and visual working memory tasks. Analyses revealed that ART errors were significantly correlated with performance in both auditory and visual working memory tasks, albeit with a stronger correlation observed with auditory working memory. While these experiments have only scratched the surface of the theoretical and computational constraints on auditory working memory, they provide a valuable proof of concept for ART. Further research with ART has the potential to deepen our understanding of auditory working memory, as well as to explore the extent to which existing models are tapping into domain-general constraints.

RViT: Robust Fusion Vision Transformer with Variational Hierarchical Denoising Process for Image Classification

Transformers designed for natural language processing have originally been explored for computer vision in recent research. Various Vision Transformers (ViTs) play an increasingly important role in the field of image tasks such as computer vision, multimodal fusion and multimedia analysis. However, to obtain promising performance, most existing ViTs usually rely on artificially filtered high-quality images, which may suffer from inherent noise risk. Generally, such well-constructed images are not always available in every situation. To this end, we propose a Robust ViT (RViT) to focus on the relevant and robust representation learning for image classification tasks. Specifically, we first develop a novel Denoising VTUnet module, where we conceptualize the nonrobust noise as the uncertainty under the variational conditions. Furthermore, we design a fusion transformer backbone with a tailored fusion attention mechanism to perform image classification based on the extracted robust representations effectively. To demonstrate the superiority of our model, the compared experiments are conducted on several popular datasets. Benefiting from the sequence regularity of the Transformer and captured robust feature, the proposed method exceeds compared Transformer-based models with superior performance in visual tasks.

MixImages: An Urban Perception AI Method Based on Polarization Multimodalities.

Intelligent urban perception is one of the hot topics. Most previous urban perception models based on semantic segmentation mainly used RGB images as unimodal inputs. However, in natural urban scenes, the interplay of light and shadow often leads to confused RGB features, which diminish the model's perception ability. Multimodal polarization data encompass information dimensions beyond RGB, which can enhance the representation of shadow regions, serving as additional data for assistance. Additionally, in recent years, transformers have achieved outstanding performance in visual tasks, and their large, effective receptive field can provide more discriminative cues for shadow regions. For these reasons, this study proposes a novel semantic segmentation model called MixImages, which can combine polarization data for pixel-level perception. We conducted comprehensive experiments on a polarization dataset of urban scenes. The results showed that the proposed MixImages can achieve an accuracy advantage of 3.43% over the control group model using only RGB images in the unimodal benchmark while gaining a performance improvement of 4.29% in the multimodal benchmark. Additionally, to provide a reference for specific downstream tasks, we also tested the impact of different combinations of polarization types on the overall segmentation accuracy. The proposed MixImages can be a new option for conducting urban scene perception tasks.

Speech, Nonspeech Audio, and Visual Interruptions of a Tracking Task: A Replication and Extension of Nees and Sampsell (2021)

Interruptions from technology—such as alerts from mobile communication devices—are a pervasive aspect of modern life. Interruptions can be detrimental to performance of the ongoing, interrupted task. Designers often can choose whether interruptions are delivered as visual or auditory alerts. Contradictory theories have emerged regarding whether auditory or visual alerts are more harmful to performance of ongoing visual tasks. Multiple Resources Theory predicts better overall performance with auditory alerts, but Auditory Preemption Theory predicts better overall performance with visual alerts. Nees and Sampsell previously found that multitasking was superior with nonspeech auditory alerts as compared to visual alerts. In the current experiment, their methods were replicated and extended to include a speech auditory alerts condition. Performance of the ongoing tracking task was worse with interruption from visual alerts, and perceived workload also was highest in this condition. Reaction time to alerts was fastest with visual alerts. There also was converging evidence to suggest that performance with speech alerts was superior to performance with nonspeech tonal alerts. The current experiment replicated the results of Nees and Sampsell and extended their findings to speech alert sounds. Like in their study, the pattern of findings here support Multiple Resources Theory over Auditory Preemption Theory.

Daylighting Through Electrochromic Glazing Under Overcast Skies: Impacts on Visual Task Performance and Perception

ABSTRACT Electrochromic (EC) glazing allows modulating the intensity of visual and solar transmission by dynamically switching between bleached (clear) and tinted (colored) states, leading to changes in spectral power distribution (SPD), correlated color temperature (CCT), and illuminance of the incoming daylight. In this experimental study, we investigated the impact of spectral composition of daylight filtered through a 6-pane EC façade on visual task performance and visual perception. In a semi-controlled test room, 19 subjects were exposed to five setting scenarios of the EC façade (fully bleached, mixed setting after fully bleached, fully tinted, mixed setting after fully tinted, and user-controlled). Test subjects performed a series of visual tasks, and their assessments of the indoor environment and of the outside view were collected along with luminous and spectral measurements under each condition. The analysis provided statistical evidence that the changes in spectral composition of daylight do not have a practically relevant effect on visual task performance in terms of visual acuity and color naming accuracy. However, the daylight filtered through the fully tinted glazing had a substantive impact on visual perception, evoking negative responses to the color rendering of the indoor environment and of the outside view. A mixed settings of the EC façade could improve a natural assessment of the incoming light compared to the fully tinted state, achieving better ratings in terms of perception of the indoor and outdoor environments. When given control of the EC glazing, subjects expressed higher acceptance and satisfaction compared to the tinted and mixed scenarios.

The Effect of Icon Color Combinations in Information Interfaces on Task Performance under Varying Levels of Cognitive Load

In human–computer interaction interfaces, icons serve as highly symbolic elements that convey information and significantly influence the performance of visual search and other tasks. Thus, the selection of appropriate colors plays a crucial role in the design of human–computer interfaces. This study aimed to investigate the effects of icon color combinations on human visual search task performance across various cognitive load conditions. The experiment was divided into two parts, involving a cognitive load test experiment and an icon search task, wherein the former required participants to select the target icon from 16 de-colored icons, and the latter involved selecting the target icon from 16 color combinations of 8 colors (background colors: black, red, blue, and purple; foreground colors: white, yellow, green, and turquoise). A total of 20 participants (11 females and 9 males, all aged between 18 and 24 years) were recruited for this experiment to perform both tasks sequentially. Through a comprehensive consideration of factors such as the NASA-TLX scale, retrieval time, etc., the experimental results revealed that cognitive load increased with time pressure, resulting in varying optimal color combinations for different cognitive load levels. Additionally, background colors did not affect task response time under different cognitive loads, whereas white foreground color was superior to turquoise foreground color in a medium cognitive load environment. Meanwhile, our results recommend prioritizing white-on-black as the preferred color combination, given that it demonstrated excellent cognitive performance in all three cognitive load environments. Conversely, white-on-blue is not recommended as a color combination for the design of high cognitive load environments. Concerning medium cognitive load environments, yellow-on-red or white-on-purple color combinations were preferred. Finally, in low cognitive load environments, white-on-purple is recommended as the primary color combination. Overall, this study provides a theoretical reference for the future design of interactive interface icons across various contexts.

When Load is Low, Working Memory is Shielded From Long-Term Memory's Influence.

Previous studies found that episodic long-term memory (eLTM) enhances working memory (WM) performance when both novel and previously learnt word pairs must be retained on a short-term basis. However, there is uncertainty regarding how and when WM draws on eLTM. Three possibilities are (a) that people draw on eLTM only if WM capacity is exceeded; (b) that there is always a contribution of eLTM to WM performance, irrespective of whether prior knowledge is helpful or not; or (c) benefits of prior knowledge are specific to comparisons between conditions which are similarly ambiguous concerning whether LTM may be useful. We built on the assumption that under conditions of a contribution from LTM, these LTM traces of memoranda could benefit or hamper performance in WM tasks depending on the match between the traces stored in LTM and the ones to-be stored in WM in the current trial, yielding proactive facilitation (PF) and proactive interference (PI), respectively. Across four experiments, we familiarized participants with some items before they completed a separate WM task. In accordance with possibility (a) we show that there are indeed conditions in which only WM contributes to performance. Performance deteriorated with the addition of stimuli from eLTM when WM load was low, but not when it was high; and an exchange of information between LTM and WM occurred only when WM capacity was exceeded, with PI and PF effects affecting immediate memory performance in verbal and visual tasks only at higher set sizes.

Simulating vision impairment in virtual reality: a comparison of visual task performance with real and simulated tunnel vision

In this work, we explore the potential and limitations of simulating gaze-contingent tunnel vision conditions using Virtual Reality (VR) with built-in eye tracking technology. This approach promises an easy and accessible way of expanding study populations and test groups for visual training, visual aids, or accessibility evaluations. However, it is crucial to assess the validity and reliability of simulating these types of visual impairments and evaluate the extend to which participants with simulated tunnel vision can represent real patients. Two age-matched participant groups were acquired: The first group (n = 8, aged 20–60, average 49.1 ± 13.2) consisted of patients diagnosed with Retinitis pigmentosa (RP). The second group (n = 8, aged 27–59, average 46.5 ± 10.8) consisted of visually healthy participants with simulated tunnel vision. Both groups carried out different visual tasks in a virtual environment for 30 min per day over the course of four weeks. Task performances as well as gaze characteristics were evaluated in both groups over the course of the study. Using the ’two one-sided tests for equivalence’ method, the two groups were found to perform similar in all three visual tasks. Significant differences between groups were found in different aspects of their gaze behavior, though most of these aspects seem to converge over time. Our study evaluates the potential and limitations of using Virtual Reality technology to simulate the effects of tunnel vision within controlled virtual environments. We find that the simulation accurately represents performance of RP patients in the context of group averages, but fails to fully replicate effects on gaze behavior.

Effects of partial sleep deprivation on visual search behavior and complex task performance under low and high threat.

Effects of partial sleep deprivation on visual search behavior and complex task performance under low and high threat.

A Dual-Branch Autoencoder Network for Underwater Low-Light Polarized Image Enhancement

Underwater detection faces uncomfortable illumination conditions, and traditional optical images sensitive to intensity often cannot work well in these conditions. Polarization imaging is a good solution for underwater detection under adverse lighting conditions. However, the process of obtaining polarization information causes it to be more sensitive to noise; serious noise reduces the quality of polarized images and subsequent performance in advanced visual tasks. Unfortunately, the flourishing low-light image enhancement methods applied to intensity images have not demonstrated satisfactory performance when transferred to polarized images. In this paper, we propose a low-light image enhancement paradigm based on the antagonistic properties of polarization parameters. Furthermore, we develop a dual-branch network that relies on a gradient residual dense feature extraction module (GRD) designed for polarized image characteristics and polarization loss, effectively avoiding noise introduced during the direct amplification of brightness, and capable of restoring target contour details. To facilitate a data-driven learning method, we propose a simulation method for underwater low-light polarized images. Extensive experimental results on real-world datasets demonstrate the effectiveness of our proposed approach and its superiority against other state-of-the-art methods.

AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models

Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specific domain knowledge and have a weaker understanding of localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide fine-grained semantic and design a prompt learner to fine-tune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state-of-the-art performance with an accuracy of 86.1%, an image-level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset.

Supplemental Material for Effects of Partial Sleep Deprivation on Visual Search Behavior and Complex Task Performance Under Low and High Threat

Supplemental Material for Effects of Partial Sleep Deprivation on Visual Search Behavior and Complex Task Performance Under Low and High Threat

Maximal aerobic exercise and acute visual performance in females: Implications for concussion side-line testing

PurposeMany sports-related concussion (SRC) outcomes can be prevented by removing affected athletes and allowing return after full recovery. Diagnosing concussions on the side-line is challenging, as tools often rely on visual performance assessment. Since acute exercise can affect vision, it is vital to determine if exercise can mask potential brain injury if visual performance assessments are used. The aim of this study was to determine the effect of a single bout of maximal aerobic exercise on acute visual performance. MethodsEighty previously sedentary females were randomly assigned to an experimental group (N = 40) or control group (N = 40) and completed an identical visual task test battery. Two weeks later, participants returned to participate in the test battery, with the experimental group doing so immediately following a maximal treadmill exercise protocol. ResultsSignificant (P < 0.05) improvements were found in accommodation facility, saccadic eye movement, speed of recognition, peripheral awareness, and hand-eye coordination (P < 0.001 for all), but not in visual memory (P = 0.556). All visual tasks, barring visual memory, loaded onto a single factor and approximated the root mean square error of approximation (RMSEA) threshold value (RMSEA = 0.112 [90 %CI: 0.047–0.182]), and a significant large main effect was found on all factors as a universal visual task performance (P < 0.001; d = 1.01). ConclusionThis study shows that exercise can affect performance across multiple, but not all, visual tasks and should be utilised with caution in comprehensive side-line SRC assessments. Visual memory may be more stable to the effects of acute exercise and be considered for inclusion in side-line SRC assessments.

FAM: Improving columnar vision transformer with feature attention mechanism

Vision Transformer has garnered outstanding performance in visual tasks due to its capability for global modeling of image information. However, during the self-attention computation of image tokens, a common issue of attention map homogenization arises, impacting the final performance of the model as attention maps propagate through feature maps layer by layer. In this research, we propose a token-based approach to adjust the output of attention sub-layer, focusing on the feature dimensions, to address the homogenization problem. Furthermore, different network architectures exhibit variations in their approaches to modeling image features. Specifically, Vision Transformers excel at modeling long-range relationships, while convolutional neural networks possess local receptive fields. Therefore, this paper introduces a plug-and-play convolutional operator-based component, integrated into the Vision Transformer, to validate the impact of structural enhancements on model performance. Experimental results on image recognition and adversarial attack tasks respectively demonstrate the effectiveness and robustness of the two proposed methods. Additionally, the analysis of information entropy on the feature maps of the model’s final layer indicates that the improved model exhibits higher information richness, making it more conducive to the classifier’s discriminative capabilities.