Visual Task Performance Research Articles

Exploring neural architectures for simultaneously recognizing multiple visual attributes

Much experimental evidence in neuroscience has suggested a division of higher visual processing into a ventral pathway specialized for object recognition and a dorsal pathway specialized for spatial recognition. Previous computational studies have suggested that neural networks with two segregated pathways (branches) have better performance in visual recognition tasks than neural networks with a single pathway (branch). One previously proposed possibility is that two pathways increase the learning efficiency of a network by allowing separate networks to process information about different visual attributes separately. However, most of these previous studies were limited, considering recognition of only two visual attributes, identity and location, simultaneously with a restricted number of classes in each attribute. We investigate whether it is always advantageous to use two-pathway networks when recognizing other visual attributes as well as examine whether the advantage of using two-pathway networks would be different when there are a different number of classes in each attribute. We find that it is always advantageous to use segregated pathways to process different visual attributes separately, with this advantage increasing with a greater number of classes. Thus, using a computational approach, we demonstrate that it is computationally advantageous to have separate pathways if the amount of variations of a given visual attribute is high or that attribute needs to be finely discriminated. Hence, when the size of the computer vision model is limited, designing a segregated pathway (branch) for a given visual attribute should only be used when it is computationally advantageous to do so.

Underwater image enhancement through color deviation detection-guided peak flattening

Underwater images frequently exhibit color deviation and blurring. Color deviation arises from the selective spectral absorption of light, whereas blurring results from particle scattering within the water. These challenges impede the performance of high-level visual perception tasks. To improve the underwater image quality, we propose an adaptive method for underwater image enhancement that employs color deviation detection-guided peak flattening. Utilizing six features identified through statistical analysis of the latest underwater real-image datasets, we develop a color deviation detection model for underwater images, which employs ensemble learning to quantify the extent of color deviation. Subsequently, we devise the peak flattening algorithm to achieve histogram adaptive partition conversion. The conversion range is determined by the estimated color deviation value, eliminating the need for iteration. For channels with severe color deviation, we employ the weighted fusion to restore partial gray distribution. Comparisons with state-of-the-art methods demonstrate that the proposed method significantly improves contrast and color fidelity, particularly in images with severe degradation.

A comparative analysis of partial sleep restriction versus split sleep regimen on cognitive processing, declarative memory, and affective behaviour in nursing students

Objectives: Sleep plays a regulatory role in functions such as abstraction, fluid intelligence, and declarative memory. This research seeks to assess the influence of two sleep restriction schedules—partial nocturnal sleep restriction and a split sleep (SS) regimen—on fluid intelligence and various memory functions (including encoding, consolidation, and retrieval) in shift-working nurses. Materials and Methods: The research involved 46 4th-year nursing students (23 males and 23 females) assigned to night duties, categorised into two groups: Partial sleep restriction (5 h of night sleep with 4 h of daytime recovery sleep) and SS (5 h of night sleep with 1.5 h of an afternoon nap and recovery sleep of 2.5 h following the tests). Instruments such as the Pittsburgh sleep quality index (PSQI), Karolinska sleepiness scale (KSS), and Raven’s Progressive Matrices were utilised to evaluate sleep quality, sleepiness levels, overall mood, benefits of daytime naps, and fluid intelligence. Memory-related activities comprised a picture-encoding task and a factual knowledge task, evaluated through subjective assessments and two alternative choice questions. Results: The PSQI global scores highlighted notable differences, with female nursing students in the partial sleep group scoring lower (9 ± 2.311) than their counterparts in the SS group (5 ± 1.09). Sleepiness, gauged by the KSSKSS score, was higher in the partial sleep group (7 ± 3.2) compared to female nursing students in the SS regimen (5 ± 5.8). Results from Raven’s progressive matrices pointed to delayed abstraction in the partial sleep restriction group (9.31 ± 6.24) in contrast to the SS group (9.01 ± 5.59), indicating poorer performance in visual attentive tasks. Positive and negative affect schedule scores unveiled heightened negativity in mood due to sleep restriction in the partial sleep group, with less hostility observed in those with an afternoon nap, albeit not reaching statistical significance. Positive moods exhibited fluctuation, with attentiveness declining in the partial sleep group. The study affirmed the advantages of a daytime nap on long-term memory. Female nurses in the SS regimen displayed statistically significant picture encoding accuracy (78 ± 3.65) with faster reaction times (2 ± 6.37) compared to the partial sleep group. Males in the SS regimen recorded a higher percentage of hits (88 ± 5.16). Learning sessions at 3 pm, coinciding with the circadian dip, affected the partial sleep groups, whereas the nap mitigated such effects for the SS group. Allowing a 1.5-h afternoon nap synchronised with the circadian dip enhanced memory in the SS group. Conclusion: By comparing partial nocturnal sleep restriction and a SS regimen, this study unveils their distinct effects on fluid intelligence and memory processes among shift-working nurses. The results provide valuable insights into the degree of dependence of basic cortical functions on sleep for healthcare professionals navigating demanding schedules, underscoring the significance of accounting for both nocturnal sleep duration and daytime naps to optimise cognitive performance.

IF-USOD: Multimodal information fusion interactive feature enhancement architecture for underwater salient object detection

Underwater salient object detection (USOD) has garnered increasing attention due to its superior performance in various underwater visual tasks. Despite the growing interest, research on USOD remains in its nascent stages, with existing methods often struggling to capture long-range contextual features of salient objects. Additionally, these methods frequently overlook the complementary nature of multimodal information. The multimodal information fusion can render previously indiscernible objects more detectable, as capturing complementary features from diverse source images enables a more accurate depiction of objects. In this work, we explore an innovative approach that integrates RGB and depth information, coupled with interactive feature enhancement, to advance the detection of underwater salient objects. Our method first leverages the strengths of both transformer and convolutional neural network architectures to extract features from source images. Here, we employ a two-stage training strategy designed to optimize feature fusion. Subsequently, we utilize self-attention and cross-attention mechanisms to model the correlations among the extracted features, thereby amplifying the relevant features. Finally, to fully exploit features across different network layers, we introduce a cross-scale learning strategy to facilitate multi-scale feature fusion, which improves the detection accuracy of underwater salient objects by generating both coarse and fine salient predictions. Extensive experimental evaluations demonstrate the state-of-the-art model performance of our proposed method.

Frequency domain-based latent diffusion model for underwater image enhancement

The degradation of underwater images, due to complex factors, negatively impacts the performance of underwater visual tasks. However, most underwater image enhancement methods (UIE) have been confined to the spatial domain, disregarding the frequency domain. This limitation hampers the full exploitation of the model’s learning and representational capabilities. To address this, a two-stage frequency domain-based latent diffusion model (FD-LDM) is introduced for UIE. Firstly, the model employs a lightweight parameter estimation network (L-PEN) to estimate the degradation parameters of underwater images, thereby mitigating the impact of color bias on the diffusion model. Subsequently, considering the varying degrees of recovery between high and low-frequency images, high and low-frequency priors are extracted in the second stage and integrated with the refined latent diffusion model to enhance the images further. Extensive experiments have confirmed the method’s effectiveness and robustness, particularly under color bias scenes.

The intrinsic propagation directionality of fMRI infra-slow activity during visual tasks

The temporal order of propagation in the blood-oxygen-level-dependent (BOLD) infra-slow activity (ISA, 0.01–0.1 Hz) of functional magnetic resonance imaging (fMRI) can indicate the functional organization of the brain. While prior studies have revealed the temporal order of propagation of BOLD ISA during rest, how it emerges during cognitive tasks remains unclear. Furthermore, its differences between the gray and white matters at the whole-brain scale are unexplored. In this study, we probed the propagation of BOLD ISA using a publicly available fMRI dataset from participants performing visual detection and discrimination tasks (N = 46, 29 females). We examined the temporal order of propagation based on ISA oscillatory phase differences among brain parcels. During visual task performance, ISA in both the gray and white matters propagated in a direction from the visual cortex to the association cortex, including the default mode network (DMN). This result differs from the previously reported propagation direction during rest that traveled from the visual and somatosensory cortices to the DMN, suggesting that the functional organization may change when performing cognitive tasks. In addition, the propagation in the white matter represented more complex patterns than that in the gray matter, exhibiting that the cingulum preceded DMN. Our results may help the understanding of how task performance alters the sensory-DMN propagation according of ISA.

ALFusion: Adaptive fusion for infrared and visible images under complex lighting conditions

In the task of infrared and visible image fusion, source images often exhibit complex and variable characteristics due to scene illumination. To address the challenges posed by complex lighting conditions and enhance the quality of fused images, we develop the ALFusion method, which combines dynamic convolution and Transformer for infrared and visible image fusion. The core idea is to utilize the adaptability of dynamic convolution coupled with the superior long-range modeling capabilities of the Transformer to design a hybrid feature extractor. This extractor dynamically and comprehensively captures features from source images under various lighting conditions. To enhance the effectiveness of mixed features, we integrate an elaborate multi-scale attention enhancement module into the skip connections of the U-Net architecture. In this module, we use convolutional kernels of various sizes to expand the receptive field and incorporate an attention mechanism to enhance and highlight the combined use of dynamic convolution and the Transformer. Considering the performance in advanced visual tasks, an illumination detection network is integrated into the loss function, strategically balancing the pixel fusion ratio and optimizing visual impacts under varied lighting conditions, leveraging information from different source images. The fusion results indicate that our method consistently delivers superior background contrast and enhanced texture and structural features across diverse lighting conditions. Ablation and complementary experiments have been conducted, further affirming the effectiveness of our proposed method and highlighting its potential in advanced visual tasks.

A Comparative Study of Swin Transformer and CNN Models for Crowd Counting

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

Correction to “Effects of partial sleep deprivation on visual search behavior and complex task performance under low and high threat,” by Nieuwenhuys (2024).

Correction to “Effects of partial sleep deprivation on visual search behavior and complex task performance under low and high threat,” by Nieuwenhuys (2024).

Dopaminergic modulation of visual attention in the prefrontal cortex

An essential cognitive ability that enables creatures to interpret pertinent information from their surroundings only is visual attention. The prefrontal cortex (PFC) is involved in modulating visual attention, and this review focuses on the dopaminergic aspects of this process. The PFC is crucial for top-down attention management, improving visual responses, and coordinating neural activity. This is especially true of the frontal eye field (FEF). Dopamine is an important neuromodulator that directly influences the PFC's visual signal processing by adjusting sensory input and the variability of neuronal response. The substantia nigra (SN) and ventral tegmental area (VTA) send dopaminergic pathways to the prefrontal cortex, where D1 and D2 receptors have different functions in attention regulation. Dopamine receptor modification has been shown in studies involving rodents and primates to have a major effect on visual attention and cognitive task performance. The exact mechanisms underlying dopamine's involvement in executive control are still unknown despite a great deal of research, which calls for more study, especially in rodent models. The present review highlights the significance of dopamine in the domain of high-level cognitive regulation and advocates for more investigation to clarify its mechanisms in visual attention.

SAVE: Encoding spatial interactions for vision transformers

Transformers have achieved impressive performance in visual tasks. Position encoding, which equips vectors (elements of input tokens, queries, keys, or values) with sequence specificity, effectively alleviates the lack of permutation relation in transformers. In this work, we first clarify that both position encoding and additional position-specific operations will introduce positional information when participating in self-attention. On this basis, most existing position encoding methods are equivalent to special affine transformations. However, this encoding method lacks the correlation of vector content interaction. We further propose Spatial Aggregation Vector Encoding (SAVE) that employs transition matrices to recombine vectors. We design two simple yet effective modes to merge other vectors, with each one serving as an anchor. The aggregated vectors control spatial contextual connections by establishing two-dimensional relationships. Our SAVE can be plug-and-play in vision transformers, even with other position encoding methods. Comparative results on three image classification datasets show that the proposed SAVE performs comparably to current position encoding methods. Experiments on detection tasks show that the SAVE improves the downstream performance of transformer-based methods. Code is available at https://github.com/maxiao0234/SAVE.

Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles

Drone vision is widely used in change detection, disaster response, and military reconnaissance due to its wide field of view and flexibility. However, under haze and thin cloud conditions, image quality is usually degraded due to atmospheric scattering. This results in issues like color distortion, reduced contrast, and lower clarity, which negatively impact the performance of subsequent advanced visual tasks. To improve the quality of unmanned aerial vehicle (UAV) images, we propose a dehazing method based on calibration of the atmospheric scattering model. We designed two specialized neural network structures to estimate the two unknown parameters in the atmospheric scattering model: the atmospheric light intensity A and medium transmission t. However, calculation errors always occur in both processes for estimating the two unknown parameters. The error accumulation for atmospheric light and medium transmission will cause the deviation in color fidelity and brightness. Therefore, we designed an encoder-decoder structure for irradiance guidance, which not only eliminates error accumulation but also enhances the detail in the restored image, achieving higher-quality dehazing results. Quantitative and qualitative evaluations indicate that our dehazing method outperforms existing techniques, effectively eliminating haze from drone images and significantly enhancing image clarity and quality in hazy conditions. Specifically, the compared experiment on the R100 dataset demonstrates that the proposed method improved the peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM) metrics by 6.9 dB and 0.08 over the second-best method, respectively. On the N100 dataset, the method improved the PSNR and SSIM metrics by 8.7 dB and 0.05 over the second-best method, respectively.

Individual Differences in the Impact of Distracting Environmental Sounds on the Performance of a Continuous Visual Task in Older Adults.

Background/Objectives: Vulnerability to sound distraction is commonly reported in older adults with dementia and tends to be associated with adverse impacts on daily activity. However, study outcome heterogeneity is increasingly evident, with preserved resistance to distraction also evident. Contributory factors may include individual differences in distractibility in older adulthood per se, and failure to consider the influence of how difficult a person found the test. Methods: We therefore measured distractibility in a group of older adults by comparing the performance of a primary visual task (Swansea Test of Attentional Control), which includes an adaptive algorithm to take into account how difficult a person finds the test under both no-sound and sound conditions. Results: Analysis revealed no significant difference in group mean performance between no-sound versus sound conditions [t (33) = 0.181, p = 0.858; Cohen's effect size d = -0.028], but individual differences in performance both within and between sound and no-sound conditions were evident, indicating that for older adults, distracting sounds can be neutral, detrimental, or advantageous with respect to visual task performance. It was not possible to determine individual thresholds for whether sound versus no-sound conditions affected a person's actual behaviour. Conclusions: Nevertheless, our findings indicate how variable such effects may be in older adults, which in turn may help to explain outcome heterogeneity in studies including people living with dementia. Furthermore, such within-group heterogeneity highlights the importance of considering a person's individual performance in order to better understand their behaviour and initiate interventions as required.

Time courses of brain plasticity underpinning visual motion perceptual learning

Visual perceptual learning (VPL) refers to a long-term improvement of visual task performance through training or experience, reflecting brain plasticity even in adults. In human subjects, VPL has been mostly studied using functional magnetic resonance imaging (fMRI). However, due to the low temporal resolution of fMRI, how VPL affects the time course of visual information processing is largely unknown. To address this issue, we trained human subjects to perform a visual motion direction discrimination task. Their behavioral performance and magnetoencephalography (MEG) signals responding to the motion stimuli were measured before, immediately after, and two weeks after training. Training induced a long-lasting behavioral improvement for the trained direction. Based on the MEG signals from occipital sensors, we found that, for the trained motion direction, VPL increased the motion direction decoding accuracy, reduced the motion direction decoding latency, enhanced the direction-selective channel response, and narrowed the tuning profile. Following the MEG source reconstruction, we showed that VPL enhanced the cortical response in early visual cortex (EVC) and strengthened the feedforward connection from EVC to V3A. These VPL-induced neural changes co-occurred in 160–230 ms after stimulus onset. Complementary to previous fMRI findings on VPL, this study provides a comprehensive description on the neural mechanisms of visual motion perceptual learning from a temporal perspective and reveals how VPL shapes the time course of visual motion processing in the adult human brain.

LGIT: local–global interaction transformer for low-light image denoising

Transformer-based methods effectively capture global dependencies in images, demonstrating outstanding performance in multiple visual tasks. However, existing Transformers cannot effectively denoise large noisy images captured under low-light conditions owing to (1) the global self-attention mechanism causing high computational complexity in the spatial dimension owing to a quadratic increase in computation with the number of tokens; (2) the channel-wise self-attention computation unable to optimise the spatial correlations in images. We propose a local–global interaction Transformer (LGIT) that employs an adaptive strategy to select relevant patches for global interaction, achieving low computational complexity in global self-attention computation. A top-N patch cross-attention model (TPCA) is designed based on superpixel segmentation guidance. TPCA selects top-N patches most similar to the target image patch and applies cross attention to aggregate information from them into the target patch, effectively enhancing the utilisation of the image’s nonlocal self-similarity. A mixed-scale dual-gated feedforward network (MDGFF) is introduced for the effective extraction of multiscale local correlations. TPCA and MDGFF were combined to construct a hierarchical encoder-decoder network, LGIT, to compute self-attention within and across patches at different scales. Extensive experiments using real-world image-denoising datasets demonstrated that LGIT outperformed state-of-the-art (SOTA) convolutional neural network (CNN) and Transformer-based methods in qualitative and quantitative results.

Introducing ART: A new method for testing auditory memory with circular reproduction tasks.

Theories of visual working memory have seen significant progress through the use of continuous reproduction tasks. However, these tasks have mainly focused on studying visual features, with limited examples existing in the auditory domain. Therefore, it is unknown to what extent newly developed memory models reflect domain-general limitations or are specific to the visual domain. To address this gap, we developed a novel methodology: the Auditory Reproduction Task (ART). This task utilizes Shepard tones, which create an infinite rising or falling tone illusion by dissecting pitch chroma and height, to create a 1-360° auditory circular space. In Experiment 1, we validated the perceptual circularity and uniformity of this auditory stimulus space. In Experiment 2, we demonstrated that auditory working memory shows similar set size effects to visual working memory-report error increased at a set size of 2 relative to 1, caused by swap errors. In Experiment 3, we tested the validity of ART by correlating reproduction errors with commonly used auditory and visual working memory tasks. Analyses revealed that ART errors were significantly correlated with performance in both auditory and visual working memory tasks, albeit with a stronger correlation observed with auditory working memory. While these experiments have only scratched the surface of the theoretical and computational constraints on auditory working memory, they provide a valuable proof of concept for ART. Further research with ART has the potential to deepen our understanding of auditory working memory, as well as to explore the extent to which existing models are tapping into domain-general constraints.

RViT: Robust Fusion Vision Transformer with Variational Hierarchical Denoising Process for Image Classification

Transformers designed for natural language processing have originally been explored for computer vision in recent research. Various Vision Transformers (ViTs) play an increasingly important role in the field of image tasks such as computer vision, multimodal fusion and multimedia analysis. However, to obtain promising performance, most existing ViTs usually rely on artificially filtered high-quality images, which may suffer from inherent noise risk. Generally, such well-constructed images are not always available in every situation. To this end, we propose a Robust ViT (RViT) to focus on the relevant and robust representation learning for image classification tasks. Specifically, we first develop a novel Denoising VTUnet module, where we conceptualize the nonrobust noise as the uncertainty under the variational conditions. Furthermore, we design a fusion transformer backbone with a tailored fusion attention mechanism to perform image classification based on the extracted robust representations effectively. To demonstrate the superiority of our model, the compared experiments are conducted on several popular datasets. Benefiting from the sequence regularity of the Transformer and captured robust feature, the proposed method exceeds compared Transformer-based models with superior performance in visual tasks.

MixImages: An Urban Perception AI Method Based on Polarization Multimodalities.

Intelligent urban perception is one of the hot topics. Most previous urban perception models based on semantic segmentation mainly used RGB images as unimodal inputs. However, in natural urban scenes, the interplay of light and shadow often leads to confused RGB features, which diminish the model's perception ability. Multimodal polarization data encompass information dimensions beyond RGB, which can enhance the representation of shadow regions, serving as additional data for assistance. Additionally, in recent years, transformers have achieved outstanding performance in visual tasks, and their large, effective receptive field can provide more discriminative cues for shadow regions. For these reasons, this study proposes a novel semantic segmentation model called MixImages, which can combine polarization data for pixel-level perception. We conducted comprehensive experiments on a polarization dataset of urban scenes. The results showed that the proposed MixImages can achieve an accuracy advantage of 3.43% over the control group model using only RGB images in the unimodal benchmark while gaining a performance improvement of 4.29% in the multimodal benchmark. Additionally, to provide a reference for specific downstream tasks, we also tested the impact of different combinations of polarization types on the overall segmentation accuracy. The proposed MixImages can be a new option for conducting urban scene perception tasks.

Speech, Nonspeech Audio, and Visual Interruptions of a Tracking Task: A Replication and Extension of Nees and Sampsell (2021)

Interruptions from technology—such as alerts from mobile communication devices—are a pervasive aspect of modern life. Interruptions can be detrimental to performance of the ongoing, interrupted task. Designers often can choose whether interruptions are delivered as visual or auditory alerts. Contradictory theories have emerged regarding whether auditory or visual alerts are more harmful to performance of ongoing visual tasks. Multiple Resources Theory predicts better overall performance with auditory alerts, but Auditory Preemption Theory predicts better overall performance with visual alerts. Nees and Sampsell previously found that multitasking was superior with nonspeech auditory alerts as compared to visual alerts. In the current experiment, their methods were replicated and extended to include a speech auditory alerts condition. Performance of the ongoing tracking task was worse with interruption from visual alerts, and perceived workload also was highest in this condition. Reaction time to alerts was fastest with visual alerts. There also was converging evidence to suggest that performance with speech alerts was superior to performance with nonspeech tonal alerts. The current experiment replicated the results of Nees and Sampsell and extended their findings to speech alert sounds. Like in their study, the pattern of findings here support Multiple Resources Theory over Auditory Preemption Theory.

Daylighting Through Electrochromic Glazing Under Overcast Skies: Impacts on Visual Task Performance and Perception

ABSTRACT Electrochromic (EC) glazing allows modulating the intensity of visual and solar transmission by dynamically switching between bleached (clear) and tinted (colored) states, leading to changes in spectral power distribution (SPD), correlated color temperature (CCT), and illuminance of the incoming daylight. In this experimental study, we investigated the impact of spectral composition of daylight filtered through a 6-pane EC façade on visual task performance and visual perception. In a semi-controlled test room, 19 subjects were exposed to five setting scenarios of the EC façade (fully bleached, mixed setting after fully bleached, fully tinted, mixed setting after fully tinted, and user-controlled). Test subjects performed a series of visual tasks, and their assessments of the indoor environment and of the outside view were collected along with luminous and spectral measurements under each condition. The analysis provided statistical evidence that the changes in spectral composition of daylight do not have a practically relevant effect on visual task performance in terms of visual acuity and color naming accuracy. However, the daylight filtered through the fully tinted glazing had a substantive impact on visual perception, evoking negative responses to the color rendering of the indoor environment and of the outside view. A mixed settings of the EC façade could improve a natural assessment of the incoming light compared to the fully tinted state, achieving better ratings in terms of perception of the indoor and outdoor environments. When given control of the EC glazing, subjects expressed higher acceptance and satisfaction compared to the tinted and mixed scenarios.