Visual Task Performance Research Articles

Low-light image enhancement via adaptive frequency decomposition network

Images captured in low light conditions suffer from low visibility, blurred details and strong noise, resulting in unpleasant visual appearance and poor performance of high level visual tasks. To address these problems, existing approaches have attempted to enhance the visibility of low-light images using convolutional neural networks (CNN). However, due to the insufficient consideration of the characteristics of the information of different frequency layers in the image, most of them yield blurry details and amplified noise. In this work, to fully extract and utilize these information, we proposed a novel Adaptive Frequency Decomposition Network (AFDNet) for low-light image enhancement. An Adaptive Frequency Decomposition (AFD) module is designed to adaptively extract low and high frequency information of different granularities. Specifically, the low-frequency information is employed for contrast enhancement and noise suppression in low-scale space and high-frequency information is for detail restoration in high-scale space. Meanwhile, a new frequency loss function are proposed to guarantee AFDNet’s recovery capability for different frequency information. Extensive experiments on various publicly available datasets show that AFDNet outperforms the existing state-of-the-art methods both quantitatively and visually. In addition, our results showed that the performance of the face detection can be effectively improved by using AFDNet as pre-processing.

Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition.

The teacher-free online Knowledge Distillation (KD) aims to train an ensemble of multiple student models collaboratively and distill knowledge from each other. Although existing online KD methods achieve desirable performance, they often focus on class probabilities as the core knowledge type, ignoring the valuable feature representational information. We present a Mutual Contrastive Learning (MCL) framework for online KD. The core idea of MCL is to perform mutual interaction and transfer of contrastive distributions among a cohort of networks in an online manner. Our MCL can aggregate cross-network embedding information and maximize the lower bound to the mutual information between two networks. This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks. Beyond the final layer, we extend MCL to intermediate layers and perform an adaptive layer-matching mechanism trained by meta-optimization. Experiments on image classification and transfer learning to visual recognition tasks show that layer-wise MCL can lead to consistent performance gains against state-of-the-art online KD approaches. The superiority demonstrates that layer-wise MCL can guide the network to generate better feature representations. Our code is publicly avaliable at https://github.com/winycg/L-MCL.

Diagnostic Performance of an Eye-Tracking Assisted Visual Inference Language Test in the Assessment of Cognitive Decline due to Alzheimer's Disease.

The assessment of language changes associated with visual search impairment can be an important diagnostic tool in the Alzheimer's disease (AD) continuum. Investigate the performance of an eye-tracking assisted visual inference language task in differentiating subjects with mild cognitive impairment (MCI) or AD dementia from cognitively unimpaired older adults (controls). We assessed a group of 95 older adults (49 MCI, 18 mild dementia due to AD, and 28 controls). The subjects performed the same task under multiple experimental conditions which generate correlated responses that need to be taken into account. Thus, we performed a non-parametric repeated measures ANOVA model for verbal answers, and a linear mixed model (LMM) or its generalized version for the analysis of eye tracking variables. Significant differences were found in verbal answers across all diagnostic groups independently of type of inference, i.e., logic or pragmatic. Also, eye-tracking parameters were able to discriminate AD from MCI and controls. AD patients did more visits to challenge stimulus (Control-AD, -0.622, SE = 0.190, p = 0.004; MCI-AD, -0.514, SE = 0.173, p = 0.011), more visits to the correct response stimulus (Control-AD, -1.363, SE = 0.383, p = 0.002; MCI-AD, -0.946, SE = 0.349, p = 0.022), more fixations on distractors (Control-AD, -4.580, SE = 1.172, p = 0.001; MCI-AD, -2.940, SE = 1.070, p = 0.020), and a longer time to first fixation on the correct response stimulus (Control-AD, -0.622, SE = 0.190, p = 0.004; MCI-AD, -0.514, SE = 0.173, p = 0.011). The analysis of oculomotor behavior along with language assessment protocols may increase the sensitivity for detection of subtle deficits in the MCI-AD continuum, representing an important diagnostic tool.

Toward Data-Model-Agnostic Autonomous Machine-Generated Data Labeling and Annotation Platform: COVID-19 Autoannotation Use Case

Quick, early, and precise detection is important for diagnosis to control the spread of COVID-19 infection. Artificial Intelligence (AI) technology could certainly be used as a modulating tool to ease the detection, and help with the preventive steps further. Convolutional neural networks (CNNs) have achieved state-of-the-art performance in many visual recognition tasks. Nevertheless, most of these state-of-the-art networks highly rely on the availability of a high amount of labeled data, being an essential step in supervised machine learning tasks. Conventionally, this manual, mundane, and time-consuming process of annotating images is done by humans. Learning to localize or detect COVID-19 infection masks in our specific case study typically requires the collection of CT scan data that has been labeled with bounding boxes or similar annotations, which generally is limited. A technique that could perform such learning with much less annotations, and transfer the learned proposals that are algorithm-driven to generate more synthetic annotated samples would be helpful & quite valuable. We present such a technique inspired by weakly trained mask region based convolutional neural networks (R-CNN) architecture for localization, in which the number of images with their pixel-level masks can be a small proportion of the total dataset, and then further improvise CNNs by inversely generating dense annotations on-the-go using an algorithmic-based computational approach. We focus on alleviating the bottleneck associated with deep learning models needing annotated data for training in an intuitive reverse engineering fashion through this work. Our proposed solution can certainly provide the prospect of automated labeling on-the-fly, thereby reducing much of the manual work. As a result, one can quickly train a precise COVID-19 infection detector with the leverage of autonomous frame-by-frame machine generated annotations. The model achieved mean precision accuracy (%) of 0.99, 0.931, and 0.8 for train, validation, and test set, respectively. The results demonstrate that the proposed method can be adopted in a clinical setting for assisting radiologists, and also our fully autonomous approach can be generalized to any detection/recognition tasks at ease.

Perception of verticality in the post-COVID-19 condition correlates to infection severity.

SARS-CoV-2 infection affects multiple systems, including musculoskeletal, neurological, and respiratory systems. Changes associated with physical inactivity due to prolonged hospitalization can affect the functional capacity of individuals with long coronavirus disease 2019 (COVID-19) or post-COVID-19 condition and may cause changes in some postural control functions, such as verticality. This study aimed to evaluate the perception of verticality in individuals with long COVID. Cross-sectional study. This study included 60 participants with post-COVID-19 condition divided into 2 groups: hospitalized group (n = 24), those hospitalized owing to SARS-CoV-2 infection; and non-hospitalized group (n = 36), those infected with SARS-CoV-2 but not hospitalized. All participants were examined using a post-COVID-19 functional status (PCFS), sit-to-stand test, grip strength assessment, painful and tactile sensory assessments, visual acuity assessment, and vestibular assessment. Verticality perception was evaluated using the subjective visual vertical (SVV) and subjective haptic vertical (SHV) tests. In both tests, the absolute values (positive values only) and true values (positive and negative values) were considered. To verify potential confounders that could influence the verticality of the results, logistic regression models were used for categorical variables and multiple linear regressions were used for continuous variables. For analysis between groups, the independent samples test (Mann-Whitney U test) was used. There were no confounders between clinical variables and verticality in either group. There was a significant increase in absolute SVV (mean deviation [MD]: 2.83; P < .0001) and true SVV (MD: -4.18; P = .005) in the hospitalized group compared to the non-hospitalized group. Furthermore, there was a significant increase in the true SHV (MD: -3.6; P = .026) in the hospitalized group compared to that in the non-hospitalized group. Less accurate visual and haptic verticality perception task performance was observed in hospitalized patients with post-COVID-19 condition.

CSA: A Channel-Separated Attention Module for Enhancing MRI Reconstruction.

Channel attention mechanisms have been proven to effectively enhance network performance in various visual tasks, including the Magnetic Resonance Imaging (MRI) reconstruction task. Channel attention mechanisms typically involve channel dimensionality reduction and cross-channel interaction operations to achieve complexity reduction and generate more effective weights of channels. However, the operations may negatively impact MRI reconstruction performance since it was found that there is no discernible correlation between adjacent channels and the low information value in some feature maps. Therefore, we proposed the Channel-Separated Attention (CSA) module tailored for MRI reconstruction networks. Each layer of the CSA module avoids compressing channels, thereby allowing for lossless information transmission. Additionally, we employed the Hadamard product to realize that each channel's importance weight was generated solely based on itself, avoiding cross-channel interaction and reducing the computational complexity. We replaced the original channel attention module with the CSA module in an advanced MRI reconstruction network and noticed that CSA module achieved superior reconstruction performance with fewer parameters. Furthermore, we conducted comparative experiments with state-of-the-art channel attention modules on an identical network backbone, CSA module achieved competitive reconstruction outcomes with only approximately 1.036% parameters of the Squeeze-and-Excitation (SE) module. Overall, the CSA module makes an optimal trade-off between complexity and reconstruction quality to efficiently and effectively enhance MRI reconstruction. The code is available at https://github.com/smd1997/CSA-Net.

DEGAN: Decompose-Enhance-GAN Network for Simultaneous Low-Light Image Lightening and Denoising

Images taken in low-light situations frequently have a significant quality reduction. Taking care of these degradation problems in low-light images is essential for raising their visual quality and enhancing high-level visual task performance. However, because of the inherent information loss in dark images, conventional Retinex-based approaches for low-light image enhancement frequently fail to accomplish real denoising. This research introduces DEGANet, a revolutionary deep-learning framework created particularly for improving and denoising low-light images. To overcome these restrictions, DEGANet makes use of the strength of a Generative Adversarial Network (GAN). The Decom-Net, Enhance-Net, and an Adversarial Generative Network (GAN) are three linked subnets that make up our novel Retinex-based DEGANet architecture. The Decom-Net is in charge of separating the reflectance and illumination components from the input low-light image. This decomposition enables Enhance-Net to effectively enhance the illumination component, thereby improving the overall image quality. Due to the complicated noise patterns, fluctuating intensities, and intrinsic information loss in low-light images, denoising them presents a significant challenge. By incorporating a GAN into our architecture, DEGANet is able to effectively denoise and smooth the enhanced image as well as retrieve the original data and fill in the gaps, producing an output that is aesthetically beautiful while maintaining key features. Through a comprehensive set of studies, we demonstrate that DEGANet exceeds current state-of-the-art methods in both terms of image enhancement and denoising quality.

Learning Semantic Degradation-Aware Guidance for Recognition-Driven Unsupervised Low-Light Image Enhancement

Low-light images suffer severe degradation of low lightness and noise corruption, causing unsatisfactory visual quality and visual recognition performance. To solve this problem while meeting the unavailability of paired datasets in wide-range scenarios, unsupervised low-light image enhancement (ULLIE) techniques have been developed. However, these methods are primarily guided to alleviate the degradation effect on visual quality rather than semantic levels, hence limiting their performance in visual recognition tasks. To this end, we propose to learn a Semantic Degradation-Aware Guidance (SDAG) that perceives the low-light degradation effect on semantic levels in a self-supervised manner, which is further utilized to guide the ULLIE methods. The proposed SDAG utilizes the low-light degradation factors as augmented signals to degrade the low-light images, and then capture their degradation effect on semantic levels. Specifically, our SDAG employs the subsequent pre-trained recognition model extractor to extract semantic representations, and then learns to self-reconstruct the enhanced low-light image and its augmented degraded images. By constraining the relative reconstruction effect between the original enhanced image and the augmented formats, our SDAG learns to be aware of the degradation effect on semantic levels in a relative comparison manner. Moreover, our SDAG is general and can be plugged into the training paradigm of the existing ULLIE methods. Extensive experiments demonstrate its effectiveness for improving the ULLIE approaches on the downstream recognition tasks while maintaining a competitive visual quality. Code will be available at https://github.com/zheng980629/SDAG.

Dual attentional transformer for video visual relation prediction

Video visual relation detecti on (VidVRD) is to detect visual relations among instances as well as the trajectories of the corresponding subjects and objects in the video. Most current works improve the accuracy of tracking the objects but neglect the other key challenge, predicting the reliable visual relations in the videos, a vital meant for downstream tasks further. In this paper, we propose a dual attentional transformer network (VRD-DAT) for predicting the visual relations, also known as the predicates, in multi-relation videos. Specifically, our network first respectively targets modeling action visual predicates (Act-T) and spatial locating visual relations (Spa-T) via two parallel visual transformer structures simultaneously. Then, an attentional weighting module obtains the final precise merged visual relations. We conduct extensive experiments on two public datasets, ImageNet-VidVRD and VidOR, to demonstrate our model is capable of outperforming other state-of-the-art methods effectively on the task of video visual relation prediction. Quantitative and qualitative results also show that with more accurate visual relations, the performance of the video visual relation detection task can be further boosted.

Contrastive fusion representation learning for foreground object detection

This paper investigates the problem of an effective and robust fusion representation method for foreground moving object detection. Many fusion representation learning approaches pay attention to the similarity measurement between the fusion results and source images in texture details and pixel intensity, ignoring the harmful information, e.g., noise, blur, and extreme illumination. Therefore, the aggregated features of infrared and visible images will introduce much harmful information, affecting the model performance of downstream visual tasks. This paper tackles these problems by proposing a contrastive fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike the traditional fusion optimization mechanism, the former aims to extract valuable features and reject harmful features via the maximum mutual information theory. The latter is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Experimental results and comparisons with the state-of-the-art on three public datasets (i.e., TNO, MF, and cross-modal FOD dataset of infrared and visible images), validate the effectiveness, robustness, and overall superiority of the proposed contrast fusion representation learning method. Concisely, our contrastive fusion representation learning has gained 53.9%,43.2%,46.4%,52.3%,2.2%,87.1%,3.5% on EI, SF, DF, AG, MI, and Nabf metrics compared with the best competitors.

The effect of performance pressure and error-feedback on anxiety and performance in an interceptive task.

Whilst the disruptive effects of anxiety on attention and performance have been well documented, the antecedents to anxiety in motivated performance scenarios are less well understood. We therefore sought to understand the cognitive appraisals that mediate the relationship between pressurised performance situations and the onset of anxiety. We tested the effects of performance pressure and error feedback on appraisals of the probability and cost of failure, the experience of anxiety, and subsequent impacts on visual attention, movement kinematics, and task performance during a virtual reality interception task. A series of linear mixed effects models indicated that failure feedback and situational pressure influenced appraisals of the probability and cost of failure, which subsequently predicted the onset of anxious states. We did not, however, observe downstream effects on performance and attention. The findings support the predictions of Attentional Control Theory Sport, that (i) momentary errors lead to negative appraisals of the probability of future failure; and (ii) that appraisals of both the cost and probability of future failure are important predictors of anxiety. The results contribute to a better understanding of the precursors to anxiety and the feedback loops that may maintain anxious states.

The Impact of Ruminative Thinking on Verbal and Visual Task Performance

Research shows the detrimental effects of rumination on individuals’ cognitive performance. However, there is limited research regarding rumination interference on task performance. The aim of this study was to investigate the link between rumination and cognition and explore if its impact is better described by Baddeley’s working memory model or Borkovec’s cognitive avoidance theory of worry. A laboratory experiment was conducted comparing three ruminative valence groups (negative, neutral, and positive). The aim was to explore individuals’ performance on a verbal and visual free-recall task and determine interference effects. Task scores pre- and post-rumination were collected from 69 university student participants and two 3 × 2 ANOVAs were conducted. The results suggest rumination may not interfere with individuals’ verbal and visual task performance, indicating the link between rumination and cognition may not be as significant as previously suggested. Furthermore, the results suggest that Baddeley’s model is a better description of rumination impacts on task performance compared to Borkovec’s theoretical model.

Autistic traits are associated with enhanced working memory capacity for abstract visual stimuli

We tested whether the association between autistic traits and enhanced performance in visual-perceptual tasks extends to visual working memory capacity. We predicted that any positive effect of autistic traits on visual working memory performance would be greatest during domain-specific tasks, in which visual resources must be relied upon. We used a visual ‘matrix’ task, involving recall of black-and-white chequered patterns which increased in size, to establish participants' capacity (span). We assessed 144 young adults' (M = 22.0 years, SD = 2.5) performance on abstract, ‘low semantic’ versus ‘high semantic’ task versions. The latter offered multimodal coding due to the availability of long-term memory resources that could supplement visual working memory. Participants also completed measures of autistic traits and trait anxiety. Autistic traits, especially Attention to Detail, Attention Switching, and Communication, positively predicted visual working memory capacity, specifically in the low semantic task, which relies on visual working memory resources. Autistic traits are therefore associated with enhanced processing and recall of visual information. The benefit is removed, however, when multimodal coding may be incorporated, emphasising the visual nature of the benefit. Strengths in focused attention to detail therefore appear to benefit domain-specific visual working memory task performance.

Earcons to reduce mode confusions in partially automated vehicles: Development and application of an evaluation method

Partially automated vehicles have systems that can ensure lateral and longitudinal control through adaptive cruise control and lane centering assist, meaning that there are three possible levels (modes) of automation: manual driving, automated longitudinal control, and automated lateral and longitudinal control. Confusions can occur when drivers fail to perceive or comprehend the automation mode indicated by the interface. Earcons reflecting the hierarchy of these automation modes through the use of pitch, rhythm, and number of note variations could improve mode awareness. The goal of the present study was therefore to develop and test a method for assessing the efficiency of earcons as indicators of automation mode, based on a mode awareness model. This method consisted in checking during a controlled laboratory situation, that the earcons were perceived correctly (Experiment 1), comprehended in isolation (Experiment 2), and comprehended during the performance of a parallel visual task that mimicked the visual demand of driving with partial automation and having to react to changes of visual icons (Experiment 3). This method was applied to earcons indicating the hierarchy of automation modes. The results of these experiments indicated that the earcons were efficiently perceived. They were comprehended in isolation and had a provoked a small decrement to a visual task. The parameters used here to create earcons for the purpose of reducing mode confusions were therefore validated to future simulator studies. The method for assessing them could be applied elsewhere, to ensure that other auditory signals adequately contribute to all dimensions of mode awareness before being integrated to a vehicle.

Detecting Adversarial Examples by Input Transformations, Defense Perturbations, and Voting.

Over the past few years, convolutional neural networks (CNNs) have proved to reach superhuman performance in visual recognition tasks. However, CNNs can easily be fooled by adversarial examples (AEs), i.e., maliciously crafted images that force the networks to predict an incorrect output while being extremely similar to those for which a correct output is predicted. Regular AEs are not robust to input image transformations, which can then be used to detect whether an AE is presented to the network. Nevertheless, it is still possible to generate AEs that are robust to such transformations. This article extensively explores the detection of AEs via image transformations and proposes a novel methodology, called defense perturbation, to detect robust AEs with the same input transformations the AEs are robust to. Such a defense perturbation is shown to be an effective counter-measure to robust AEs. Furthermore, multinetwork AEs are introduced. This kind of AEs can be used to simultaneously fool multiple networks, which is critical in systems that use network redundancy, such as those based on architectures with majority voting over multiple CNNs. An extensive set of experiments based on state-of-the-art CNNs trained on the Imagenet dataset is finally reported.

Presaccadic attention sharpens visual acuity

Visual perception is limited by spatial resolution, the ability to discriminate fine details. Spatial resolution not only declines with eccentricity but also differs for polar angle locations around the visual field, also known as ‘performance fields'. To compensate for poor peripheral resolution, we make rapid eye movements—saccades—to bring peripheral objects into high-acuity foveal vision. Already before saccade onset, visual attention shifts to the saccade target location and prioritizes visual processing. This presaccadic shift of attention improves performance in many visual tasks, but whether it changes resolution is unknown. Here, we investigated whether presaccadic attention sharpens peripheral spatial resolution; and if so, whether such effect interacts with performance fields asymmetries. We measured acuity thresholds in an orientation discrimination task during fixation and saccade preparation around the visual field. The results revealed that presaccadic attention sharpens acuity, which can facilitate a smooth transition from peripheral to foveal representation. This acuity enhancement is similar across the four cardinal locations; thus, the typically robust effect of presaccadic attention does not change polar angle differences in resolution.

Development of a quantitative measurement on visual clutter in see through display.

With the wide use of transmission displays to improve operation performance, the display information highlights clutter because of the contradiction between the massive amount of information and limited display area. Our study aimed to develop a quantitative measurement for declutter design and appraisal. Using the ergonomics research system of characters and symbols in a see-through cockpit display, we set the simulated flight task interface at four pixel scale levels by enlarging all the display elements in a certain ratio. Flight task videos of 12 clutter degrees were recorded using each flight interface matched with three flight scene complexity levels. A total of 60 pilots completed the visual search tasks in the flight task video while the eye tracker was used to record the view path in real time. Visual search performance was analyzed to study the effect of various clutter factors and levels on pilots' performance in visual search tasks, and acquire quantitative clutter measure parameters. GLM univariate test revealed that there were significant differences among the fixation time in areas of interest (AOI), total Fixation point number, total fixation time at four pixel scale levels, and three flight scene complexity levels (P < 0.05). Visual search performance declined after the cutoff point, while the clutter degree increased. According to the visual search performance data, the recommend feature congestion upper pixel number limit in a 600*800 display was 18,576, and the pixel ratio was 3.87%. A quantitative measurement for declutter design and appraisal of cockpit displays was developed, which can be used to support see-through display design.

Automatic Postoperative Brain Tumor Segmentation with Limited Data using Transfer Learning and Triplet Attention

Accurate brain tumor segmentation is clinically important for diagnosis and treatment planning. Convolutional neural networks (CNNs) have achieved promising performance in various visual recognition tasks. Training such networks usually requires large amount of labeled data, which is often challenging for medical applications. In this work, we address the segmentation problem by applying transfer learning to downstream segmentation tasks. Specifically, we explore how knowledge acquired from a large preoperative dataset can be transferred to postoperative tumor segmentation on a smaller dataset. To this end, we have developed a 3D CNN for brain tumor segmentation, and fine-tuned the pretrained models on the target domain data. To better exploit the inter-channel and spatial information, triplet attention has been incorporated and extended into existing segmentation network. Extensive experiments on our dataset demonstrate the effectiveness of transfer learning and attention modules for improved postoperative tumor segmentation performance when only limited amount of annotated data is available.

Effects of anti-fatigue lenses on performance, muscle activity and subjective discomfort responses during a seated computer task

Anti-fatigue lenses (AFL) intend to provide health benefits in association with computer work. Their effects on visual and muscle discomfort mechanisms and task performance remain unclear. Twenty-three computer users (n = 12 males) underwent two 90-min computer sessions with AFL or placebo lenses. Eye strain, body discomfort, typing performance, upper trapezius (UT) activation amplitude and variability, and neck posture were analyzed for time, condition, and sex effects. No significant effects of condition were observed on the dependent measures evaluated. Discomfort increased over time (neck/shoulder: p < .001), more so in females (eye strain: p < .001). Females’ UT activation amplitude was negatively correlated to eye strain in the placebo condition (p = .05).

A Robust Feature Downsampling Module for Remote-Sensing Visual Tasks

Remote sensing (RS) images present unique challenges for computer vision due to lower resolution, smaller objects, and fewer features. Mainstream backbone networks show promising results for traditional visual tasks. However, they use convolution to reduce feature map dimensionality, which can result in information loss for small objects in RS images and decreased performance. To address this problem, we propose a new and universal downsampling module named Robust Feature Downsampling (RFD). RFD fuses multiple feature maps extracted by different downsampling techniques, creating a more robust feature map with a complementary set of features. Leveraging this, we overcome the limitations of conventional convolutional downsampling, resulting in more accurate and robust analysis of RS images. We develop two versions of RFD module, Shallow RFD (SRFD) and Deep RFD (DRFD), tailored to adapt to different stages of feature capture and improve feature robustness. We replace the downsampling layers of existing mainstream backbones with RFD module and conduct comparative experiments on several public RS image datasets. The results show significant improvements compared to baseline approaches in RS image classification, object detection, and semantic segmentation. Specifically, our RFD module achieved an average performance gain of 1.5% on NWPU-RESISC45 classification dataset without utilizing any additional pretraining data, resulting in state-of-the-art performance on this dataset. Moreover, in detection and segmentation tasks on DOTA and iSAID datasets, our RFD module outperforms the baseline approaches by 2-7% when utilizing pretraining data from NWPU-RESISC45. These results highlight the value of RFD module in enhancing the performance of RS visual tasks.