Visual Task Research Articles

A survey of neurosymbolic visual reasoning with scene graphs and common sense knowledge

Combining deep learning and common sense knowledge via neurosymbolic integration is essential for semantically rich scene representation and intuitive visual reasoning. This survey paper delves into data- and knowledge-driven scene representation and visual reasoning approaches based on deep learning, common sense knowledge and neurosymbolic integration. It explores how scene graph generation, a process that detects and analyses objects, visual relationships and attributes in scenes, serves as a symbolic scene representation. This representation forms the basis for higher-level visual reasoning tasks such as visual question answering, image captioning, image retrieval, image generation, and multimodal event processing. Infusing common sense knowledge, particularly through the use of heterogeneous knowledge graphs, improves the accuracy, expressiveness and reasoning ability of the representation and allows for intuitive downstream reasoning. Neurosymbolic integration in these approaches ranges from loose to tight coupling of neural and symbolic components. The paper reviews and categorises the state-of-the-art knowledge-based neurosymbolic approaches for scene representation based on the types of deep learning architecture, common sense knowledge source and neurosymbolic integration used. The paper also discusses the visual reasoning tasks, datasets, evaluation metrics, key challenges and future directions, providing a comprehensive review of this research area and motivating further research into knowledge-enhanced and data-driven neurosymbolic scene representation and visual reasoning.

WatchCap: Improving Scanning Efficiency in People with Low Vision through Compensatory Head Movement Stimulation

Individuals with low vision (LV) frequently face challenges in scanning performance, which in turn complicates daily activities requiring visual recognition. Although those with PVL can theoretically compensate for these scanning deficiencies through the use of active head movements, few practical applications have sought to capitalize on this potential, especially during visual recognition tasks. In this paper, we present WatchCap, a novel device that leverages the hanger reflex phenomenon to naturally elicit head movements through stimulation feedback. Our user studies, conducted with both sighted individuals in a simulated environment and people with glaucoma-related PVL, demonstrated that WatchCap's scanning-contingent stimulation enhances visual exploration. This improvement is evidenced by the fixation and saccade-related features and positive feedback from participants, which did not cause discomfort to the users. This study highlights the promise of facilitating head movements to aid those with LVs in visual recognition tasks. Critically, since WatchCap functions independently of predefined or task-specific cues, it has a wide scope of applicability, even in ambient task situations. This independence positions WatchCap to complement existing tools aimed at detailed visual information acquisition, allowing integration with existing tools and facilitating a comprehensive approach to assisting individuals with LV.

Quantum Convolutional Neural Networks for Line Orientation Classification in Pixelated Images

Quantum Convolutional Neural Networks (QCNNs) offer a promising avenue for image classification tasks due to their potential to leverage quantum properties for enhanced computational capabilities. In this paper, we explore the application of QCNNs for line orientation classification in pixelated images. Specifically, we investigate the differentiation between horizontal and vertical lines, a fundamental task in image processing and computer vision. We propose a QCNN architecture tailored to this task, leveraging quantum convolutional layers to extract features from pixelated images and classify line orientations. We demonstrate the effectiveness of our approach through experimental evaluation on benchmark datasets, comparing the performance of QCNNs with different optimizers. Our study integrated the QCNN operator and optimizer into Qiskit Machine Learning’s Neural Network Classifier, leveraging quantum computing techniques for classification tasks. Our QCNN model demonstrated a training accuracy of 71.43% and a test accuracy of 60.0%. Noteworthy observations include the failure of the SPSA optimizer to converge within the designated iterations, requiring twice the iterations compared to the COBYLA optimizer for convergence.

Knowledge is power: Open-world knowledge representation learning for knowledge-based visual reasoning

Knowledge-based visual reasoning requires the ability to associate outside knowledge that is not present in a given image for cross-modal visual understanding. Two deficiencies of the existing approaches are that (1) they only employ or construct elementary and explicit but superficial knowledge graphs while lacking complex and implicit but indispensable cross-modal knowledge for visual reasoning, and (2) they also cannot reason new/unseen images or questions in open environments and are often violated in real-world applications. How to represent and leverage tacit multimodal knowledge for open-world visual reasoning scenarios has been less studied. In this paper, we propose a novel open-world knowledge representation learning method to not only construct implicit knowledge representations from the given images and their questions but also enable knowledge transfer from a known given scene to an unknown scene for answer prediction. Extensive experiments conducted on six benchmarks demonstrate the superiority of our approach over other state-of-the-art methods. We apply our approach to other visual reasoning tasks, and the experimental results show that our approach, with its good performance, can support related reasoning applications.

Enhancing Emergency Vehicle Detection: A Deep Learning Approach with Multimodal Fusion

Emergency vehicle detection plays a critical role in ensuring timely responses and reducing accidents in modern urban environments. However, traditional methods that rely solely on visual cues face challenges, particularly in adverse conditions. The objective of this research is to enhance emergency vehicle detection by leveraging the synergies between acoustic and visual information. By incorporating advanced deep learning techniques for both acoustic and visual data, our aim is to significantly improve the accuracy and response times. To achieve this goal, we developed an attention-based temporal spectrum network (ATSN) with an attention mechanism specifically designed for ambulance siren sound detection. In parallel, we enhanced visual detection tasks by implementing a Multi-Level Spatial Fusion YOLO (MLSF-YOLO) architecture. To combine the acoustic and visual information effectively, we employed a stacking ensemble learning technique, creating a robust framework for emergency vehicle detection. This approach capitalizes on the strengths of both modalities, allowing for a comprehensive analysis that surpasses existing methods. Through our research, we achieved remarkable results, including a misdetection rate of only 3.81% and an accuracy of 96.19% when applied to visual data containing emergency vehicles. These findings represent significant progress in real-world applications, demonstrating the effectiveness of our approach in improving emergency vehicle detection systems.

Attention capture by own name decreases with speech compression

Auditory stimuli that are relevant to a listener have the potential to capture focal attention even when unattended, the listener’s own name being a particularly effective stimulus. We report two experiments to test the attention-capturing potential of the listener’s own name in normal speech and time-compressed speech. In Experiment 1, 39 participants were tested with a visual word categorization task with uncompressed spoken names as background auditory distractors. Participants’ word categorization performance was slower when hearing their own name rather than other names, and in a final test, they were faster at detecting their own name than other names. Experiment 2 used the same task paradigm, but the auditory distractors were time-compressed names. Three compression levels were tested with 25 participants in each condition. Participants’ word categorization performance was again slower when hearing their own name than when hearing other names; the slowing was strongest with slight compression and weakest with intense compression. Personally relevant time-compressed speech has the potential to capture attention, but the degree of capture depends on the level of compression. Attention capture by time-compressed speech has practical significance and provides partial evidence for the duplex-mechanism account of auditory distraction.

How we perceive the width of grasped objects: Insights into the central processes that govern proprioceptive judgements.

Low-level proprioceptive judgements involve a single frame of reference, whereas high-level proprioceptive judgements are made across different frames of reference. The present study systematically compared low-level (grasp grasp) and high-level (vision grasp, grasp vision) proprioceptive tasks, and quantified the consistency of grasp vision and possible reciprocal nature of related high-level proprioceptive tasks. Experiment 1 (n = 30) compared performance across vision grasp, a grasp vision and a grasp grasp tasks. Experiment 2 (n = 30) compared performance on the grasp vision task between hands and over time. Participants were accurate (mean absolute error 0.27 cm [0.20 to 0.34]; mean [95% CI]) and precise ( = 0.95 [0.93 to 0.96]) for grasp grasp judgements, with a strong correlation between outcomes (r = -0.85 [-0.93 to -0.70]). Accuracy and precision decreased in the two high-level tasks ( = 0.86 and 0.89; mean absolute error = 1.34 and 1.41 cm), with most participants overestimating perceived width for the vision grasp task and underestimating it for grasp vision task. There was minimal correlation between accuracy and precision for these two tasks. Converging evidence indicated performance was largely reciprocal (inverse) between the vision grasp and grasp vision tasks. Performance on the grasp vision task was consistent between dominant and non-dominant hands, and across repeated sessions a day or week apart. Overall, there are fundamental differences between low- and high-level proprioceptive judgements that reflect fundamental differences in the cortical processes that underpin these perceptions. Moreover, the central transformations that govern high-level proprioceptive judgements of grasp are personalised, stable and reciprocal for reciprocaltasks. KEY POINTS: Low-level proprioceptive judgements involve a single frame of reference (e.g. indicating the width of a grasped object by selecting from a series of objects of different width), whereas high-level proprioceptive judgements are made across different frames of reference (e.g. indicating the width of a grasped object by selecting from a series of visible lines of different length). We highlight fundamental differences in the precision and accuracy of low- and high-level proprioceptive judgements. We provide converging evidence that the neural transformations between frames of reference that govern high-level proprioceptive judgements of grasp are personalised, stable and reciprocal for reciprocal tasks. This stability is likely key to precise judgements and accurate predictions in high-level proprioception.

TinyM 2 Net-V2: A Compact Low-power Software Hardware Architecture for M ulti m odal Deep Neural Networks

With the evaluation of Artificial Intelligence (AI), there has been a resurgence of interest in how to use AI algorithms on low-power embedded systems to broaden potential use cases of the Internet of Things (IoT). To mimic multimodal human perception, multimodal deep neural networks (M-DNN) have recently become very popular with the classification task due to their impressive performance for computer vision and audio processing tasks. This article presents TinyM 2 Net-V2 —a compact low-power software hardware architecture for m ulti m odal deep neural networks for resource-constrained tiny devices. To compress the models to implement on tiny devices, cyclicly sparsification and hybrid quantization (4-bits weights and 8-bits activations) methods are used. Although model compression techniques are an active research area, we are the first to demonstrate their efficacy for multimodal deep neural networks, using cyclicly sparsification and hybrid quantization of weights/activations. TinyM 2 Net-V2 shows that even a tiny multimodal deep neural network model can improve the classification accuracy more than that of any unimodal counterparts. Parameterized M-DNN model architecture was designed to be evaluated in two different case-studies: vehicle detection from multimodal images and audios and COVID-19 detection from multimodal audio recordings. The most compressed TinyM 2 Net-V2 achieves 92.5% COVID-19 detection accuracy (6.8% improvement from the unimodal full precision model) and 90.6% vehicle classification accuracy (7.7% improvement from the unimodal full precision model). A parameterized and flexible FPGA hardware accelerator was designed as well for TinyM 2 Net-V2 models. To the best of our knowledge, this is the first work accelerating multimodal deep neural network models on low-power Artix-7 FPGA hardware. We achieved energy efficiency of 9.04 GOP/s/W and 15.38 GOP/s/W for case-study 1 and case-study 2, respectively, which is comparable to the state-of-the-art results. Finally, we compared our tiny FPGA hardware implementation results with off-the-shelf resource-constrained devices and showed our implementation is faster and consumed less power compared to the off-the-shelf resource-constrained devices.

A Novel Lightweight Model for Underwater Image Enhancement.

Underwater images suffer from low contrast and color distortion. In order to improve the quality of underwater images and reduce storage and computational resources, this paper proposes a lightweight model Rep-UWnet to enhance underwater images. The model consists of a fully connected convolutional network and three densely connected RepConv blocks in series, with the input images connected to the output of each block with a Skip connection. First, the original underwater image is subjected to feature extraction by the SimSPPF module and is processed through feature summation with the original one to be produced as the input image. Then, the first convolutional layer with a kernel size of 3 × 3, generates 64 feature maps, and the multi-scale hybrid convolutional attention module enhances the useful features by reweighting the features of different channels. Second, three RepConv blocks are connected to reduce the number of parameters in extracting features and increase the test speed. Finally, a convolutional layer with 3 kernels generates enhanced underwater images. Our method reduces the number of parameters from 2.7 M to 0.45 M (around 83% reduction) but outperforms state-of-the-art algorithms by extensive experiments. Furthermore, we demonstrate our Rep-UWnet effectively improves high-level vision tasks like edge detection and single image depth estimation. This method not only surpasses the contrast method in objective quality, but also significantly improves the contrast, colorimetry, and clarity of underwater images in subjective quality.

An Investigation on Hardware-Aware Vision Transformer Scaling

Vision Transformer (ViT) has demonstrated promising performance in various computer vision tasks, and recently attracted a lot of research attention. Many recent works have focused on proposing new architectures to improve ViT and deploying it into real-world applications. However, little effort has been made to analyze and understand ViT’s architecture design space and its implication for hardware costs on different devices. In this work, by simply scaling ViT’s depth, width, input size, and other basic configurations, we show that a scaled vanilla ViT model without bells and whistles can achieve comparable or superior accuracy-efficiency trade-off than most of the latest ViT variants. Specifically, compared with DeiT-Tiny, our scaled model achieves a ↑ 1.9% higher ImageNet top-1 accuracy under the same FLOPs and a ↑ 3.7% better ImageNet top-1 accuracy under the same latency on an NVIDIA Edge GPU TX2. Motivated by this, we further investigate the extracted scaling strategies from the following two aspects: (1) can these scaling strategies be transferred across different real hardware devices ? and (2) can these scaling strategies be transferred to different ViT variants and tasks ?. For (1), our exploration, based on various devices with different resource budgets, indicates that the transferability effectiveness depends on the underlying device together with its corresponding deployment tool. For (2), we validate the effective transferability of the aforementioned scaling strategies obtained from a vanilla ViT model on top of an image classification task to the PiT model, a strong ViT variant targeting efficiency as well as object detection and video classification tasks. In particular, when transferred to PiT, our scaling strategies lead to a boosted ImageNet top-1 accuracy of from 74.6% to 76.7% (↑ 2.1%) under the same 0.7G FLOPs. When transferred to the COCO object detection task, the average precision is boosted by ↑ 0.7% under a similar throughput on a V100 GPU.

Separate cue- and alpha-related mechanisms for distractor suppression.

Research on selective attention has largely focused on the enhancement of behaviorally important information, with less focus on the suppression of distracting information. Enhancement and suppression can operate through a push-pull relationship attributable to competitive interactions among neural populations. There has been considerable debate, however, regarding (i) whether suppression can be voluntarily deployed, independent of enhancement, and (ii) whether the voluntary deployment of suppression is associated with neural processes that occur prior to distractor onset. Here, we investigated the interplay between pre- and post-distractor neural processes, while male and female human subjects performed a visual search task with a cue that indicated the location of an upcoming distractor. We utilized two established EEG markers of suppression: the distractor positivity (PD) and alpha power (∼8-15 Hz). The PD-a component of event-related potentials-has been linked with successful distractor suppression, and increased alpha power has been linked with attenuated sensory processing. Cueing the location of an upcoming distractor speeded responses and led to an earlier PD, consistent with earlier suppression due to the strategic use of a spatial cue. In comparison, higher pre-distractor alpha power contralateral to distractors led to a later PD, consistent with later suppression. Lower alpha power contralateral to distractors instead led to distractor-related attentional capture. Lateralization of alpha power was not linked to the spatial cue. This observation, combined with differences in the timing of successful suppression-as indexed by earlier and later PD components-demonstrates that cue-related, voluntary suppression can occur separate from alpha-related gating of sensory processing.Significance Statement Selective suppression of distracting information is important for survival. There are ongoing debates, however, regarding whether and how selective enhancement can be voluntarily deployed independent of selective enhancement. Here, we recorded EEG while subjects performed a visual search task with cues that indicated the location of upcoming distractors. Behavioral and electrophysiological results revealed that foreknowledge of a distractor's location speeded suppression (relative to non-cued trials), thereby facilitating target detection. The results further revealed that such cue-related suppression of a distractor can occur separate from alpha-related gating of sensory processing. That is, alpha-related suppression in the present task occurred independent of the spatially informative cue (i.e., on both cued and non-cued trials) and later than cue-related suppression (as indexed by later and earlier PD components).

Evidence of elevated situational awareness for active duty soldiers during navigation of a virtual environment.

U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians' physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups' physiological or cognitive markers and performance for these groups.

Revisiting the Deviation Effects of Irrelevant Sound on Serial and Nonserial Tasks.

Two types of disruptive effects of irrelevant sound on visual tasks have been reported: the changing-state effect and the deviation effect. The idea that the deviation effect, which arises from attentional capture, is independent of task requirements, whereas the changing-state effect is specific to tasks that require serial processing, has been examined by comparing tasks that do or do not require serial-order processing. While many previous studies used the missing-item task as the nonserial task, it is unclear whether other cognitive tasks lead to similar results regarding the different task specificity of both effects. Kattner et al. (Memory and Cognition, 2023) used the mental-arithmetic task as the nonserial task, and failed to demonstrate the deviation effect. However, there were several procedural factors that could account for the lack of deviation effect, such as differences in design and procedures (e.g., conducted online, intermixed conditions). In the present study, we aimed to investigate whether the deviation effect could be observed in both the serial-recall and mental-arithmetic tasks when these procedural factors were modified. We found strong evidence of the deviation effect in both the serial-recall and the mental-arithmetic tasks when stimulus presentation and experimental design were aligned with previous studies that demonstrated the deviation effect (e.g., conducted in-person, blockwise presentation of sound, etc.). The results support the idea that the deviation effect is not task-specific.

Visual Reinforcement Learning for Dynamic Object Detection

Abstract. Object detection is a widely studied task in computer vision. Current methods often focus on images captured from appropriate viewpoints. However, there is a large disparity between objects observed from different viewpoints in the real world. Dynamic Object Detection (DOD) method automatically adjusts the camera viewpoint in a visual scene to sequentially find optimal viewpoints. Currently, the DOD tasks are usually modeled as a sequential decision-making problem and solved using reinforcement learning methods. Existing approaches face challenges with sparse rewards and training instability. To tackle these issues, we proposed a single-step reward function and a lightweight network, respectively. The single-step reward function, which provides timely feedback, gives an efficient training process for DOD tasks. The lightweight network with few parameters can ensure the stability of the training process. To evaluate the effectiveness of our method, we developed a simulation dataset based on UE4, which consists of 1800 training images and 450 testing images. The dataset includes five object categories: vans, cars, trailers, box trucks and SUVs. Experiments demonstrate that our method outperforms SOTA object detectors on our simulation dataset. Specifically, the average precisions(APs) are improved from 89.1% to 96.0% when using the YOLOv8 object detector.

Attentional bias to uncertainty-based information: a conceptual replication of Fergus et al. (2013)

AbstractAttentional biases to negative information are often observed in emotional disorders such as anxiety and depression. Recently, Fergus et al. (Cognitive Therapy and Research 37:735-741, 2013) demonstrated that participants scoring high in Intolerance of Uncertainty (IU), a transdiagnostic risk factor for emotional disorders, were faster at identifying targets with uncertainty-based information (e.g., word stimuli such as ‘maybe’), relative to neutral information (e.g. word stimuli related to household items). Moreover, this effect was specific to IU over other anxious traits such as worry. This report aimed to conceptually replicate the results of Fergus et al. (Cognitive Therapy and Research 37:735-741, 2013). The current experiment was conducted with a student sample (n = 126). As in the original study, a visual search task with uncertainty-based and neutral information (e.g., word stimuli) was used. Reaction times were collected. IU and trait anxiety was measured via self-report. We partially replicated the results of Fergus et al. (Cognitive Therapy and Research 37:735-741, 2013). We found that higher IU was significantly associated with faster identification of uncertainty-based information, relative to neutral information. However, we also observed a similar pattern of results for trait anxiety. Additionally, we observed no specificity of IU or trait anxiety in predicting faster identification of uncertainty-based information, relative to neutral information. Despite the lack of specificity of IU or trait anxiety in predicting attentional bias to uncertainty-based information, these findings highlight that negative emotionality may be generally associated with attentional biases to uncertainty-based information in the absence of direct threat.

Transcranial alternating current stimulation over frontal eye fields mimics attentional modulation of visual processing.

Attentional control over sensory processing has been linked to neural alpha oscillations and related inhibition of cerebral cortex. Despite the wide consensus on the functional relevance of alpha oscillations for attention, precise neural mechanisms of how alpha oscillations shape perception and how this top-down modulation is implemented in cortical networks remain unclear. Here, we tested the hypothesis that alpha oscillations in frontal eye fields (FEF) are causally involved in the top-down regulation of visual processing in humans (male and female). We applied sham-controlled, intermittent transcranial alternating current stimulation (tACS) over bilateral FEF at either 10 Hz (alpha) or 40 Hz (gamma) to manipulate attentional preparation in a visual discrimination task. Under each stimulation condition, we measured psychometric functions for contrast perception and introduced a novel linear mixed modeling approach for statistical control of neurosensory side effects of the electric stimulation. TACS at alpha frequency reduced the slope of the psychometric function, resulting in improved sub-threshold and impaired super-threshold contrast perception. Side effects on the psychometric functions were complex and showed large interindividual variability. Controlling for the impact of side effects on the psychometric parameters by using covariates in the linear mixed model analysis reduced this variability and strengthened the perceptual effect. We propose that alpha tACS over FEF mimicked a state of endogenous attention by strengthening a fronto-occipitoparietal network in the alpha band. We speculate that this network modulation enhanced phasic gating in occipitoparietal cortex leading to increased variability of single-trial psychometric thresholds, measurable as a reduction of psychometric slope.Significance statement Attention is fundamental to the voluntary control of perception and behavior. Yet, precise underlying neural mechanisms remain unclear. Here, we provide evidence for a vital role of frontal alpha oscillations in the regulation of gating of visual information by using intermittent transcranial alternating current stimulation (tACS). We show that modulation of frontal alpha oscillations affected the slope of psychometric functions of visual contrast perception, leading to contrast-dependent improvement and impairment of perception. Our data adds to work on alpha oscillations in spatial attention and studies on the psychometrics of attention. Furthermore, we introduce a novel approach for the statistical control of tACS side effects and thereby contribute to the ongoing debate on outcome variability in studies using transcranial neurostimulation methods.

A Novel Pretrained General-purpose Vision Language Model for the Vietnamese Language

Lying in the cross-section of computer vision and natural language processing, vision language models are capable of processing images and text at once. These models are helpful in various tasks: text generation from image and vice versa, image-text retrieval, or visual navigation. Besides building a model trained on a dataset for a task, people also study general-purpose models to utilize many datasets for multitasks. Their two primary applications are image captioning and visual question answering. For English, large datasets and foundation models are already abundant. However, for Vietnamese, they are still limited. To expand the language range, this work proposes a pretrained general-purpose image-text model named VisualRoBERTa. A dataset of 600k images with captions (translated MS COCO 2017 from English to Vietnamese) is introduced to pretrain VisualRoBERTa. The model’s architecture is built using Convolutional Neural Network and Transformer blocks. Fine-tuning VisualRoBERTa shows promising results on the ViVQA dataset with 34.49% accuracy, 0.4173 BLEU 4, and 0.4390 RougeL (in visual question answering task), and best outcomes on the sViIC dataset with 0.6685 BLEU 4, 0.6320 RougeL (in image captioning task).

Dog Breed Identification Using Deep Learning

Deep learning has revolutionized computer vision tasks, including image classification. In this study, we propose a deep learning approach for the identification of dog breeds from images. Leveraging transfer learning with the ResNet50V2 model, our methodology involves preprocessing and augmenting a dataset of dog images to enhance model generalization. Through rigorous experimentation and training, our model achieves competitive accuracy rates in identifying over 60 unique dog breeds. Results indicate the efficacy of our approach in breed classification tasks, with potential applications in pet identification systems and animal welfare initiatives. Keywords: Deep learning, Dog breed identification, Transfer learning, Computer vision.

Graph Analysis of the Visual Cortical Network during Naturalistic Movie Viewing Reveals Increased Integration and Decreased Segregation Following Mild TBI.

Traditional neuroimaging methods have identified alterations in brain activity patterns following mild traumatic brain injury (mTBI), particularly during rest, complex tasks, and normal vision. However, studies using graph theory to examine brain network changes in mTBI have produced varied results, influenced by the specific networks and task demands analyzed. In our study, we employed functional MRI to observe 17 mTBI patients and 54 healthy individuals as they viewed a simple, non-narrative underwater film, simulating everyday visual tasks. This approach revealed significant mTBI-related changes in network connectivity, efficiency, and organization. Specifically, the mTBI group exhibited higher overall connectivity and local network specialization, suggesting enhanced information integration without overwhelming the brain's processing capabilities. Conversely, these patients showed reduced network segregation, indicating a less compartmentalized brain function compared to healthy controls. These patterns were consistent across various visual cortex subnetworks, except in primary visual areas. Our findings highlight the potential of using naturalistic stimuli in graph-based neuroimaging to understand brain network alterations in mTBI and possibly other conditions affecting brain integration.

Deep Feature Retention Module Network for Texture Classification

Texture describes the unique features of an image. Therefore, texture classification is a crucial task in computer vision. Various CNN-based deep learning methods have been developed to classify textures. During training, the deep-learning model undergoes an end-to-end procedure of learning features from low to high levels. Most CNN architectures depend on high-level features for the final classification. Hence, other low- and mid-level information was not prioritized for the final classification. However, in the case of texture classification, it is essential to determine detailed feature information within the pattern to classify textures as they have diversity and irregularity in images within the same class. Therefore, the feature information at the low- and mid-levels can also provide meaningful information to distinguish the classes. In this study, we introduce a CNN model with a feature retention module (FRM) to utilize features from numerous levels. FRM maintains the texture information extracted at each level and extracts feature information through filters of various sizes. We used three texture datasets to evaluate the proposed model combined with the FRM. The experimental results showed that learning using different levels of features together assists in improving learning performance more than learning using high-level features.