Visual Task Research Articles

Deep Learning Approach for Suspicious Activity Detection from Surveillance Video

The present generation seeks ways to live their lives free of fear. This often prompts the need for enhanced law enforcement and security systems. Suspicion activity recognition in surveillance footage has become an essential component in modern security systems, focused around the problem of recognizing inappropriate behaviors without too much human time. The advent of deep learning has made an impact in computer vision tasks within such a way the real-time detection by the evaluation of large amounts of video footage becomes realistic. In this context the paper presents a model in which deep learning, particularly Convolutional Neural Networks (CNNs), are utilized for the behavioral pattern recognition from security footage. In addition, it also improves the efficacy of detection and reduces the shortcomings associated with these systems where only the post-event analysis is performed manually or automatically. In this paper, we demonstrate a novel IP model that analyzes live video and performs normal-suspicious–threat triage in real-time. Results indicate the validity of the method in different environmental and illumination conditions. An important objective that this system is designed for is to change the way security is practiced and delivered in today’s intelligent systems providing actionable intelligence and the ability to prevent threats rather than just respond to them making this a high growth opportunity for marketing high-risk and sensitive areas.

A Comparative Study of Swin Transformer and CNN Models for Crowd Counting

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

Learned scalable video coding for humans and machines

AbstractVideo coding has traditionally been developed to support services such as video streaming, videoconferencing, digital TV, and so on. The main intent was to enable human viewing of the encoded content. However, with the advances in deep neural networks (DNNs), encoded video is increasingly being used for automatic video analytics performed by machines. In applications such as automatic traffic monitoring, analytics such as vehicle detection, tracking and counting, would run continuously, while human viewing could be required occasionally to review potential incidents. To support such applications, a new paradigm for video coding is needed that will facilitate efficient representation and compression of video for both machine and human use in a scalable manner. In this manuscript, we introduce an end-to-end learnable video codec that supports a machine vision task in its base layer, while its enhancement layer, together with the base layer, supports input reconstruction for human viewing. The proposed system is constructed based on the concept of conditional coding to achieve better compression gains. Comprehensive experimental evaluations conducted on four standard video datasets demonstrate that our framework outperforms both state-of-the-art learned and conventional video codecs in its base layer, while maintaining comparable performance on the human vision task in its enhancement layer.

The role of selective attention in implicit learning: evidence for a contextual cueing effect of task-irrelevant features.

With attentional mechanisms, humans select and de-select information from the environment. But does selective attention modulate implicit learning? We tested whether the implicit acquisition of contingencies between features are modulated by the task-relevance of those features. We implemented the contingencies in a novel variant of the contextual cueing paradigm. In such a visual search task, participants could use non-spatial cues to predict target location, and then had to discriminate target shapes. In Experiment 1, the predictive feature for target location was the shape of the distractors (task-relevant). In Experiment 2, the color feature of distractors (task-irrelevant) cued target location. Results showed that participants learned to predict the target location from both the task-relevant and the task-irrelevant feature. Subsequent testing did not suggest explicit knowledge of the contingencies. For the purpose of further testing the significance of task-relevance in a cue competition situation, in Experiment 3, we provided two redundantly predictive cues, shape (task-relevant) and color (task-irrelevant) simultaneously, and subsequently tested them separately. There were no observed costs of single predictive cues when compared to compound cues. The results were not indicative of overshadowing effects, on the group and individual level, or of reciprocal overshadowing. We conclude that the acquisition of contingencies occurs independently of task-relevance and discuss this finding in the framework of the event coding literature.

TPTE: Text-Guided Patch Token Exploitation for Unsupervised Fine-Grained Representation Learning

Recent advances in pre-trained vision-language models have successfully boosted the performance of unsupervised image representation in many vision tasks. Most of existing works focus on learning global visual features with Transformers and neglect detailed local cues, leading to suboptimal performance in fine-grained vision tasks. In this article, we propose a text-guided patch token exploitation framework to enhance the discriminative power of unsupervised representation by exploiting more detailed local features. Our text-guided decoder extracts local features with the guidance of texts or learned prompts describing discriminative object parts. We hence introduce a local-global relation distillation loss to promote the joint optimization of local and global features. The proposed method allows to flexibly extract either global or global-local features as the image representation. It significantly outperforms previous methods in fine-grained image retrieval and base-to-new fine-grained classification tasks. For instance, our Recall@1 metric surpasses the recent unsupervised retrieval method STML by 6.0% on the SOP dataset. The code is publicly available at https://github.com/maosnhehe/TPTE .

MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images

Semantic segmentation of remote sensing images is a fundamental task in computer vision, holding substantial relevance in applications such as land cover surveys, environmental protection, and urban building planning. In recent years, multi-modal fusion-based models have garnered considerable attention, exhibiting superior segmentation performance when compared with traditional single-modal techniques. Nonetheless, the majority of these multi-modal models, which rely on Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) for feature fusion, face limitations in terms of remote modeling capabilities or computational complexity. This paper presents a novel Mamba-based multi-modal fusion network called MFMamba for semantic segmentation of remote sensing images. Specifically, the network employs a dual-branch encoding structure, consisting of a CNN-based main encoder for extracting local features from high-resolution remote sensing images (HRRSIs) and of a Mamba-based auxiliary encoder for capturing global features on its corresponding digital surface model (DSM). To capitalize on the distinct attributes of the multi-modal remote sensing data from both branches, a feature fusion block (FFB) is designed to synergistically enhance and integrate the features extracted from the dual-branch structure at each stage. Extensive experiments on the Vaihingen and the Potsdam datasets have verified the effectiveness and superiority of MFMamba in semantic segmentation of remote sensing images. Compared with state-of-the-art methods, MFMamba achieves higher overall accuracy (OA) and a higher mean F1 score (mF1) and mean intersection over union (mIoU), while maintaining low computational complexity.

Improving Visual Object Detection using General UFIR Filtering

Object detection is a fundamental task in computer vision, which involves the identification and localization of objects within image frames or video sequences. The problem is complicated by large variations in the video camera bounding box, which can be thought of as colored measurement noise (CMN). In this paper, we use the general unbiased finite impulse response (GUFIR) approach to improve detection performance under CMN. The results are compared to the general Kalman filter (GKF) and two detection methods: “Faster-RCNN” and “Tensorflow PASCAL Visual Object Classes (VOC)”. Experimental testing is carried out using the benchmark data ”Car4”. It is shown that GUFIR significantly improves the detection accuracy and demonstrates the properties of the effective tool for visual object tracking.

Design of Travel Route Planning Model Based on Deep Learning

Tourism route planning is affected by the factors such as tourism destination and tourists’ preference, which leads to poor automatic matching of routes. A mathematical model of tourism route planning based on deep learning is proposed. Under the condition of total route constraints, according to the historical tourism preference information and prior information of tourists, a big data model for the distribution of personalized characteristics of tourists is established, spatial constraints and time constraints parameters are input, and the feature matching of geographic location information and tourist interest parameter information is carried out by using the deep learning method, and the statistical feature quantity of the parameters of the tourism route planning model is extracted. Under the control of the deep learning and geographic information data set, Carry out multi-constraint and multi-objective hierarchical analysis of tourists and tourism destinations, and realize the optimization design of tourism route planning algorithm. The simulation results show that the accuracy of this method is high and the deviation is small. It improves the satisfaction level of tourists and helps users complete a series of visual analysis tasks such as route mining, route planning and destination analysis.

Associations between sex and lifestyle activities with cognitive reserve in mid-life adults with genetic risk for Alzheimer's disease.

Females have a higher age-adjusted incidence of Alzheimer's Disease (AD) than males, even when accounting for longer lifespan and, therefore, stand to benefit the most from dementia prevention efforts. As exposure to many modifiable risk factors for dementia begins in mid-life, interventions must be implemented from middle-age. Building cognitive reserve, particularly through stimulating avocational activities and occupational attainment presents a crucial, underexplored, dementia prevention approach for mid-life. It is currently unknown, however, whether modifiable lifestyle factors can protect against AD processes, from mid-life, differentially for females and males who carry inherited risk for late-life dementia. To address this gap, this study investigated the impact of biological sex and APOE4 carrier status on the relationship between stimulating activities, occupational attainment, and cognition in mid-life. We leveraged the PREVENT-Dementia program, the world's largest study investigating the origins and early diagnosis of dementia in mid-life at-risk individuals (N = 700; 40-59 years). Cognitive performance was measured using the Cognito Battery and the Visual Short Term Memory Binding task. Mid-life specific reserve contributors were assessed via the Lifetime of Experiences Questionnaire. Females had significantly better episodic and relational memory (p < 0.001), and lower occupational attainment than males (p < 0.001). Engagement in stimulating activities was positively associated with episodic and relational memory, regardless of sex and APOE4 status (β = 0.05, CI 0.03-0.07, p < 0.001). APOE4 carriers showed significant sex differences in the association between occupational attainment and episodic and relational memory (β = 0.38, CI 0.12-0.63, p = 0.003). APOE4 carrier females with higher occupational attainment showed better cognition (β = 0.16, CI -0.002-0.32, p = 0.053), whereas APOE4 carrier males showed the opposite effect (β = -0.20, CI -0.40 - -0.001, p = 0.049). Our findings suggest that occupational attainment in mid-life contributes to cognitive reserve against inherited risk of dementia in females, but not males. They highlight the need for high precision approaches that consider biological sex and APOE4 carrier status to inform Alzheimer's disease prevention strategies and clinical trials.

Visual inspection time as an accessible measure of processing speed: A validation study in children with cerebral palsy.

This study examined the validity of a visual inspection time (IT) task as a measure of processing speed (PS) in a sample of children with and without cerebral palsy (CP). IT tasks measure visualization speed without focusing on the motor response time to indicate decision making about the properties of those stimuli. Participants were 113 children ages 8-16, including 45 with congenital CP, and 68 typically developing peers. Measures were a standard visual IT task that required dual key responding and a modified version using an assistive technology button with response option scanning. Performance on these measures was examined against traditional Wechsler PS measures (Coding, Symbol Search). IT performance shared considerable variance with traditional paper-pencil PS measures for the group with CP, but not necessarily in the typically developing group. Concurrent validity was found for both IT task versions with traditional PS measures in the group with CP. IT classification accuracy for lowered PS showed modest sensitivity and good specificity particularly for the modified IT task. As measures of PS in children with CP who are unable to validly participate in traditional PS tasks, IT tasks demonstrate adequate concurrent validity and may serve as a beneficial alternative measure of PS in this population.

The Influence of Visualization in Book Content on Language Processing and Memory

This study explored the effects of visualization and phonetic processing on comprehension and memory retention in learning Mandarin as a second language. Through an experiment involving three conditions: pinyin, black-and-white pictures, and colored pictures, the study evaluated participants’ accuracy and reaction time in completing visual and phonetic-based tasks. The results showed that visual cues, especially colored pictures, increased response accuracy and decreased reaction time compared to pinyin and black-and-white pictures. These findings suggest that visualization not only aids vocabulary recall but also facilitates faster cognitive processing, potentially reducing students’ cognitive load. Implications for Mandarin teaching include the importance of using appropriate visual teaching materials to support learning and memory retention at the beginner level. These results contribute to the growing body of evidence supporting the effectiveness of multimodal learning and the importance of tailoring teaching methods to learners’ cognitive characteristics.

Neural networks for abstraction and reasoning.

For half a century, artificial intelligence research has attempted to reproduce the human qualities of abstraction and reasoning - creating computer systems that can learn new concepts from a minimal set of examples, in settings where humans find this easy. While specific neural networks are able to solve an impressive range of problems, broad generalisation to situations outside their training data has proved elusive. In this work, we look at several novel approaches for solving the Abstraction & Reasoning Corpus (ARC). This is a dataset of abstract visual reasoning tasks introduced to test algorithms on broad generalization. Despite three international competitions with $100,000 in prizes, the best algorithms still fail to solve a majority of ARC tasks. The best solvers today rely on complex hand-crafted rules, without using machine learning at all. We revisit whether recent advances in neural networks allow progress on this task, or whether an entirely different class of models are required. First, we adapt the DreamCoder neurosymbolic reasoning solver to ARC. DreamCoder automatically writes programs in a bespoke domain-specific language to perform reasoning, using a neural network to mimic human intuition. We present the Perceptual Abstraction and Reasoning Language (PeARL) language, which allows DreamCoder to solve ARC tasks, and propose a new recognition model that allows us to significantly improve on the previous best implementation. We also propose a new encoding and augmentation scheme that allows large language models (LLMs) to solve ARC tasks, and find that the largest models can solve some ARC tasks. LLMs are able to solve a different group of problems to state-of-the-art solvers, and provide an interesting way to complement other approaches. We perform an ensemble analysis, combining systems to achieve better results than any system alone and analysing individual strengths. However, it is sobering to see that approaches based on neural networks still lag behind existing hand-crafted solvers, and we suggest avenues for future improvements. Our findings with the ensemble model may indicate that a diversity of methods might be necessary to solve problems in ARC. Humans likely employ diverse strategies to solve ARC. Studies involving human participants to identify the strategies they employ to solve ARC could provide valuable insights for future AI approaches. Finally, we publish the arckit Python library to make future research on ARC easier.

No-Reference Quality Assessment Based on Dual-Channel Convolutional Neural Network for Underwater Image Enhancement

Underwater images are important for underwater vision tasks, yet their quality often degrades during imaging, promoting the generation of Underwater Image Enhancement (UIE) algorithms. This paper proposes a Dual-Channel Convolutional Neural Network (DC-CNN)-based quality assessment method to evaluate the performance of different UIE algorithms. Specifically, inspired by the intrinsic image decomposition, the enhanced underwater image is decomposed into reflectance with color information and illumination with texture information based on the Retinex theory. Afterward, we design a DC-CNN with two branches to learn color and texture features from reflectance and illumination, respectively, reflecting the distortion characteristics of enhanced underwater images. To integrate the learned features, a feature fusion module and attention mechanism are conducted to align efficiently and reasonably with human visual perception characteristics. Finally, a quality regression module is used to establish the mapping relationship between the extracted features and quality scores. Experimental results on two public enhanced underwater image datasets (i.e., UIQE and SAUD) show that the proposed DC-CNN method outperforms a variety of the existing quality assessment methods.

Foreground separation knowledge distillation for object detection

In recent years, deep learning models have become predominant methods for computer vision tasks, but the large computation and storage requirements of many models make them challenging to deploy on devices with limited resources. Knowledge distillation (KD) is a widely used approach for model compression. However, when applied in the object detection problems, the existing KD methods either directly applies the feature map or simply separate the foreground from the background by using a binary mask, aligning the attention between the teacher and the student models. Unfortunately, these methods either completely overlook or fail to thoroughly eliminate noise, resulting in unsatisfactory model accuracy for student models. To address this issue, we propose a foreground separation distillation (FSD) method in this paper. The FSD method enables student models to distinguish between foreground and background using Gaussian heatmaps, reducing irrelevant information in the learning process. Additionally, FSD also extracts the channel feature by converting the spatial feature maps into probabilistic forms to fully utilize the knowledge in each channel of a well-trained teacher. Experimental results demonstrate that the YOLOX detector enhanced with our distillation method achieved superior performance on both the fall detection and the VOC2007 datasets. For example, YOLOX with FSD achieved 73.1% mean average precision (mAP) on the Fall Detection dataset, which is 1.6% higher than the baseline. The code of FSD is accessible via https://doi.org/10.5281/zenodo.13829676.

Debiasing vision-language models for vision tasks: a survey

Debiasing vision-language models for vision tasks: a survey

Person in Uniforms Re-Identification

Person in Uniforms Re-identification (PU-ReID) is an emerging computer vision task for various intelligent video surveillance applications. PU-ReID is much understudied due to the absence of large-scale annotated datasets, also this task is extremely challenging because many individuals captured in surveillance videos wear same clothing, introducing significant interference for retrieval tasks owing to the high visual similarity of outfits and subtle differences among individuals. This research initiates the exploration of person in uniform re-identification, a novel and challenging task tailored for real industrial scenarios. To address these issues, a novel framework is proposed for PU-ReID, which aims to reduce the visual impact of similar uniforms and learn the unique cues derived from human parts and detailed visual features. Specifically, several novel techniques are built in this study: first, a uniform feature separation method with orthogonal constraints is proposed to extract non-uniform features. Second, multi-view subspace feature alignment is introduced to integrate soft-biometrics including optics-related visual features, contextual information of human parts, and cloth-invariant biometric features. In addition, to close the gap between academic research and real world settings, a new person in uniforms ReID dataset named PU-151 is constructed, which consists of 151 gas station employees in uniforms from 1,488 videos. At last, extensive experiments conducted on five datasets demonstrate that the proposed approach significantly outperforms the state-of-the-art methods. This advancement can drive further developments in re-identification and person search technologies.

SSDSNet: A Dual-Stream State-Space Network for Efficient Image Denoising with Long-Range Dependency Modeling

Abstract In recent years, significant progress has been made in image denoising, driven by advancements in modern deep neural networks such as Convolutional Neural Networks&amp;#xD;(CNNs) and Transformers. However, existing denoising models often face limitations due to inherent local inductive biases or the quadratic computational complexity of their architectures. Recently, Selective Structured State Space Models, exemplified by Mamba, have shown considerable potential in modeling long-range dependencies with linear complexity. Despite this promise, their application in low-level computer vision tasks remains underexplored. In this study, we introduce a straightforward yet effective model, the Dual-Stream State-Space Module (DSSM), specifically designed for image denoising. The DSSM serves as the core component of our model, combining convolutional operations with the&amp;#xD;Mamba model to enhance the capability of capturing both local features and global context.&amp;#xD;This dual-stream approach leverages the inherent structure of local image patches while effectively modeling long-range dependencies, resulting in feature representations tailored for denoising tasks. Extensive experiments demonstrate the effectiveness of our method. For instance, on the SIDD dataset, our model achieves performance on par with state-of-the-art methods, maintaining a global receptive field while operating with similar computational costs.&amp;#xD;

FA-HRNet: A New Fusion Attention Approach for Vegetation Semantic Segmentation and Analysis

Semantic segmentation of vegetation in aerial remote sensing images is a critical aspect of vegetation mapping. Accurate vegetation segmentation effectively informs real-world production and construction activities. However, the presence of species heterogeneity, seasonal variations, and feature disparities within remote sensing images poses significant challenges for vision tasks. Traditional machine learning-based methods often struggle to capture deep-level features for the segmentation. This work proposes a novel deep learning network named FA-HRNet that leverages the fusion of attention mechanism and a multi-branch network structure for vegetation detection and segmentation. Quantitative analysis from multiple datasets reveals that our method outperforms existing approaches, with improvements in MIoU and PA by 2.17% and 4.85%, respectively, compared with the baseline network. Our approach exhibits significant advantages over the other methods regarding cross-region and cross-scale capabilities, providing a reliable vegetation coverage ratio for ecological analysis.

I-DINO: High-Quality Object Detection for Indoor Scenes

Object Detection in Complex Indoor Scenes is designed to identify and categorize objects in indoor settings, with applications in areas such as smart homes, security surveillance, and home service robots. It forms the basis for advanced visual tasks including visual question answering, video description generation, and instance segmentation. Nonetheless, the task faces substantial hurdles due to background clutter, overlapping objects, and significant size differences. To tackle these challenges, this study introduces an indoor object detection approach utilizing an enhanced DINO framework. To cater to the needs of indoor object detection, an Indoor-COCO dataset was developed from the COCO object detection dataset. The model incorporates an advanced Res2net as the backbone feature extraction network, complemented by a deformable attention mechanism to better capture detailed object features. An upgraded Bi-FPN module is employed to replace the conventional feature fusion module, and SIoU loss is utilized to expedite convergence. The experimental outcomes indicate that the refined model attains an mAP of 62.3%, marking a 5.2% improvement over the baseline model. These findings illustrate that the DINO-based indoor object detection model exhibits robust generalization abilities and practical utility for multi-scale object detection in complex environments.

Intelligent multi-robot collaborative transport system

The research on multi-robot systems has been divided into various fields, such as communication, navigation, task allocation, and collaborative transport. While significant progress has been made in each area, there has been limited research integrating these fields to build a fully autonomous multi-robot collaborative transport system. Therefore, we identify the key issues and propose a multi-robot collaborative transport system founded on ROS1 and conduct validation in a simulated environment, laying a solid foundation for the system to run on real robots. The primary contributions of this study include three key areas: (1) modeling and validating robot collaborative transport, (2) developing a visual task allocation system leveraging FastDDS service, and (3) resolving path collision issues in multi-robot navigation through both traditional methods and reinforcement learning techniques. Extensive experimental evaluations demonstrate that the proposed intelligent multi-robot collaborative transport system can autonomously navigate to target points for collaborative transport task. Performance assessments, based on the error between the target point and the object’s arrival point as well as the transport trajectory error, reveal that the system effectively completes the assigned tasks.