Local Region Information Research Articles

Deep learning based fine-grained recognition technology for basketball movements

The technical essentials of basketball are very complex, in order to accomplish the identification of fine-grained basketball movements and improve the quality of basketball training, the study innovatively proposes a region screening strategy to obtain the local region information of image frames, and draws on the SlowFast network structure to design the static and dynamic detail models, and finally utilizes the 3D attention feature fusion module to complete the multi-feature relationship processing and recognition. Performance testing experiments have confirmed that the dual stream model based on region filtering has achieved a maximum improvement in average accuracy of 16.35 % and 60.36 % in the recognition of RGB images, preserving relatively complete local effective region information of RGB images. The model based on channel domain attention mechanism improves its average accuracy to over 60 %. The model recognition accuracy calculation time after dual stream fusion is reduced by about 50 %. The model based on 3D attention feature fusion module exhibits stronger robustness when the feature dimension changes. And the loss value of this model rapidly decreases and converges to the minimum loss value, with its accuracy and recall reaching 90.03 % and 88.47 % respectively, and its AUC value reaching a maximum of 0.761. The design of the study contributes to the digital development of teaching and training in basketball and enriches the theoretical body of knowledge on deep learning and action recognition.

MPSA: Multi-Position Supervised Soft Attention-based convolutional neural network for histopathological image classification

In recent years, significant achievements have been made in the field of histopathological image analysis using convolutional neural networks (CNNs). However, existing CNNs fail to fully capture the important local structures and regional information in histopathological images due to the complex tissue structures and variable pathological features present in these images. They often treat all regions equally, which further exacerbates the challenge of accurately analyzing such images. Current network model can’t extract deep layer features efficiently without guiding. To alleviate this problem, we propose a novel network model called Multi-Position Supervised Soft Attention (MPSA). MPSA adds regions of interest (RoI) labels at multiple feature layers for deep supervision, and then uses the supervised layers as soft attention to guide the learning of the classification network, enabling the network to accurately extract features of the lesion target. Additionally, we design a Multi-level Attention Feature Enhancement Module (MAFEM), which combines multiple levels of attention mechanisms to enhance the performance of the convolutional neural network in histopathological image classification. MAFEM includes spatial attention, soft attention of the main branch, and our proposed soft attention for multi-branch feature fusion. The proposed soft attention for multi-branch feature fusion aims to enhance the predictive performance of the classification model by activating relevant neurons in the diagnostic area in a highly activated state, while effectively avoiding noise activation. This innovative approach ensures that the model can focus on the most pertinent information, leading to improved classification accuracy. We conducted classification experiments on the liver cancer histopathological images dataset and the results showed that our method achieved a classification accuracy of 95.79%, indicating that it is very effective in the analysis of liver histopathological images. Our proposed network architecture has also demonstrated good generalization ability in other medical datasets, achieving a classification accuracy of 84.41% on the ultrasound carotid plaque dataset.

Region-based compressive distributed storage in Mobile CrowdSensing

Compressive Sensing-based distributed data storage for Mobile CrowdSensing (MCS) primarily addresses the problem that data needs to be stored temporarily on mobile devices when the central server cannot work as expected. In the crowdsensing-aware platform, the emerging distributed storage scheme mainly recovers the data fields of the entire area. However, when only local area information is required, it is still necessary to recover the entire area, creating unnecessary waste. Based on this observation, this paper proposes a Region-based Distributed Data Storage with global Denoising (R-DDS-D) algorithm. Region-based distributed data storage strategy is individually coded in regional blocks, allowing users to reconstruct the regional information they need selectively. Simultaneously, global denoising can be achieved through the correlation between small regions, which can further improve the accuracy of reconstructed data. In addition, not all local region information is equally significant, an extended algorithm for sampling allocation is proposed considering the importance of different regions to further improve accuracy. Theoretical analysis demonstrates that R-DDS-D can recover the signal with high probability, and the sampling allocation method can more reasonably allocate sampling resources. Experimentally, the R-DDS-D scheme offers greater flexibility, recruiting cost savings, and smoother reconstruction performance for recovering local regions, and the sampling allocation method further enhances the reconstruction quality in the case of uneven data distribution.

Patch-Based Auxiliary Node Classification for Domain Adaptive Object Detection

Domain adaptive object detection (DAOD) aims to leverage labeled source domain data to train object detection models that can generalize well to unlabeled target domains. Recently, many researchers have considered implementing fine-grained pixel-level domain adaptation using graph representations. Existing methods construct semantically complete graphs and align them across domains via graph matching. This work introduced an auxiliary node classification task before domain alignment through graph matching, which utilizes the inherent information of graph nodes to classify them, in order to avoid suboptimal graph matching results caused by node class confusion. However, previous methods neglected the contextual information of graph nodes, leading to biased node classification and suboptimal graph matching. To solve this issue, we propose a novel patch-based auxiliary node classification method for DAOD. Unlike existing methods that use only the inherent information of nodes for node classification, our method exploits the local region information of nodes and employs multi-layer convolutional neural networks to learn the local region feature representation of nodes, enriching the node context information. Thus, accurate and robust node classification results are produced and the risk of class confusion is reduced. Moreover, we propose a progressive strategy to fuse the inherent features and the learned local region features of nodes, which ensures that the network can stably and reliably utilize local region features for accurate node classification. In this paper, we conduct abundant experiments on various DAOD scenarios and demonstrate that our proposed model outperforms existing works.

Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training

Image captioning aims at generating descriptive and meaningful textual descriptions of images, enabling a broad range of vision-language applications. Prior works have demonstrated that harnessing the power of Contrastive Image Language Pre-training (CLIP) offers a promising approach to achieving zero-shot captioning, eliminating the need for expensive caption annotations. However, the widely observed modality gap in the latent space of CLIP harms the performance of zero-shot captioning by breaking the alignment between paired image-text features. To address this issue, we conduct an analysis on the CLIP latent space which leads to two findings. Firstly, we observe that the CLIP's visual feature of image subregions can achieve closer proximity to the paired caption due to the inherent information loss in text descriptions. In addition, we show that the modality gap between a paired image-text can be empirically modeled as a zero-mean Gaussian distribution. Motivated by the findings, we propose a novel zero-shot image captioning framework with text-only training to reduce the modality gap. In particular, we introduce a subregion feature aggregation to leverage local region information, which produces a compact visual representation for matching text representation. Moreover, we incorporate a noise injection and CLIP reranking strategy to boost captioning performance. We also extend our framework to build a zero-shot VQA pipeline, demonstrating its generality. Through extensive experiments on common captioning and VQA datasets such as MSCOCO, Flickr30k and VQAV2, we show that our method achieves remarkable performance improvements. Code is available at https://github.com/Artanic30/MacCap.

Multi-axis interactive multidimensional attention network for vehicle re-identification

Learning fine-grained discriminative information is essential to address the challenges of small inter-class differences and large intra-class differences in vehicle re-identification (Re-ID). Attentional mechanism is often used to capture important global information in images rather than fine-grained discriminative information. Studies have shown that the multi-axis interaction of information can enhance the feature representation ability of networks. This paper explores how to use the multi-axis interaction of information to facilitate more effective learning of attention and how to capture important detailed information in local regions. We propose a multi-axis interactive multidimensional attention network (MIMA-Net) for vehicle Re-ID. The network allows information to interact on multiple axes and calibrates the weight distribution of features from multiple dimensions to learn subtle discriminative information in vehicle parts/regions. The window-channel attention module (W-CAM) in MIMA-Net facilitates the learning of channel attention by interacting first across locations and then across channels, while the channel group-spatial attention module (CG-SAM) facilitates the learning of spatial attention by interacting first across channels and then across locations. These two modules perform window partitioning in a priori manner and channel semantic aggregation in an adaptive manner to learn discriminative semantic features in parts, respectively. These two approaches complement each other to strengthen the feature representation ability of MIMA-Net. Extensive experiments on three large public datasets, VeRi-776, VehicleID, and VERI-Wild, verify the effectiveness of our MIMA-Net and show that our method achieves state-of-the-art performance.

Self-Prompting Tracking: A Fast and Efficient Tracking Pipeline for UAV Videos

In the realm of visual tracking, remote sensing videos captured by Unmanned Aerial Vehicles (UAVs) have seen significant advancements with wide applications. However, there remain challenges to conventional Transformer-based trackers in balancing tracking accuracy and inference speed. This problem is further exacerbated when Transformers are extensively implemented at larger model scales. To address this challenge, we present a fast and efficient UAV tracking framework, denoted as SiamPT, aiming to reduce the number of Transformer layers without losing the discriminative ability of the model. To realize it, we transfer the conventional prompting theories in multi-model tracking into UAV tracking, where a novel self-prompting method is proposed by utilizing the target’s inherent characteristics in the search branch to discriminate targets from the background. Specifically, a self-distribution strategy is introduced to capture feature-level relationships, which segment tokens into distinct smaller patches. Subsequently, salient tokens within the full attention map are identified as foreground targets, enabling the fusion of local region information. These fused tokens serve as prompters to enhance the identification of distractors, thereby avoiding the demand for model expansion. SiamPT has demonstrated impressive results on the UAV123 benchmark, achieving success and precision rates of 0.694 and 0.890 respectively, while maintaining an inference speed of 91.0 FPS.

Hierarchical Dynamic Graph Convolutional Network With Interpretability for EEG-Based Emotion Recognition.

Graph convolutional networks (GCNs) have shown great prowess in learning topological relationships among electroencephalogram (EEG) channels for EEG-based emotion recognition. However, most existing GCN-only methods are designed with a single spatial pattern, lacking connectivity enhancement within local functional regions and ignoring the data dependencies of EEG original data. In this article, hierarchical dynamic GCN (HD-GCN) is proposed to explore dynamic multilevel spatial information among EEG channels, with discriminative features of EEG signals as auxiliary information. Specifically, representation learning in topological space consists of two branches: one for extracting global dynamic information and one for exploring augmentation information in local functional regions. In each branch, a layerwise adjacency matrix is utilized to enrich the expressive power of GCN. Furthermore, a data-dependent auxiliary information module (AIM) is developed to capture multidimensional fusion features. Extensive experiments on two public datasets, SJTU emotion EEG dataset (SEED) and DREAMER, demonstrate that the proposed method consistently exceeds state-of-the-art methods. Interpretability analysis of the proposed model is performed, discovering the active brain regions and important electrode pairs related to emotion.

TC-SfM: Robust Track-Community-Based Structure-from-Motion.

Structure-from-Motion (SfM) aims to recover 3D scene structures and camera poses based on the correspondences between input images, and thus the ambiguity caused by duplicate structures (i.e., different structures with strong visual resemblance) always results in incorrect camera poses and 3D structures. To deal with the ambiguity, most existing studies resort to additional constraint information or implicit inference by analyzing two-view geometries or feature points. In this paper, we propose to exploit high-level information in the scene, i.e., the spatial contextual information of local regions, to guide the reconstruction. Specifically, a novel structure is proposed, namely, track-community, in which each community consists of a group of tracks and represents a local segment in the scene. A community detection algorithm is performed on the track-graph to partition the scene into segments. Then, the potential ambiguous segments are detected by analyzing the neighborhood of tracks and corrected by checking the pose consistency. Finally, we perform partial reconstruction on each segment and align them with a novel bidirectional consistency cost function which considers both 3D-3D correspondences and pairwise relative camera poses. Experimental results demonstrate that our approach can robustly alleviate reconstruction failure resulting from visually indistinguishable structures and accurately merge the partial reconstructions.

Local Entropy-Based Coupled Anisotropic Diffusion for Detail-And Edge-Preserving Smoothing

It is important in image restoration to remove noise while preserving sharp edges and fine details such as blurred thin edges and low-contrast fine feature. The Perona–Malik (P-M) model is a well-known anisotropic diffusion denoising model, which can effectively remove noise while preserving edges. However, its diffusion coefficient only associates with the gradient of each pixel but not with the local region information; thus, the P-M model is not able to effectively preserve the important details of image. To address this problem, this paper proposes an anisotropic diffusion denoising model based on local entropy. The diffusion coefficient of the new model not only depends on the gradient of image but also on the local region information described by local entropy. On this basis, a coupled anisotropic diffusion scheme is proposed for detail-and edge-preserving smoothing. Experimental results show that the proposed model not only can effectively remove noise while preserving the boundaries better but also can maintain important details in an image very well.

Global Plus Local Jointly Regularized Support Vector Data Description for Novelty Detection.

In many practice application, the cost for acquiring abnormal data is quite expensive, thus the one-class classification (OCC) problem attracts great attention. As one of the solutions, support vector data description (SVDD) gains a continuous focus in outlier detection since it is based on the data description. For the sphere obtained by SVDD, both the center and the volume (or radius) strongly depend on the support vectors, while the support vectors are sensitive to the tradeoff parameter C. Hence, how to select this parameter is a rather challenging problem. In order to address this problem, we define several distance metrics relative to the image region in Gaussian kernel space. With the distance metrics, two probability densities relative to the global region and the local region are designed, respectively. Then, the information quantity and the information entropy are developed for regularizing the tradeoff parameter. This novel SVDD is called global plus local jointly regularized support vector data description (GL-SVDD), in which both the global region information and the local image region information jointly penalize the images as possible outliers. Finally, we use the UCI dataset and the hyperspectral data of cherry fruit to evaluate the performance of several OCC approaches. Experimental results show that GL-SVDD is encouraging.

Fire Safety Detection Based on CAGSA-YOLO Network

The layout of a city is complex, and indoor spaces have thousands of aspects that make them susceptible to fire. If a fire breaks out, it is difficult to quell, so a fire in the city will cause great harm. However, the traditional fire detection algorithm has a low detection efficiency and high detection rate of small targets, and disasters have occurred during detection. Therefore, this paper proposes a fire safety detection algorithm based on CAGSA-YOLO and constructs a fire safety dataset to integrate common fire safety tools into fire detection, which has a preventive detection effect before a fire occurs. In the improved algorithm, the upsampling in the original YOLOv5 is replaced with the CARAFE module. By adjusting its internal Parameter contrast, the algorithm pays more attention to local regional information and obtains stronger feature maps. Secondly, a new scale detection layer is added to detect objects larger than 4 × 4. Furthermore, the sampling Ghost lightweight design replaces C3 with the C3Ghost module without reducing the mAP. Finally, a lighter SA mechanism is introduced to optimize visual information processing resources. Using the fire safety dataset, the precision, recall, and mAP of the improved model increase to 89.7%, 80.1%, and 85.1%, respectively. At the same time, the size of the improved model is reduced by 0.6 M to 13.8 M, and the Param is reduced from 7.1 M to 6.6 M.

SIRN: An iterative reasoning network for transmission lines based on scene prior knowledge

High-accuracy positioning and identification of transmission line components is the premise for their status detection and fault diagnosis. However, due to the limitations of imbalance object scale and distribution in aerial images, the problem of detecting dense-tiny objects still needs to be solved. Considering the prior knowledge of fixed connection scenes, we proposed a novel method named scene iterative reasoning network (SIRN) based on self-adaptive clustering and local scene knowledge. It consists of the coarse-grained detector (CGD), scene self-adaptive clustering (SSAC) subnetwork, and scene structure iterative reasoning (SSIR) subnetwork. The CGD was proposed to obtain global coarse detection results. Then, the SSAC subnetwork utilized unsupervised self-adaptive clustering to obtain accurate dense regions based on the CGD results. Finally, the SSIR subnetwork was designed to extract and fuse the scene semantic structure information of local regions for realizing accurate dense-tiny object detection. In summary, the SIRN model utilizes an iterative inference strategy to detect dense regions accurately. By leveraging scene structure information, the model effectively transforms the challenge of detecting dense objects into an advantage, resulting in precise detection. The SIRN model is compatible with single-stage and two-stage object detection models, achieving a notable mean average precision (mAP) improvement ranging from 4.4% to 12.2% compared to the baseline models. Compared to other state-of-the-art object detection models, SIRN demonstrates significant advantages in accuracy and error rate metrics. In addition, qualitative and quantitative experiments show that the SIRN model considerably enhanced the detection of dense-tiny objects. The code is available at https://github.com/CharmingWang/SIRN.

Multi-scale-average-filter-assisted level set segmentation model with local region restoration achievements

Segmentation of noisy images having light in the background it is a challenging task for the existing segmentation approaches and methods. In this paper, we suggest a novel variational method for joint restoration and segmentation of noisy images which are having intensity and inhomogeneity in the existence of high contrast light in the background. The proposed model combines statistical local region information of circular regions centered at each pixel with a multi-phase segmentation technique enabling inhomogeneous image restoration. The proposed model is written in the fuzzy set framework and resolved through alternating direction minimization approach of multipliers. Through experiments, we have tested the performance of the suggested approach on diverse types of synthetic and real images in the existence of intensity and in-homogeneity; and evaluate the precision, as well as, the robustness of the suggested model. Furthermore, the outcomes are, then, compared with other state-of-the-art models including two-phase and multi-phase approaches and show that our method has superiority for images in the existence of noise and inhomogeneity. Our empirical evaluation and experiments, using real images, evaluate and assess the efficiency of the suggested model against several other closest rivals. We observed that the suggested model can precisely segment all the images having brightness, diffuse edges, high contrast light in the background, and inhomogeneity.

3D Skeleton and Two Streams Approach to Person Re-identification Using Optimized Region Matching

Person re-identification (Re-ID) is a challenging and arduous task due to non-overlapping views, complex background, and uncontrollable occlusion in video surveillance. An existing method for capturing pedestrian local region information is to divide person regions into horizontal stripes, which may lead to invalid features and erroneous learning. To solve this problem, this paper proposes a 3D skeleton and a two-stream approach to person Re-ID. The first stream of the method uses the 3D skeleton for background filtering and region segmentation. The second stream uses Siamese net to extract the global descriptor. The features of the two streams are fused to preserve the integrity of the person. An optimized region matching method for metric learning is designed. Extensive comparing experiments were conducted with state-of-the-art Re-ID methods on the Market-1501, CUHK03, and DukeMTMC-reID datasets. Experimental results show that the proposed method outperforms the existing methods in recognition accuracy.

Deep spatio-temporal 3D densenet with multiscale ConvLSTM-Resnet network for citywide traffic flow forecasting

Reliable traffic flow forecasting is paramount in Intelligent Transportation Systems (ITS) as it can effectively improve traffic efficiency and social security. Its vital challenge is to effectively integrate various factors (such as multiple temporal correlations, complex spatial correlation, high heterogeneous) to infer the evolution trend of future traffic flow. Inspired by spatio-temporal prediction in computer vision, we regard traffic data slices at each moment as “traffic frames”. This paper presents an end-to-end architecture named Spatio-Temporal 3D Densenet Multiscale ConvLSTM-Resnet Network (ST-3DDMCRN) to predict future traffic flow accurately. Specifically, a 3D densenet network is applied simultaneously to capture the traffic frame’s local regional spatio-temporal information. Traditional Resnet networks cannot capture long-range spatial correlation, a novel multiscale ConvLSTM-Resnet network is developed to overcome this problem, extracting traffic frame’s nearby and long-range spatial dependencies. In addition, considering the spatio-temporal heterogeneity of traffic frames, a Region-Squeeze-and-Excitation (RSE) unit is designed to accurately quantify the difference of the contributions of the correlations in space. The experiment result on two datasets in the real world illustrates the ST-3DDMCRN model outperforms the state-of-art baselines for the citywide traffic flow prediction. Furthermore, to validate the model’s generality, we utilize the model to predict the passenger pickup/dropoff demand task, the prediction results are more accurate than the baseline methods.

Evaluation and Stratification for Chinese International Education Quality with Deep Learning Model.

In the process of human communication, language learning and communication play a fundamental, leading, broad, and long-lasting role. It serves as a link and a bridge between countries and peoples, allowing for greater understanding and camaraderie. The significance of Chinese in international commercial and cultural exchanges has been more obvious in the current era, as China's comprehensive strength continues to improve. Its cultural value and practical value have been continuously improved, the international community has an increasing demand for learning Chinese, and the cause of international Chinese language education has developed rapidly. China must strengthen the international dissemination of Chinese, so that the world can better understand and accept China, so that China can better integrate into the world. In this context, how to evaluate and stratify the quality of Chinese international education has become an important research topic. Relying on the hot deep learning technology in recent years, this work designs a neural network for evaluating the quality of international Chinese education. The content of this work is as follows: aiming at the serious defect of the current mainstream feature classification network that only uses the top-level features extracted by a single convolution layer to classify, which leads to the loss of classification accuracy. A multiscale feature pyramid fusion network is built in this paper, starting with the working mechanism of a convolutional neural network. It is capable of fully extracting and combining the representations of the network's shallow and deep outputs, based on first- and second-order characteristics of global and local discriminative region information. Second, the network structure has the bottleneck layer module and the batch normalization layer module, both of which are made up of varying numbers of 1 × 1 convolution kernels.

Deep Learning-Based Identification of Maize Leaf Diseases Is Improved by an Attention Mechanism: Self-Attention.

Maize leaf diseases significantly reduce maize yield; therefore, monitoring and identifying the diseases during the growing season are crucial. Some of the current studies are based on images with simple backgrounds, and the realistic field settings are full of background noise, making this task challenging. We collected low-cost red, green, and blue (RGB) images from our experimental fields and public dataset, and they contain a total of four categories, namely, southern corn leaf blight (SCLB), gray leaf spot (GLS), southern corn rust (SR), and healthy (H). This article proposes a model different from convolutional neural networks (CNNs) based on transformer and self-attention. It represents visual information of local regions of images by tokens, calculates the correlation (called attention) of information between local regions with an attention mechanism, and finally integrates global information to make the classification. The results show that our model achieves the best performance compared to five mainstream CNNs at a meager computational cost, and the attention mechanism plays an extremely important role. The disease lesions information was effectively emphasized, and the background noise was suppressed. The proposed model is more suitable for fine-grained maize leaf disease identification in a complex background, and we demonstrated this idea from three perspectives, namely, theoretical, experimental, and visualization.

Breast Cancer Molecular Subtype Prediction on Pathological Images with Discriminative Patch Selection and Multi-Instance Learning.

Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient’s paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable block sampling error is risky due to the tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&E pathological whole slide images (WSI) using the AI method is useful and critical to assist pathologists to pre-screen proper paraffin block for IHC. It is a challenging task since only WSI-level labels of molecular subtypes from IHC can be obtained without detailed local region information. Gigapixel WSIs are divided into a huge amount of patches to be computationally feasible for deep learning, while with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selection and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&E WSIs. Firstly, co-teaching strategy using two networks was adopted to learn molecular subtype representations and filter out some noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating local patch with global slide constraint information was used to fine-tune MIL framework on obtained discriminative patches and further improve the prediction performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed AI method and our models outperformed even senior pathologists, which has the potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic.

Cross-Database Micro-Expression Recognition: A Benchmark

Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis. CDMER is more challenging than the conventional micro-expression recognition (MER), because the training and testing samples in CDMER come from different micro-expression databases, resulting in inconsistency of the feature distributions between the training and testing sets. In this paper, we contribute to this topic from three aspects. First, we establish a CDMER experimental evaluation protocol aiming to allow the researchers to conveniently work on this topic and evaluate their proposed methods under the same standard. Second, we conduct benchmark experiments by using NINE state-of-the-art domain adaptation (DA) methods and SIX popular spatiotemporal descriptors for investigating CDMER problem from two different perspectives. Third, we propose a novel DA method called region selective transfer regression (RSTR) to deal with the CDMER task. The overall superior performance of RSTR over the state-of-the-art DA methods demonstrates that taking into consideration the facial local region information used in RSTR contributes to developing effective DA methods for dealing with CDMER problem.