Feature Map Research Articles

Multi-scale Lesion Feature Fusion and Location-Aware for Chest Multi-disease Detection.

Accurately identifying and locating lesions in chest X-rays has the potential to significantly enhance diagnostic efficiency, quality, and interpretability. However, current methods primarily focus on detecting of specific diseases in chest X-rays, disregarding the presence of multiple diseases in a single chest X-ray scan. Moreover, the diversity in lesion locations and attributes introduces complexity in accurately discerning specific traits for each lesion, leading to diminished accuracy when detecting multiple diseases. To address these issues, we propose a novel detection framework that enhances multi-scale lesion feature extraction and fusion, improving lesion position perception and subsequently boosting chest multi-disease detection performance. Initially, we construct a multi-scale lesion feature extraction network to tackle the uniqueness of various lesion features and locations, strengthening the global semantic correlation between lesion features and their positions. Following this, we introduce an instance-aware semantic enhancement network that dynamically amalgamates instance-specific features with high-level semantic representations across various scales. This adaptive integration effectively mitigates the loss of detailed information within lesion regions. Additionally, we perform lesion region feature mapping using candidate boxes to preserve crucial positional information, enhancing the accuracy of chest disease detection across multiple scales. Experimental results on the VinDr-CXR dataset reveal a 6% increment in mean average precision (mAP) and an 8.4% improvement in mean recall (mR) when compared to state-of-the-art baselines. This demonstrates the effectiveness of the model in accurately detecting multiple chest diseases by capturing specific features and location information.

Research on sound quality of roller chain transmission system based on multi-source transfer learning.

To establish the sound quality evaluation model of roller chain transmission system, we collect the running noise under different working conditions. After the noise samples are preprocessed, a group of experienced testers are organized to evaluate them subjectively. Mel frequency cepstral coefficient (MFCC) of each noise sample is calculated, and the MFCC feature map is used as an objective evaluation. Combining with the subjective and objective evaluation results of the roller chain system noise, we can get the original dataset of its sound quality research. However, the number of high-quality noise samples is relatively small. Based on the sound quality research of various chain transmission systems, a novel method called multi-source transfer learning convolutional neural network (MSTL-CNN) is proposed. By transferring knowledge from multiple source tasks to target task, the difficulty of small sample sound quality prediction is solved. Compared with the problem that single source task transfer learning has too much error on some samples, MSTL-CNN can give full play to the advantages of all transfer learning models. The results also show that the MSTL-CNN proposed in this paper is significantly better than the traditional sound quality evaluation methods.

Tire Defect Detection Based on Low and High-level Feature Fusion

Abstract Recently, object detection based on deep learning has made great progress in the field of defect detection. Due to its complex texture background and varied defect features, existing defect detection methods based on object detection face great challenges in the detection of tire internal defects. In this paper, a tire defect detection model based on low and high-level feature fusion is proposed. First, a multi-head feature extraction module is proposed, which extracts abundant effective information from multiple dimensions. Second, a spatial semantic fusion upsampling module is proposed to alleviate the problem of information loss in the upsampling process. Finally, a novel prediction head is designed to expand the receptive field by compressing the size of the feature map to improve the detection accuracy of large defects. Experimental results show that the mAP of the proposed method achieves 94.03% on the tire internal defect dataset, and the average detection time is 36.74ms, which can meet the needs of industrial online detection.

Non-Contact Blood Pressure Estimation from Radar Signals by a Stacked Deformable Convolution Network.

This study introduces a contactless blood pressure monitoring approach that combines conventional radar signal processing with novel deep learning architectures. During the preprocessing phase, datasets suitable for synchronization are created by integrating Kalman filtering, multiscale bandpass filters, and a periodic extraction method in the time domain. These data comprise data on chest micro variations, encapsulating a complex array of physiological and biomedical information reflective of cardiac micromotions. The Radar-based Stacked Deformable convolution Network (RSD-Net) integrates channel and spatial self attention mechanisms within a deformable convolutional framework to enhance feature extraction from radar signals. The network architecture systematically employs deformable convolutions for initial deep feature extraction from individual signals. Subsequently, continuous blood pressure estimation is conducted using self attention mechanisms on feature map from single source coupled with multi-feature map channel attention. The performance of model is corroborated via the open-source dataset procured using a non-invasive 24GHz six-port continuous wave radar system. The dataset, encompassing readings from 30 healthy individuals subjected to diverse conditions including rest, the Valsalva maneuver, apnea, and tilt-table examinations. It serves to substantiate the validity and resilience of the proposed method in the non-contact assessment of continuous blood pressure. Evaluation metrics reveal Pearson correlation coefficients of 0.838 for systolic and 0.797 for diastolic blood pressure predictions. The Mean Error (ME) and Standard Deviation (SD) for systolic and diastolic blood pressure measurements are -0.32 ±6.14mmHg and -0.20 ±5.50mmHg, respectively. The ablation study assesses the contribution of different structural components of the RSD-Net, validating their significance in the overall of model performance.

Asymmetric Convolution: An Efficient and Generalized Method to Fuse Feature Maps in Multiple Vision Tasks.

Fusing features from different sources is a critical aspect of many computer vision tasks. Existing approaches can be roughly categorized as parameter-free or learnable operations. However, parameter-free modules are limited in their ability to benefit from offline learning, leading to poor performance in some challenging situations. Learnable fusing methods are often space-consuming and timeconsuming, particularly when fusing features with different shapes. To address these shortcomings, we conducted an in-depth analysis of the limitations associated with both fusion methods. Based on our findings, we propose a generalized module named Asymmetric Convolution Module (ACM). This module can learn to encode effective priors during offline training and efficiently fuse feature maps with different shapes in specific tasks. Specifically, we propose a mathematically equivalent method for replacing costly convolutions on concatenated features. This method can be widely applied to fuse feature maps across different shapes. Furthermore, distinguished from parameter-free operations that can only fuse two features of the same type, our ACM is general, flexible, and can fuse multiple features of different types. To demonstrate the generality and efficiency of ACM, we integrate it into several state-of-the-art models on three representative vision tasks: visual object tracking, referring video object segmentation, and monocular 3D object detection. Extensive experimental results on three tasks and several datasets demonstrate that our new module can bring significant improvements and noteworthy efficiency.

Anomaly classification based on self-supervised learning and its application

BackgroundAnomaly classification remains a challenging task in computer vision applications across diverse practical fields such as industrial detection and security check. The purpose of this study is to develop a new anomaly classification method based on self-supervised learning (Self-ACM), which is expected to enhance anomaly classification accuracy significantly. MethodsThe feature maps of images were abstracted using VGG16 model pre-trained on ImageNet, which were then fed into the innovative model to capture the normality distribution within the dataset domain. Then, a self-supervised adversarial anomaly classification learning framework was proposed to facilitate the acquisition of a higher-level semantic representations for improved anomaly detection. Thirdly, we collected and constructed a novel terahertz (THZ) dataset, which serves as a pioneering resource for benchmarking anomaly classification tasks in the field. ResultsThrough a series of rigorous experiments, our findings unequivocally demonstrate the following key insights: Firstly, harnessing feature maps as input data yielded a significant enhancement in anomaly detection performance, underscoring the effectiveness of this approach. Secondly, the integration of self-supervision enriched the dataset with invaluable information, empowering both the discriminator and generator to acquire superior feature representations. The culmination of these advancements is our novel method achieving unparalleled state-of-the-art performance across multiple benchmark datasets. This breakthrough underscores the transformative impact of our approach on anomaly detection methodologies, solidifying its position as a pioneering solution in the field. ConclusionThe Self-ACM strategy not only advances anomaly detection methodologies but also offers a remarkable contribution to dataset creation, setting a new standard for anomaly classification research.

Self-supervised monocular visual odometry based on cross-correlation

Visual odometry constitutes a critical component in enabling autonomous navigation. However, the existing methods are limited by the feature extraction and matching accuracy, and cannot show good real-time performance while combining accuracy and robustness. In this paper, we propose a novel monocular visual odometry framework based on cross-correlation. The framework starts with a parameter-sharing Siamese network to build feature extractors that can simultaneously process multiple images as inputs. Moreover, we design cross-correlation modules and define a cross-correlation matrix to describe the strength of correlation between different parts of the input feature maps, reflecting the rotational and translational transformations of the input images. Furthermore, a novel loss function is introduced to impose constraints on the network. Additionally, a fully convolutional network is designed for pose estimation, computing poses alterations from the structure of the cross-correlation matrix. Channel attention and spatial attention mechanisms are introduced to improve the performance. More importantly, our method innovatively uses time intervals as labels, enables self-supervised training, and relies only on a monocular camera. Experimental results on the KITTI visual odometry dataset and the Oxford Robotcar Dataset show that our method produces competitive performance, demonstrating the superiority of the proposed method.

A Novel Lightweight Model for Underwater Image Enhancement

Underwater images suffer from low contrast and color distortion. In order to improve the quality of underwater images and reduce storage and computational resources, this paper proposes a lightweight model Rep-UWnet to enhance underwater images. The model consists of a fully connected convolutional network and three densely connected RepConv blocks in series, with the input images connected to the output of each block with a Skip connection. First, the original underwater image is subjected to feature extraction by the SimSPPF module and is processed through feature summation with the original one to be produced as the input image. Then, the first convolutional layer with a kernel size of 3 × 3, generates 64 feature maps, and the multi-scale hybrid convolutional attention module enhances the useful features by reweighting the features of different channels. Second, three RepConv blocks are connected to reduce the number of parameters in extracting features and increase the test speed. Finally, a convolutional layer with 3 kernels generates enhanced underwater images. Our method reduces the number of parameters from 2.7 M to 0.45 M (around 83% reduction) but outperforms state-of-the-art algorithms by extensive experiments. Furthermore, we demonstrate our Rep-UWnet effectively improves high-level vision tasks like edge detection and single image depth estimation. This method not only surpasses the contrast method in objective quality, but also significantly improves the contrast, colorimetry, and clarity of underwater images in subjective quality.

Building footprint extraction from aerial images using an edge-aware YOLO-v8 network

Abstract. Building footprint extraction is a critical indicator for assessing urban infrastructure, and extracting building footprints from remote sensing imagery can have significant practical applications. However, achieving rapid and accurate extraction of building footprints remains highly challenging, especially in scenarios with complex scenes, dense building distributions, and small targets. The instance segmentation models of the YOLO series offer strong real-time performance, reducing considerable time and effort in practical applications. Therefore, we propose building footprint extraction based on an enhance YOLO-v8 network. This study focuses on three enhancements to the YOLO-v8 network to improve extraction accuracy. Building upon the YOLO-v8 framework, we have incorporated the Feature Pyramid Network (FPN) module into feature maps at all scales to efficiently propagate high-level semantic information. Additionally, we introduce the Triple Feature Encoder (TFE) module, which integrates spatial detail information from feature maps at three different scales to enhance the network's ability to extract multi-scale information. Finally, we explore the integration of the Prewitt model, a conventional edge detection operator, to assist in extracting edge features in target regions of feature maps. This integration aims to reduce the jagged edges frequently seen in the outcomes of the original YOLO-v8. Furthermore, the Prewitt operator's noise suppression capability helps mitigate the influence of non-target areas in the feature maps. The proposed framework achieves an instance segmentation accuracy of mAP50 is 84.6% and mAP50:95 is 51.4% on public datasets, outperforming the original YOLO-v8 network.

Feature Maps Need More Attention: A Spatial-Channel Mutual Attention-Guided Transformer Network for Face Super-Resolution

Recently, transformer-based face super-resolution (FSR) approaches have achieved promising success in restoring degraded facial details due to their high capability for capturing both local and global dependencies. However, while existing methods focus on introducing sophisticated structures, they neglect the potential feature map information, limiting FSR performance. To circumvent this problem, we carefully design a pair of guiding blocks to dig for possible feature map information to enhance features before feeding them to transformer blocks. Relying on the guiding blocks, we propose a spatial-channel mutual attention-guided transformer network for FSR, for which the backbone architecture is a multi-scale connected encoder–decoder. Specifically, we devise a novel Spatial-Channel Mutual Attention-guided Transformer Module (SCATM), which is composed of a Spatial-Channel Mutual Attention Guiding Block (SCAGB) and a Channel-wise Multi-head Transformer Block (CMTB). SCATM on the top layer (SCATM-T) aims to promote both local facial details and global facial structures, while SCATM on the bottom layer (SCATM-B) seeks to optimize the encoded features. Considering that different scale features are complementary, we further develop a Multi-scale Feature Fusion Module (MFFM), which fuses features from different scales for better restoration performance. Quantitative and qualitative experimental results on various datasets indicate that the proposed method outperforms other state-of-the-art FSR methods.

A Deep Neural Network for Road Extraction with the Capability to Remove Foreign Objects with Similar Spectra

Abstract. Existing road extraction methods based on deep learning often struggle with distinguishing ground objects that share similar spectral information, such as roads and buildings. Consequently, this study proposes a dual encoder-decoder deep neural network to address road extraction in complex backgrounds. In the feature extraction stage, the first encoder-decoder designed for extracting road features. The second encoder-decoder utilized for extracting building features. During the feature fusion stage, road features and building features are integrated using a subtraction method. The resultant road features, constrained by building features, enhance the preservation of accurate road feature information. Within the feature fusion stage, road feature maps and building feature maps designated for fusion are input into the convolutional block attention module. This step aims to amplify the features of different channels and extract key information from diverse spatial positions. Subsequently, feature fusion is executed using the element-by-element subtraction method. The outcome is road features constrained by building features, thus preserving more precise road feature information. Experimental results demonstrate that the model successfully learns both road and building features concurrently. It effectively distinguishes between easily confused roads and buildings with similar spectral information, ultimately enhancing the accuracy of road extraction.

An Efficient Semantic Segmentation Method for Remote-Sensing Imagery Using Improved Coordinate Attention

Semantic segmentation stands as a prominent domain within remote sensing that is currently garnering significant attention. This paper introduces a pioneering semantic segmentation model based on TransUNet architecture with improved coordinate attention for remote-sensing imagery. It is composed of an encoding stage and a decoding stage. Notably, an enhanced and improved coordinate attention module is employed by integrating two pooling methods to generate weights. Subsequently, the feature map undergoes reweighting to accentuate foreground information and suppress background information. To address the issue of time complexity, this paper introduces an improvement to the transformer model by sparsifying the attention matrix. This reduces the computing expense of calculating attention, making the model more efficient. Additionally, the paper uses a combined loss function that is designed to enhance the training performance of the model. The experimental results conducted on three public datasets manifest the efficiency of the proposed method. The results indicate that it excels in delivering outstanding performance for semantic segmentation tasks pertaining to remote-sensing images.

Segmentation and Tracking Based on Equalized Memory Matching Network and Its Application in Electric Substation Inspection

With the wide application of deep learning, power inspection technology has made great progress. However, substation inspection videos often present challenges such as complex backgrounds, uneven lighting distribution, variations in the appearance of power equipment targets, and occlusions, which increase the difficulty of object segmentation and tracking, thereby adversely affecting the accuracy and reliability of power equipment condition monitoring. In this paper, a pixel-level equalized memory matching network (PEMMN) for power intelligent inspection segmentation and tracking is proposed. Firstly, an equalized memory matching network is designed to collect historical information about the target using a memory bank, in which a pixel-level equalized matching method is used to ensure that the reference frame information can be transferred to the current frame reliably, guiding the segmentation tracker to focus on the most informative region in the current frame. Then, to prevent memory explosion and the accumulation of segmentation template errors, a mask quality evaluation module is introduced to obtain the confidence level of the current segmentation result so as to selectively store the frames with high segmentation quality to ensure the reliability of the memory update. Finally, the synthetic feature map generated by the PEMMN and the mask quality assessment strategy are unified into the segmentation tracking framework to achieve accurate segmentation and robust tracking. Experimental results show that the method performs excellently on real substation inspection scenarios and three generalized datasets and has high practical value.

Segmentation of the left atrial appendage based on fusion attention.

In clinical practice, the morphology of the left atrial appendage (LAA) plays an important role in the selection of LAA closure devices for LAA closure procedures. The morphology determination is influenced by the segmentation results. The LAA occupies only a small part of the entire 3D medical image, and the segmentation results are more likely to be biased towards the background region, making the segmentation of the LAA challenging. In this paper, we propose a lightweight attention mechanism called fusion attention, which imitates human visual behavior. We process the 3D image of the LAA using a method that involves overview observation followed by detailed observation. In the overview observation stage, the image features are pooled along the three dimensions of length, width, and height. The obtained features from the three dimensions are then separately input into the spatial attention and channel attention modules to learn the regions of interest. In the detailed observation stage, the attention results from the previous stage are fused using element-wise multiplication and combined with the original feature map to enhance feature learning. The fusion attention mechanism was evaluated on a left atrial appendage dataset provided by Liaoning Provincial People's Hospital, resulting in an average Dice coefficient of 0.8855. The results indicate that the fusion attention mechanism achieves better segmentation results on 3D images compared to existing lightweight attention mechanisms.

Capsule Attention Based Detection of Contraband in X-Ray Images

Aiming at the problem of low detection accuracy caused by the different postures, different sizes, complex backgrounds and overlapping occlusions of contraband in the security checking process, a Matrix capsule network based on attention mechanism (MCAM) is designed by introducing attention mechanism into the capsule network. Firstly, a multi-feature-extraction (MFE) module is designed to solve the difficulty of detecting contraband with different sizes and complex backgrounds; then the Conv2d with Attention for ConvCap (CACC) is constructed in the convolutional layer of the capsule, and the weight information is computed in the channel dimensions of the feature map to give the contraband regions higher coefficients to enhance the contraband detection ability when the poses are different and the occlusion is severe; finally, a new capsule detection layer is designed by utilizing the pose matrix of the capsule to give full play to the detection ability of the matrix capsule network. The mAPs of the proposed model on SIXray, SIXray10, and SIXray100 are 85.10%, 64.05%, and 53.67%, respectively, which are 42.79%, 19.73%, and 9.41% higher than those of the original network, higher than those of the current mainstream detection networks, and the number of parameters of the network and the amount of computation are also lower.

Semantic Coarse-to-Fine Granularity Learning for Two-Stage Few-Shot Anomaly Detection

Anomaly detection is critical in industrial inspection, where identifying defects significantly impacts product quality and safety. Existing models, primarily based on convolutional neural networks (CNNs), struggle with noise sensitivity and insufficient resolution for fine-grained feature discrimination. To address these issues, we propose a two-stage few-shot anomaly detection network that enhances semantic feature granularity and generalization. The network includes a coarse-grained anomaly detection module, a multi-scale channel attention module, and a fine-grained detection module. The coarse-grained module identifies abnormal regions, serving as the initial filter. The multi-scale channel attention module focuses on anomalous features, enhancing sensitivity to fine-grained characteristics. This step overcomes limitations in discerning subtle yet critical anomalies. The fine-grained detection module refines feature maps, enhancing generalization. Experimental results on the MVTec dataset show an image-level Area under the region of convergence (AUROC) of 92.3% and a pixel-level AUROC of 95.3%, a 1% to 2% improvement over leading FSAD methods.

Research on power generation prediction of hydropower in river basin based on multi-head attention graph convolutional neural network

For the power generation prediction of traditional hydropower stations, most of them only use time series prediction and neglect to study the spatial topological relationship of hydropower stations in the river basin, so that it is difficult to fully explore the characteristic relationship of space power stations. In this paper, a research method for power generation prediction of hydropower in river basin hydropower stations based on multi-head attention map convolutional neural network is proposed. This method establishes a first-level node neighborhood feature map based on the spatial geographic distribution information of hydropower stations in the basin, and uses the method of graph convolution to carry out node feature mining and feature learning, so as to transform the power generation capacity evaluation problem of the hydropower station in the basin into the node prediction problem in the graph, which is different from the global normalization rule. the multi-head attention mechanism introduced further improves the information aggregation quality of the graph node, and uses the historical temperature, power generation, electricity price, unit status and other data of each hydropower station in the basin for training. The reasoning results show that the proposed method achieves higher accuracy than other schemes, and the power prediction method is conducive to the formulation of power plans of hydropower stations in the basin, and can also play a positive role in guiding the site selection of hydropower plants.

A Novel Multi-Scale Feature Map Fusion for Oil Spill Detection of SAR Remote Sensing

The efficient and timely identification of oil spill areas is crucial for ocean environmental protection. Synthetic aperture radar (SAR) is widely used in oil spill detection due to its all-weather monitoring capability. Meanwhile, existing deep learning-based oil spill detection methods mainly rely on the classical U-Net framework and have achieved impressive results. However, SAR images exhibit high noise, blurry boundaries, and irregular shapes of target areas, as well as speckles and shadows, which lead to the loss of performance in existing algorithms. In this paper, we propose a novel network architecture to achieve more precise segmentation of oil spill areas by reintroducing rich semantic contextual information before obtaining the final segmentation mask. Specifically, the proposed architecture can re-fuse feature maps from different levels at the decoder end. We design a multi-convolutional layer (MCL) module to extract basic feature information from SAR images, and a feature extraction module (FEM) module further extracts and fuses feature maps generated by the U-Net decoder at different levels. Through these operations, the network can learn rich global and local contextual information, enable sufficient interaction of feature information at different stages, enhance the model’s contextual awareness, and improve its ability to recognize complex textures and blurry boundaries, thereby enhancing the segmentation accuracy of SAR images. Compared to many U-Net based segmentation networks, our method shows promising results and achieves state-of-the-art performance on multiple evaluation metrics.

An intelligent assessment method of criminal psychological attribution based on unbalance data

Criminal cases often exhibit imbalance and cannot be extended by data augmentation when classified into attribution types. To solve the problem of unbalance data in offenders’ attribution classification, this paper proposes a criminal psychological attribution assessment model by an improved Balanced TF-Distinguishing IDF method (B-TF-dIDF) and constructed a hybrid network with attention method to fuse numerical and text features for improving the accuracy. First, as a statistical method, B-TF-dIDF is presented to reduce the impact of class-imbalance for extraction of numerical features, in which a balanced element is added to reduce the effects of incorrect type keywords on classification, and a distinguishing element is added to discriminate the types of keywords. Then, an improved hybrid network model composed of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) is constructed to balance the influence of different lengths of text samples for extracting the semantic features of criminal texts. For evaluating different feature weights by their importance, Spatial Attention is used to improve CNN in the feature maps. Moreover, the self-attention is also performed to re-evaluate the mixed features. Finally, the softmax classifier provides a scientific basis for developing a hierarchical treatment mechanism further. Additionally, we build a criminal data set with labels from real cases for testing. The experiment proved that the proposed model is better than other related methods in various evaluation indicators, including the micro and macro scopes. Moreover, the F1 of minority samples has increased by 6%–8%, indicating that the proposed method can reduce the impact of class-imbalance.

Multi-Scale Dense Graph Attention Network for Hyperspectral Classification

In recent years, numerous deep learning-based methods have gained increasing attention in hyperspectral classification, particularly the Graph Neural Network, which exhibits superior capabilities in structural description. However, a single graph structure is not suitable for hyperspectral feature representation. Therefore, we propose a novel Multiple-Scale graph network structure, known as the Multi-Scale Dense Graph Attention network for hyperspectral classification. Firstly, semi-supervised local Fisher discriminant analysis and superpixel segmentation were employed for dimensionality reduction and multi-scale graph construction, respectively. Secondly, Spectral-Spatial convolution is applied to extract shallow features from the image. Subsequently, an improved graph self-attention network is sequentially applied to each scale graph, and the different scale graphs are densely connected through spatial feature alignment modules, designed using twice matrix multiplication. Finally, the combined pixel-level feature map from multiple graph spaces is derived, and Spectral-Spatial convolution is employed to fuse the abundant feature maps for hyperspectral classification. Experimental results on various hyperspectral datasets demonstrate the superiority of our MSDesGATnet over many state-of-the-art methods. The code is available at https://github.com/l7170/MSDesGAT.git.