Multi-level Feature Extraction Research Articles

SAR Target Recognition Method based on Adaptive Weighted Decision Fusion of Deep Features

Background: This paper proposes a synthetic aperture radar (SAR) target recognition method based on adaptive weighted decision fusion of multi-level deep features. Methods: The trained ResNet-18 is employed to extract multi-level deep features from SAR images. Afterwards, based on the joint sparse representation (JSR) model, the multi-level deep features are represented to obtain the corresponding reconstruction error vectors. Considering the differences in the abilities of different levels of features to distinguish the target, the reconstruction error vectors are analyzed based on entropy theory, and their corresponding weights are adaptively obtained. Finally, the fused reconstruction error result is obtained through adaptively weighted fusion, and the target label is determined accordingly. Results: Experiments are conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset under different conditions, and the proposed method is compared with published methods, including multi-feature decision fusion, JSR-based decision fusion and other types of ResNets. Conclusion: The experimental results under standard operating condition (SOC) and extended operating conditions (EOCs) including depression angle variance and noise corruption validate the advantages of the proposed method.

Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation

Accurate segmentation of medical images is vital for disease detection and treatment. Convolutional Neural Networks (CNN) and Transformer models are widely used in medical image segmentation due to their exceptional capabilities in image recognition and segmentation. However, CNNs often lack an understanding of the global context and may lose spatial details of the target, while Transformers struggle with local information processing, leading to reduced geometric detail of the target. To address these issues, this research presents a Global-Local Fusion network model (GLFUnet) based on the U-Net framework and attention mechanisms. The model employs a dual-branch network that utilizes ConvNeXt and Swin Transformer to simultaneously extract multi-level features from pathological images. It enhances ConvNeXt’s local feature extraction with spatial and global attention up-sampling modules, while improving Swin Transformer’s global context dependency with channel attention. The Attention Feature Fusion module and skip connections efficiently merge local detailed and global coarse features from CNN and Transformer branches at various scales. The fused features are then progressively restored to the original image resolution for pixel-level prediction. Comprehensive experiments on datasets of stomach and liver cancer demonstrate GLFUnet’s superior performance and adaptability in medical image segmentation, holding promise for clinical analysis and disease diagnosis.

Zero-Shot Sketch-Based Remote-Sensing Image Retrieval Based on Multi-Level and Attention-Guided Tokenization

Effectively and efficiently retrieving images from remote-sensing databases is a critical challenge in the realm of remote-sensing big data. Utilizing hand-drawn sketches as retrieval inputs offers intuitive and user-friendly advantages, yet the potential of multi-level feature integration from sketches remains underexplored, leading to suboptimal retrieval performance. To address this gap, our study introduces a novel zero-shot, sketch-based retrieval method for remote-sensing images, leveraging multi-level feature extraction, self-attention-guided tokenization and filtering, and cross-modality attention update. This approach employs only vision information and does not require semantic knowledge concerning the sketch and image. It starts by employing multi-level self-attention guided feature extraction to tokenize the query sketches, as well as self-attention feature extraction to tokenize the candidate images. It then employs cross-attention mechanisms to establish token correspondence between these two modalities, facilitating the computation of sketch-to-image similarity. Our method significantly outperforms existing sketch-based remote-sensing image retrieval techniques, as evidenced by tests on multiple datasets. Notably, it also exhibits robust zero-shot learning capabilities in handling unseen categories and strong domain adaptation capabilities in handling unseen novel remote-sensing data. The method’s scalability can be further enhanced by the pre-calculation of retrieval tokens for all candidate images in a database. This research underscores the significant potential of multi-level, attention-guided tokenization in cross-modal remote-sensing image retrieval. For broader accessibility and research facilitation, we have made the code and dataset used in this study publicly available online.

Debris Flow Susceptibility Evaluation Based on Multi-level Feature Extraction CNN Model: A Case Study of Nujiang Prefecture, China

Debris Flow Susceptibility Evaluation Based on Multi-level Feature Extraction CNN Model: A Case Study of Nujiang Prefecture, China

Lattice 123 pattern for automated Alzheimer’s detection using EEG signal

AbstractThis paper presents an innovative feature engineering framework based on lattice structures for the automated identification of Alzheimer's disease (AD) using electroencephalogram (EEG) signals. Inspired by the Shannon information entropy theorem, we apply a probabilistic function to create the novel Lattice123 pattern, generating two directed graphs with minimum and maximum distance-based kernels. Using these graphs and three kernel functions (signum, upper ternary, and lower ternary), we generate six feature vectors for each input signal block to extract textural features. Multilevel discrete wavelet transform (MDWT) was used to generate low-level wavelet subbands. Our proposed model mirrors deep learning approaches, facilitating feature extraction in frequency and spatial domains at various levels. We used iterative neighborhood component analysis to select the most discriminative features from the extracted vectors. An iterative hard majority voting and a greedy algorithm were used to generate voted vectors to select the optimal channel-wise and overall results. Our proposed model yielded a classification accuracy of more than 98% and a geometric mean of more than 96%. Our proposed Lattice123 pattern, dynamic graph generation, and MDWT-based multilevel feature extraction can detect AD accurately as the proposed pattern can extract subtle changes from the EEG signal accurately. Our prototype is ready to be validated using a large and diverse database.

Development of Smart Agriculture to detect the Arabica Coffee Leaf Disease using IAFSA based MSAB with Channel and Spatial Attention Network

Plant diseases provide challenges for the agriculture sector, notably to produce Arabica coffee. Recognising issues on Arabica coffee leaves is a first step in avoiding and curing illnesses to prevent crop loss. With the extraordinary advancements achieved in convolutional neural networks (CNN) in recent years, Arabica coffee leaf damage can now be identified without the aid of a specialist. However, the local characteristics that convolutional layers in CNNs record are typically redundant and unable to make efficient use of global data to support the prediction process. The proposed Hybrid Attention UNet, also known as CMSAMB-UNet due to its feature extraction and global modelling capabilities, integrates both the Channel and Spatial Attention Module (CSAM) as well as the Multi-head Self-Attention Block (MSAB). In this study, CMSAMB-UNet is built on Resnet50 to extract multi-level features from plant picture data. Two shallow layers of feature maps are used with CSAM according to local attention. used throughout the feature extraction process to enrich the features and adaptively disregard unwanted features. In order to recreate the spatial feature connection of the input pictures using high-resolution feature maps, two global attention maps produced by MSAB are combined.

Using a Hybrid Convolutional Neural Network with a Transformer Model for Tomato Leaf Disease Detection

Diseases of tomato leaves can seriously damage crop yield and financial rewards. The timely and accurate detection of tomato diseases is a major challenge in agriculture. Hence, the early and accurate diagnosis of tomato diseases is crucial. The emergence of deep learning has dramatically helped in plant disease detection. However, the accuracy of deep learning models largely depends on the quantity and quality of training data. To solve the inter-class imbalance problem and improve the generalization ability of the classification model, this paper proposes a cycle-consistent generative-adversarial-network-based Transformer model to generate diseased tomato leaf images for data augmentation. In addition, this paper uses a Transformer model and densely connected CNN architecture to extract multilevel local features. The Transformer module is utilized to capture global dependencies and contextual information accurately to expand the sensory field of the model. Experiments show that the proposed model achieved 99.45% accuracy on the PlantVillage dataset. The 2018 Artificial Intelligence Challenger dataset and the private dataset attained accuracies of 98.30% and 95.4%, and the proposed classification model achieved a higher accuracy and smaller model size compared to previous deep learning models. The classification model is generalizable and robust and can provide a stable theoretical framework for crop disease prevention and control.

Automatic segmentation of curtain wall frame using a context collaboration pyramid network

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an “encoder-decoder” architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module.

Semantic segmentation using cross-stage feature reweighting and efficient self-attention

Recently, vision transformers have demonstrated strong performance in various computer vision tasks. The success of ViTs can be attribute to the ability of capturing long-range dependencies. However, transformer-based approaches often yield segmentation maps with incomplete object structures because of restricted cross-stage information propagation and lack of low-level details. To address these problems, we introduce a CNN-transformer semantic segmentation architecture which adopts a CNN backbone for multi-level feature extraction and a transformer encoder that focuses on global perception learning. Transformer embeddings of all stages are integrated to compute feature weights for dynamic cross-stage feature reweighting. As a result, high-level semantic context and low-level spatial details can be embedded into each stage to preserve multi-level information. An efficient attention-based feature fusion mechanism is developed to combine reweighted transformer embeddings with CNN features to generate segmentation maps with more complete object structure. Different from regular self-attention that has quadratic computational complexity, our efficient self-attention method achieves similar performance with linear complexity. Experimental results on ADE20K and Cityscapes datasets show that the proposed segmentation approach demonstrates superior performance against most state-of-the-art networks.

Alzheimer's MRI Recognition Based on Convolutional Neural Networks

Alzheimer's disease (AD) is a progressively worsening neurodegenerative disorder, where early diagnosis is crucial for implementing effective disease management strategies. In multimodal diagnostic pathways, Magnetic Resonance Imaging (MRI) plays a pivotal role as a non-invasive imaging technique in disease identification and progression monitoring. Deep learning, particularly Convolutional Neural Networks (CNN), can extract key biomarkers from complex imaging data. By training CNN models to automatically interpret MRI scans, radiology experts can utilize these advanced analytical tools for more efficient and consistent pathological assessments, thereby enhancing clinical decision support systems. This study develops a deep learning-based method for the automatic classification of Alzheimer's brain MRI images, using convolutional neural networks to categorize MRI images into four stages: no dementia, very mild dementia, mild dementia, and moderate dementia. A CNN model is constructed, learning distinctive features of the images through multi-level feature extraction and performing feature map visualization. Early stopping is employed to prevent overfitting. The model is trained on a training set and evaluated on a test set, with performance metrics including confusion matrix, accuracy, precision, recall, F1 score, Kappa coefficient, Matthew’s coefficient, ROC curve with AUC value, and PQ curve with AP value. The results show that the proposed model effectively differentiates between various categories of MRI images, providing valuable tools for early diagnosis and condition monitoring.

MobileInst: Video Instance Segmentation on the Mobile

Video instance segmentation on mobile devices is an important yet very challenging edge AI problem. It mainly suffers from (1) heavy computation and memory costs for frame-by-frame pixel-level instance perception and (2) complicated heuristics for tracking objects. To address these issues, we present MobileInst, a lightweight and mobile-friendly framework for video instance segmentation on mobile devices. Firstly, MobileInst adopts a mobile vision transformer to extract multi-level semantic features and presents an efficient query-based dual-transformer instance decoder for mask kernels and a semantic-enhanced mask decoder to generate instance segmentation per frame. Secondly, MobileInst exploits simple yet effective kernel reuse and kernel association to track objects for video instance segmentation. Further, we propose temporal query passing to enhance the tracking ability for kernels. We conduct experiments on COCO and YouTube-VIS datasets to demonstrate the superiority of MobileInst and evaluate the inference latency on one single CPU core of the Snapdragon 778G Mobile Platform, without other methods of acceleration. On the COCO dataset, MobileInst achieves 31.2 mask AP and 433 ms on the mobile CPU, which reduces the latency by 50% compared to the previous SOTA. For video instance segmentation, MobileInst achieves 35.0 AP and 30.1 AP on YouTube-VIS 2019 & 2021.

Orthogonal Dictionary Guided Shape Completion Network for Point Cloud

Point cloud shape completion, which aims to reconstruct the missing regions of the incomplete point clouds with plausible shapes, is an ill-posed and challenging task that benefits many downstream 3D applications. Prior approaches achieve this goal by employing a two-stage completion framework, generating a coarse yet complete seed point cloud through an encoder-decoder network, followed by refinement and upsampling. However, the encoded features suffer from information loss of the missing portion, leading to an inability of the decoder to reconstruct seed points with detailed geometric clues. To tackle this issue, we propose a novel Orthogonal Dictionary Guided Shape Completion Network (ODGNet). The proposed ODGNet consists of a Seed Generation U-Net, which leverages multi-level feature extraction and concatenation to significantly enhance the representation capability of seed points, and Orthogonal Dictionaries that can learn shape priors from training samples and thus compensate for the information loss of the missing portions during inference. Our design is simple but to the point, extensive experiment results indicate that the proposed method can reconstruct point clouds with more details and outperform previous state-of-the-art counterparts. The implementation code is available at https://github.com/corecai163/ODGNet.

MFTN: Multi-Level Feature Transfer Network Based on MRI-Transformer for MR Image Super-resolution

Due to the unique environment and inherent properties of magnetic resonance imaging (MRI) instruments, MR images typically have lower resolution. Therefore, improving the resolution of MR images is beneficial for assisting doctors in diagnosing the condition. Currently, the existing MR image super-resolution (SR) methods still have the problem of insufficient detail reconstruction. To overcome this issue, this paper proposes a multi-level feature transfer network (MFTN) based on MRI-Transformer to realize SR of low-resolution MRI data. MFTN consists of a multi-scale feature reconstruction network (MFRN) and a multi-level feature extraction branch (MFEB). MFRN is constructed as a pyramid structure to gradually reconstruct image features at different scales by integrating the features obtained from MFEB, and MFEB is constructed to provide detail information at different scales for low resolution MR image SR reconstruction by constructing multiple MRI-Transformer modules. Each MRI-Transformer module is designed to learn the transfer features from the reference image by establishing feature correlations between the reference image and low-resolution MR image. In addition, a contrast learning constraint item is added to the loss function to enhance the texture details of the SR image. A large number of experiments show that our network can effectively reconstruct high-quality MR Images and achieves better performance compared to some state-of-the-art methods. The source code of this work will be released on GitHub.

Feature Optimization-Based Machine Learning Approach for Czech Land Cover Classification Using Sentinel-2 Images

The Czech Republic is one of the countries along the Belt and Road Initiative, and classifying land cover in the Czech Republic helps to understand the distribution of its forest resources, laying the foundation for forestry cooperation between China and the Czech Republic. This study aims to develop a practical approach for land cover classification in the Czech Republic, with the goal of efficiently acquiring spatial distribution information regarding its forest resources. This approach is based on multi-level feature extraction and selection, integrated with advanced machine learning or deep learning models. To accomplish this goal, the study concentrated on two typical experimental regions in the Czech Republic and conducted a series of classification experiments, using Sentinel-2 and DEM data in 2018 as the main data sources. Initially, this study extracted various features, including spectral, vegetation, and terrain features, from the study area, then assessed and selected key features based on their importance. Additionally, this study also explored multi-level spatial contextual features to improve classification performance. The extracted features include texture and morphological features, as well as deep semantic information learned by utilizing a deep learning model, 3D CNN. Finally, an AdaBoost ensemble learning model with the random forest as the base classifier is designed to produce land cover classification maps, thus obtaining the spatial distribution of forest resources. The experimental results demonstrate that feature optimization significantly enhances the extraction of high-quality features of surface objects, thereby improving classification performance. Specifically, morphological and texture features can effectively enhance the discriminability between different features of surface objects, thereby improving classification accuracy. Utilizing deep learning networks enables more efficient extraction of deep feature information, further enhancing classification accuracy. Moreover, employing an ensemble learning model effectively boosts the accuracy of the original classification results from different individual classifiers. Ultimately, the classification accuracy of the two experimental areas reaches 92.84% and 93.83%, respectively. The user accuracies for forests are 92.24% and 93.14%, while the producer accuracies are 97.71% and 97.02%. This study applies the proposed approach for nationwide classification in the Czech Republic, resulting in an overall classification accuracy of 90.98%, with forest user accuracy at 91.97% and producer accuracy at 96.2%. The results in this study demonstrate the feasibility of combining feature optimization with the 3D Convolutional Neural Network (3DCNN) model for land cover classification. This study can serve as a reference for research methods in deep learning for land cover classification, utilizing optimized features.

A hybrid approach for Android malware detection using improved multi-scale convolutional neural networks and residual networks

The open-source nature of Android, along with its coarse-grained permission management and widespread use, has heightened its vulnerability to malware threats. However, many traditional approaches face limitations in extracting comprehensive features, which hinders accurate and efficient Android malware detection. In this paper, we propose an effective hybrid approach combining an improved multi-scale convolutional neural network (MSCNN) with residual networks (ResNet) to defend against Android malware. The approach comprises an enhanced feature extraction network and a detection network. Initially, we introduce MSCNN, a novel deep learning model for comprehensive feature extraction. The MSCNN extends into three branches at distinct levels, concatenating features from low to high dimensions. This multilevel structure preserves rich semantic features while avoiding complex feature selection and analysis. Further, ResNet is employed as the detection network, and the hybrid models’ performance is evaluated by comparison with a single ResNet. Finally, we validate the effectiveness of our approach by comparing our experimental results with state-of-the-arts. The experimental results show our approach effectively detects Android malware with high accuracy (99.20%) and precision (99.49%), and utilizing MSCNN as a multilevel feature extraction network significantly enhances the performance of the hybrid models, particularly, the F1-Score of MSCNN+ResNet18 increases by 4.8%.

Bearing Fault Diagnosis Based on VMD-WPT-ViT for Wind Turbine

As an important part of wind turbine, the fault diagnosis of rolling bearings is helpful to maintain the stable operation of the fan. However, most of the existing fault diagnosis methods focus on one-dimensional time series data, which is not only difficult to extract multi-level feature information, but also fails to consider the noise problem in the signal. Therefore, to solve the above problems, a noise reduction algorithm combining variational mode decomposition (VMD) and wavelet packet threshold noise reduction (WPT) is proposed, and Vision-Transformer (ViT) model is combined to realize the fault diagnosis of rolling bearings of wind turbines. VMD is used to decompose the original signal and find out the noisy component. WPT is used to de-noise the component, and the de-noised component and other components are reconstructed to form a pure signal and converted into a two-dimensional image. Finally, a diagnostic model is built based on Vision-transformer. The experimental results show that the proposed method can effectively improve the diagnostic accuracy by extracting the advanced global features of the signal.

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Multi-scale object detection is critical for analyzing remote sensing images. Traditional feature pyramid networks, which are aimed at accommodating objects of varying sizes through multi-level feature extraction, face significant challenges due to the diverse scale variations present in remote sensing images. This situation often forces single-level features to span a broad spectrum of object sizes, complicating accurate localization and classification. To tackle these challenges, this paper proposes an innovative algorithm that incorporates an adaptive multi-scale feature enhancement and fusion module (ASEM), which enhances remote sensing image object detection through sophisticated multi-scale feature fusion. Our method begins by employing a feature pyramid to gather coarse multi-scale features. Subsequently, it integrates a fine-grained feature extraction module at each level, utilizing atrous convolutions with varied dilation rates to refine multi-scale features, which markedly improves the information capture from widely varied object scales. Furthermore, an adaptive enhancement module is applied to the features of each level by employing an attention mechanism for feature fusion. This strategy concentrates on the features of critical scale, which significantly enhance the effectiveness of capturing essential feature information. Compared with the baseline method, namely, Rotated FasterRCNN, our method achieved an mAP of 74.21% ( 0.81%) on the DOTA-v1.0 dataset and an mAP of 84.90% (+9.2%) on the HRSC2016 dataset. These results validated the effectiveness and practicality of our method and demonstrated its significant application value in multi-scale remote sensing object detection tasks.

ViT-UperNet: a hybrid vision transformer with unified-perceptual-parsing network for medical image segmentation

The existing image semantic segmentation models have low accuracy in detecting tiny targets or multi-targets at overlapping regions. This work proposes a hybrid vision transformer with unified-perceptual-parsing network (ViT-UperNet) for medical image segmentation. A self-attention mechanism is embedded in a vision transformer to extract multi-level features. The image features are extracted hierarchically from low to high dimensions using 4 groups of Transformer blocks with different numbers. Then, it uses a unified-perceptual-parsing network based on a feature pyramid network (FPN) and a pyramid pooling module (PPM) for the fusion of multi-scale contextual features and semantic segmentation. FPN can naturally use hierarchical features, and generate strong semantic information on all scales. PPM can better use the global prior knowledge to understand complex scenes, and extract features with global context information to improve segmentation results. In the training process, a scalable self-supervised learner named masked autoencoder is used for pre-training, which strengthens the visual representation ability and improves the efficiency of the feature learning. Experiments are conducted on cardiac magnetic resonance image segmentation where the left and right atrium and ventricle are selected for segmentation. The pixels accuracy is 93.85%, the Dice coefficient is 92.61% and Hausdorff distance is 11.16, which are improved compared with the other methods. The results show the superiority of Vit-UperNet in medical images segmentation, especially for the low-recognition and serious-occlusion targets.

Multi-level feature extraction and reconstruction for 3D MRI image super-resolution

Magnetic resonance imaging (MRI) is an essential radiology technique in clinical diagnosis, but its spatial resolution may not suffice to meet the growing need for precise diagnosis due to hardware limitations and thicker slice thickness. Therefore, it is crucial to explore suitable methods to increase the resolution of MRI images. Recently, deep learning has yielded many impressive results in MRI image super-resolution (SR) reconstruction. However, current SR networks mainly use convolutions to extract relatively single image features, which may not be optimal for further enhancing the quality of image reconstruction. In this work, we propose a multi-level feature extraction and reconstruction (MFER) method to restore the degraded high-resolution details of MRI images. Specifically, to comprehensively extract different types of features, we design the triple-mixed convolution by leveraging the strengths and uniqueness of different filter operations. For the features of each level, we then apply deconvolutions to upsample them separately at the tail of the network, followed by the feature calibration of spatial and channel attention. Besides, we also use a soft cross-scale residual operation to improve the effectiveness of parameter optimization. Experiments on lesion-free and glioma datasets indicate that our method obtains superior quantitative performance and visual effects when compared with state-of-the-art MRI image SR methods.

Multi-Level Feature Extraction Networks for Hyperspectral Image Classification

Hyperspectral image (HSI) classification plays a key role in the field of earth observation missions. Recently, transformer-based approaches have been widely used for HSI classification due to their ability to model long-range sequences. However, these methods face two main challenges. First, they treat HSI as linear vectors, disregarding their 3D attributes and spatial structure. Second, the repeated concatenation of encoders leads to information loss and gradient vanishing. To overcome these challenges, we propose a new solution called the multi-level feature extraction network (MLFEN). MLFEN consists of two sub-networks: the hybrid convolutional attention module (HCAM) and the enhanced dense vision transformer (EDVT). HCAM incorporates a band shift strategy to eliminate the edge effect of convolution and utilizes hybrid convolutional blocks to capture the 3D properties and spatial structure of HSI. Additionally, an attention module is introduced to identify strongly discriminative features. EDVT reconfigures the organization of original encoders by incorporating dense connections and adaptive feature fusion components, enabling faster propagation of information and mitigating the problem of gradient vanishing. Furthermore, we propose a novel sparse loss function to better fit the data distribution. Extensive experiments conducted on three public datasets demonstrate the significant advancements achieved by MLFEN.