Multi-level Feature Extraction Research Articles

Accurate segmentation of liver tumor from multi-modality non-contrast images using a dual-stream multi-level fusion framework

The use of multi-modality non-contrast images (i.e., T1FS, T2FS and DWI) for segmenting liver tumors provides a solution by eliminating the use of contrast agents and is crucial for clinical diagnosis. However, this remains a challenging task to discover the most useful information to fuse multi-modality images for accurate segmentation due to inter-modal interference. In this paper, we propose a dual-stream multi-level fusion framework (DM-FF) to, for the first time, accurately segment liver tumors from non-contrast multi-modality images directly. Our DM-FF first designs an attention-based encoder–decoder to effectively extract multi-level feature maps corresponding to a specified representation of each modality. Then, DM-FF creates two types of fusion modules, in which a module fuses learned features to obtain a shared representation across multi-modality images to exploit commonalities and improve the performance, and a module fuses the decision evidence of segment to discover differences between modalities to prevent interference caused by modality’s conflict. By integrating these three components, DM-FF enables multi-modality non-contrast images to cooperate with each other and enables an accurate segmentation. Evaluation on 250 patients including different types of tumors from two MRI scanners, DM-FF achieves a Dice of 81.20%, and improves performance (Dice by at least 11%) when comparing the eight state-of-the-art segmentation architectures. The results indicate that our DM-FF significantly promotes the development and deployment of non-contrast liver tumor technology.

A new dual-channel convolutional neural network and its application in rolling bearing fault diagnosis

Abstract Recently, deep learning has received widespread attention in the field of bearing fault diagnosis due to its powerful feature learning capability. However, when the actual working conditions are complex and variable, the fault information in a single domain is limited, making it difficult to achieve high accuracy. To overcome these challenges, this paper proposes a bearing fault diagnosis method based on the Markov transition field, continuous wavelet transform (CWT), and dual-channel convolutional neural network (CNN). The method combines the descriptive ability of the Markov model for state transfer, the time-frequency analysis ability of CWT for signal, and the excellent performance of CNN with attention mechanism in feature extraction and classification. Specifically, we first propose a multi-channel Markov transition field method, which is combined with CWT to obtain two different representations of two-dimensional (2D) images. To comprehensively mine fault information, we further propose a dual-channel CNN with an attention mechanism. The design of this network structure aims to extract multi-level features from two types of 2D images. At the same time, we designed and embedded an attention mechanism to enable the network to focus more on extracting effective features, thereby improving the performance and accuracy of the network. To verify the effectiveness of the proposed method, three datasets were used for empirical research. The results show that this method exhibits superior performance in bearing fault diagnosis and has higher accuracy compared to traditional methods.

SAM-CFFNet: SAM-Based Cross-Feature Fusion Network for Intelligent Identification of Landslides

Landslides are common hazardous geological events, and accurate and efficient landslide identification methods are important for hazard assessment and post-disaster response to geological disasters. Deep learning (DL) methods based on remote sensing data are currently widely used in landslide identification tasks. The recently proposed segment anything model (SAM) has shown strong generalization capabilities in zero-shot semantic segmentation. Nevertheless, SAM heavily relies on user-provided prompts, and performs poorly in identifying landslides on remote sensing images. In this study, we propose a SAM-based cross-feature fusion network (SAM-CFFNet) for the landslide identification task. The model utilizes SAM’s image encoder to extract multi-level features and our proposed cross-feature fusion decoder (CFFD) to generate high-precision segmentation results. The CFFD enhances landslide information through fine-tuning and cross-fusing multi-level features while leveraging a shallow feature extractor (SFE) to supplement texture details and improve recognition performance. SAM-CFFNet achieves high-precision landslide identification without the need for prompts while retaining SAM’s robust feature extraction capabilities. Experimental results on three open-source landslide datasets show that SAM-CFFNet outperformed other comparative models in terms of landslide identification accuracy and achieved an intersection over union (IoU) of 77.13%, 55.26%, and 73.87% on the three datasets, respectively. Our ablation studies confirm the effectiveness of each module designed in our model. Moreover, we validated the justification for our CFFD design through comparative analysis with diverse decoders. SAM-CFFNet achieves precise landslide identification using remote sensing images, demonstrating the potential application of the SAM-based model in geohazard analysis.

DSWIN: Automated hunger detection model based on hand-crafted decomposed shifted windows architecture using EEG signals

Hunger is a physiological state that arises from complex interactions of multiple factors, including higher brain center control. We purposed to develop an accurate and efficient machine-learning model for the automated detection of hunger using EEG signals. We prospectively acquired 14-channel EEG (sampling frequency 128 Hz) from 43 and 48 fasted and post-prandial healthy subjects (hungry vs. control groups, respectively) using the EMOTIV EPOC+ mobile brain cap system. To augment the hunger response, fasted subjects were also shown video images of food during EEG recording. The EEG signals were divided into 15-second segments. 877 and 852 participants/subjects were in the hungry and control groups. We created a novel handcrafted architecture—decomposed shifted window (DSWIN)—that combined swin patch division with tunable Q-factor wavelet transform-based signal decomposition for multilevel feature extraction of EEG signals. Textural and statistical features were extracted from the multiple patches and decomposed signals using a novel penta pattern-based extractor and statistical moments, respectively, and then merged. Iterative neighborhood component analysis (INCA) and iterative ReliefF (IRF) were applied. Twenty-eight selected feature vectors were generated, which were then fed to a shallow k-nearest neighbors (kNN) classifier to calculate channel-wise prediction vectors. From the 28 channel-wise prediction vectors, another 26 modes of function-based voted results were calculated using iterative hard majority voting, and the best overall model result was selected using a greedy algorithm. Our model attained 99.54% and 82.71% binary classification accuracies of hungry status vs. control using 10-fold and leave-one-subject-out cross-validations, respectively.

An adaptive detection approach for multi-scale defects on wind turbine blade surface

Multi-scale defects will inevitably appear in wind turbine blade defect detection within practical detection applications. An adaptive multi-scale detection approach is proposed to accurately classify and locate defects on the wind turbine blade surface. The proposed approach includes two main detection procedures: rough detection and precise detection. In the rough detection, the multi-level features extraction module with an adaptive bounding box proposal module is used to depict multi-scale defect regions and train a binary classifier to distinguish defects from non-defects. At the precise detection stage, three defect categories and four coordinates representing defect locations are obtained based on a multi-class defect classifier and regression of rough location boxes. The proposed method is evaluated on a real wind turbine blade surface defect dataset collected in a commercial wind farm and annotated manually. Results show that (1) the proposed model can detect the class of the blade multi-scale defects and outperforms other schemes with 96.89% mAP in the same model training epochs, (2) the positioning performance analysis of the model for multi-scale defects is conducted to validate the accuracy of the proposed model for multi-scale defect location.

Semantic aware-based instruction embedding for binary code similarity detection.

Binary code similarity detection plays a crucial role in various applications within binary security, including vulnerability detection, malicious software analysis, etc. However, existing methods suffer from limited differentiation in binary embedding representations across different compilation environments, lacking dynamic high-level semantics. Moreover, current approaches often neglect multi-level semantic feature extraction, thereby failing to acquire precise semantic information about the binary code. To address these limitations, this paper introduces a novel detection solution called BinBcla. This method employs an enhanced pre-training model to generate instruction embeddings with dynamic semantics for binary functions. Subsequently, multi-feature fusion technique is utilized to extract local semantic information and long-distance global features from the code, respectively, employing self-attention to comprehend the structure information of the code. Finally, an improved cosine similarity method is employed to learn relationships among all elements of the distance vectors, thereby enhancing the model's robustness to new sample functions. Experiments are conducted across different architectures, compilers, and optimization levels. The results indicate that BinBcla achieves higher accuracy, precision and F1 score compared to existing methods.

SAR Target Recognition Method based on Adaptive Weighted Decision Fusion of Deep Features

Background: This paper proposes a synthetic aperture radar (SAR) target recognition method based on adaptive weighted decision fusion of multi-level deep features. Methods: The trained ResNet-18 is employed to extract multi-level deep features from SAR images. Afterwards, based on the joint sparse representation (JSR) model, the multi-level deep features are represented to obtain the corresponding reconstruction error vectors. Considering the differences in the abilities of different levels of features to distinguish the target, the reconstruction error vectors are analyzed based on entropy theory, and their corresponding weights are adaptively obtained. Finally, the fused reconstruction error result is obtained through adaptively weighted fusion, and the target label is determined accordingly. Results: Experiments are conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset under different conditions, and the proposed method is compared with published methods, including multi-feature decision fusion, JSR-based decision fusion and other types of ResNets. Conclusion: The experimental results under standard operating condition (SOC) and extended operating conditions (EOCs) including depression angle variance and noise corruption validate the advantages of the proposed method.

Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation

Accurate segmentation of medical images is vital for disease detection and treatment. Convolutional Neural Networks (CNN) and Transformer models are widely used in medical image segmentation due to their exceptional capabilities in image recognition and segmentation. However, CNNs often lack an understanding of the global context and may lose spatial details of the target, while Transformers struggle with local information processing, leading to reduced geometric detail of the target. To address these issues, this research presents a Global-Local Fusion network model (GLFUnet) based on the U-Net framework and attention mechanisms. The model employs a dual-branch network that utilizes ConvNeXt and Swin Transformer to simultaneously extract multi-level features from pathological images. It enhances ConvNeXt’s local feature extraction with spatial and global attention up-sampling modules, while improving Swin Transformer’s global context dependency with channel attention. The Attention Feature Fusion module and skip connections efficiently merge local detailed and global coarse features from CNN and Transformer branches at various scales. The fused features are then progressively restored to the original image resolution for pixel-level prediction. Comprehensive experiments on datasets of stomach and liver cancer demonstrate GLFUnet’s superior performance and adaptability in medical image segmentation, holding promise for clinical analysis and disease diagnosis.

Zero-Shot Sketch-Based Remote-Sensing Image Retrieval Based on Multi-Level and Attention-Guided Tokenization

Effectively and efficiently retrieving images from remote-sensing databases is a critical challenge in the realm of remote-sensing big data. Utilizing hand-drawn sketches as retrieval inputs offers intuitive and user-friendly advantages, yet the potential of multi-level feature integration from sketches remains underexplored, leading to suboptimal retrieval performance. To address this gap, our study introduces a novel zero-shot, sketch-based retrieval method for remote-sensing images, leveraging multi-level feature extraction, self-attention-guided tokenization and filtering, and cross-modality attention update. This approach employs only vision information and does not require semantic knowledge concerning the sketch and image. It starts by employing multi-level self-attention guided feature extraction to tokenize the query sketches, as well as self-attention feature extraction to tokenize the candidate images. It then employs cross-attention mechanisms to establish token correspondence between these two modalities, facilitating the computation of sketch-to-image similarity. Our method significantly outperforms existing sketch-based remote-sensing image retrieval techniques, as evidenced by tests on multiple datasets. Notably, it also exhibits robust zero-shot learning capabilities in handling unseen categories and strong domain adaptation capabilities in handling unseen novel remote-sensing data. The method’s scalability can be further enhanced by the pre-calculation of retrieval tokens for all candidate images in a database. This research underscores the significant potential of multi-level, attention-guided tokenization in cross-modal remote-sensing image retrieval. For broader accessibility and research facilitation, we have made the code and dataset used in this study publicly available online.

Debris Flow Susceptibility Evaluation Based on Multi-level Feature Extraction CNN Model: A Case Study of Nujiang Prefecture, China

Debris Flow Susceptibility Evaluation Based on Multi-level Feature Extraction CNN Model: A Case Study of Nujiang Prefecture, China

Lattice 123 pattern for automated Alzheimer’s detection using EEG signal

AbstractThis paper presents an innovative feature engineering framework based on lattice structures for the automated identification of Alzheimer's disease (AD) using electroencephalogram (EEG) signals. Inspired by the Shannon information entropy theorem, we apply a probabilistic function to create the novel Lattice123 pattern, generating two directed graphs with minimum and maximum distance-based kernels. Using these graphs and three kernel functions (signum, upper ternary, and lower ternary), we generate six feature vectors for each input signal block to extract textural features. Multilevel discrete wavelet transform (MDWT) was used to generate low-level wavelet subbands. Our proposed model mirrors deep learning approaches, facilitating feature extraction in frequency and spatial domains at various levels. We used iterative neighborhood component analysis to select the most discriminative features from the extracted vectors. An iterative hard majority voting and a greedy algorithm were used to generate voted vectors to select the optimal channel-wise and overall results. Our proposed model yielded a classification accuracy of more than 98% and a geometric mean of more than 96%. Our proposed Lattice123 pattern, dynamic graph generation, and MDWT-based multilevel feature extraction can detect AD accurately as the proposed pattern can extract subtle changes from the EEG signal accurately. Our prototype is ready to be validated using a large and diverse database.

Development of Smart Agriculture to detect the Arabica Coffee Leaf Disease using IAFSA based MSAB with Channel and Spatial Attention Network

Plant diseases provide challenges for the agriculture sector, notably to produce Arabica coffee. Recognising issues on Arabica coffee leaves is a first step in avoiding and curing illnesses to prevent crop loss. With the extraordinary advancements achieved in convolutional neural networks (CNN) in recent years, Arabica coffee leaf damage can now be identified without the aid of a specialist. However, the local characteristics that convolutional layers in CNNs record are typically redundant and unable to make efficient use of global data to support the prediction process. The proposed Hybrid Attention UNet, also known as CMSAMB-UNet due to its feature extraction and global modelling capabilities, integrates both the Channel and Spatial Attention Module (CSAM) as well as the Multi-head Self-Attention Block (MSAB). In this study, CMSAMB-UNet is built on Resnet50 to extract multi-level features from plant picture data. Two shallow layers of feature maps are used with CSAM according to local attention. used throughout the feature extraction process to enrich the features and adaptively disregard unwanted features. In order to recreate the spatial feature connection of the input pictures using high-resolution feature maps, two global attention maps produced by MSAB are combined.

Using a Hybrid Convolutional Neural Network with a Transformer Model for Tomato Leaf Disease Detection

Diseases of tomato leaves can seriously damage crop yield and financial rewards. The timely and accurate detection of tomato diseases is a major challenge in agriculture. Hence, the early and accurate diagnosis of tomato diseases is crucial. The emergence of deep learning has dramatically helped in plant disease detection. However, the accuracy of deep learning models largely depends on the quantity and quality of training data. To solve the inter-class imbalance problem and improve the generalization ability of the classification model, this paper proposes a cycle-consistent generative-adversarial-network-based Transformer model to generate diseased tomato leaf images for data augmentation. In addition, this paper uses a Transformer model and densely connected CNN architecture to extract multilevel local features. The Transformer module is utilized to capture global dependencies and contextual information accurately to expand the sensory field of the model. Experiments show that the proposed model achieved 99.45% accuracy on the PlantVillage dataset. The 2018 Artificial Intelligence Challenger dataset and the private dataset attained accuracies of 98.30% and 95.4%, and the proposed classification model achieved a higher accuracy and smaller model size compared to previous deep learning models. The classification model is generalizable and robust and can provide a stable theoretical framework for crop disease prevention and control.

Automatic segmentation of curtain wall frame using a context collaboration pyramid network

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an “encoder-decoder” architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module.

Semantic segmentation using cross-stage feature reweighting and efficient self-attention

Recently, vision transformers have demonstrated strong performance in various computer vision tasks. The success of ViTs can be attribute to the ability of capturing long-range dependencies. However, transformer-based approaches often yield segmentation maps with incomplete object structures because of restricted cross-stage information propagation and lack of low-level details. To address these problems, we introduce a CNN-transformer semantic segmentation architecture which adopts a CNN backbone for multi-level feature extraction and a transformer encoder that focuses on global perception learning. Transformer embeddings of all stages are integrated to compute feature weights for dynamic cross-stage feature reweighting. As a result, high-level semantic context and low-level spatial details can be embedded into each stage to preserve multi-level information. An efficient attention-based feature fusion mechanism is developed to combine reweighted transformer embeddings with CNN features to generate segmentation maps with more complete object structure. Different from regular self-attention that has quadratic computational complexity, our efficient self-attention method achieves similar performance with linear complexity. Experimental results on ADE20K and Cityscapes datasets show that the proposed segmentation approach demonstrates superior performance against most state-of-the-art networks.

Alzheimer's MRI Recognition Based on Convolutional Neural Networks

Alzheimer's disease (AD) is a progressively worsening neurodegenerative disorder, where early diagnosis is crucial for implementing effective disease management strategies. In multimodal diagnostic pathways, Magnetic Resonance Imaging (MRI) plays a pivotal role as a non-invasive imaging technique in disease identification and progression monitoring. Deep learning, particularly Convolutional Neural Networks (CNN), can extract key biomarkers from complex imaging data. By training CNN models to automatically interpret MRI scans, radiology experts can utilize these advanced analytical tools for more efficient and consistent pathological assessments, thereby enhancing clinical decision support systems. This study develops a deep learning-based method for the automatic classification of Alzheimer's brain MRI images, using convolutional neural networks to categorize MRI images into four stages: no dementia, very mild dementia, mild dementia, and moderate dementia. A CNN model is constructed, learning distinctive features of the images through multi-level feature extraction and performing feature map visualization. Early stopping is employed to prevent overfitting. The model is trained on a training set and evaluated on a test set, with performance metrics including confusion matrix, accuracy, precision, recall, F1 score, Kappa coefficient, Matthew’s coefficient, ROC curve with AUC value, and PQ curve with AP value. The results show that the proposed model effectively differentiates between various categories of MRI images, providing valuable tools for early diagnosis and condition monitoring.

MobileInst: Video Instance Segmentation on the Mobile

Video instance segmentation on mobile devices is an important yet very challenging edge AI problem. It mainly suffers from (1) heavy computation and memory costs for frame-by-frame pixel-level instance perception and (2) complicated heuristics for tracking objects. To address these issues, we present MobileInst, a lightweight and mobile-friendly framework for video instance segmentation on mobile devices. Firstly, MobileInst adopts a mobile vision transformer to extract multi-level semantic features and presents an efficient query-based dual-transformer instance decoder for mask kernels and a semantic-enhanced mask decoder to generate instance segmentation per frame. Secondly, MobileInst exploits simple yet effective kernel reuse and kernel association to track objects for video instance segmentation. Further, we propose temporal query passing to enhance the tracking ability for kernels. We conduct experiments on COCO and YouTube-VIS datasets to demonstrate the superiority of MobileInst and evaluate the inference latency on one single CPU core of the Snapdragon 778G Mobile Platform, without other methods of acceleration. On the COCO dataset, MobileInst achieves 31.2 mask AP and 433 ms on the mobile CPU, which reduces the latency by 50% compared to the previous SOTA. For video instance segmentation, MobileInst achieves 35.0 AP and 30.1 AP on YouTube-VIS 2019 & 2021.

MFTN: Multi-Level Feature Transfer Network Based on MRI-Transformer for MR Image Super-resolution

Due to the unique environment and inherent properties of magnetic resonance imaging (MRI) instruments, MR images typically have lower resolution. Therefore, improving the resolution of MR images is beneficial for assisting doctors in diagnosing the condition. Currently, the existing MR image super-resolution (SR) methods still have the problem of insufficient detail reconstruction. To overcome this issue, this paper proposes a multi-level feature transfer network (MFTN) based on MRI-Transformer to realize SR of low-resolution MRI data. MFTN consists of a multi-scale feature reconstruction network (MFRN) and a multi-level feature extraction branch (MFEB). MFRN is constructed as a pyramid structure to gradually reconstruct image features at different scales by integrating the features obtained from MFEB, and MFEB is constructed to provide detail information at different scales for low resolution MR image SR reconstruction by constructing multiple MRI-Transformer modules. Each MRI-Transformer module is designed to learn the transfer features from the reference image by establishing feature correlations between the reference image and low-resolution MR image. In addition, a contrast learning constraint item is added to the loss function to enhance the texture details of the SR image. A large number of experiments show that our network can effectively reconstruct high-quality MR Images and achieves better performance compared to some state-of-the-art methods. The source code of this work will be released on GitHub.

Orthogonal Dictionary Guided Shape Completion Network for Point Cloud

Point cloud shape completion, which aims to reconstruct the missing regions of the incomplete point clouds with plausible shapes, is an ill-posed and challenging task that benefits many downstream 3D applications. Prior approaches achieve this goal by employing a two-stage completion framework, generating a coarse yet complete seed point cloud through an encoder-decoder network, followed by refinement and upsampling. However, the encoded features suffer from information loss of the missing portion, leading to an inability of the decoder to reconstruct seed points with detailed geometric clues. To tackle this issue, we propose a novel Orthogonal Dictionary Guided Shape Completion Network (ODGNet). The proposed ODGNet consists of a Seed Generation U-Net, which leverages multi-level feature extraction and concatenation to significantly enhance the representation capability of seed points, and Orthogonal Dictionaries that can learn shape priors from training samples and thus compensate for the information loss of the missing portions during inference. Our design is simple but to the point, extensive experiment results indicate that the proposed method can reconstruct point clouds with more details and outperform previous state-of-the-art counterparts. The implementation code is available at https://github.com/corecai163/ODGNet.

Feature Optimization-Based Machine Learning Approach for Czech Land Cover Classification Using Sentinel-2 Images

The Czech Republic is one of the countries along the Belt and Road Initiative, and classifying land cover in the Czech Republic helps to understand the distribution of its forest resources, laying the foundation for forestry cooperation between China and the Czech Republic. This study aims to develop a practical approach for land cover classification in the Czech Republic, with the goal of efficiently acquiring spatial distribution information regarding its forest resources. This approach is based on multi-level feature extraction and selection, integrated with advanced machine learning or deep learning models. To accomplish this goal, the study concentrated on two typical experimental regions in the Czech Republic and conducted a series of classification experiments, using Sentinel-2 and DEM data in 2018 as the main data sources. Initially, this study extracted various features, including spectral, vegetation, and terrain features, from the study area, then assessed and selected key features based on their importance. Additionally, this study also explored multi-level spatial contextual features to improve classification performance. The extracted features include texture and morphological features, as well as deep semantic information learned by utilizing a deep learning model, 3D CNN. Finally, an AdaBoost ensemble learning model with the random forest as the base classifier is designed to produce land cover classification maps, thus obtaining the spatial distribution of forest resources. The experimental results demonstrate that feature optimization significantly enhances the extraction of high-quality features of surface objects, thereby improving classification performance. Specifically, morphological and texture features can effectively enhance the discriminability between different features of surface objects, thereby improving classification accuracy. Utilizing deep learning networks enables more efficient extraction of deep feature information, further enhancing classification accuracy. Moreover, employing an ensemble learning model effectively boosts the accuracy of the original classification results from different individual classifiers. Ultimately, the classification accuracy of the two experimental areas reaches 92.84% and 93.83%, respectively. The user accuracies for forests are 92.24% and 93.14%, while the producer accuracies are 97.71% and 97.02%. This study applies the proposed approach for nationwide classification in the Czech Republic, resulting in an overall classification accuracy of 90.98%, with forest user accuracy at 91.97% and producer accuracy at 96.2%. The results in this study demonstrate the feasibility of combining feature optimization with the 3D Convolutional Neural Network (3DCNN) model for land cover classification. This study can serve as a reference for research methods in deep learning for land cover classification, utilizing optimized features.