Multi-level Features Research Articles

Do the best of all together: Hierarchical spatial-frequency fusion transformers for animal re-identification

Animal re-identification is fundamental in the ecology and ethology community for understanding the Earth’s ecosystems. Due to the uncertainty of photographing animals in the wild, such as the position and angle of capture and the variations in animal individuals as well as in their poses and habitats, building image-based visual models for re-identifying animal individuals is a challenging task. We propose a novel framework, hierarchical spatial-frequency fusion transformers, to address animal re-identification. We first use spatial and frequency representation learning to capture effective deep features and then employ hierarchical transformer-based representation learning to consolidate low-level detailed information into high-level semantic information from a global perspective. Through this process, we construct a deeply supervised nonlinear aggregation method to enhance finer multi-scale, multi-level and cross-domain features. Our method illustrates the case of “do the best of all together”, meaning that only when both the frequency and spatial domains are combined can we achieve the best performance. The experimental results demonstrate that our approach achieves significantly higher performance than other state-of-the-art methods.

Speech self-supervised representations benchmarking: A case for larger probing heads

Self-supervised learning (SSL) leverages large datasets of unlabeled speech to reach impressive performance with reduced amounts of annotated data. The high number of proposed approaches fostered the emergence of comprehensive benchmarks that evaluate their performance on a set of downstream tasks exploring various aspects of the speech signal. However, while the number of considered tasks has been growing, most proposals rely upon a single downstream architecture that maps the frozen SSL representations to the task labels. This study examines how benchmarking results are affected by changes in the probing head architecture. Interestingly, we found that altering the downstream architecture structure leads to significant fluctuations in the performance ranking of the evaluated models. Against common practices in speech SSL benchmarking, we evaluate larger-capacity probing heads, showing their impact on performance, inference costs, generalization, and multi-level feature exploitation.

Boundary-Aware Gradient Operator Network for Medical Image Segmentation.

Medical image segmentation is a crucial task in computer-aided diagnosis. Although convolutional neural networks (CNNs) have made significant progress in the field of medical image segmentation, the convolution kernels of CNNs are optimized from random initialization without explicitly encoding gradient information, leading to a lack of specificity for certain features, such as blurred boundary features. Furthermore, the frequently applied down-sampling operation also loses the fine structural features in shallow layers. Therefore, we propose a boundary-aware gradient operator network (BG-Net) for medical image segmentation, in which the gradient convolution (GConv) and the boundary-aware mechanism (BAM) modules are developed to simulate image boundary features and the remote dependencies between channels. The GConv module transforms the gradient operator into a convolutional operation that can extract gradient features; it attempts to extract more features such as images boundaries and textures, thereby fully utilizing limited input to capture more features representing boundaries. In addition, the BAM can increase the amount of global contextual information while suppressing invalid information by focusing on feature dependencies and the weight ratios between channels. Thus, the boundary perception ability of BG-Net is improved. Finally, we use a multi-modal fusion mechanism to effectively fuse lightweight gradient convolution and U-shaped branch features into a multilevel feature, enabling global dependencies and low-level spatial details to be effectively captured in a shallower manner. We conduct extensive experiments on eight datasets that broadly cover medical images to evaluate the effectiveness of the proposed BG-Net. The experimental results demonstrate that BG-Net outperforms the state-of-the-art methods, particularly those focused on boundary segmentation.

Protein multi-level structure feature-integrated deep learning method for mutational effect prediction.

Through iterative rounds of mutation and selection, proteins can be engineered to enhance their desired biological functions. Nevertheless, identifying optimal mutation sites for directed evolution remains challenging due to the vastness of the protein sequence landscape and the epistatic mutational effects across residues. To address this challenge, we introduce MLSmut, a deep learning-based approach that leverages multi-level structural features of proteins. MLSmut extracts salient information from protein co-evolution, sequence semantics, and geometric features to predict the mutational effect. Extensive benchmark evaluations on 10 single-site and two multi-site deep mutation scanning datasets demonstrate that MLSmut surpasses existing methods in predicting mutational outcomes. To overcome the limited training data availability, we employ a two-stage training strategy: initial coarse-tuning on a large corpus of unlabeled protein data followed by fine-tuning on a curated dataset of 40-100 experimental measurements. This approach enables our model to achieve satisfactory performance on downstream protein prediction tasks. Importantly, our model holds the potential to predict the mutational effects of any protein sequence. Collectively, these findings suggest that our approach can substantially reduce the reliance on laborious wet lab experiments and deepen our understanding of the intricate relationships between mutations and protein function.

Non-local feature aggregation quaternion network for single image deraining

The existing deraining methods are based on convolutional neural networks (CNN) learning the mapping relationship between rainy and clean images. However, the real-valued CNN processes the color images as three independent channels separately, which fails to fully leverage color information. Additionally, sliding-window-based neural networks cannot effectively model the non-local characteristics of an image. In this work, we proposed a non-local feature aggregation quaternion network (NLAQNet), which is composed of two concurrent sub-networks: the Quaternion Local Detail Repair Network (QLDRNet) and the Multi-Level Feature Aggregation Network (MLFANet). Furthermore, in the subnetwork of QLDRNet, the Local Detail Repair Block (LDRB) is proposed to repair the backdrop of an image that has not been damaged by rain streaks. Finally, within the MLFANet subnetwork, we have introduced two specialized blocks, namely the Non-Local Feature Aggregation Block (NLAB) and the Feature Aggregation Block (Mix), specifically designed to address the restoration of rain-streak-damaged image backgrounds. Extensive experiments demonstrate that the proposed network delivers strong performance in both qualitative and quantitative evaluations on existing datasets. The code is available at https://github.com/xionggonghe/NLAQNet.

Research on the Application of Deep Learning in Human Spinal Image Segmentation

Abstract Traditional segmentation methods can only segment grayscale images, which limits their application; The segmentation process often depends on the doctor’s experience, which can lead to subjective factors affecting the results; Therefore, the accuracy and efficiency of segmentation are difficult to achieve practical application results. The deep learning model is a structural model that mimics the neural connections within the human brain. The deep learning model can accurately extract multi-level features of key information in images from low-level to high-level, and provide feedback on data interpretation, thereby achieving accurate and efficient image segmentation results. Introducing deep learning algorithms into medical image segmentation can accurately express the key information at a deeper level in spinal images, achieving better image segmentation results.

Colorectal cancer image recognition algorithm based on improved transformer

Aiming at the problems of the complex background of colorectal cancer tissue cell images and the difficulty of detection caused by the low differentiation of cancer cell regions, a deep learning method is used to detect the cancer cell regions. By integrating the skip feedback connection structure into U-Net and combining it with the Swin Transformer for feature extraction, we improve the multi-level feature extraction capabilities of the model. This algorithm enables end-to-end recognition of colorectal adenocarcinoma tissue images and achieves an accuracy of 95.8% on the NCT-CRC-HE-100K dataset, demonstrating its potential to significantly support colorectal cancer detection and treatment.

A Privacy-Preserving AIoT Framework for Fall Detection and Classification Using Hierarchical Learning With Multilevel Feature Fusion

A Privacy-Preserving AIoT Framework for Fall Detection and Classification Using Hierarchical Learning With Multilevel Feature Fusion

FRAP 2.0: a community engagement method applied to cervical cancer disparities among Hispanic women

focused Rapid Assessment Process (fRAP) 2.0 is a community engagement approach combining geospatial mapping with rapid qualitative assessment in cyclical fashion within communities to capture multifactorial and multilevel features impacting...

MPE-YOLO: enhanced small target detection in aerial imaging

Aerial image target detection is essential for urban planning, traffic monitoring, and disaster assessment. However, existing detection algorithms struggle with small target recognition and accuracy in complex environments. To address this issue, this paper proposes an improved model based on YOLOv8, named MPE-YOLO. Initially, a multilevel feature integrator (MFI) module is employed to enhance the representation of small target features, which meticulously moderates information loss during the feature fusion process. For the backbone network of the model, a perception enhancement convolution (PEC) module is introduced to replace traditional convolutional layers, thereby expanding the network’s fine-grained feature processing capability. Furthermore, an enhanced scope-C2f (ES-C2f) module is designed, utilizing channel expansion and stacking of multiscale convolutional kernels to enhance the network’s ability to capture small target details. After a series of experiments on the VisDrone, RSOD, and AI-TOD datasets, the model has not only demonstrated superior performance in aerial image detection tasks compared to existing advanced algorithms but also achieved a lightweight model structure. The experimental results demonstrate the potential of MPE-YOLO in enhancing the accuracy and operational efficiency of aerial target detection. Code will be available online (https://github.com/zhanderen/MPE-YOLO).

Predicting Surgical Outcomes Using Wearables

Accurate prediction of readmission and severe complication risks during the pre-operative stage is crucial for clinical decision-making. In addition to traditional static clinical data, the activity and physiological data collected by commercial wearable devices such as Fitbit wristbands provide valuable complementary information. However, data from wearables are often incomplete and noisy, presenting challenges in extracting clinically meaningful features from fine-grained time series data. In this paper, we propose a pipeline to develop predictive models using pre-operative wearable data to predict surgical outcomes in patients undergoing pancreatic surgeries. The core of this pipeline is a multi-level feature engineering framework designed to extract high-level features from wearable data, which are then combined with static clinical characteristics to form the input for the predictive model. In a clinical study involving 61 patients, our approach improved the Area Under the Receiver Operating Characteristic (AUROC) by 44% compared to the existing risk calculator employed in clinical practice.

A radiomics approach for predicting gait freezing in Parkinson's disease based on resting-state functional magnetic resonance imaging indices: a cross-sectional study.

Freezing of gait is a significant and debilitating motor symptom often observed in individuals with Parkinson's disease. Resting-state functional magnetic resonance imaging, along with its multi-level feature indices, has provided a fresh perspective and valuable insight into the study of freezing of gait in Parkinson's disease. It has been revealed that Parkinson's disease is accompanied by widespread irregularities in inherent brain network activity. However, the effective integration of the multi-level indices of resting-state functional magnetic resonance imaging into clinical settings for the diagnosis of freezing of gait in Parkinson's disease remains a challenge. Although previous studies have demonstrated that radiomics can extract optimal features as biomarkers to identify or predict diseases, a knowledge gap still exists in the field of freezing of gait in Parkinson's disease. This cross-sectional study aimed to evaluate the ability of radiomics features based on multi-level indices of resting-state functional magnetic resonance imaging, along with clinical features, to distinguish between Parkinson's disease patients with and without freezing of gait. We recruited 28 patients with Parkinson's disease who had freezing of gait (15 men and 13 women, average age 63 years) and 30 patients with Parkinson's disease who had no freezing of gait (16 men and 14 women, average age 64 years). Magnetic resonance imaging scans were obtained using a 3.0T scanner to extract the mean amplitude of low-frequency fluctuations, mean regional homogeneity, and degree centrality. Neurological and clinical characteristics were also evaluated. We used the least absolute shrinkage and selection operator algorithm to extract features and established feedforward neural network models based solely on resting-state functional magnetic resonance imaging indicators. We then performed predictive analysis of three distinct groups based on resting-state functional magnetic resonance imaging indicators indicators combined with clinical features. Subsequently, we conducted 100 additional five-fold cross-validations to determine the most effective model for each classification task and evaluated the performance of the model using the area under the receiver operating characteristic curve. The results showed that when differentiating patients with Parkinson's disease who had freezing of gait from those who did not have freezing of gait, or from healthy controls, the models using only the mean regional homogeneity values achieved the highest area under the receiver operating characteristic curve values of 0.750 (with an accuracy of 70.9%) and 0.759 (with an accuracy of 65.3%), respectively. When classifying patients with Parkinson's disease who had freezing of gait from those who had no freezing of gait, the model using the mean amplitude of low-frequency fluctuation values combined with two clinical features achieved the highest area under the receiver operating characteristic curve of 0.847 (with an accuracy of 74.3%). The most significant features for patients with Parkinson's disease who had freezing of gait were amplitude of low-frequency fluctuation alterations in the left parahippocampal gyrus and two clinical characteristics: Montreal Cognitive Assessment and Hamilton Depression Scale scores. Our findings suggest that radiomics features derived from resting-state functional magnetic resonance imaging indices and clinical information can serve as valuable indices for the identification of freezing of gait in Parkinson's disease.

Сопоставление разноуровневых признаковых пространств романов Р. Брэдбери «Вино из одуванчиков» и «Лето, прощай»

The aim of the study is to determine the similarities and differences of the multilevel semantic spaces formed by the signs of English verbal nouns in the works of Ray Bradbury "Dandelion Wine" and "Farewell Summer". The author provides a definition of the term "feature space", identifies three parameters of variability (feature, level, cluster variability of levels) for a comparative analysis of the feature spaces of the studied works. The article examines the signs of five linguistic levels: phonetic, morphological, diachronic, etymological and semantic. The scientific novelty of the study lies in the fact that in this work the feature space is considered as a certain set of multi-level features related by statistically relevant relationships. In this context, it is first demonstrated in relation to works of art. The results revealed that the novel "Dandelion Wine" contains a higher number of English verbal nouns, with the multilevel characteristics of these nouns forming fewer clusters. On the other hand, the novel "Farewell Summer" contains twice as few English lexical units, but with more clusters. Thus, we can state that the "Farewell Summer" feature space is significantly more differentiated and, accordingly, there will be much more oppositions between the features.

CycPeptMP: enhancing membrane permeability prediction of cyclic peptides with multi-level molecular features and data augmentation.

Cyclic peptides are versatile therapeutic agents that boast high binding affinity, minimal toxicity, and the potential to engage challenging protein targets. However, the pharmaceutical utility of cyclic peptides is limited by their low membrane permeability-an essential indicator of oral bioavailability and intracellular targeting. Current machine learning-based models of cyclic peptide permeability show variable performance owing to the limitations of experimental data. Furthermore, these methods use features derived from the whole molecule that have traditionally been used to predict small molecules and ignore the unique structural properties of cyclic peptides. This study presents CycPeptMP: an accurate and efficient method to predict cyclic peptide membrane permeability. We designed features for cyclic peptides at the atom-, monomer-, and peptide-levels and seamlessly integrated these into a fusion model using deep learning technology. Additionally, we applied various data augmentation techniques to enhance model training efficiency using the latest data. The fusion model exhibited excellent prediction performance for the logarithm of permeability, with a mean absolute error of $0.355$ and correlation coefficient of $0.883$. Ablation studies demonstrated that all feature levels contributed and were relatively essential to predicting membrane permeability, confirming the effectiveness of augmentation to improve prediction accuracy. A comparison with a molecular dynamics-based method showed that CycPeptMP accurately predicted peptide permeability, which is otherwise difficult to predict using simulations.

Multi-level temporal feature fusion with feature exchange strategy for multiple object tracking

Multi-level temporal feature fusion with feature exchange strategy for multiple object tracking

IDSSI: Image Deturbulence with Semantic and Spatial–Temporal Information

The propagation of light through turbulent media results in refractive index fluctuations, causing distortion and blurring of images captured by imaging equipment. Consequently, it is of paramount importance to mitigate turbulence effects. Deep convolutional neural networks incorporating attention mechanisms have demonstrated remarkable success in dynamic video restoration. However, in most networks, the attention mechanism is limited to capturing simple contextual features, failing to adequately exploit multi-level image features. This paper introduces IDSSI, an effective model that utilizes semantic and spatio–temporal information for turbulence mitigation. A novel two-branch feature extraction structure is proposed, capable of extracting multi-scale enhanced features with semantic information under limited supervision. These perceived semantic features are subsequently fused into global features, enhancing multi-scale perception for turbulence repair. Furthermore, a new Spatial–Temporal feature learning strategy is proposed. This approach facilitates the extraction and modulation of temporal information by obtaining edge cues, effectively concatenating and merging with spatial features. This strategy serves as an efficient alternative to 3D convolution. Experimental results on relevant datasets demonstrate the superiority of the proposed model over current state-of-the-art turbulence repair methods.

MGB-Unet: An Improved Multiscale Unet with Bottleneck Transformer for Myositis Segmentation from Ultrasound Images.

Myositis is the inflammation of the muscles that can arise from various sources with diverse symptoms and require different treatments. For treatment to achieve optimal results, it is essential to obtain an accurate diagnosis promptly. This paper presents a new supervised segmentation architecture that can efficiently perform precise segmentation and classification of myositis from ultrasound images with few computational resources. The architecture of our model includes a unique encoder-decoder structure that integrates the Bottleneck Transformer (BOT) with a newly developed Residual block named Multi-Conv Ghost switchable bottleneck Residual Block (MCG_RB). This block effectively captures and analyzes ultrasound image input inside the encoder segment at several resolutions. Moreover, the BOT module is a transformer-style attention module designed to bridge the feature gap between the encoding and decoding stages. Furthermore, multi-level features are retrieved using the MCG-RB module, which combines multi-convolution with ghost switchable residual connections of convolutions for both the encoding and decoding stages. The suggested method attains state-of-the-art performance on a benchmark set of myositis ultrasound images across all parameters, including accuracy, precision, recall, dice coefficient, and Jaccard index. Despite its limited training data, the suggested approach demonstrates remarkable generalizability by yielding exceptional results. The proposed model showed a substantial enhancement in accuracy when compared to segmentation state-of-the-art methods such as Unet++, DeepLabV3, and the Duck-Net. The dice coefficient and Jaccard index obtained improvements of up to 3%, 6%, and 7%, respectively, surpassing the other methods.

A lightweight Color-changing melon ripeness detection algorithm based on model pruning and knowledge distillation: leveraging dilated residual and multi-screening path aggregation.

Color-changing melons are a kind of cucurbit plant that combines ornamental and food. With the aim of increasing the efficiency of harvesting Color-changing melon fruits while reducing the deployment cost of detection models on agricultural equipment, this study presents an improved YOLOv8s network approach that uses model pruning and knowledge distillation techniques. The method first merges Dilated Wise Residual (DWR) and Dilated Reparam Block (DRB) to reconstruct the C2f module in the Backbone for better feature fusion. Next, we designed a multilevel scale fusion feature pyramid network (HS-PAN) to enrich semantic information and strengthen localization information to enhance the detection of Color-changing melon fruits with different maturity levels. Finally, we used Layer-Adaptive Sparsity Pruning and Block-Correlation Knowledge Distillation to simplify the model and recover its accuracy. In the Color-changing melon images dataset, the mAP0.5 of the improved model reaches 96.1%, the detection speed is 9.1% faster than YOLOv8s, the number of Params is reduced from 6.47M to 1.14M, the number of computed FLOPs is reduced from 22.8GFLOPs to 7.5GFLOPs. The model's size has also decreased from 12.64MB to 2.47MB, and the performance of the improved YOLOv8 is significantly more outstanding than other lightweight networks. The experimental results verify the effectiveness of the proposed method in complex scenarios, which provides a reference basis and technical support for the subsequent automatic picking of Color-changing melons.

Multi-Level Feature Extraction and Classification for Lane Changing Behavior Prediction and POD-Based Evaluation

Multi-Level Feature Extraction and Classification for Lane Changing Behavior Prediction and POD-Based Evaluation

A Boundary-Enhanced Decouple Fusion Segmentation Network for Diagnosis of Adenomatous Polyps.

Adenomatous polyps, a common premalignant lesion, are often classified into villous adenoma (VA) and tubular adenoma (TA). VA has a higher risk of malignancy, whereas TA typically grows slowly and has a lower likelihood of cancerous transformation. Accurate classification is essential for tailored treatment. In this study, we develop a deep learning-based approach for the localization and classification of adenomatous polyps using endoscopic images. Specifically, a pre-trained EGE-UNet is first adopted to extract regions of interest from original images. Multi-level feature maps are then extracted by the feature extraction pipeline (FEP). The deep-level features are fed into the Pyramid Pooling Module (PPM) to capture global contextual information, and the squeeze body edge (SBE) module is then used to decouple the body and edge parts of features, enabling separate analysis of their distinct characteristics. The Group Aggregation Bridge (GAB) and Boundary Enhancement Module (BEM) are then applied to enhance the body features and edge features, respectively, emphasizing their structural and morphological characteristics. By combining the features of the body and edge parts, the final output can be obtained. Experiments show the proposed method achieved promising results on two private datasets. For adenoma vs. non-adenoma classification, It achieved a mIoU of 91.41%, mPA of 96.33%, mHD of 11.63, and mASD of 2.33. For adenoma subclassification (non-adenomas vs. villous adenomas vs. tubular adenomas), it achieved a mIoU of 91.21%, mPA of 94.83%, mHD of 13.75, and mASD of 2.56. These results demonstrate the potential of our approach for precise adenomatous polyp classification.