Residual Connections Research Articles

Insights into the dependence of post-stroke motor recovery on the initial corticospinal tract connectivity from a computational model

There is a consensus that motor recovery post-stroke primarily depends on the degree of the initial connectivity of the ipsilesional corticospinal tract (CST). Indeed, if the residual CST connectivity is sufficient to convey motor commands, the neuromotor system continues to use the CST predominantly, and motor function recovers up to 80%. In contrast, if the residual CST connectivity is insufficient, hand/arm dexterity barely recovers, even as the phases of stroke progress. Instead, the functional upregulation of the reticulospinal tract (RST) often occurs. In this study, we construct a computational model that reproduces the dependence of post-stroke motor recovery on the initial CST connectivity. The model emulates biologically plausible evolutions of primary motor descending tracts, based on activity-dependent or use-dependent plasticity and the preferential use of more strongly connected neural circuits. The model replicates several elements of the empirical evidence presented by the Fugl-Meyer Assessment (FMA) subscores, which evaluate the capabilities for out-of-synergy and in-synergy movements. These capabilities presumably change differently depending on the degree of the initial CST connectivity post-stroke, providing insights into the interactive dynamics of the primary descending motor tracts. We discuss findings derived from the proposed model in relation to the well-known proportional recovery rule. This modeling study aims to present a way to differentiate individuals who can achieve 70 to 80% recovery in the chronic phase from those who cannot by examining the interactive evolution of out-of-synergy and in-synergy movement capabilities during the subacute phase, as assessed by the FMA.

Artificial intelligence in oncourology: integrated deep learning technologies in the tasks of segmentation of three-dimensional images of kidney tumors

In order to improve the accuracy of cancer diagnosis, a new convolutional neural network architecture is presented, which provides automatic segmentation and detection of kidney tumors on three-dimensional images obtained by computed tomography. The proposed approach is based on the integration of three complementary technologies: multilevel convolutional processing, residual connections and U-Net architectural principles. This approach ensures efficient processing of voluminous medical data. An original neural network system for segmentation of kidney images obtained by computed tomography and detection of kidney tumors has been built. To validate the system, a comprehensive experiment was conducted using the publicly available KiTS19 dataset (Kidney Tumor Segmentation 2019) provided by the University of Minnesota Clinic through the Grand Challenge platform. The dataset includes 300 labeled images of kidneys obtained by computed tomography, with confirmed diagnoses. The experiment consisted of the following stages: dataset preprocessing, including normalization and augmentation; system training for 210 cases; validation on an independent sample of 90 cases. The results of the experiment demonstrate the high diagnostic efficiency of the system: the accuracy of automatic segmentation of anatomical structures of the kidneys was 96 % (according to the Dice coefficient); the accuracy of detection and segmentation of tumor formations has reached 91 % (according to the Dice coefficient). The results obtained can be applied in the following areas of clinical practice: preoperative planning and navigation during organ – preserving operations; automated screening of studies using computed tomography for early detection of kidney tumors; quantitative assessment of the dynamics of tumor growth in monitoring the course of the disease; support for clinical decision-making in oncourology.

A Multi-Scale Convolutional Residual Time-Frequency Calibration Method for Low-Accuracy Air Pollution Data

Air pollution concerns have led to the widespread deployment of air quality monitoring stations. While high-cost government stations provide accurate data, their deployment is limited, whereas low-cost sensors offer widespread coverage but with lower accuracy. To enhance the accuracy of measurement data from low-cost air monitoring sensors, this study proposes a Multi-Scale Convolutional Residual Time-Frequency Calibration Method (MCRTF-CM), focusing on the PM2.5 sensor as an example. This method leverages multi-scale convolution in the feature extractor to capture diverse features at various scales using parallel convolutional kernels. Residual connections merge the original and multi-scale features, preserving the initial input for enhanced stability. The calibration module employs Gated Recurrent Units (GRUs) to capture long-term dependencies in time-series data through reset and update gates. Additionally, the Frequency Enhanced Channel Attention Mechanism (FECAM) uses Discrete Cosine Transform (DCT) to convert time-domain data to frequency-domain, assigning weights to different frequency components to enhance key features and suppress irrelevant ones. Experimental results demonstrate that MCRTF-CM outperforms optimal Long Short-Term Memory (LSTM) networks, reducing RMSE, MAE, MSE, and MAPE by 13.59%, 14.04%, 25.33%, and 8.22%, respectively, indicating its better performance.

MixCFormer: A CNN–Transformer Hybrid with Mixup Augmentation for Enhanced Finger Vein Attack Detection

Finger vein recognition has gained significant attention for its importance in enhancing security, safeguarding privacy, and ensuring reliable liveness detection. As a foundation of vein recognition systems, vein detection faces challenges, including low feature extraction efficiency, limited robustness, and a heavy reliance on real-world data. Additionally, environmental variability and advancements in spoofing technologies further exacerbate data privacy and security concerns. To address these challenges, this paper proposes MixCFormer, a hybrid CNN–transformer architecture that incorporates Mixup data augmentation to improve the accuracy of finger vein liveness detection and reduce dependency on large-scale real datasets. First, the MixCFormer model applies baseline drift elimination, morphological filtering, and Butterworth filtering techniques to minimize the impact of background noise and illumination variations, thereby enhancing the clarity and recognizability of vein features. Next, finger vein video data are transformed into feature sequences, optimizing feature extraction and matching efficiency, effectively capturing dynamic time-series information and improving discrimination between live and forged samples. Furthermore, Mixup data augmentation is used to expand sample diversity and decrease dependency on extensive real datasets, thereby enhancing the model’s ability to recognize forged samples across diverse attack scenarios. Finally, the CNN and transformer architecture leverages both local and global feature extraction capabilities to capture vein feature correlations and dependencies. Residual connections improve feature propagation, enhancing the stability of feature representations in liveness detection. Rigorous experimental evaluations demonstrate that MixCFormer achieves a detection accuracy of 99.51% on finger vein datasets, significantly outperforming existing methods.

An Enhanced Model for Mammogram Image Denoising Using a CNN Autoencoder with Residual Connections

Image denoising has a wide range of applications in obtaining better-quality noisy data, such as medical imaging and low-light photography. This work proposes a new CNN autoencoder architecture also equipped with a residual connection that advances state-of-the-art performance on this denoising benchmark. The proposed model is compared with state-of-the-art methods, including conditional GAN, CNNs with advanced loss functions, and self-supervised learning models in terms of PSNR, SSIM, and computational loss. It achieves the best results on metrics of PSNR at 44.82, SSIM at 0.970, and a loss value as low as 0.015 on a custom dataset of medical images. These results reveal the efficiency of the architecture in noise reduction while retaining the critical information about the structure and, hence, have many promises in real-world applications. The emphasis here is on an application of residual connections fused with deep learning models having superior state-of-the-art performance in image denoising.

Remaining useful life prediction method of bearings based on the interactive learning strategy

To address the issues of low accuracy, high time cost, and different failure modes in predicting the remaining useful life of rolling bearings, a remaining useful life prediction model based on an interactive learning convolution network with a feature attention mechanism (ILCANet) was proposed in this paper. The model divided time series data into an odd part and an even part, and the interactive learning strategy improved the ability of the model to extract features from long series. The binary tree structure was used to increase the number of network layers and to subdivide sequence features. In the model, residual connection was introduced to prevent the gradient from disappearing. In addition, in order to determine the degree of contribution of different features, a feature attention mechanism was embedded to assign weights to features. Experiments were conducted with PHM2012 and XJTU-SY datasets, and the results showed that the proposed method had better prediction accuracy in RUL prediction than other prediction methods.

Dual Strategy Based Improved Swarm Intelligence and Stacked LSTM With Residual Connection for Land Use Land Cover Classification

Dual Strategy Based Improved Swarm Intelligence and Stacked LSTM With Residual Connection for Land Use Land Cover Classification

Vegetation segmentation using oblique photogrammetry point clouds based on RSPT network

ABSTRACT Vegetation segmentation via point cloud data can provide important information for urban planning and environmental protection. The point cloud dataset is obtained using light detection and ranging (LiDAR) or RGB-D images. Oblique photogrammetry has received little attention as another important source of point cloud data. We present a pointwise annotated oblique photogrammetry point-cloud dataset that contains rich RGB information, texture, and structural features. This dataset contains five regions of Bengbu, China, with more than twenty thousand samples in this paper. Obviously, previous indoor point cloud semantic segmentation models are no longer applicable to oblique photogrammetry point clouds. A random sampling point transformer (RSPT) network is proposed to enhance vegetation segmentation accuracy. The RSPT model offers both efficient and lightweight architecture. In RSPT, random point sampling is utilized to downsample point clouds, and a local feature aggregation module based on self-attention is designed to extract additional representation features. The network also incorporated residual and dense connections (ResiDense) to capture both local and comprehensive features. Compared to state-of-the-art models, RSPT achieves notable improvements. The intersection over union (IoU) metric increased from 96.0% to 96.5%, the F1-score increased from 90.8% to 97.0%, and the overall accuracy (OA) increased from 91.9% to 96.9%.

Accurate recognition of human abnormal behaviours using adaptive 3D residual attention network with gated recurrent units (GRU) in the video sequences

ABSTRACT Abnormal or violent behaviour by individuals with mental disorders presents significant risks to public safety, necessitating advanced systems capable of detecting such behaviours in real time. Traditional single-sensing methods for human activity recognition often struggle with issues like signal noise, dropped data, and limited scalability, which hinder their ability to accurately detect abnormal behaviours in dynamic and complex environments. This paper introduces a novel solution that addresses these challenges by proposing an adaptive 3D residual attention network (A3D-RAN) combined with Gated Recurrent Units (GRUs). The A3D-RAN utilises an adaptive attention mechanism to focus on the most relevant regions in video sequences, while residual connections improve feature reuse and maintain gradient flow, enabling fine-grained detail capture. GRUs are integrated to efficiently model long-term temporal dependencies, ensuring a more comprehensive understanding of human behaviour across time. Through extensive experimentation on real-world datasets, our model achieved a remarkable accuracy of 97%, significantly surpassing the 78% accuracy of standalone A3D-RAN implementations. Moreover, the robustness of the model under challenging conditions – such as occlusions and lighting variations – demonstrates its potential for real-world surveillance applications. By employing the Improved War Strategy Optimization (IWSO) Algorithm for parameter tuning, we further enhanced performance, reaching an unprecedented accuracy of 99%. This breakthrough underscores the practical value of our approach in improving public safety and security through accurate and timely detection of abnormal behaviours in surveillance systems.

Predicting the toxic side effects of drug interactions using chemical structures and protein sequences

The study aims to address the critical issue of toxic side effects resulting from drug combinations, which can significantly increase health risks, clinical complications, and lead to drug being withdrawn from the market. A model named TSEDDI (toxic side effects of drug-drug interaction) has been developed to improve the identification of drug pairs that may induce toxicity or adverse reactions. By utilizing drug chemical structures and diverse proteins, we employ a convolutional neural network (CNN) to extract features from molecular images, enzyme proteins, transporter proteins, and target proteins. Furthermore, we introduce a weighted binary cross entropy loss function to tackle class imbalance and integrate the multi-head attention mechanism with residual connections to enhance model performance. Our model outperformed advanced baseline models in predicting drug-drug interaction (DDI) side effects, achieving an accuracy of 0.9059 (± 0.0010) and consistently excelling across various evaluation metrics. The case study confirms the potential mechanisms by which four pairs of drugs cause side effects, thus demonstrating the effectiveness of our model in predicting DDI side effects. The TSEDDI model combines multiple attention mechanisms and residual connections, enhancing its ability to detect toxic and adverse effects related to DDIs. As a result, it becomes a valuable resource for promptly identifying adverse reactions in clinical trials. Future research could investigate drug substructures prone to toxic side effects.

Leveraging RegNetX002 for Automated Classification of COVID-19 and Other Lung Opacities in Chest X-ray Images

The global corona virus epidemic has affected medical treatment. Because there are so many variations of newly developing infectious viruses, both the dangers they pose, and the effectiveness of vaccinations need much attention. To better comprehend this illness and investigate its transmission, individual diagnosis, and maybe other fascinating related concerns, it is helpful to learn more complex and interpretable models using COVID-19 data. On the other hand, the lack of accurately classified data is creating certain complications and challenges. Utilizing pre-trained Deep Neural Network (DNN) models on big datasets such as ImageNet has been done in earlier publications. This study evaluated the X002 version of the RegNet CNN architecture. Due to its less computationally costly design and very small number of parameters, the RegNet model is more computationally feasible for imaging applications compared to other CNN models. Self-regulation is a regulatory module that RegNet employs; it retrieves spatio-temporal data from the network's intermediate levels. Furthermore, RegNet's weight residual connections, batch normalization, and regularization mechanism approaches make it fast, scalable, and versatile. Our suggested approach successfully classified COVID-19, normal, pneumonia, and other lung opacities with an astounding accuracy of 95.37%, as determined after thorough testing and review.

EDCLoc: a prediction model for mRNA subcellular localization using improved focal loss to address multi-label class imbalance

BackgroundThe subcellular localization of mRNA plays a crucial role in gene expression regulation and various cellular processes. However, existing wet lab techniques like RNA-FISH are usually time-consuming, labor-intensive, and limited to specific tissue types. Researchers have developed several computational methods to predict mRNA subcellular localization to address this. These methods face the problem of class imbalance in multi-label classification, causing models to favor majority classes and overlook minority classes during training. Additionally, traditional feature extraction methods have high computational costs, incomplete features, and may lead to the loss of critical information. On the other hand, deep learning methods face challenges related to hardware performance and training time when handling complex sequences. They may suffer from the curse of dimensionality and overfitting problems. Therefore, there is an urgent need for more efficient and accurate prediction models.ResultsTo address these issues, we propose a multi-label classifier, EDCLoc, for predicting mRNA subcellular localization. EDCLoc reduces training pressure through a stepwise pooling strategy and applies grouped convolution blocks of varying sizes at different levels, combined with residual connections, to achieve efficient feature extraction and gradient propagation. The model employs global max pooling at the end to further reduce feature dimensions and highlight key features. To tackle class imbalance, we improved the focal loss function to enhance the model’s focus on minority classes. Evaluation results show that EDCLoc outperforms existing methods in most subcellular regions. Additionally, the position weight matrix extracted by multi-scale CNN filters can match known RNA-binding protein motifs, demonstrating EDCLoc’s effectiveness in capturing key sequence features.ConclusionsEDCLoc outperforms existing prediction tools in most subcellular regions and effectively mitigates class imbalance issues in multi-label classification. These advantages make EDCLoc a reliable choice for multi-label mRNA subcellular localization. The dataset and source code used in this study are available at https://github.com/DellCode233/EDCLoc.

Enhanced Disc Herniation Classification Using Grey Wolf Optimization Based on Hybrid Feature Extraction and Deep Learning Methods.

The early diagnosis and treatment of lumbar disc herniation is much more likely to yield favorable results, allowing the hernia to be treated before it develops further. The aim of this study was to classify lumbar disc herniations in a computer-aided, fully automated manner using magnetic resonance images (MRIs). This study presents a hybrid method integrating residual network (ResNet50), grey wolf optimization (GWO), and machine learning classifiers such as multi-layer perceptron (MLP) and support vector machine (SVM) to improve classification performance. The proposed approach begins with feature extraction using ResNet50, a deep convolutional neural network known for its robust feature representation capabilities. ResNet50's residual connections allow for effective training and high-quality feature extraction from input images. Following feature extraction, the GWO algorithm, inspired by the social hierarchy and hunting behavior of grey wolves, is employed to optimize the feature set by selecting the most relevant features. Finally, the optimized feature set is fed into machine learning classifiers (MLP and SVM) for classification. The use of various activation functions (e.g., ReLU, identity, logistic, and tanh) in MLP and various kernel functions (e.g., linear, rbf, sigmoid, and polynomial) in SVM allows for a thorough evaluation of the classifiers' performance. The proposed methodology demonstrates significant improvements in metrics such as accuracy, precision, recall, and F1 score, outperforming traditional approaches in several cases. These results highlight the effectiveness of combining deep learning-based feature extraction with optimization and machine learning classifiers. Compared to other methods, such as capsule networks (CapsNet), EfficientNetB6, and DenseNet169, the proposed ResNet50-GWO-SVM approach achieved superior performance across all metrics, including accuracy, precision, recall, and F1 score, demonstrating its robustness and effectiveness in classification tasks.

A mutual inclusion mechanism for precise boundary segmentation in medical images.

Accurate image segmentation is crucial in medical imaging for quantifying diseases, assessing prognosis, and evaluating treatment outcomes. However, existing methods often fall short in integrating global and local features in a meaningful way, failing to give sufficient attention to abnormal regions and boundary details in medical images. These limitations hinder the effectiveness of segmentation techniques in clinical settings. To address these issues, we propose a novel deep learning-based approach, MIPC-Net, designed for precise boundary segmentation in medical images. Our approach, inspired by radiologists' working patterns, introduces two distinct modules: 1. Mutual Inclusion of Position and Channel Attention (MIPC) Module: To improve boundary segmentation precision, we present the MIPC module. This module enhances the focus on channel information while extracting position features and vice versa, effectively enhancing the segmentation of boundaries in medical images. 2. Skip-Residue Module: To optimize the restoration of medical images, we introduce Skip-Residue, a global residual connection. This module improves the integration of the encoder and decoder by filtering out irrelevant information and recovering the most crucial information lost during the feature extraction process. We evaluate the performance of MIPC-Net on three publicly accessible datasets: Synapse, ISIC2018-Task, and Segpc. The evaluation uses metrics such as the Dice coefficient (DSC) and Hausdorff Distance (HD). Our ablation study confirms that each module contributes to the overall improvement of segmentation quality. Notably, with the integration of both modules, our model outperforms state-of-the-art methods across all metrics. Specifically, MIPC-Net achieves a 2.23 mm reduction in Hausdorff Distance on the Synapse dataset, highlighting the model's enhanced capability for precise image boundary segmentation. The introduction of the novel MIPC and Skip-Residue modules significantly improves feature extraction accuracy, leading to better boundary recognition in medical image segmentation tasks. Our approach demonstrates substantial improvements over existing methods, as evidenced by the results on benchmark datasets.

MGT: Machine Learning Accelerates Performance Prediction of Alloy Catalytic Materials.

The application of deep learning technology in the field of materials science provides a new method for predicting the adsorption energy of high-performance alloy catalysts in hydrogen evolution reactions and material discovery. The activity and selectivity of catalytic materials are mainly influenced by the properties and positions of active sites and adsorption sites. However, current deep learning models have not sufficiently focused on the importance of active atoms and adsorbates, instead placing more emphasis on the overall structure of the catalytic materials. In this paper, the overall molecular graph and a masked graph, which ignores fixed atoms, are separately input into the Masked Graph Transformer (MGT) network to enhance the model's ability to recognize key sites in catalytic reactions. Second, we introduce a nonlinear message-passing mechanism to improve the dot-product attention in the Transformer and capture the directional information on the relative positions of nodes by integrating molecular geometric information through deep tensor products. Subsequently, we constructed the NLMP-TransNet framework, which combines MPNN and Transformer and optimizes the model's learning and prediction capabilities through weight sharing and residual connections. The MGT achieves an error rate of 0.5447 eV on the small data set OC20-Ni, surpassing existing technologies. Ablation studies confirm the necessity of focusing on site features for accurate adsorption energy prediction. Code is available at https://github.com/KristinSun/OCP-MGT.git.

DeepMoIC: multi-omics data integration via deep graph convolutional networks for cancer subtype classification

BackgroundAchieving precise cancer subtype classification is imperative for effective prognosis and treatment. Multi-omics studies, encompassing diverse data modalities, have emerged as powerful tools for unraveling the complexities of cancer. However, owing to the intricacies of biological data, multi-omics datasets generally show variations in data types, scales, and distributions. These intractable problems lead to challenges in exploring intact representations from heterogeneous data, which often result in inaccuracies in multi-omics information analysis.ResultsTo address the challenges of multi-omics research, our approach DeepMoIC presents a novel framework derived from deep Graph Convolutional Network (GCN). Leveraging autoencoder modules, DeepMoIC extracts compact representations from omics data and incorporates a patient similarity network through the similarity network fusion algorithm. To handle non-Euclidean data and explore high-order omics information effectively, we design a Deep GCN module with two strategies: residual connection and identity mapping. With extracted higher-order representations, our approach consistently outperforms state-of-the-art models on a pan-cancer dataset and 3 cancer subtype datasets.ConclusionThe introduction of Deep GCN shows encouraging performance in terms of supervised multi-omics feature learning, offering promising insights for precision medicine in cancer research. DeepMoIC can potentially be an important tool in the field of cancer subtype classification because of its capacity to handle complex multi-omics data and produce reliable classification findings.

Dysarthric speech recognition: an investigation on using depthwise separable convolutions and residual connections

Dysarthric speech recognition: an investigation on using depthwise separable convolutions and residual connections

An airborne gravity gradient compensation method based on residual backpropagation

Abstract Airborne gravity gradient dynamic measurement error compensation is a crucial aspect of data processing in gravity gradient dynamic measurements. This study introduces a deep learning approach based on a residual backpropagation (Res-BP) neural network for post-error compensation in airborne gravity gradient dynamic measurement. The network employs residual connections to facilitate identity mapping, thereby enabling gradient propagation across layers. This strategy preserves the original information while acquiring additional information through nonlinear operations, effectively mitigating the gradient vanishing issue and enhancing the neural network's fitting capability. The method proposed in this paper is applied to both simulation data from a gravity gradiometer and high-altitude dynamic measured data of an airborne gravity gradient. Compared to traditional neural network multilayer perceptron (MLP), the Res-BP method significantly improves compensation accuracy through its application in high flight experiment of the southern section of
Zhangguangcai Ridge on the western side of Mudanjiang City, Heilongjiang Province.

Unified Domain Adaptation for Specialized Indoor Scene Inpainting Using a Pre-Trained Model

Image inpainting for indoor environments presents unique challenges due to complex spatial relationships, diverse lighting conditions, and domain-specific object configurations. This paper introduces a resource-efficient post-processing framework that enhances domain-specific image inpainting through an adaptation mechanism. Our architecture integrates a convolutional neural network with residual connections optimized via a multi-term objective function combining perceptual losses and adaptive loss weighting. Experiments on our curated dataset of 4000 indoor household scenes demonstrate improved performance, with training completed in 20 min on commodity GPU hardware with 0.14 s of inference latency per image. The framework exhibits enhanced results across standard metrics (FID, SSIM, LPIPS, MAE, and PSNR), showing improvements in structural coherence and perceptual quality while preserving cross-domain generalization abilities. Our methodology offers a novel approach for efficient domain adaptation in image inpainting, particularly suitable for real-world applications under computational constraints. This work advances the development of domain-aware image restoration systems and provides architectural insights for specialized image processing frameworks.

A Typical Infrared Background Radiation Prediction Model Based on RF-VMD and Optimized Hybrid Neural Network

ABSTRACT The short-term prediction and adjustment of a target’s infrared radiation hold significant value in military camouflage applications. Existing radiation prediction models generally require real-time environmental and meteorological data support, resulting in lag in active camouflage. To meet the demand for active camouflage of background infrared (IR) radiation, a short-term background IR radiation prediction method based on historical data is proposed. First, a random forest (RF) is used to filter the collected multidimensional meteorological parameters. Variational mode decomposition (VMD) is applied for time-frequency analysis on these parameters, optimizing them with Bayesian algorithms and decomposing them into multivariate intrinsic mode functions (IMFs) with similar frequencies to reduce the impact of nonlinearity in the data. Based on the superimposed IMFs as inputs, a hybrid deep neural network prediction model is established. The model optimizes the CNN-LSTM network with residual connections and introduces a multi-head self-attention mechanism to enhance spatiotemporal feature extraction of the multidimensional meteorological parameters, focusing on key temporal feature regions. According to the experimental results, the constructed model demonstrates high prediction accuracy and adaptability across different background environments, with a low parameter count and fast prediction capability, meeting the practical application needs for various complex backgrounds.