Residual Connections Research Articles

Innovative Approach to Detecting Autism Spectrum Disorder Using Explainable Features and Smart Web Application

Autism Spectrum Disorder (ASD) is a complex developmental condition marked by challenges in social interaction, communication, and behavior, often involving restricted interests and repetitive actions. The diversity in symptoms and skill profiles across individuals creates a diagnostic landscape that requires a multifaceted approach for accurate understanding and intervention. This study employed advanced machine-learning techniques to enhance the accuracy and reliability of ASD diagnosis. We used a standard dataset comprising 1054 patient samples and 20 variables. The research methodology involved rigorous preprocessing, including selecting key variables through data mining (DM) visualization techniques including Chi-Square tests, analysis of variance, and correlation analysis, along with outlier removal to ensure robust model performance. The proposed DM and logistic regression (LR) with Shapley Additive exPlanations (DMLRS) model achieved the highest accuracy at 99%, outperforming state-of-the-art methods. eXplainable artificial intelligence was incorporated using Shapley Additive exPlanations to enhance interpretability. The model was compared with other approaches, including XGBoost, Deep Models with Residual Connections and Ensemble (DMRCE), and fast lightweight automated machine learning systems. Each method was fine-tuned, and performance was verified using k-fold cross-validation. In addition, a real-time web application was developed that integrates the DMLRS model with the Django framework for ASD diagnosis. This app represents a significant advancement in medical informatics, offering a practical, user-friendly, and innovative solution for early detection and diagnosis.

Feature Intensification Using Perception-Guided Regional Classification for Remote Sensing Image Super-Resolution

In recent years, super-resolution technology has gained widespread attention in the field of remote sensing. Despite advancements, current methods often employ uniform reconstruction techniques across entire remote sensing images, neglecting the inherent variability in spatial frequency distributions, particularly the distinction between high-frequency texture regions and smoother areas, leading to computational inefficiency, which introduces redundant computations and fails to optimize the reconstruction process for regions of higher complexity. To address these issues, we propose the Perception-guided Classification Feature Intensification (PCFI) network. PCFI integrates two key components: a compressed sensing classifier that optimizes speed and performance, and a deep texture interaction fusion module that enhances content interaction and detail extraction. This network mitigates the tendency of Transformers to favor global information over local details, achieving improved image information integration through residual connections across windows. Furthermore, a classifier is employed to segment sub-image blocks prior to super-resolution, enabling efficient large-scale processing. The experimental results on the AID dataset indicate that PCFI achieves state-of-the-art performance, with a PSNR of 30.87 dB and an SSIM of 0.8131, while also delivering a 4.33% improvement in processing speed compared to the second-best method.

B cell epitope prediction by capturing spatial clustering property of the epitopes using graph attention network.

Knowledge of B cell epitopes is critical to vaccine design, diagnostics, and therapeutics. As experimental validation for epitopes is time-consuming and costly, many in silico tools have been developed to computationally predict the B cell epitopes. While most methods show poor performance, deep learning methods in recent years have shown promising results. We developed a method called EpiGraph that outperformed previous methods, including those that showed a significant improvement in performance in recent years. Our model's performance can be attributed to the following factors: (1) a combination of structure and sequence feature embeddings obtained from pretrained ESM-IF1 and ESM-2 models could capture the structural and evolutionary features of B cell epitopes, (2) a graph attention network could learn the spatial proximity of B cell epitopes with high graph homophily, and (3) residual connections in the model framework mitigate the over-smoothing problem in the graph neural network. Our model achieved the highest performance on an independent benchmark dataset. The results were also consistent on a different dataset. The datasets and source codes are available at https://github.com/sj584/EpiGraph . A user-friendly web server is freely available at http://epigraph.kaist.ac.kr .

DeepAge: Harnessing Deep Neural Network for Epigenetic Age Estimation From DNA Methylation Data of Human Blood Samples

Accurate prediction of biological age from DNA methylation data is a critical endeavor in understanding the molecular mechanisms of aging and developing age-related disease interventions. Traditional epigenetic clocks rely on linear regression or basic machine learning models, which often fail to capture the complex, non-linear interactions within methylation data. This study introduces DeepAge, a novel deep learning framework utilizing Temporal Convolutional Networks (TCNs) to enhance the prediction of biological age from DNA methylation profiles using selected CpGs by a Dual-Correlation based apparoach. DeepAge leverages a sequence-based approach with dilated convolutions to effectively capture long-range dependencies between CpG sites, addressing the limitations of prior models by incorporating advanced network architectures including residual connections and dropout regularization. The dual correlation feature selection enhances our model's predictive capabilities by identifying the most age-relevant CpG sites. Our model outperforms existing epigenetic clocks across multiple datasets, offering significant improvements in accuracy and providing deeper insights into the epigenetic determinants of aging. The proposed method not only sets a new standard in age estimation but also highlights the potential of deep learning in biologically relevant feature extraction and interpretation, contributing to the broader field of computational biology and precision medicine.

Data Augmentation and Fault Diagnosis for Imbalanced Industrial Process Data Based on Residual Wasserstein Generative Adversarial Network With Gradient Penalty

ABSTRACTIn practical industrial applications, equipment usually operates normally and failures are relatively rare, resulting in serious imbalances in the collected data. This imbalance leads to issues such as overfitting, instability, and poor robustness, significantly reducing the accuracy and stability of fault diagnosis system. To address these challenges, this research proposes a method for imbalanced data augmentation and industrial process fault diagnosis based on improved Generative Adversarial Network (GAN). The method adopts Wasserstein distance with gradient penalty and integrates residual connections into the architecture of the generator. This innovation not only helps improve gradient transfer in the generator, but also significantly enhances the data generation capabilities of the generative model through improving the stability of training. Limited industrial process data is used by a generative model to produce synthetic samples with high similarity and diversity. These high‐quality samples improve fault diagnosis by enriching the imbalanced dataset. Experimental results on two industrial datasets confirm the method's effectiveness in enhancing fault diagnosis performance with limited data.

Robotic Grasping Technology Integrating Large Kernel Convolution and Residual Connections

Robotic Grasping Technology Integrating Large Kernel Convolution and Residual Connections

Research on a Joint Extraction Method of Track Circuit Entities and Relations Integrating Global Pointer and Tensor Learning

To address the issue of efficiently reusing the massive amount of unstructured knowledge generated during the handling of track circuit equipment faults and to automate the construction of knowledge graphs in the railway maintenance domain, it is crucial to leverage knowledge extraction techniques to efficiently extract relational triplets from fault maintenance text data. Given the current lag in joint extraction technology within the railway domain and the inefficiency in resource utilization, this paper proposes a joint extraction model for track circuit entities and relations, integrating Global Pointer and tensor learning. Taking into account the associative characteristics of semantic relations, the nesting of domain-specific terms in the railway sector, and semantic diversity, this research views the relation extraction task as a tensor learning process and the entity recognition task as a span-based Global Pointer search process. First, a multi-layer dilate gated convolutional neural network with residual connections is used to extract key features and fuse the weighted information from the 12 different semantic layers of the RoBERTa-wwm-ext model, fully exploiting the performance of each encoding layer. Next, the Tucker decomposition method is utilized to capture the semantic correlations between relations, and an Efficient Global Pointer is employed to globally predict the start and end positions of subject and object entities, incorporating relative position information through rotary position embedding (RoPE). Finally, comparative experiments with existing mainstream joint extraction models were conducted, and the proposed model’s excellent performance was validated on the English public datasets NYT and WebNLG, the Chinese public dataset DuIE, and a private track circuit dataset. The F1 scores on the NYT, WebNLG, and DuIE public datasets reached 92.1%, 92.7%, and 78.2%, respectively.

A temporal convolution network-based just-in-time learning method for industrial quality variable prediction

Real-time acquisition of quality variables is paramount for enhancing control and optimization of industrial processes. Process modeling methods, such as soft sensors, offer a means to predict difficult-to-obtain quality variables using easily measurable process parameters. However, the dynamic nature of industrial processes poses significant challenges to modeling. For instance, conventional models are typically trained offline using historical data, rendering them incapable of adapting to real-time changes in data distribution or environmental conditions. To tackle this challenge, we introduce a novel approach termed the Residual Temporal Attention Temporal Convolution Network (RTA-TCN) and propose a just-in-time learning method based on RTA-TCN for industrial process modeling. The RTA-TCN model incorporates temporal attention into TCN, enabling the integration of previous time-step process variables into the current ones, as well as the fusion of internally relevant features among inputs. Moreover, to prevent the partial loss of original information during feature integration, residual connections are introduced into the temporal attention mechanism. These connections facilitate the retention of original feature information to a maximal extent while integrating relevant features. Consequently, the proposed RTA-TCN demonstrates significant advantages in handling the non-linearity and long-term dynamic dependencies inherent in industrial variables. Additionally, the proposed just-in-time learning method leverages RTA-TCN as a local model and updates it in real-time using online industrial data. This just-in-time learning method enables effective adaptation to varying data distributions and environmental conditions. We validate the performance of our method using two industrial datasets (Debutanizer Column and Sulfur Recovery Unit).

REDalign: accurate RNA structural alignment using residual encoder-decoder network

BackgroundRNA secondary structural alignment serves as a foundational procedure in identifying conserved structural motifs among RNA sequences, crucially advancing our understanding of novel RNAs via comparative genomic analysis. While various computational strategies for RNA structural alignment exist, they often come with high computational complexity. Specifically, when addressing a set of RNAs with unknown structures, the task of simultaneously predicting their consensus secondary structure and determining the optimal sequence alignment requires an overwhelming computational effort of O(L6)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$O(L^6)$$\\end{document} for each RNA pair. Such an extremely high computational complexity makes these methods impractical for large-scale analysis despite their accurate alignment capabilities.ResultsIn this paper, we introduce REDalign, an innovative approach based on deep learning for RNA secondary structural alignment. By utilizing a residual encoder-decoder network, REDalign can efficiently capture consensus structures and optimize structural alignments. In this learning model, the encoder network leverages a hierarchical pyramid to assimilate high-level structural features. Concurrently, the decoder network, enhanced with residual skip connections, integrates multi-level encoded features to learn detailed feature hierarchies with fewer parameter sets. REDalign significantly reduces computational complexity compared to Sankoff-style algorithms and effectively handles non-nested structures, including pseudoknots, which are challenging for traditional alignment methods. Extensive evaluations demonstrate that REDalign provides superior accuracy and substantial computational efficiency.ConclusionREDalign presents a significant advancement in RNA secondary structural alignment, balancing high alignment accuracy with lower computational demands. Its ability to handle complex RNA structures, including pseudoknots, makes it an effective tool for large-scale RNA analysis, with potential implications for accelerating discoveries in RNA research and comparative genomics.

Early Warning of Energy Storage Battery Fault Based on Improved Autoformer and Adaptive Threshold

To enhance voltage prediction accuracy in energy storage batteries and address the limitations of fixed threshold warning methods, a fault warning approach based on an improved Autoformer model and adaptive thresholds is proposed. First, a spatiotemporal filtering layer is introduced into the autocorrelation mechanism to analyze the trend features of voltage sequences across different frequency domains. Additionally, an adaptive gating residual connection is used to link the sublayer and current layer output features, which helps to improve the model's adaptive feature selection capability. This innovation enables the development of a robust voltage prediction model based on the enhanced Autoformer. Then, a similarity‐based adaptive threshold, using interval estimation, is employed to rapidly track variations in battery voltage, enabling dynamic adjustment of voltage thresholds. Finally, the proposed method is validated with real voltage data from an operational energy storage station. The experimental results shows that the proposed model has higher accuracy and robustness compared to similar methods. The adaptive threshold can reduce the false alarm rate by ≈18% and issue alarms at three sampling points ahead of the battery management system alarm, improving fault warning accuracy and illustrating that early fault warning is effectively and practically carried out using the method.

Residual channel attention based sample adaptation few-shot learning for hyperspectral image classification.

Few-shot learning (FSL) uses prior knowledge and supervised experience to effectively classify hyperspectral images (HSIs), thereby reducing the cost of large numbers of labeled samples. However, existing few-shot methods ignore the correlation between cross-domain feature channels, and the feature representation ability is insufficient. To address above issue, this paper proposes a novel Residual Channel Attention Based Sample Adaptation Few-Shot Learning for Hyperspectral Image Classification(RCASA-FSL) for hyperspectral image classification (HSIC), which can capture and enhance cross-domain dependencies through multi-layer residual connection and random-based feature recalibration. Specifically, a Deep Residual Feature Channel Attention Mechanism (DRFCAM) is designed to obtain cross-domain dependencies by residual concatenation, and further the residual structure is stacked for mining depth discrimination information. Furthermore, a new Random-based Feature Recalibration Module (RFRM) is proposed to reassign the feature weights via random matrix, which fully explore feature weight relationships to guide the sample adaptation process. Besides, we design a joint loss function with combining the FSL loss and domain adaptive loss for further optimization model. Experiments conducted on several standard hyperspectral datasets demonstrate that the proposed RCASA-FSL is superior to other FSL techniques in both quantitative and qualitative aspects.

Investigation of hybrid modeling and its transferability in building load prediction used for district heating systems

In the district heating systems, the historical operation data of the buildings in those areas would be partially or entirely missing. The traditional data-driven model is hard to predict the ground truth results because the historical data is not available for model training. However, utilizing the physics-based methods for load calculation takes a long time to process and encounters low accuracy issues. This paper investigates several hybrid models that integrate the data-driven model and the physics-based models with different fusion methods. The physics-based models calculate envelope load and infiltration load, based on Fourier's law and the grand canonical ensemble theory, respectively. After undergoing load processing, features fusion, and residual connection, the best advanced hybrid models generate 21.35%, 16.35%, and 12.73% better prediction results compared with the data-driven model. Moreover, the advanced hybride models also perform strong transferability across all the data quantity groups. In terms of practical application, the advanced hybrid models could be deployed with effective generalization in limited data scenarios and robust transfer capabilities. The selected best model constructed by hybrid modeling displays the highest performance and saves the total training costs with strong transferability.

Associative graph convolution network for point cloud analysis

Since point cloud is the raw output of most 3D sensors, its effective analysis is in huge demand in the field of autonomous driving and robotic manipulation. However, directly processing point clouds is challenging because point clouds are a kind of disordered and unstructured geometric data. Recently, numerous graph convolution neural networks are proposed for introducing graph structure to point clouds yet far from perfect. Specially, DGCNN tries to learn local geometric of points in semantic space and recomputes the graph using nearest neighbors in the feature space in each layer. However, it discards all the information of the previous graph after each graph update, which neglects the relations between each dynamic update. To this end, we propose an associative graph convolution neural network (AGCN) which mainly consists of associative graph convolution (AGConv) and two kinds of residual connections. AGConv additionally considers the information from the previous graph when computing the edge function on current local neighborhoods in each layer, and it can precisely and continuously capture the local geometric features on point clouds. Residual connections further explore the semantic relations between layers for effective learning on point clouds. Extensive experiments on several benchmark datasets show that our network achieves competitive classification and segmentation results.

Bearing fault diagnosis based on wavelet adaptive threshold filtering and multi-channel fusion cross-attention neural network.

In industrial applications, it is difficult to extract the fault feature directly when the rolling bearing works under strong background noise. In addition, single-channel vibration sensor data pose limitations in providing a comprehensive representation of bearing fault features; how to effectively fuse data of each channel and extract features is a challenge. To solve the above-mentioned problems, a fault diagnosis method based on wavelet adaptive threshold filtering and multi-channel fusion cross-attention neural network is proposed in this paper. First, the multi-scale discrete wavelet transform is applied to obtain the wavelet coefficients of each channel. Adaptive threshold filtering is conducted to filter out noise and extract symbolic features. The threshold updates with the training of the network. Then, the wavelet coefficients are reconstructed and the channel attention is performed to further extract the symbolic features of the fault signal. Finally, the multi-channel fault signals are fused by a cross-attention module. This module can fully extract the features of each channel and fuse multi-channel data. To improve the generalization ability of the network, residual connections are added. To verify the effectiveness of the proposed method, experiments are carried out on the rolling bearing datasets of Case Western Reserve University and Xi'an Jiaotong University. In addition, the gas turbine main bearing dataset is also applied to prove the reliability of this method.

VMC‐UNet: A Vision Mamba‐CNN U‐Net for Tumor Segmentation in Breast Ultrasound Image

ABSTRACTBreast cancer remains one of the most significant health threats to women, making precise segmentation of target tumors critical for early clinical intervention and postoperative monitoring. While numerous convolutional neural networks (CNNs) and vision transformers have been developed to segment breast tumors from ultrasound images, both architectures encounter difficulties in effectively modeling long‐range dependencies, which are essential for accurate segmentation. Drawing inspiration from the Mamba architecture, we introduce the Vision Mamba‐CNN U‐Net (VMC‐UNet) for breast tumor segmentation. This innovative hybrid framework merges the long‐range dependency modeling capabilities of Mamba with the detailed local representation power of CNNs. A key feature of our approach is the implementation of a residual connection method within the U‐Net architecture, utilizing the visual state space (VSS) module to extract long‐range dependency features from convolutional feature maps effectively. Additionally, to better integrate texture and structural features, we have designed a bilinear multi‐scale attention module (BMSA), which significantly enhances the network's ability to capture and utilize intricate feature details across multiple scales. Extensive experiments conducted on three public datasets demonstrate that the proposed VMC‐UNet surpasses other state‐of‐the‐art methods in breast tumor segmentation, achieving Dice coefficients of 81.52% for BUSI, 88.00% for BUS, and 88.96% for STU. The source code is accessible at https://github.com/windywindyw/VMC‐UNet.

MSSTGNN: Multi-Scaled Spatio-Temporal Graph Neural Networks for short- and long-term traffic prediction

Accurate traffic prediction plays a crucial role in ensuring traffic safety and minimizing property damage. The utilization of STGNNs in traffic prediction has gained significant attention from researchers aiming to capture the intricate time-varying relationships within traffic data. While existing STGNNs commonly rely on Euclidean distance to assess the similarity between nodes, which may fall short in reflecting POI or regional functions. The traffic network exhibits static from a macro perspective, whereas undergoes dynamic changes in the micro perspective. Previous researchers incorporating self-attention to capture time-varying features for constructing dynamic graphs have faced challenges in overlooking the connections between nodes due to the Softmax polarization effect, which tends to amplify extreme value differences, and fails to accurately represent the true relationships between nodes. To solve this problem, we introduce the Multi-Scaled Spatio-Temporal Graph Neural Networks (MSSTGNN), which aims to comprehensively capture characteristics within traffic from multiscale viewpoints to construct multi-perspective graphs. We employ a trainable matrix to enhance the predefined adjacency matrices and to construct an optimal dynamic graph based on both trend and period. Additionally, a graph aggregate technique is proposed to effectively merge trend and periodic dynamic graphs. TCN is developed to model nonstationary traffic data, and we leverage the skip and residual connections to increase the model depth. A two-stage learning approach and a novel MSELoss function is designed to enhance the model's performance. The experimental results demonstrate that the MSSTGNN model outperforms the existing methods, achieving state-of-the-art performances across multiple real-world datasets.

A Multi-Scale Feature Fusion Deep Learning Network for the Extraction of Cropland Based on Landsat Data

In the face of global population growth and climate change, the protection and rational utilization of cropland are crucial for food security and ecological balance. However, the complex topography and unique ecological environment of the Qinghai-Tibet Plateau results in a lack of high-precision cropland monitoring data. Therefore, this paper constructs a high-quality cropland dataset for the YarlungZangbo-Lhasa-Nyangqv River region of the Qinghai-Tibet Plateau and proposes an MSC-ResUNet model for cropland extraction based on Landsat data. The dataset is annotated at the pixel level, comprising 61 Landsat 8 images in 2023. The MSC-ResUNet model innovatively combines multiscale features through residual connections and multiscale skip connections, effectively capturing features ranging from low-level spatial details to high-level semantic information and further enhances performance by incorporating depthwise separable convolutions as part of the feature fusion process. Experimental results indicate that MSC-ResUNet achieves superior accuracy compared to other models, with F1 scores of 0.826 and 0.856, and MCC values of 0.816 and 0.847, in regional robustness and temporal transferability tests, respectively. Performance analysis across different months and band combinations demonstrates that the model maintains high recognition accuracy during both growing and non-growing seasons, despite the study area’s complex landforms and diverse crops.

TTMGNet: Tree Topology Mamba-Guided Network Collaborative Hierarchical Incremental Aggregation for Change Detection

Change detection (CD) identifies surface changes by analyzing bi-temporal remote sensing (RS) images of the same region and is essential for effective urban planning, ensuring the optimal allocation of resources, and supporting disaster management efforts. However, deep-learning-based CD methods struggle with background noise and pseudo-changes due to local receptive field limitations or computing resource constraints, which limits long-range dependency capture and feature integration, normally resulting in fragmented detections and high false positive rates. To address these challenges, we propose a tree topology Mamba-guided network (TTMGNet) based on Mamba architecture, which combines the Mamba architecture for effectively capturing global features, a unique tree topology structure for retaining fine local details, and a hierarchical feature fusion mechanism that enhances multi-scale feature integration and robustness against noise. Specifically, the a Tree Topology Mamba Feature Extractor (TTMFE) leverages the similarity of pixels to generate minimum spanning tree (MST) topology sequences, guiding information aggregation and transmission. This approach utilizes a Tree Topology State Space Model (TTSSM) to embed spatial and positional information while preserving the global feature extraction capability, thereby retaining local features. Subsequently, the Hierarchical Incremental Aggregation Module is utilized to gradually align and merge features from deep to shallow layers to facilitate hierarchical feature integration. Through residual connections and cross-channel attention (CCA), HIAM enhances the interaction between neighboring feature maps, ensuring that critical features are retained and effectively utilized during the fusion process, thereby enabling more accurate detection results in CD. The proposed TTMGNet achieved F1 scores of 92.31% on LEVIR-CD, 90.94% on WHU-CD, and 77.25% on CL-CD, outperforming current mainstream methods in suppressing the impact of background noise and pseudo-change and more accurately identifying change regions.

A multimodal educational robots driven via dynamic attention.

With the development of artificial intelligence and robotics technology, the application of educational robots in teaching is becoming increasingly popular. However, effectively evaluating and optimizing multimodal educational robots remains a challenge. This study introduces Res-ALBEF, a multimodal educational robot framework driven by dynamic attention. Res-ALBEF enhances the ALBEF (Align Before Fuse) method by incorporating residual connections to align visual and textual data more effectively before fusion. In addition, the model integrates a VGG19-based convolutional network for image feature extraction and utilizes a dynamic attention mechanism to dynamically focus on relevant parts of multimodal inputs. Our model was trained using a diverse dataset consisting of 50,000 multimodal educational instances, covering a variety of subjects and instructional content. The evaluation on an independent validation set of 10,000 samples demonstrated significant performance improvements: the model achieved an overall accuracy of 97.38% in educational content recognition. These results highlight the model's ability to improve alignment and fusion of multimodal information, making it a robust solution for multimodal educational robots.

Facial emotion recognition using deep quantum and advanced transfer learning mechanism.

Facial expressions have become a common way for interaction among humans. People cannot comprehend and predict the emotions or expressions of individuals through simple vision. Thus, in psychology, detecting facial expressions or emotion analysis demands an assessment and evaluation of decisions for identifying the emotions of a person or any group during communication. With the recent evolution of technology, AI (Artificial Intelligence) has gained significant usage, wherein DL (Deep Learning) based algorithms are employed for detecting facial expressions. The study proposes a system design that detects facial expressions by extracting relevant features using a Modified ResNet model. The proposed system stacks building-blocks with residual connections and employs an advanced extraction method with quantum computing, which significantly reduces computation time compared to conventional methods. The backbone stem utilizes a quantum convolutional layer comprised of several parameterized quantum-filters. Additionally, the research integrates residual connections in the ResNet-18 model with the Modified up Sampled Bottle Neck Process (MuS-BNP), retaining computational efficacy while benefiting from residual connections. The proposed model demonstrates superior performance by overcoming the issue of maximum similarity within varied facial expressions. The system's ability to accurately detect and differentiate between expressions is measured using performance metrics such as accuracy, F1-score, recall, and precision. This performance analysis confirms the efficacy of the proposed system, highlighting the advantages of quantum computing in feature extraction and the integration of residual connections. The model achieves quantum superiority, providing faster and more accurate computations compared to existing methodologies. The results suggest that the proposed approach offers a promising solution for facial expression recognition tasks, significantly improving both speed and accuracy.