Multi-scale Deep Learning Research Articles

A method for predicting remaining useful life using enhanced Savitzky–Golay filter and improved deep learning framework

Ensuring operational integrity in large-scale equipment hinges on effective fault prediction and health management. Prognostics and health management (PHM) face the challenge of accurately predicting remaining useful life (RUL) using multivariate sensor data. Traditional methods often require extensive prior knowledge for indicator construction and processing. Deep learning offers a promising alternative. This study presents a multi-channel multi-scale deep learning approach. Initially, an improved Savitzky‒Golay filter (ISG) addresses challenges posed by large and rapidly changing data volumes, enhancing data preprocessing. Subsequently, a framework integrates convolutional neural networks (CNNs) with long short-term memory (LSTM) to capture hierarchical signal information and make integrated predictions. The CNN extracts spatial features from multi-channel input data, while the LSTM captures temporal dependencies. By fusing outputs from both components, the framework enhances predictive accuracy and robustness for complex operational datasets. Experimental validation on the C-MAPSS dataset tests various fusion methods and CNN depths, determining parameters and evaluating filtering effectiveness. Comparative analyses show promising performance, particularly under dynamic conditions. While not optimal for predicting multiple fault types, it outperforms classical algorithms, especially in single fault type prediction tasks.

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

3D object instance segmentation plays a vital role in various applications such as autonomous driving, robotics and virtual reality. However, tabletop scenes exhibit diverse object complexities and size variations. The challenge is to enhance the accuracy of segmenting these scenes for multiple object instances. This limitation directly impacts robots’ capabilities to effectively grasp and manipulate objects. In this paper, we propose a multi-scale deep learning and clustering-based approach for object instance segmentation in tabletop scenes. Our approach incorporates a multi-scale neighborhood feature sampling (MNFS) module specifically designed to extract local features, and a clustering algorithm to eliminate noise and preserve instance integrity. Furthermore, we combine the strength of both methods through ScoreNet and non-maximal suppression. We conducted extensive experiments on TO-Scene, the first large-scale dataset of 3D tabletop scenes, and observed an average mIoU improvement of approximately 4.07% compared to existing methods. This highlights the superior performance of our proposed method. In addition, we tested our algorithm on a real-scene robotics platform and showed that it has good performance and generalization capabilities to support future applications such as robot grasping.

Streamflow Prediction in Human‐Regulated Catchments Using Multiscale Deep Learning Modeling With Anthropogenic Similarities

Streamflow Prediction in Human‐Regulated Catchments Using Multiscale Deep Learning Modeling With Anthropogenic Similarities

A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion

Medical imaging is indispensable for accurate diagnosis and effective treatment, with modalities like MRI and CT providing diverse yet complementary information. Traditional image fusion methods, while essential in consolidating information from multiple modalities, often suffer from poor image quality and loss of crucial details due to inadequate handling of semantic information and limited feature extraction capabilities. This paper introduces a novel medical image fusion technique leveraging unsupervised image segmentation to enhance the semantic understanding of the fusion process. The proposed method, named DUSMIF, employs a multi-branch, multi-scale deep learning architecture that integrates advanced attention mechanisms to refine the feature extraction and fusion processes. An innovative approach that utilizes unsupervised image segmentation to extract semantic information is introduced, which is then integrated into the fusion process. This not only enhances the semantic relevance of the fused images but also improves the overall fusion quality. The paper proposes a sophisticated network structure that extracts and fuses features at multiple scales and across multiple branches. This structure is designed to capture a comprehensive range of image details and contextual information, significantly improving the fusion outcomes. Multiple attention mechanisms are incorporated to selectively emphasize important features and integrate them effectively across different modalities and scales. This approach ensures that the fused images maintain high quality and detail fidelity. A joint loss function combining content loss, structural similarity loss, and semantic loss is formulated. This function not only guides the network in preserving image brightness and texture but also ensures that the fused image closely resembles the source images in both content and structure. The proposed method demonstrates superior performance over existing fusion techniques in objective assessments and subjective evaluations, confirming its effectiveness in enhancing the diagnostic utility of fused medical images.

An Improved Archimedes Optimization-aided Multi-scale Deep Learning Segmentation with dilated ensemble CNN classification for detecting lung cancer using CT images

ABSTRACT Early detection of lung cancer is necessary to prevent deaths caused by lung cancer. But, the identification of cancer in lungs using Computed Tomography (CT) scan based on some deep learning algorithms does not provide accurate results. A novel adaptive deep learning is developed with heuristic improvement. The proposed framework constitutes three sections as (a) Image acquisition, (b) Segmentation of Lung nodule, and (c) Classifying lung cancer. The raw CT images are congregated through standard data sources. It is then followed by nodule segmentation process, which is conducted by Adaptive Multi-Scale Dilated Trans-Unet3+. For increasing the segmentation accuracy, the parameters in this model is optimized by proposing Modified Transfer Operator-based Archimedes Optimization (MTO-AO). At the end, the segmented images are subjected to classification procedure, namely, Advanced Dilated Ensemble Convolutional Neural Networks (ADECNN), in which it is constructed with Inception, ResNet and MobileNet, where the hyper parameters is tuned by MTO-AO. From the three networks, the final result is estimated by high ranking-based classification. Hence, the performance is investigated using multiple measures and compared among different approaches. Thus, the findings of model demonstrate to prove the system’s efficiency of detecting cancer and help the patient to get the appropriate treatment.

Multi-Scale Deep Learning-Based University Financial System: Hardware Design and Database Integration

With the increasing scale of college enrollment and the increasing complexity of college teaching management, college finance department should innovate the traditional financial management mode while adapting to the reform of teaching management, and make use of the openness and real-time characteristics of Internet to improve the quality of college financial management and reduce the risk of college financial management. To this end, this paper designs a university financial system based on multi-scale deep learning. In the hardware design, the system adds multiple sensors and scans all the information in the financial database using a coordinator. In the software design, the weights that can connect the financial information of the same attribute are set by establishing a database form; according to the multilayer perceptual network topology, a full interconnection model based on multi-scale deep learning is designed to realize the system’s deep extraction of data. The experimental results show that the financial risk is based on the risk warning capability for university finance, and compared with the system under the traditional design, the university finance system designed in this time has the most categories of financial information parameters extracted.

Multi-scale deep learning ensemble for segmentation of endometriotic lesions

Ultrasound is a readily available, non-invasive and low-cost screening for the identification of endometriosis lesions, but its diagnostic specificity strongly depends on the experience of the operator. For this reason, computer-aided diagnosis tools based on Artificial Intelligence techniques can provide significant help to the clinical staff, both in terms of workload reduction and in increasing the overall accuracy of this type of examination and its outcome. However, although these techniques are spreading rapidly in a variety of domains, their application to endometriosis is still very limited. To fill this gap, we propose and evaluate a novel multi-scale ensemble approach for the automatic segmentation of endometriosis lesions from transvaginal ultrasounds. The peculiarity of the method lies in its high discrimination capability, obtained by combining, in a fusion fashion, multiple Convolutional Neural Networks trained on data at different granularity. The experimental validation carried out shows that: (i) the proposed method allows to significantly improve the performance of the individual neural networks, even in the presence of a limited training set; (ii) with a Dice coefficient of 82%, it represents a valid solution to increase the diagnostic efficacy of the ultrasound examination against such a pathology.

Innovative Financial Management in Higher Education: A Multi-Scale Deep Learning Approach for Risk Reduction and Quality Enhancement

This paper presents a novel university financial management system leveraging multi-scale deep learning. With rising college enrollment and teaching complexities, traditional financial models require adaptation to mitigate risks and improve management quality. The system integrates hardware and software innovations: multiple sensors enhance data scanning, coordinated by a central coordinator, ensuring comprehensive financial database coverage. Software-wise, a structured database establishes attribute-based financial connections, crucial for weight assignment. Employing a multilayer perceptual network topology, a full interconnection model based on multi-scale deep learning facilitates profound data extraction. Experimental evaluations demonstrate the system's superior financial risk assessment capabilities compared to traditional approaches, extracting a broader spectrum of financial parameters for comprehensive risk warnings. By embracing multi-scale deep learning, this system promises significant advancements in university financial management, enhancing adaptability and risk mitigation in college finance departments.

Revealing the reconstruction mechanism of AgPd nanoalloys under fluorination based on a multiscale deep learning potential.

The design of heterogeneous catalysts generally involves optimizing the reactivity descriptor of adsorption energy, which is inevitably governed by the structure of surface-active sites. A prerequisite for understanding the structure-properties relationship is the precise identification of real surface-active site structures, rather than relying on conceived structures derived from bulk alloy properties. However, it remains a formidable challenge due to the dynamic nature of nanoalloys during catalytic reactions and the lack of accurate and efficient interatomic potentials for simulations. Herein, a generalizable deep-learning potential for the Ag-Pd-F system is developed based on a dataset encompassing the bulk, surface, nanocluster, amorphous, and point defected configurations with diverse compositions to achieve a comprehensive description of interatomic interactions, facilitating precise prediction of adsorption energy, surface energy, formation energy, and diffusion energy barrier and is utilized to investigate the structural evolutions of AgPd nanoalloys during fluorination. The structural evolutions involve the inward diffusion of F, the outward diffusion of Ag in Ag@Pd nanoalloys, the formation of surface AgFx species in mixed and Janus AgPd nanoalloys, and the shape deformation from cuboctahedron to sphere in Ag and Pd@Ag nanoalloys. Moreover, the effects of atomic diffusion and dislocation formation and migration on the reconstructing pathway of nanoalloys are highlighted. It is demonstrated that the stress relaxation upon F adsorption serves as the intrinsic driving factor governing the surface reconstruction of AgPd nanoalloys.

Bringing vision to climate: A hierarchical model for water depth monitoring in headwater streams

Stream-observing cameras have recently been deployed in stream systems to monitor water depth dynamics. However, most existing image-based water depth monitoring methods require additional gauging equipment, extensive manual annotations, or complex manual calibration. In this paper, we propose the hierarchical model, a novel multi-modal and multi-scale deep learning framework for monitoring water depth in headwater streams with only a field camera capable of night vision and no additional equipment. In particular, the hierarchical model integrates long-term dynamic patterns extracted from large-scale meteorological data with short-term dynamic patterns extracted from small-scale stream image data to jointly monitor water depth at a fine-level temporal resolution. In order to overcome the issue of limited availability of images, we introduce a transfer learning strategy and incorporate more accurate long-term patterns that enable the hierarchical model to perform competitively even with a small number of images. We evaluate our method on a real-world headwater stream monitoring dataset from the West Brook study area in western Massachusetts, United States. Our extensive experiments demonstrate that the hierarchical model outperforms several state-of-the-art methods for water depth monitoring, and that more accurate long-term patterns can better guide the monitoring of short-term patterns with excellent flexibility and less computational cost. The mean absolute error of our hierarchical model achieves a remarkable level of 4.9cm at the study site with 0.89m average water depths, and only 12.5cm at more drastically varied site with 3.95m average depths.

A Data-Driven Autonomous Assessment Framework for Education Quality Based on Multiscale Deep Learning

Nowadays, computational intelligence-assisted autonomous assessment of education quality has become a more and more general concern in the area of smart education management. As education quality assessment is a complicated process with multiple heterogeneous factors, it remains challenging to make effective assessment using simple information modality and criteria. To deal with this issue, this paper introduces multiscale deep learning, and proposes a novel data-driven autonomous assessment framework for education quality. In particular, four aspects of heterogeneous indicators are selected as the basic indexes and a dilated convolutional neural network structure is formulated to perform multiscale feature extraction. Then, the structural equation is adopted to make multivariate characterization and output the final assessment results. At last, some simulations are carried out on realistic education operation data to evaluate the efficiency of the proposed autonomous assessment framework. Two aspects of findings can be deduced from the results. For one thing, multisource fusion of multiple indicators well makes sense to autonomous education assessment. For another, multiscale deep learning can provide some beneficial promotion to assessment efficiency.

Interpretable Multi-Scale Deep Learning for RNA Methylation Analysis across Multiple Species.

RNA modification plays a crucial role in cellular regulation. However, traditional high-throughput sequencing methods for elucidating their functional mechanisms are time-consuming and labor-intensive, despite extensive research. Moreover, existing methods often limit their focus to specific species, neglecting the simultaneous exploration of RNA modifications across diverse species. Therefore, a versatile computational approach is necessary for interpretable analysis of RNA modifications across species. A multi-scale biological language-based deep learning model is proposed for interpretable, sequential-level prediction of diverse RNA modifications. Benchmark comparisons across species demonstrate the model's superiority in predicting various RNA methylation types over current state-of-the-art methods. The cross-species validation and attention weight visualization also highlight the model's capability to capture sequential and functional semantics from genomic backgrounds. Our analysis of RNA modifications helps us find the potential existence of "biological grammars" in each modification type, which could be effective for mapping methylation-related sequential patterns and understanding the underlying biological mechanisms of RNA modifications.

Numerical approximation based on deep convolutional neural network for high‐dimensional fully nonlinear merged PDEs and 2BSDEs

This paper proposes two efficient approximation methods to solve high‐dimensional fully nonlinear partial differential equations (NPDEs) and second‐order backward stochastic differential equations (2BSDEs), where such high‐dimensional fully NPDEs are extremely difficult to solve because the computational cost of standard approximation methods grows exponentially with the number of dimensions. Therefore, we consider the following methods to overcome this difficulty. For the merged fully NPDEs and 2BSDEs system, combined with the time forward discretization and ReLU function, we use multiscale deep learning fusion and convolutional neural network (CNN) techniques to obtain two numerical approximation schemes, respectively. Finally, three practical high‐dimensional test problems involving Allen–Cahn, Black–Scholes–Barenblatt, and Hamilton–Jacobi–Bellman equations are given so that the first proposed method exhibits higher efficiency and accuracy than the existing method, while the second proposed method can extend the dimensionality of the completely NPDEs–2BSDEs system over 400 dimensions, from which the numerical results highlight the effectiveness of proposed methods.

The MS-RadarFormer: A Transformer-Based Multi-Scale Deep Learning Model for Radar Echo Extrapolation

As a spatial–temporal sequence prediction task, radar echo extrapolation aims to predict radar echoes’ future movement and intensity changes based on historical radar observations. Two urgent issues still need to be addressed in deep learning radar echo extrapolation models. First, the predicted radar echo sequences often exhibit echo-blurring phenomena. Second, over time, the output echo intensities from the model gradually weaken. In this paper, we propose a novel model called the MS-RadarFormer, a Transformer-based multi-scale deep learning model for radar echo extrapolation, to mitigate the two above issues. We introduce a multi-scale design in the encoder–decoder structure and a Spatial–Temporal Attention block to improve the precision of radar echoes and establish long-term dependencies of radar echo features. The model uses a non-autoregressive approach for echo prediction, avoiding accumulation errors during the recursive generation of future echoes. Compared to the baseline, our model shows an average improvement of 15.8% in the critical success index (CSI), an average decrease of 8.3% in the false alarm rate (FAR), and an average improvement of 16.2% in the Heidke skill score (HSS).

Multiscale Deep Learning for Detection and Recognition: A Comprehensive Survey.

Recently, the multiscale problem in computer vision has gradually attracted people's attention. This article focuses on multiscale representation for object detection and recognition, comprehensively introduces the development of multiscale deep learning, and constructs an easy-to-understand, but powerful knowledge structure. First, we give the definition of scale, explain the multiscale mechanism of human vision, and then lead to the multiscale problem discussed in computer vision. Second, advanced multiscale representation methods are introduced, including pyramid representation, scale-space representation, and multiscale geometric representation. Third, the theory of multiscale deep learning is presented, which mainly discusses the multiscale modeling in convolutional neural networks (CNNs) and Vision Transformers (ViTs). Fourth, we compare the performance of multiple multiscale methods on different tasks, illustrating the effectiveness of different multiscale structural designs. Finally, based on the in-depth understanding of the existing methods, we point out several open issues and future directions for multiscale deep learning.

A Hybrid Model for Long-Term Wind Power Forecasting Utilizing NWP Subsequence Correction and Multi-Scale Deep Learning Regression Methods

The accuracy of long-term wind power forecasting (WPF) is crucial for the efficient operation of grid systems. However, wind power generation is highly stochastic and intermittent due to the influence of weather, which makes long-term WPF less effective. Numerical weather prediction (NWP) data contains valuable weather forecast information, which can mitigate the negative effects of stochastic weather fluctuations on WPF. However, the accuracy of NWP data decreases over time, and multiple NWP data can have redundancy and errors that make it challenging to extract valid information. Reducing the variety and errors in NWP data and using more effective information extraction methods are essential for improving long-term WPF performance. In this article, we propose a novel long-term WPF hybrid model that corrects NWP wind speed and uses multi-scale deep learning regression prediction to exclude excessive NWP data. We use only the corrected NWP wind speed data to establish a nonlinear mapping relationship with actual power data. The validation case study shows that our proposed model reduces the mean squared error (MSE) and mean absolute error (MAE) by 65.0% and 43.8%, respectively, compared to the current state-of-the-art time series forecasting model in a seven-day forecasting scenario.

Enhancing aircraft engine remaining useful life prediction via multiscale deep transfer learning with limited data

Abstract Predicting the remaining useful life (RUL) of the aircraft engine based on historical data plays a pivotal role in formulating maintenance strategies and mitigating the risk of critical failures. None the less, attaining precise RUL predictions often encounters challenges due to the scarcity of historical condition monitoring data. This paper introduces a multiscale deep transfer learning framework via integrating domain adaptation principles. The framework encompasses three integral components: a feature extraction module, an encoding module, and an RUL prediction module. During pre-training phase, the framework leverages a multiscale convolutional neural network to extract distinctive features from data across varying scales. The ensuing parameter transfer adopts a domain adaptation strategy centered around maximum mean discrepancy. This method efficiently facilitates the acquisition of domain-invariant features from the source and target domains. The refined domain adaptation Transformer-based multiscale convolutional neural network model exhibits enhanced suitability for predicting RUL in the target domain under the condition of limited samples. Experiments on the C-MAPSS dataset have shown that the proposed method significantly outperforms state-of-the-art methods.

Transformer encoder with multiscale deep learning for pain classification using physiological signals.

Pain, a pervasive global health concern, affects a large segment of population worldwide. Accurate pain assessment remains a challenge due to the limitations of conventional self-report scales, which often yield inconsistent results and are susceptible to bias. Recognizing this gap, our study introduces PainAttnNet, a novel deep-learning model designed for precise pain intensity classification using physiological signals. We investigate whether PainAttnNet would outperform existing models in capturing temporal dependencies. The model integrates multiscale convolutional networks, squeeze-and-excitation residual networks, and a transformer encoder block. This integration is pivotal for extracting robust features across multiple time windows, emphasizing feature interdependencies, and enhancing temporal dependency analysis. Evaluation of PainAttnNet on the BioVid heat pain dataset confirm the model's superior performance over the existing models. The results establish PainAttnNet as a promising tool for automating and refining pain assessments. Our research not only introduces a novel computational approach but also sets the stage for more individualized and accurate pain assessment and management in the future.

Skin cancer detection using multi-scale deep learning and transfer learning

Skin cancer detection using multi-scale deep learning and transfer learning

Estimation of 2D profile dynamics of electrostatic potential fluctuations using multi-scale deep learning

Abstract Dynamics in magnetically confined plasmas are dominated by turbulence driven by spatial inhomogeneities in density and temperature. Simultaneous measurement of velocity field and density fluctuations is necessary to observe the particle transport, but the measurement of the velocity field fluctuations is often challenging. Here, we propose a method to estimation velocity field fluctuations from density fluctuations by using plasma turbulence simulations and a deep technique learning. In order to take multi-scale characteristics into account, the several number of spatial filters are used in the convolutional neural network. The velocity field fluctuations are successfully predicted, and the particle transport estimated from the predicted velocity field fluctuations is within 93.1% accuracy. The deep learning could be used for the prediction of physical variables which are difficult to be measured.