Multi-scale Network Model Research Articles

A multi-scale temporal convolutional capsule network with parameter-free attention module-dynamic routing for intelligent diagnosis of rolling bearing

Abstract We proposed a multi-scale temporal convolutional capsule network model coupled with a parameter-free attention module and dynamic routing mechanism to analyze complex vibration signals for diagnosing the health of bearings. The proposed method utilizes a capsule network as the fundamental architecture. Instead of a convolutional neural network, a temporal convolutional network is employed. Additionally, a multi-scale feature fusion module is integrated into the capsule network structure to dynamically extract various layers of features from fault samples, enhancing the discriminatory capability of abnormal data. Subsequently, the parameter-free attention module and dynamic routing mechanism are employed to construct digital capsules. This allows the smallest unit capsule in a single layer to carry more information, enhance the similarity between the instance primary capsule and the fault capsule, reduce the interference of irrelevant features to the model, and improve the accuracy of fault type recognition. Finally, a multi-scale temporal convolutional capsule network model that integrates feature extraction and pattern recognition is established to perform end-to-end diagnosis of the bearing. Experimental findings suggest that the proposed method outperforms other deep learning methods in terms of accuracy and robustness. It can provide a theoretical basis and implementation path for the detection and diagnosis of train wheelset bearing time series abnormal data.

Multi-Scale convolutional neural network for finger vein recognition

With the continuous advancement of science and technology, an increasing number of deep learning methods are being applied in the field of finger vein recognition to describe the structural characteristics of finger veins. However, some deep learning methods fail to adequately extract longer texture features. during the feature extraction process, resulting in a decrease in the uniqueness of extracted finger vein features. Additionally, these methods tend to extract global information while neglecting the importance of local texture information. To address the aforementioned issues, this paper introduces a multiscale convolution network (MCNet) model based on finger vein structure. On one hand, a multiscale feature extraction (MFE) model based on the rectangular and square convolution kernels are employed to extract structural information from finger veins and to simultaneously enhance the features of longer texture features. On the other hand, the paper introduces a cross-information fusion attention (CFA) block that combines spatial and channel information, in order to enhance local details information and the network’s ability to extract vein patterns. The experimental results on the public datasets FV-USM, SDUMLA-HMT, and HKPU validate the effectiveness of MCNet with the recognition rates of 99.86%, 99.11%, and 99.15% respectively.

Multi-scale spiking network model of human cerebral cortex.

Although the structure of cortical networks provides the necessary substrate for their neuronal activity, the structure alone does not suffice to understand the activity. Leveraging the increasing availability of human data, we developed a multi-scale, spiking network model of human cortex to investigate the relationship between structure and dynamics. In this model, each area in one hemisphere of the Desikan-Killiany parcellation is represented by a $1\,\mathrm{mm^{2}}$ column with a layered structure. The model aggregates data across multiple modalities, including electron microscopy, electrophysiology, morphological reconstructions, and diffusion tensor imaging, into a coherent framework. It predicts activity on all scales from the single-neuron spiking activity to the area-level functional connectivity. We compared the model activity with human electrophysiological data and human resting-state functional magnetic resonance imaging (fMRI) data. This comparison reveals that the model can reproduce aspects of both spiking statistics and fMRI correlations if the inter-areal connections are sufficiently strong. Furthermore, we study the propagation of a single-spike perturbation and macroscopic fluctuations through the network. The open-source model serves as an integrative platform for further refinements and future in silico studies of human cortical structure, dynamics, and function.

A clustered federated learning framework for collaborative fault diagnosis of wind turbines

Data-driven approaches demonstrate significant potential in accurately diagnosing faults in wind turbines. To enhance diagnostic performance and reduce communication costs in federated learning with data heterogeneity among different clients, we introduce a clustered federated learning framework to wind turbine fault diagnosis. Initially, a lightweight multiscale separable residual network (LMSRN) model is proposed for each local client. The LMSRN model integrates a multiscale spatial feature derivation unit and a depthwise separable feature extraction unit. Subsequently, to tackle data heterogeneity among clients, canonical correlation coefficients of representations are extracted from the intermediate layers of local LMSRN models, and a representational canonical correlation clustering (RCCC) method is proposed to assess the similarity of local LMSRN models and group them into clusters. Finally, a global model is trained for each cluster. Real-world wind turbine data experiments showcase the superior performance of the proposed clustered federated learning framework over traditional methods in terms of diagnostic accuracy and computational speed. Additionally, the optimal choice of the number of clusters is also discussed.

Prediction of mechanical properties of rolled steel based on dual-attention multiscale convolutional neural network

The mechanical properties of carbon steel sheet directly depend on the chemical composition and process parameters of the steel. There is a complex nonlinear relationship between chemical composition, process parameters, and mechanical properties, and the establishment of a model with the ability to automatically distinguish the importance of different parameters, and to have good prediction accuracy and generalisation in different data sets is one of the important issues in the field of carbon steel mechanical properties prediction. In this paper, we proposed a multi-scale convolutional network model based on dual-attention mechanism (DAM-MSDSC), where the dual-attention module mines from features and channels at the same time, making the model focus on the important features. The multiscale feature fusion module effectively improves the model prediction accuracy by mining features from the output of the middle layer of the network to obtain features containing multiscale fusion features. In hot-rolled steel, for the two parameters of carbon content and temperature, it is confirmed that the change of carbon content has a greater impact on TS, while FTT and CT have a greater impact on YS. Through comparative experiments on different datasets, the proposed model had the best prediction performance, in which the correct rates of YS, TS, and EL are 98 %, 98.7 %, and 99.4 %, respectively, in cold rolled steel, and 97.8 %, 98.6 %, and 99.2 %, respectively, in hot rolled steel.

Using the MSFNet Model to Explore the Temporal and Spatial Evolution of Crop Planting Area and Increase Its Contribution to the Application of UAV Remote Sensing

Remote sensing technology can be used to monitor changes in crop planting areas to guide agricultural production management and help achieve regional carbon neutrality. Agricultural UAV remote sensing technology is efficient, accurate, and flexible, which can quickly collect and transmit high-resolution data in real time to help precision agriculture management. It is widely used in crop monitoring, yield prediction, and irrigation management. However, the application of remote sensing technology faces challenges such as a high imbalance of land cover types, scarcity of labeled samples, and complex and changeable coverage types of long-term remote sensing images, which have brought great limitations to the monitoring of cultivated land cover changes. In order to solve the abovementioned problems, this paper proposed a multi-scale fusion network (MSFNet) model based on multi-scale input and feature fusion based on cultivated land time series images, and further combined MSFNet and Model Diagnostic Meta Learning (MAML) methods, using particle swarm optimization (PSO) to optimize the parameters of the neural network. The proposed method is applied to remote sensing of crops and tomatoes. The experimental results showed that the average accuracy, F1-score, and average IoU of the MSFNet model optimized by PSO + MAML (PSML) were 94.902%, 91.901%, and 90.557%, respectively. Compared with other schemes such as U-Net, PSPNet, and DeepLabv3+, this method has a better effect in solving the problem of complex ground objects and the scarcity of remote sensing image samples and provides technical support for the application of subsequent agricultural UAV remote sensing technology. The study found that the change in different crop planting areas was closely related to different climatic conditions and regional policies, which helps to guide the management of cultivated land use and provides technical support for the realization of regional carbon neutrality.

State of Health Estimations for Lithium-Ion Batteries Based on MSCNN

Lithium-ion batteries, essential components in new energy vehicles and energy storage stations, play a crucial role in health-status investigation and ensuring safe operation. To address challenges such as limited estimation accuracy and a weak generalization ability in conventional battery state of health (SOH) estimation methods, this study presents an integrated approach for SOH estimation that incorporates multiple health indicators and utilizes the multi-scale convolutional neural network (MSCNN) model. Initially, the aging characteristics of the battery are comprehensively analyzed, and then the health indicators are extracted from the charging data, including the temperature, time, current, voltage, etc., and the statistical transformation is performed. Subsequently, Pearson’s method is employed to analyze the correlation between these health indicators and identify those with strong correlations. A regression-prediction model based on the MSCNN model is then developed for estimating battery SOH. Finally, validation using a publicly available lithium-ion battery dataset demonstrates that, under similar operating conditions, the mean absolute error (MAE) for SOH estimation is less than 0.67%, the mean absolute percentage error (MAPE) is less than 0.37%, and the root mean square error (RMSE) is less than 0.74%. The MSCNN has good generalization for datasets with different working conditions.

GR-GAN: A unified adversarial framework for single image glare removal and denoising

In this paper, we introduce the single image glare removal (SIGR) task. SIGR aims to eliminate glare or light caused by external environment in lighting scene. To efficiently improve natural image quality and usability, many research tasks, such as image deraining and shadow removal, have been investigated a lot. However, SIGR is still underexplored. Therefore, we propose to construct a dataset and explore deep learning-based models for SIGR. Our contributions can be summarized as follows: (1) We establish a new benchmark dataset for SIGR, termed De-Glare, aiming to propel research of SIGR. This dataset comprises pairs of {glare, glare-free} images sourced from both real-world and synthetic data, utilized for training and evaluating models. (2) We conduct a comprehensive benchmarking of extensive state-of-the-art (SoTA) methods on the constructed De-Glare dataset and provide insightful analyses based on the results. (3) An innovative approach for single image glare removal employing a multi-scale generative adversarial network (GR-GAN) model is proposed. Glare images typically exhibit irregular glare shapes and cluttered backgrounds. To address these irregular glare patterns, we introduce a deformable convolution-based glare attention detector (GAD) designed to generate an attention map which specifics glare spots or rays in the input image. In pursuit of enhancing the perceptual quality of output image, GR-GAN adaptively filters out irrelevant noises and enhances salient features through a generator with cascaded pyramid neck (CPN) network. This work can provide useful insights for developing better SIGR models. Without specific tuning, our method achieves the SoTA results on multiple computer vision tasks, including the image deraining and image shadow removal.

Mapping the strain-stiffening behavior of the lung and lung cancer at microscale resolution using the crystal ribcage.

Lung diseases such as cancer substantially alter the mechanical properties of the organ with direct impact on the development, progression, diagnosis, and treatment response of diseases. Despite significant interest in the lung's material properties, measuring the stiffness of intact lungs at sub-alveolar resolution has not been possible. Recently, we developed the crystal ribcage to image functioning lungs at optical resolution while controlling physiological parameters such as air pressure. Here, we introduce a data-driven, multiscale network model that takes images of the lung at different distending pressures, acquired via the crystal ribcage, and produces corresponding absolute stiffness maps. Following validation, we report absolute stiffness maps of the functioning lung at microscale resolution in health and disease. For representative images of a healthy lung and a lung with primary cancer, we find that while the lung exhibits significant stiffness heterogeneity at the microscale, primary tumors introduce even greater heterogeneity into the lung's microenvironment. Additionally, we observe that while the healthy alveoli exhibit strain-stiffening of ∼1.75 times, the tumor's stiffness increases by a factor of six across the range of measured transpulmonary pressures. While the tumor stiffness is 1.4 times the lung stiffness at a transpulmonary pressure of three cmH2O, the tumor's mean stiffness is nearly five times greater than that of the surrounding tissue at a transpulmonary pressure of 18 cmH2O. Finally, we report that the variance in both strain and stiffness increases with transpulmonary pressure in both the healthy and cancerous lungs. Our new method allows quantitative assessment of disease-induced stiffness changes in the alveoli with implications for mechanotransduction.

Reinforced convolutional neural network fault diagnosis of industrial production systems

A new fault diagnosis method is proposed combining multiple models into a multiscale residual jagged dilated convolutional neural network with long short-term memory (MRJDCNN-LSTM). This method has strong feature extraction ability and can effectively extract and classify fault features. First, a new multi-scale residual jagged dilated convolution neural network (MRJDCNN) model is designed by incorporating the principle of residual learning into the improved jagged dilated convolution, and then it is combined with the long short term memory(LSTM) network delicately. This not only greatly improves the extraction rate of nonlinear high-latitude spatial features at different scales while preventing model degradation, but also subtly extracts time-related features, effectively reducing the loss of necessary features, and thus greatly improving the accuracy of model diagnosis. The simulation and comparative experiments of Tennessee-Eastman (TE) process and industrial coking furnace both prove that the constructed model has excellent performance.

Multi-focused image fusion algorithm based on multi-scale hybrid attention residual network.

In order to improve the detection performance of image fusion in focus areas and realize end-to-end decision diagram optimization, we design a multi-focus image fusion network based on deep learning. The network is trained using unsupervised learning and a multi-scale hybrid attention residual network model is introduced to enable solving for features at different levels of the image. In the training stage, multi-scale features are extracted from two source images with different focal points using hybrid multi-scale residual blocks (MSRB), and the up-down projection module (UDP) is introduced to obtain multi-scale edge information, then the extracted features are operated to obtain deeper image features. These blocks can effectively utilize multi-scale feature information without increasing the number of parameters. The deep features of the image are extracted in its test phase, input to the spatial frequency domain to calculate and measure the activity level and obtain the initial decision map, and use post-processing techniques to eliminate the edge errors. Finally, the decision map is generated and optimized, and the final fused image is obtained by combining the optimized decision map with the source image. The comparative experiments show that our proposed model achieves better fusion performance in subjective evaluation, and the quality of the obtained fused images is more robust with richer details. The objective evaluation metrics work better and the image fusion quality is higher.

CASF-MNet: multi-scale network with cross attention mechanism and spatial dimension feature fusion for maize leaf disease detection

Maize is an important food crop, and accurate identification of maize leaf diseases is significant for disease control. Aiming at the complexity of the color and texture features of diseases with inter-class similarity and intra-class variability, a two-channel multi-scale network model with a cross-attention mechanism and spatial dimension feature fusion (CASF-MNet) was designed. Firstly, aiming at the complexity of color features, the HSV color Sub-network (CSNet) was designed to extract the color features of disease spots. Then, aiming at the complexity of texture features, Texture Sub-Network (TSNet) and Cross Attention mechanism (CAM) were proposed to enhance the texture features of disease spots. Finally, to better fuse color and texture features, a cross-spatial feature fusion mechanism (FFM) was proposed. In the five-fold cross-validation of the self-built dataset, the average F1-Score and average Accuracy of CASF-MNet reach 97.17% and 97.02%. In the test experiments of two public datasets, the F1-Score of CASF-MNet reaches 98.13% and 98.71%, and the Accuracy reaches 97.44% and 98.67%, which are better than the seven most advanced leaf disease identification methods. It is proved that CASF-MNet is advanced and stable, and can assist disease detection personnel in maize disease detection in field environments.

A novel multi-scale CNN and Bi-LSTM arbitration dense network model for low-rate DDoS attack detection

Low-rate distributed denial of service attacks, as known as LDDoS attacks, pose the notorious security risks in cloud computing network. They overload the cloud servers and degrade network service quality with the stealthy strategy. Furthermore, this kind of small ratio and pulse-like abnormal traffic leads to a serious data scale problem. As a result, the existing models for detecting minority and adversary LDDoS attacks are insufficient in both detection accuracy and time consumption. This paper proposes a novel multi-scale Convolutional Neural Networks (CNN) and bidirectional Long-short Term Memory (bi-LSTM) arbitration dense network model (called MSCBL-ADN) for learning and detecting LDDoS attack behaviors under the condition of limited dataset and time consumption. The MSCBL-ADN incorporates CNN for preliminary spatial feature extraction and embedding-based bi-LSTM for time relationship extraction. And then, it employs arbitration network to re-weigh feature importance for higher accuracy. At last, it uses 2-block dense connection network to perform final classification. The experimental results conducted on popular ISCX-2016-SlowDos dataset have demonstrated that the proposed MSCBL-ADN model has a significant improvement with high detection accuracy and superior time performance over the state-of-the-art models.

Quantitative characterization of fractures and holes in core rolling scan images based on the MFAPNet deep learning model

The exploration and comprehensive assessment of fractured-vuggy reservoir information have perennially constituted focal points and challenges within the domain of oil and gas reservoir evaluation. The verification of geological phenomena, identification of various fracture and hole types, and the quantitative characterization thereof currently present pressing challenges. This study meticulously examines the deep carbonate reservoirs within the Dengying Formation in the Penglai gas region of the Sichuan Basin. The Core Rolling Scan images reveal five discernible features: unfilled holes, filled holes, filled fractures, open fractures, and algae. The analysis pinpoints three primary challenges in semantic segmentation recognition: the amalgamation of feature scales, class imbalance, and the scarcity of datasets with substantial sample sizes. To address these challenges, this paper introduces a Multi-Scale Feature Aggregation Pyramid Network model (MFAPNet), achieving a pixel accuracy of 68.04% in recognizing the aforementioned five types. Lastly, the model is employed in calculating core porosity, exposing a scaling relationship between wellbore image porosity and core porosity ranging from 1.5 to 3 times. To a certain extent, it reveals the correlation between the wellbore image logging data and the actual formation of the Dengying Formation in the Penglai Gas Field of the Sichuan Basin, and also provides a basis for the subsequent logging evaluation of the formation. The partial code and CHA355 dataset are publicly available at https://github.com/zyng886/MFAPNet.

Fault Diagnosis of Rotating Machinery under Complex Conditions Based on Multi-Scale Convolutional Neural Networks

In complex operational scenarios involving variable speeds, burdens, and noise, rotating machinery necessitates extended maintenance. Extracting stable and effective fault-sensitive features under such intricate conditions presents a significant challenge. To tackle this issue, the paper introduces the Multi-Scale Convolutional Neural Network (MSCNN) model tailored specifically for such complexities. The approach in this paper simultaneously captures multi-scale vibration signal features using the innovative Multi-Scale Bifurcation (MSB) module and subsequently aggregates them through a multi-scale fusion layer. This model effectively addresses the common problem of low CNN accuracy in fault diagnosis. The paper validates methodology using a bearing dataset provided by Case Western Reserve University (CWRU) and demonstrates superior performance compared to classical CNN models and other alternatives, achieving an impressive 99.56% classification accuracy for normal signals.

A pore network-based multiscale coupled model for rapid permeability prediction of tight sandstone gas

Tight sandstone has multiscale pore structures. Gas transport in tight sandstones involves several length scales and the complicated physics. Gas transport characteristics in tight sandstone can be represented by apparent gas permeability. Microstructure-based accurate and efficient calculation of gas permeability is challenging. By combining CT and SEM images along with statistical analysis, we present a novel pore network-based multiscale coupled model (MCPNM) to rapidly predict the apparent gas permeability. CT images are adopted to extract the large-scale pore network (LPNM) and the clay component, and SEM image is used to get the properties of small-scale pores. Upscaled model (UM) of small-scale pores is built via statistical analysis and then assigned to the clay domains. The LPNM and UM are coupled as MCPNM by the cross-scale connection structure with variable diameter. The pore spaces at several length scales and the flow characteristics in them are included in the MCPNM. We validate the MCPNM by comparing the calculated apparent gas permeability to the results of available multiscale pore network models and the experimental data. Compared with the available multiscale pore network models, MCPNM solves the accuracy/efficiency trade-off of tight gas permeability prediction. The effects of pressure, temperature, and gas type on gas permeability are studied. The MCPNM simplifies the permeability calculation process and can accurately and rapidly predict tight sandstone gas permeability.

Leveraging deep learning techniques for ship pipeline valve leak monitoring

The monitoring of ship pipeline valve leaks holds immense significance as it serves to enhance ship safety, mitigate energy and material losses, and protect the health and sustainability of the marine environment. In comparison to traditional expert-based fault diagnosis methods, this study presents a deep learning-based fault feature extraction approach for end-to-end valve leak fault diagnosis. To address the challenge of incomplete leak fault description by a single sensor, a Multi-Channel Multi-Scale Convolutional Neural Network (MCMS-CNN) model is established. Diverging from existing methods, the proposed MCMS-CNN model automatically extracts fault features from grayscale images obtained from two sensors and performs feature-level fusion for fault classification, thereby mitigating the impact of data redundancy and noise and enhancing fault recognition accuracy. The effectiveness of the proposed approach is verified through valve leak experiments, and comparisons are conducted with different models and information fusion methods. The results demonstrate that the proposed MCMS-CNN model exhibits advantages in terms of accuracy and robustness, confirming its practicality in ship pipeline valve leak fault diagnosis.

Multiscale network modeling reveals the gene regulatory landscape driving cancer prognosis in 32 cancer types.

Cancer is a complex disease with diverse molecular mechanisms that affect patient prognosis. Network-based approaches are effective in revealing a holistic picture of cancer prognosis and gene interactions. However, a comprehensive landscape of coexpression networks and prognostic gene modules across multiple cancer types remains elusive. In this study, we performed a systematic analysis of coexpression networks in 32 cancer types. Our analysis identified 4749 prognostic modules that play a vital role in regulating cancer progression. Integrative epigenomic analyses revealed that these modules were regulated by interactions between gene expression and methylation. Coregulated genes of network modules were enriched in chromosome cytobands and preferentially localized in open chromatin regions. The preserved network modules formed 330 module clusters that resided in chromosome hot spots. The cancer-type-specific prognostic modules participated in unique essential biological processes in different cancer types. Overall, our study provides rich resources of prevalent gene networks and underlying multiscale regulatory mechanisms driving cancer prognosis, which lay a foundation for biomarker discovery and therapeutic target development.

MSEN: A Multi-Scale Evolutionary Network for Modeling the Evolution of Temporal Knowledge Graphs

Temporal knowledge graphs play an increasingly prominent role in scenarios such as social networks, finance, and smart cities. As such, research on temporal knowledge graphs continues to deepen. In particular, research on temporal knowledge graph reasoning holds great significance, as it can provide abundant knowledge for downstream tasks such as question answering and recommendation systems. Current reasoning research focuses primarily on interpolation and extrapolation. Extrapolation research aims to predict the likelihood of events occurring in future timestamps. Historical events are crucial for predicting future events. However, existing models struggle to fully capture the evolutionary characteristics of historical knowledge graphs. This paper proposes a multi-scale evolutionary network (MSEN) model that leverages Hierarchical Transfer aware Graph Neural Network (HT-GNN) in a local memory encoder to aggregate rich structural semantics from each timestamp’s knowledge graph. It also utilizes Time Related Graph Neural Network (TR-GNN) in a global memory encoder to model temporal-semantic dependencies of entities across the global knowledge graph, mining global evolutionary patterns. The model integrates information from both encoders to generate entity embeddings for predicting future events. The proposed MSEN model demonstrates strong performance compared to several baselines on typical benchmark datasets. Results show MSEN achieves the highest prediction accuracy.

Classification of Epileptic Seizure Types Using Multiscale Convolutional Neural Network and Long Short-Term Memory

The accurate classification of seizure types using electroencephalography (EEG) signals plays a vital role in determining a precise treatment plan and therapy for epilepsy patients. Among the available deep network models, Convolutional Neural Networks (CNNs) are the most widely adopted models for learning and representing EEG signals. However, typical CNNs have high computational complexity, leading to overfitting problems. This paper proposes the design of two effective, lightweight deep network models; the 1D multiscale neural network (1D-MSCNet) model and the Long Short-term Memory (LSTM)-based compact CNN (EEG-LSTMNet) model. The 1D-MSCNet model comprises three modules: a spectral–temporal convolution module, a spatial convolution module, and a classification module. It extracts features from input EEG trials at multiple frequency/time ranges, identifying relationships between the spatial distribution of their channels. The EEG-LSTMNet model includes three convolutional layers, namely temporal, depthwise, and separable layers, a single LSTM layer, and two fully connected classification layers to extract discriminative EEG feature representations. Both models have been applied to the same EEG trials collected from the Temple University Hospital (TUH) database. Results revealed F1-score values of 96.9% and 98.4% for the 1D-MSCNet and EEG-LSTMNet, respectively. Based on the demonstrated outcomes, both models outperform related state-of-the-art methods due to their architectures’ adoption of 1D modules and layers that reduce the computational effort needed, solve the overfitting problem, and enhance classification efficiency. Hence, both models could be valuable additions for neurologists to help them decide upon precise treatments and drugs for patients depending on their type of seizure.