Deep Learning Neural Network Research Articles

Development of optimization control system for chiller plant based on a predictive model with Multi Stack LSTM and Deep Learning Neural Network Multi Output

Net-zero emissions become necessary for every country, implementing energy efficiency is the way to achieve it. Released report by the International Energy Agency in September 2022 stated that energy efficiency and electrification are the main priorities for Indonesia in achieving NZE. Currently, more than 50 % of energy consumption in buildings is intended for cooling systems (chillers plants). Thus, energy efficiency in buildings cooling systems has great potential to help achieve NZE and support SDGs. This research is intended to find an optimization control system for chiller plant in order to reduce energy consumption of cooling system in building. The development of this optimization control system is based on predictive model with Multi Stack LSTM and Deep Learning Neural Network. This research developed four different optimization control system frameworks. Every developed framework then will be tested by carrying out simulation with Chiller Plant model. The important parameter was found by using correlation matrix. Based on the correlation matrix, it was found that the most influence parameter that affect chiller plant performance are Tcws and Twb. It leads developed Frameworks 1 become the best compared with others developed frameworks in this research, because it used Twb as a variable input in parallel with performance prediction. This developed framework has deficient error MAE, MSE, RMSE respectively 0.8079, 0.6527, 0.8079 and able to reduce energy consumption by 10.72 %.

Solar Activity Dependence of Traveling Ionospheric Disturbance Amplitudes Using a Rapid‐Run Ionosonde in High Latitudes

AbstractWe investigated the amplitude of medium scale traveling ionospheric disturbances (MSTIDs, with periods 25–100 min) and their dependence on the solar activity using 16 years data of the rapid run‐ionosonde operating at high latitudes (N, Sodankylä, Finland). A deep learning neural network was applied to ionograms to extract critical frequency of the F2 region (foF2) with a 1 min time resolution. Then, we analyzed the relative amplitude of MSTIDs (i.e., foF2/foF2), which corresponds to the amplitude of atmospheric gravity waves (AGWs) causing MSTIDs. The amplitude of AGWs propagating upward increases with height due to the decreasing density of the air, and hmF2 varies depending on local time, seasonal and solar activity conditions. To account for this effect, we calculated a corrected MSTID amplitude by normalizing the relative amplitude for the air density at the hmF2. The corrected amplitudes show no clear dependence on F10.7 during winter (0–12 UT), equinox (20‐01 UT) and summer (19‐01 UT), while a positive dependence of corrected amplitudes on F10.7 was observed during winter and equinox, in 14–22 UT and 15–19 UT, respectively. Corresponding to the dependence behaviors of corrected and relative amplitudes, two likely mechanisms of MSTIDs, AGWs from the lower atmosphere and auroral sources, are inferred. Their subsequent roles in the solar activity dependence of MSTID amplitudes were separately discussed, although in reality, the observed dependence is complex and often involves several mechanisms together.

On-Line Measurement of Tracking Poses of Heliostats in Concentrated Solar Power Plants.

The tracking pose of heliostats directly affects the stability and working efficiency of concentrated solar power (CSP) plants. Due to occlusion, over-exposure, and uneven illumination caused by mirror reflection, traditional image processing algorithms showed poor performances on the detection and segmentation of heliostats, which impede vision-based 3D measurement of tracking poses of heliostats. To tackle this issue, object detection using deep learning neural networks are exploited. An improved neural network based on YOLO-v5 framework has been designed to solve the on-line detection problem of heliostats. The model achieves a recognition accuracy of 99.7% for the test set, outperforming traditional methods significantly. Based on segmented results, the corner points of each heliostat are found out using Hough Transform and line intersection methods. The 3D poses of each heliostat are then solved out based on the image coordinates of specific feature points and the camera model. Experimental and field test results demonstrate the feasibility of this hybrid approach, which provides a low-cost solution for the monitoring and measurement of tracking poses of the heliostats in CSP.

ANI/EFP: Modeling Long-Range Interactions in ANI Neural Network with Effective Fragment Potentials.

Deep learning Neural Networks (NN) have been developed in the field of molecular modeling for the purpose of circumventing the high computational cost of quantum-mechanical calculations while rivaling their accuracies. Although these networks have found great success, they generally lack the ability to accurately describe long-range interactions, which makes them unusable for extended molecular systems. Herein, we provide a method for partially retraining the deep learning general-use neural network ANI, in which the long-range interactions are represented via atomic electrostatic potentials. The electrostatic potentials, generated with polarizable effective fragment potentials (EFP), are used as an additional input feature for the network. This new ANI/EFP network can predict solute-solvent interaction energies on a trained data set with a kcal/mol accuracy. It also shows promise in predicting the interaction energies of a solute in solvent environments that have not been included in a training data set. The proposed protocol can be taken as an example and further developed, leading to highly accurate and transferable neural network potentials capable of handling long-range interactions and extended molecular systems.

Data-driven two-dimensional stationary quantum droplets and wave propagations in the amended Gross–Pitaevskii equation with two potentials via deep neural networks learning

In this paper, we develop a systematic deep learning approach to solve two-dimensional stationary quantum droplets (QDs) and investigate their wave propagation in the two-dimensional amended Gross–Pitaevskii equation with Lee–Huang–Yang correction and two kinds of potentials. Firstly, we use the initial-value iterative neural network algorithm for two-dimensional stationary QDs of stationary equations. Then, the learned stationary QDs are used as the initial value conditions for physics-informed neural networks to explore their evolutions in the space-time region. Especially, we consider two types of potentials, one is the two-dimensional quadruple-well Gaussian potential and the other is the P T -symmetric harmonic-oscillator (HO)-Gaussian potential, which lead to spontaneous symmetry breaking and the generation of multi-component QDs. The used deep learning method can also be applied to study wave propagations of other nonlinear physical models.

Advanced Hybrid Techniques for Cyberattack Detection and Defense in IoT Networks

ABSTRACTThe Internet of Things (IoT) represents a vast network of devices connected to the Internet, making it easier for users to connect to modern technology. However, the complexity of these networks and the large volume of data pose significant challenges in protecting them from persistent cyberattacks, such as distributed denial‐of‐service (DDoS) attacks and spoofing. It has become necessary to use intrusion detection systems and protect these networks. Existing intrusion detection systems for IoT networks face many problems and limitations, including high false alarm rates and delayed detection. Also, the datasets used for training may be outdated or sparse, which reduces the model's accuracy, and mechanisms may not be used to defend the network when any intrusion is detected. To address these limitations, a new hybrid deep learning and machine learning methodology is proposed that contributes to detecting DDoS and spoofing attacks, reducing false alarms, and then implementing the necessary defensive measures. In proposed hybrid methodology consists of three stages: the first stage is to propose a hybrid method for feature selection consisting of techniques (correlation coefficient and sequential feature selector); the second stage is to propose a hybrid model by integrating deep learning neural networks with a machine learning classifier (cascaded long short‐term memory [LSTM] and Naive Bayes classifier); and in the third stage, improving network defense mechanisms and blocking ports after detecting threats and maintaining network integrity. In training and evaluating the performance of the proposed methodology, three datasets (CIC‐DDoS2019, CIC‐IoT2023, and CIC‐IoV2024) were used, and these data were also balanced to obtain effective results. The accuracy of 99.91%, 99.88%, and 99.77% was obtained. Also, a cross‐validation technique was used with the test data to ensure no overfitting. The proposed methodology has proven its high performance in detecting attacks, as it provides a powerful solution to enhance the security of IoT networks and protect them from cyberattacks, as it can be applied in many fields and to other attacks.

Adaptive optimization of thermal energy and information management in intelligent green manufacturing process based on neural network

The study of thermal energy adaptive optimization has important theoretical and practical significance. The deep learning neural network model is used to monitor and analyze the thermal energy data in the manufacturing process in real time. Through the construction of adaptive optimization algorithm, the heat input and output are systematically evaluated and adjusted, and the heat distribution scheme is dynamically optimized according to environmental changes and production needs. At the same time, information management system is introduced to realize data summary, feedback and decision support. The experimental results show that the proposed method can effectively reduce the heat energy loss, optimize the heat energy utilization, and significantly reduce the overall energy consumption compared with traditional management methods. With real-time data updates, the system improves the flexibility and responsiveness of the production process, significantly improving manufacturing efficiency. The thermal energy adaptive optimization method based on neural network provides an effective solution for intelligent green manufacturing, which can not only optimize thermal energy management, but also provide a reference for other resource conservation and environmental protection.

A Generalized Learning Approach to Deep Neural Networks

Optimization of machine learning architectures is essential in determining the efficacy and the applicability of any neural architecture to real world problems. In this work a generalized Newton's method (GNM) is presented as a powerful approach to learning in deep neural networks (DNN). This technique was compared to two popular approaches, namely the stochastic gradient descent (SGD) and the Adam algorithm, in two popular classification tasks. The performance of the proposed approach confirmed it as an attractive alternative to state-of-the-art first order solutions. Due to the good results presented in the case of shallow DNN, in the last part of the article an hybrid optimization method is presented. This method consists in combining two optimization algorithms, i.e. GNM and Adam or GNM and SGD, during the training phase within the layers of the neural network. This configuration aims to benefit from the strengths of both first- and second-order algorithms. In this case a convolutional neural network is considered and its parameters are updated with a different optimization algorithm. Also in this case, the hybrid approach returns the best performance with respect to the first order algorithms.

Power data analysis and mining technology in smart grid

This study proposes a smart grid model named “GridOptiPredict”, which aims to achieve efficient analysis and processing of power system data through deep fusion of deep learning and graph neural network, so as to improve the intelligent level and overall efficiency of power grid operation. The model integrates three core functions of load forecasting, power grid state sensing and resource optimization into one, forming a closely connected and complementary framework. Through carefully designed experimental scheme, the practical value and effectiveness of “Grid OptiPredict” model are fully verified from three aspects: accuracy of load forecasting, sensitivity of power grid state sensing and efficiency of resource allocation strategy. Experimental results show that the model has significant advantages in prediction accuracy, model stability and robustness, resource optimization, security, information security, social and economic benefits and user experience.

State Estimation of Radial Distribution Systems Based on Multiple Legendre Neural Networks

The conventional weighted least square (WLS) method is the most effective technique used in the state estimation of high voltage transmission system. Unfortunately, the application of WLS in radial distribution network encounter difficulties due to the inherent characteristics of these systems, such as the low measurement redundancy and high r/x ratio of the distribution systems. Given the structure of bulky systems that require a bulky number of measurements, the use of artificial neural networks is considered an effective alternative to estimate these values using a lesser number of measurements than conventional techniques. Due to state estimation based on ANN technique, the time-consuming gain matrix manipulation and pseudo measurements required in the conventional WLS method are no longer necessary. The efficiency of deep learning neural networks such as multi-layer perceptron (MLP) and Legendre neural network (LeNN) depends on the position of the measurements and the number of neural networks applied. Determining the applicable number of neural networks to ensure high estimation accuracy plays an important role in the estimation process. This aspect is addressed in this study, where multiple neural networks are used to improve performance compared to a single neural network. The results obtained indicate that determining the applicable number of neural networks depends on several factors such as the position of the measurements and the diversity of the data. The application of LeNN on state estimation of a 69-bus radial distribution network is used as an illustrative example to explain the distinctive feature of the proposed technique.

Automated High-Precision Recognition of Solar Filaments Based on an Improved U2-Net

Solar filaments are a significant solar activity phenomenon, typically observed in full-disk solar observations in the H-alpha band. They are closely associated with the magnetic fields of solar active regions, solar flare eruptions, and coronal mass ejections. With the increasing volume of observational data, the automated high-precision recognition of solar filaments using deep learning is crucial. In this study, we processed full-disk H-alpha solar images captured by the Chinese H-alpha Solar Explorer in 2023 to generate labels for solar filaments. The preprocessing steps included limb-darkening removal, grayscale transformation, K-means clustering, particle erosion, multiple closing operations, and hole filling. The dataset containing solar filament labels is constructed for deep learning. We developed the Attention U2-Net neural network for deep learning on the solar dataset by introducing an attention mechanism into U2-Net. In the results, Attention U2-Net achieved an average Accuracy of 0.9987, an average Precision of 0.8221, an average Recall of 0.8469, an average IoU of 0.7139, and an average F1-score of 0.8323 on the solar filament test set, showing significant improvements compared to other U-net variants.

Distance-based loss function for deep feature space learning of convolutional neural networks

Convolutional Neural Networks (CNNs) have been on the forefront of neural network research in recent years. Their breakthrough performance in fields such as image classification has gathered efforts in the development of new CNN-based architectures, but recently more attention has been directed to the study of new loss functions. Softmax loss remains the most popular loss function due mainly to its efficiency in class separation, but the function is unsatisfactory in terms of intra-class compactness. While some studies have addressed this problem, most solutions attempt to refine softmax loss or combine it with other approaches. We present a novel loss function based on distance matrices (LDMAT), softmax independent, that maximizes interclass distance and minimizes intraclass distance. The loss function operates directly on deep features, allowing their use on arbitrary classifiers. LDMAT minimizes the distance between two distance matrices, one constructed with the model’s deep features and the other calculated from the labels. The use of a distance matrix in the loss function allows a two-dimensional representation of features and imposes a fixed distance between classes, while improving intra-class compactness. A regularization method applied to the distance matrix of labels is also presented, that allows a degree of relaxation of the solution and leads to a better spreading of features in the separation space. Efficient feature extraction was observed on datasets such as MNIST, CIFAR10 and CIFAR100.

Flotation separation of lithium–ion battery electrodes predicted by a long short-term memory network using data from physicochemical kinetic simulations and experiments

Anode and cathode active materials from spent lithium–ion batteries may be recovered and potentially used in new batteries to promote recycling and resource circulation. Froth flotation was applied to pristine active materials and the black mass obtained from pretreated spent batteries. Flotation kinetics was simulated with the use of computational fluid dynamics and surface chemistry. Bubble surface coverage and entrainment in the flotation kinetics model were selected and optimized by systematic fitting to experimental data. Entrainment influences the recovery and grade of the active materials. The optimized flotation kinetics model was used for generating additional data that, along with the fitted data, were used to train a deep learning neural network. The trained network was validated using anode–cathode and black mass flotation experiments, and its predictions showed a maximum residual error of 0.18 ± 0.11 recovery. The simulation framework was developed into a desktop application that predicts the flotation behavior of active materials. It provides information for estimating results following operational and physicochemical changes and for optimizing flotation processes. Finally, recovered anode active materials from black mass were selected for coin cell tests. The coulombic efficiency of these coin cells was initially lower (86.8 %) than that of cells made with pristine graphite particles (98.4 %).

Deep learning-based segmentation for high-dose-rate brachytherapy in cervical cancer using 3D Prompt-ResUNet

Objective.To develop and evaluate a 3D Prompt-ResUNet module that utilized the prompt-based model combined with 3D nnUNet for rapid and consistent autosegmentation of high-risk clinical target volume (HRCTV) and organ at risk (OAR) in high-dose-rate brachytherapy for cervical cancer patients.Approach.We used 73 computed tomography scans and 62 magnetic resonance imaging scans from 135 (103 for training, 16 for validation, and 16 for testing) cervical cancer patients across two hospitals for HRCTV and OAR segmentation. A novel comparison of the deep learning neural networks 3D Prompt-ResUNet, nnUNet, and segment anything model-Med3D was applied for the segmentation. Evaluation was conducted in two parts: geometric and clinical assessments. Quantitative metrics included the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95%), Jaccard index (JI), and Matthews correlation coefficient (MCC). Clinical evaluation involved interobserver comparison, 4-grade expert scoring, and a double-blinded Turing test.Main results.The Prompt-ResUNet model performed most similarly to experienced radiation oncologists, outperforming less experienced ones. During testing, the DSC, HD95% (mm), JI, and MCC value (mean ± SD) for HRCTV were 0.92 ± 0.03, 2.91 ± 0.69, 0.85 ± 0.04, and 0.92 ± 0.02, respectively. For the bladder, these values were 0.93 ± 0.05, 3.07 ± 1.05, 0.87 ± 0.08, and 0.93 ± 0.05, respectively. For the rectum, they were 0.87 ± 0.03, 3.54 ± 1.46, 0.78 ± 0.05, and 0.87 ± 0.03, respectively. For the sigmoid, they were 0.76 ± 0.11, 7.54 ± 5.54, 0.63 ± 0.14, and 0.78 ± 0.09, respectively. The Prompt-ResUNet achieved a clinical viability score of at least 2 in all evaluation cases (100%) for both HRCTV and bladder and exceeded the 30% positive rate benchmark for all evaluated structures in the Turing test.Significance.The Prompt-ResUNet architecture demonstrated high consistency with ground truth in autosegmentation of HRCTV and OARs, reducing interobserver variability and shortening treatment times.

Length-scale study in deep learning prediction for non-small cell lung cancer brain metastasis

Deep learning-assisted digital pathology has demonstrated the potential to profoundly impact clinical practice, even surpassing human pathologists in performance. However, as deep neural network (DNN) architectures grow in size and complexity, their explainability decreases, posing challenges in interpreting pathology features for broader clinical insights into physiological diseases. To better assess the interpretability of digital microscopic images and guide future microscopic system design, we developed a novel method to study the predictive feature length-scale that underpins a DNN’s predictive power. We applied this method to analyze a DNN’s capability in predicting brain metastasis from early-stage non-small-cell lung cancer biopsy slides. This study quantifies DNN’s attention for brain metastasis prediction, targeting features at both the cellular scale and tissue scale in H&E-stained histological whole slide images. At the cellular scale, the predictive power of DNNs progressively increases with higher resolution and significantly decreases when the resolvable feature length exceeds 5 microns. Additionally, DNN uses more macro-scale features associated with tissue architecture and is optimized when assessing visual fields greater than 41 microns. Our study computes the length-scale requirements for optimal DNN learning on digital whole-slide microscopic images, holding the promise to guide future optical microscope designs in pathology applications and facilitating downstream deep learning analysis.

Identifying defects and varieties of Malting Barley Kernels

This study introduces a comprehensive approach for classifying individual malting barley kernels, involving dual-sided kernel imaging, a specifically designed image processing algorithm, an optimized deep neural network architecture, and a mechanical sorting system. The proposed method achieves precise classification into multiple classes, aligning with quality standards for malting material assessment. Throughout the study, various image analysis techniques were assessed, including traditional feature engineering, established transfer learning deep neural network architectures, and our custom-designed convolutional neural network tailored for barley kernel image analysis. Comparative analysis underscores the superior performance of our network model. The study reveals that our proposed deep learning network achieves a 94% accuracy in classifying barley kernel defects and varieties, outperforming well-established transfer learning models to complex architectures that attain 93% accuracy. Additionally, it surpasses the traditional machine learning approach involving feature extraction and support vector machine classifiers, which achieve accuracy below 90% in detecting defective kernels and below 70% in varietal classification. However, we also noted the traditional approach’s advantage in morphological feature recognition. This observation guides new research toward integrating morphological feature extraction techniques with modern convolutional networks. This paper presents a deep neural network designed specifically for the analysis of cereal kernel images in two applications: defect and variety classification. It emphasizes the importance of standardizing kernel orientation and merging images from both sides of the kernel, and introduces a device for image acquisition that fulfills this need.

Multifunctional Human-Computer Interaction System Based on Deep Learning-Assisted Strain Sensing Array.

Continuous and reliable monitoring of gait is crucial for health monitoring, such as postoperative recovery of bone joint surgery and early diagnosis of disease. However, existing gait analysis systems often suffer from large volumes and the requirement of special space for setting motion capture systems, limiting their application in daily life. Here, we develop an intelligent gait monitoring and analysis prediction system based on flexible piezoelectric sensors and deep learning neural networks with high sensitivity (241.29 mV/N), quick response (66 ms loading, 87 ms recovery), and excellent stability (R2 = 0.9946). The theoretical simulations and experiments confirm that the sensor provides exceptional signal feedback, which can easily acquire accurate gait data when fitted to shoe soles. By integrating high-quality gait data with a custom-built deep learning model, the system can detect and infer human motion states in real time (the recognition accuracy reaches 94.7%). To further validate the sensor's application in real life, we constructed a flexible wearable recognition system with human-computer interaction interface and a simple operation process for long-term and continuous tracking of athletes' gait, potentially aiding personalized health management, early detection of disease, and remote medical care.

Wind Field Reconstruction Method Using Incomplete Wind Data Based on Vision Mamba Decoder Network

Accurate meteorological information is crucial for the safety of civil aviation flights. Complete wind field information is particularly helpful for planning flight routes. To address the challenge of accurately reconstructing wind fields, this paper introduces a deep learning neural network method based on the Vision Mamba Decoder. The goal of the method is to reconstruct the original complete wind field from incomplete wind data distributed along air routes. This paper proposes improvements to the Vision Mamba model to fit our mission, showing that the developed model can accurately reconstruct the complete wind field. The experimental results demonstrate a mean absolute error (MAE) of wind speed of approximately 1.83 m/s, a mean relative error (MRE) of around 7.87%, an R-square value of about 0.92, and an MAE of wind direction of 5.78 degrees.

Jet Tagging with More-Interaction Particle Transformer

Abstract In this study, we introduce the More-Interaction Particle Transformer (MIParT), a novel deep learning neural network designed for jet tagging. This framework incorporates our own design, the More-Interaction Attention (MIA) mechanism, which increases the dimensionality of particle interaction embeddings. We tested MIParT using the top tagging and quark-gluon datasets. Our results show that MIParT not only matches the accuracy and AUC of LorentzNet and a series of Lorentz-equivariant methods, but also significantly outperforms the ParT model in background rejection. Specifically, it improves background rejection by approximately 25\% at a 30\% signal efficiency on the top tagging dataset and by 3\% on the quark-gluon dataset. Additionally, MIParT requires only 30\% of the parameters and 53\% of the computational complexity needed by ParT, proving that high performance can be achieved with reduced model complexity. For very large datasets, we double the dimension of particle embeddings, referring to this variant as MIParT-Large (MIParT-L). We find that MIParT-L can further capitalize on the knowledge from large datasets. From a model pre-trained on the 100M JetClass dataset, the background rejection performance of the fine-tuned MIParT-L improved by 39\% on the top tagging dataset and by 6\% on the quark-gluon dataset, surpassing that of the fine-tuned ParT. Specifically, the background rejection of fine-tuned MIParT-L improved by an additional 2\% compared to the fine-tuned ParT. The results suggest that MIParT has the potential to advance efficiency benchmarks for jet tagging and event identification in particle physics.Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Article funded by SCOAP3 and published under licence by Chinese Physical Society and the Institute of High Energy Physics of the Chinese Academy of Science and the Institute of Modern Physics of the Chinese Academy of Sciences and IOP Publishing Ltd

Enhanced plasmonic scattering imaging via deep learning-based super-resolution reconstruction for exosome imaging.

Exosome analysis plays pivotal roles in various physiological and pathological processes. Plasmonic scattering microscopy (PSM) has proven to be an excellent label-free imaging platform for exosome detection. However, accurately detecting images scattered from exosomes remains a challenging task due to noise interference. Herein, we proposed an image processing strategy based on a new blind super-resolution deep learning neural network, named ESRGAN-SE, to improve the resolution of exosome PSI images. This model can obtain super-resolution reconstructed images without increasing experimental complexity. The trained model can directly generate high-resolution plasma scattering images from low-resolution images collected in experiments. The results of experiments involving the detection of light scattered by exosomes showed that the proposed super-resolution detection method has strong generalizability and robustness. Moreover, ESRGAN-SE achieved excellent results of 35.52036, 0.09081, and 8.13176 in terms of three reference-free image quality assessment metrics, respectively. These results show that the proposed network can effectively reduce image information loss, enhance mutual information between pixels, and decrease feature differentiation. And, the single-image SNR evaluation score of 3.93078 also showed that the distinction between the target and the background was significant. The suggested model lays the foundation for a potentially successful approach to imaging analysis. This approach has the potential to greatly improve the accuracy and efficiency of exosome analysis, leading to more accurate cancer diagnosis and potentially improving patient outcomes.