Network Error Research Articles

Going Deeper, Generalizing Better: An Information-Theoretic View for Deep Learning.

Deep learning has transformed computer vision, natural language processing, and speech recognition. However, two critical questions remain obscure: 1) why do deep neural networks (DNNs) generalize better than shallow networks and 2) does it always hold that a deeper network leads to better performance? In this article, we first show that the expected generalization error of neural networks (NNs) can be upper bounded by the mutual information between the learned features in the last hidden layer and the parameters of the output layer. This bound further implies that as the number of layers increases in the network, the expected generalization error will decrease under mild conditions. Layers with strict information loss, such as the convolutional or pooling layers, reduce the generalization error for the whole network; this answers the first question. However, algorithms with zero expected generalization error do not imply a small test error. This is because the expected training error is large when the information for fitting the data is lost as the number of layers increases. This suggests that the claim "the deeper the better" is conditioned on a small training error. Finally, we show that deep learning satisfies a weak notion of stability and provides some generalization error bounds for noisy stochastic gradient decent (SGD) and binary classification in DNNs.

Development of a DNI Forecast Tool Based on Machine Learning for the Smart Operation of a Parabolic Trough Collector System with Thermal Energy Storage

In the research project Smart Solar System (S3), the Solar-Institut Jülich (SIJ) developed forecast tools to predict the direct normal irradiance (DNI) in hourly resolution for the current day. The aim of the daily DNI forecast is to use it as input to enable a smart operation of a parabolic trough collector (PTC) system with a concrete thermal energy storage (C-TES) located at the company KEAN Soft Drinks Ltd in Limassol, Cyprus. The main focus in this work is on the development of a DNI forecast tool based on long short-term memory (LSTM), which is a recurrent neural network (RNN), and its comparison with a DNI forecast tool based on analytical algorithms (non-machine learning). Only the non-machine learning DNI forecast tool with hourly update was tested in real-life PTC plant operation since end of 2022. The comparisons between the three DNI forecast tools show that the potential of using machine learning is very high. Different comparisons were made including an evaluation of the accuracy of the tool (i.e. comparison of the DNI forecast data with DNI measurement data). The DNI forecast based on the LSTM network proved to be more accurate than the non-machine learning DNI forecasts when considering errors greater than ±100 W/m2. The error of the LSTM network compared to DNI measurement data was as follows: 43.6 % of the data was within ±100 W/m2, 74.8 % was within ±200 W/m2, 89.8 % was within ±300 W/m2 and 95.8 % was within ±400 W/m2.

Expressibility-induced concentration of quantum neural tangent kernels

Quantum tangent kernel methods provide an efficient approach to analyzing the performance of quantum machine learning models in the infinite-width limit, which is of crucial importance in designing appropriate circuit architectures for certain learning tasks. Recently, they have been adapted to describe the convergence rate of training errors in quantum neural networks in an analytical manner. Here, we study the connections between the expressibility and value concentration of quantum tangent kernel models. In particular, for global loss functions, we rigorously prove that high expressibility of both the global and local quantum encodings can lead to exponential concentration of quantum tangent kernel values to zero. Whereas for local loss functions, such issue of exponential concentration persists owing to the high expressibility, but can be partially mitigated. We further carry out extensive numerical simulations to support our analytical theories. Our discoveries unveil a fundamental feature of quantum neural tangent kernels, indicating that the issue of their concentration cannot be bypassed merely by transitioning to a local encoding scheme while maintaining high expressibility. This offers valuable insights for the design of wide quantum variational circuit models in practical applications.

Improved aggregation/agglomeration-dependent percolation theory to predict nanoparticle-aided electrical conductivity in polymer nanocomposites: A combination of analytical strategy and artificial neural network

The formation of aggregates/agglomerates in polymer nanocomposites is known as an inevitable phenomenon that may affect many physical/mechanical properties. Accordingly, in this study, the impacts of the size and content of nanoparticle clusters on the electrical conductivity of polymer matrices containing spherical nanoparticles were analytically evaluated. To this end, the well-known percolation theory was improved by involving major system parameters, affected by aggregation/agglomeration, reported to be indicative of how electricity may conduct through (e. g. tunneling effect, percolation threshold, content and equivalent electrical conductivity of nanoparticle domains, etc.). The characteristics of the aggregates/agglomerates were estimated via the combination of a devised strategy, representing the structure of clusters, and the fundamentals of the percolation theory. The average electrical conductivity and density of polymer/particle interphase, formed around nanoparticle domains were calculated using a particularly developed scaling theory. Also, the hypothetical relative location of nanoparticle domains to each other was defined by calculating the average nearest distance between them, benchmarked against the tunneling distance. A two-stage validation procedure was conducted using the data elicited from the literature in order to first, evaluate the model efficiency and secondly, assess its predictive capabilities with the aid of an artificial neural network (error < 7 %).

Fatigue Characteristics Analysis of Carbon Fiber Laminates with Multiple Initial Cracks

In the entire wind turbine system, the blade acts as the central load-bearing element, with its stability and reliability being essential for the safe and effective operation of the wind power unit. Carbon fiber, known for its high strength-to-weight ratio, high modulus, and lightweight characteristics, is extensively utilized in blade manufacturing due to its superior attributes. Despite these advantages, carbon fiber composites are frequently subjected to cyclic loading, which often results in fatigue issues. The presence of internal manufacturing defects further intensifies these fatigue challenges. Considering this, the current study focuses on carbon fiber composites with multiple pre-existing cracks, conducting both static and fatigue experiments by varying the crack length, the angle between cracks, and the distance among them to understand their influence on the fatigue life under various conditions. Furthermore, this study leverages the advantages of Paris theory combined with the Extended Finite Element Method (XFEM) to simulate cracks of arbitrary shapes, introducing a fatigue simulation method for carbon fiber composite laminates with multiple cracks to analyze their fatigue characteristics. Concurrently, the Particle Swarm Optimization (PSO) algorithm is employed to determine the optimal weight configuration, and the Backpropagation neural network (BP) is used to train and adjust the weights and thresholds to minimize network errors. Building on this foundation, a surrogate model for predicting the fatigue life of carbon fiber composite laminates with multiple cracks under conditions of physical parameter uncertainty has been constructed, achieving modeling and assessment of fatigue reliability. This research offers theoretical insights and methodological guidance for the utilization of carbon fiber-reinforced composites in wind turbine blade applications.

A Multi-Mechanism Fusion Method for Robust State of Charge Estimation via Bidirectional Long Short-Term Memory Model with Mixture Kernel Mean p-Power Error Loss Optimized by Golden Jackal Optimization Algorithm

Abstract Accurate state of charge (SOC) estimation is crucial for effective battery management in various applications. The bidirectional long short-term memory (BiLSTM) as an outstanding nonlinear regression model can be used for SOC estimation. This work develops a novel multi-mechanism fusion method based on BiLSTM to further enhance its estimation performance for SOC, in which the convolutional neural network (CNN), attention mechanism, and mixture kernel mean p-power error (MKMPE) loss are introduced into the BiLSTM framework for addressing different issues. First, the introduction of CNN components aims to extract essential features from battery data, enhancing the model's comprehension of complex information. Then, the attention mechanism is used to further refine the model's perceptual ability and a robust MKMPE loss is introduced into the BiLSTM framework to replace its original mean squared error loss, and a novel robust model is developed to suppress non-Gaussian noise interference. Finally, some key hyperparameters of the proposed model are fine-tuned using the golden jackal optimization algorithm, resulting in improved estimation performance. Comparative numerical experiments are meticulously conducted in various cases to evaluate the performance of the proposed method, and the experiment results demonstrate that it can perform outstanding effectiveness in handling non-Gaussian noise scenarios.

Physics-informed multi-grid neural operator: Theory and an application to porous flow simulation

We propose a physics-informed multi-grid finite neural operator as an efficient alternative for PDE solvers. The operator aims to map a random parameter function to the corresponding solution function of a linear PDE, both distributed on a fine grid (called original grid). The training of the operator involves using the physics-informed method to train a neural network on the original grid to approximate the PDE solution. The grid is then coarsened, and a new neural network is trained on the coarse grid to approximate the prediction error of the previous network. This process is iteratively repeated until the final grid's neural network achieves adequate accuracy. The combination of all neural networks from coarse to fine grids forms the multi-grid neural operator capable of mapping random parameters to solutions on the original grid. The above process involving fine-coarse-fine grid is called one V cycle (following the traditional multi-grid PDE literature), and multiple V cycles can be included into the operator to enhance its performance. Additionally, the operator integrates the idea of using coarse-grid predictions as initial guesses for fine-grid predictions. Validation is performed using three porous flow examples of varying dimensions. The operator's prediction accuracy improves with more V cycles included, and its predictions closely match results obtained from a numerical solver while executing 10 to 100 times faster.

Robust AI Based Bio Inspired Protocol using GANs for Secure and Efficient Data Transmission in IoT to Minimize Data Loss

Objectives: To propose an AI-based protocol to enhance the reliability and security of IoT data transmission in a robust manner. Deep learning with bio-inspired methods is employed to address real-time challenges such as route optimization, data integrity, error recovery, and end-to-end delay in order to minimize data loss and maximize the transmission rate. Methods: Generative Adversarial Networks (GANs) are used to enhance the robustness of the protocol in combination with a bio-inspired Artificial Immune System (AIS) to detect IoT network anomalies and responds to malicious activities by using the adaptive learning capabilities of IS. Hybrid Automatic Repeat Request (HARQ), an error recovery method, is utilized to detect network errors in order to correct and retransmit data to the destination during error-prone network conditions. Queue learning action sets are used to discover the finest path for efficient data transmission. The OMNeT++ simulator is used to assess the performance of the proposed BIP-GANs protocol. The performance results are compared with the prevailing data transmission protocols, such as EAP-IFBA, DNN-CSO, and ALO-DHT. Findings: The suggested BIP-GANs data transmission protocol outperforms with promising results, with an energy consumption rate of 6%, 8 seconds data transmission speed, 6 second less delay rate, 98% robustness (security and confidentiality), 98.5% alive nodes, and 99% network life span, which is higher than the prevailing methods. Novelty: This research study provides a comprehensive solution to secure and efficient data transmission by integrating the AI-based bio-inspired BIP-GANs method to address the security and data transmission challenges of existing methods such as EAP-IFBA, DNN-CSO, and ALO-DHT in terms of energy consumption rate, transmission speed, delay rate, life span of the network, robustness, and number of alive nodes. Keywords: Artificial Intelligence, Generative Adversarial Networks, Error Recovery, Secured Data Transmission, Deep Learning, Bio­Inspired Algorithm

Enhancing training performance of convolutional neural network algorithm through an autoencoder-based unsupervised labeling framework for mineral exploration targeting

Convolutional neural networks (CNNs), a prominent deep learning approach, have garnered significant interest in the field of mineral potential mapping (MPM) due to their capability to capture and learn spatial features that traditional algorithms tend to overlook. The effectiveness of CNNs is closely tied to the quantity of training data available, thereby impacting the outcomes of MPM. Moreover, uncertainties arising from delineation of negative samples can compromise the reliability of MPM assessments. To deal with these challenges, we propose the utilization of an autoencoder-based anomaly detection technique for the purpose of annotating locations in an unsupervised manner. Subsequently, a CNN is trained using the unsupervised annotated data to generate a Fe prospectivity model within a specific region in Iran. To validate the effectiveness of the proposed method, we first perform an MPM with insufficient positive training samples that extract from the location of known occurrences. We then execute another MPM on a set of samples labeled based on the reconstruction error of an autoencoder network. When comparing the two prospectivity models, namely using augmented data or inadequate samples, it is evident that modeling with augmented data outperforms the MPM model trained with insufficient samples. This confirms the effectiveness of the adopted approach and shows that the unsupervised labeling technique proposed in this work can significantly improve the performance of the CNN in MPM.

Predicting the compressive strength of self-compacting concrete by developed African vulture optimization algorithm-Elman neural networks

The compressive strength of concrete depends on various factors. Since these parameters can be in a relatively wide range, it is difficult for predicting the behavior of concrete. Therefore, to solve this problem, an advanced modeling is needed. The aim of the literature is to achieve an ideal and flexible solution for predicting the behavior of concrete. Therefore, it is necessary to develop new approaches. Artificial Neural Networks (ANNs) have evolved from a theoretical method to a widely utilized technology by successful applications for a variety of issues. Actually, ANNs are a strong computing tool that provides the right solutions to problems that are difficult to use conventional methods. Inspired by the biological neural system, these networks are now widely used for solving a wide range of complicated problems in civil engineering. This study’’s target is evaluating the performance of developed African vulture optimization algorithm (DAVOA)-Elman neural networks (ENNs) by considering different input parameters in predicting the self-compacting concrete compressive strength. Hence, once 8 parameters and again to get as close as possible to the prediction conditions in the laboratory, 140 parameters entered to the improved version of Elman Neural Networks as input. According to the results, the element network has the lowest mean squares of the test error in predicting the compressive strength of 7 and 28 days in 100 repetitions. Further, in predicting both compressive strengths, the element grid with the Logsig-Purelin interlayer transfer function has the lowest test error, which determines the optimal transfer function. Moreover, the results showed that DAVOA as a reliable tool with time and cost savings have high power in predicting the desired characteristics. Also, in predicting both 7-day and 28-day compressive strength, networks built with 140 parameters have a 74.54 and 70.44% improvement in test error over 8-parameter networks, respectively, which directly affects this effect. Further parameters are considered as input to the network error rate in predicting the desired properties.

Tensor network noise characterization for near-term quantum computers

Characterization of noise in current near-term quantum devices is of paramount importance to fully use their computational power. However, direct quantum process tomography becomes unfeasible for systems composed of tens of qubits. A promising alternative method based on tensor networks was recently proposed []. In this paper, we adapt it for the characterization of noise channels on near-term quantum computers and investigate its performance thoroughly. In particular, we show how experimentally feasible tomographic samples are sufficient to accurately characterize realistic correlated noise models affecting individual layers of quantum circuits, and study its performance on systems composed of up to 20 qubits. Furthermore, we combine this noise characterization method with a recently proposed noise-aware tensor network error mitigation protocol for correcting outcomes in noisy circuits, resulting accurate estimations even on deep circuit instances. This positions the tensor-network-based noise characterization protocol as a valuable tool for practical error characterization and mitigation in the near-term quantum computing era. Published by the American Physical Society 2024

Mitigating Slow-to-Write Errors in Memristor-Mapped Graph Neural Networks Induced by Adversarial Attacks

Mitigating Slow-to-Write Errors in Memristor-Mapped Graph Neural Networks Induced by Adversarial Attacks

An integrated optimization method for measurement points layout and error modeling for digital twin of CNC machine tools

Thermal error significantly influences the accuracy of precision computer numerical control (CNC) machine tools. The key to compensating thermal error lies in selecting appropriate temperature measurement points and establishing an accurate error prediction model. Traditional methods separate measurement points selection and prediction modeling, that is, selecting temperature measurement points first and then establishing the prediction model using these points. These methods are difficult to achieve optimal matching between measurement points and the prediction model, resulting in shortcomings in modeling accuracy. To address these challenges, an integrated optimization method for measurement points layout and error modeling for digital twin of CNC machine tools is proposed. A dual-stage attention-based long short-term memory combined with convolutional neural network (DA-CLSTM) error modeling method is proposed to accurately predict the thermal error for different numbers of temperature measurement points. Then, an integrated method of virtual-real temperature measurement points layout and error modeling is proposed, which ensures the temperature measurement points and the error prediction model are closely matched, offering rational sensors layout and high-accuracy prediction. Finally, experiments are conducted on the spindle system test bench to validate the method proposed in this research. Through integrated optimization of measurement points layout and the prediction model, the proposed method achieves high-accuracy thermal error prediction across various sensor counts and maintains reliable prediction ability when the number of sensors is reduced. This method is applied to a digital twin system of MKL7150 grinding machine, and effectively improves the accuracy of actual machining.

A Recurrent Stochastic Configuration Network for Dynamic System Modeling and Its Application

In order to model nonlinear dynamic systems quickly and accurately, a recurrent stochastic configuration network (RSCN) with universal approximation ability is proposed in this paper. The method consists of network parameter learning and network structure determination. The network parameter learning strategy consists of input weights and the biases of hidden nodes generated randomly through a supervision mechanism, while the output weights of hidden nodes are determined by solving the least-squares problem. The network structure is determined incrementally by the training error of a single-layer recurrent neural network. The effectiveness of RSCN is tested and evaluated by using three nonlinear dynamic functions and historical data from a municipal solid waste incineration (MSWI) process. The experimental results show that the proposed method can improve modeling accuracy and reduce training time, which is valuable for applications in the field of industrial process modeling.

Wind Speed Prediction Using Deep Recurrent Neural Networks and Farm Platform Features for One-Hour-Ahead Forecast

This paper proposes a deep recurrent neural network (DRNN) approach to model the one-hour-ahead wind speed forecasting by using various meteorological sensory data from the North Wyke farm platform (NWFP). To refine model input, mutual information analysis is applied to eliminate irrelevant sensory data. The DRNN architecture employs three recurrent layers Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and simple Recurrent Neural Network (RNN) to capture temporal relationships. The proposed networks are tested using real-life, one-year data from the NWFP. The results showed a strong correlation between the actual and predicted wind speed for LSTM, GRU, and RNN layers-based DRNN, however, simple RNN slightly outperformed the other two recurrent layers. The distribution of the network errors over the year is also analyzed. Although the observed meteorological data between the years was from different distributions, the proposed network generalized well even though these data were altered due to global warming.

Quantifying local and global mass balance errors in physics-informed neural networks

Physics-informed neural networks (PINN) have recently become attractive for solving partial differential equations (PDEs) that describe physics laws. By including PDE-based loss functions, physics laws such as mass balance are enforced softly in PINN. This paper investigates how mass balance constraints are satisfied when PINN is used to solve the resulting PDEs. We investigate PINN’s ability to solve the 1D saturated groundwater flow equations (diffusion equations) for homogeneous and heterogeneous media and evaluate the local and global mass balance errors. We compare the obtained PINN’s solution and associated mass balance errors against a two-point finite volume numerical method and the corresponding analytical solution. We also evaluate the accuracy of PINN in solving the 1D saturated groundwater flow equation with and without incorporating hydraulic heads as training data. We demonstrate that PINN’s local and global mass balance errors are significant compared to the finite volume approach. Tuning the PINN’s hyperparameters, such as the number of collocation points, training data, hidden layers, nodes, epochs, and learning rate, did not improve the solution accuracy or the mass balance errors compared to the finite volume solution. Mass balance errors could considerably challenge the utility of PINN in applications where ensuring compliance with physical and mathematical properties is crucial.

Multi-scale keypoints detection and motion features extraction in dairy cows using ResNet101-ASPP network

Detecting Keypoints in dairy cows aims to locate and track the motion trajectories of the body's joints, which plays a crucial role in behavior analysis and lameness detection. However, real farming scenarios, characterized by occlusions and large variations in object scale may result in poor detection results. Therefore we introduce the atrous spatial pyramid pooling (ASPP) module into the shallow network of ResNet101, designed to improve the multi-scale feature extraction capability of the model. The ASPP module enhances the robustness of recognition for different dimensional sizes and occluded keypoints using different dilatation rates in the parallel atrous convolutional layers to expand the model's receptive field. Furthermore, seven types of motion features, including tracking up, gait symmetry, step height balance, motion speed variability, head swing amplitude, head-neck slope and back curvature are extracted simultaneously by monitoring and tracking the motion trajectory of distinct keypoints. Several of these features represent innovative extraction models and attributes, first proposed in this study. Multiple models are trained and tested on datasets containing 2385 frames for ablation experiments. The experiments show that, in comparison with the ResNet50, MobileNet_v2_1.0, and EfficientNet-b0 backbone networks, the training error and test error of ResNet101 improve by 4.04–30.12 pixels and 3.81–28.14 pixels. Therefore, ResNet101 is used as the benchmark for subsequent model improvement by adding the ASPP module. The training error and test error of the ResNet101-ASPP network are improved by 0.27 and 0.24 pixels, respectively, compared to the benchmark network. The prediction confidence improves by 1.65-2.50% at three different dairy cow object scales, In addition, the keypoints under different occlusion conditions improve considerably, especially for small-scale keypoints, demonstrating the capability of the ASPP module for multi-scale feature extraction. By analyzing the distribution between the seven features and health, mild lameness, and severe lameness in dairy cows, it is shown that all the different features play an important role in distinguishing between different levels of lameness.

Leveraging Linear Network Error Correction for steganographic network codes

Steganography is an ingenious method employed to conceal data within an unassuming, non-secret file or message, thereby evading detection. The concealed information is later extracted at its intended destination. Many legitimate applications of steganography exist, ranging from hiding watermarks or trademarks covertly within multimedia or other content files, to embedding sensitive information securely. In this paper, we introduce Steganographic schemes employing Linear Network Error Correction Codes (LNECCs). Initially, we emphasize the enhanced throughput achieved through the integration of steganography into joint channel-network codes, then we demonstrate the feasibility of leveraging LNECCs and their corresponding error patterns to formulate distortion-free steganographic schemes. Specifically, for a given LNECC in relation to a terminal and a resolvable error pattern, with the precondition that the minimum cut between them is positive, we delineate the corresponding distortion-free steganographic protocols. Moreover, we conduct a thorough analysis of our scheme and fortify it against potent steganalysis attacks. Finally, we propose a prospective solution for cases involving erroneous retrievals.

Telescopic Network of Zhulong for Orbit Determination and Prediction of Space Objects

The increasing proliferation of space debris, intermittent space incidents, and the rapid emergence of massive LEO satellite constellations pose significant threats to satellites in orbit. Ground-based optical observations play a crucial role in space surveillance and space situational awareness (SSA). The Zhulong telescopic observation network stands as a pivotal resource in the realm of space object tracking and prediction. This publicly available network plays a critical role in furnishing essential data for accurately delineating and forecasting the orbit of space objects in Earth orbit. Comprising a sophisticated array of hardware components including precise telescopes, optical sensors, and image sensors, the Zhulong network synergistically collaborates to achieve unparalleled levels of precision in tracking and observing space objects. Central to the network’s efficacy is its ability to extract positional information, referred to as angular data, from consecutive images. These angular data serve as the cornerstone for precise orbit determination and prediction. In this study, the CPF (Consolidated Prediction Format) orbit serves as the reference standard against which the accuracy of the angular data is evaluated. The findings reveal that the angular data error of the Zhulong network remains consistently below 3 arcseconds, attesting to its remarkable precision. Moreover, through the accumulation of angular data over time, coupled with the utilization of numerical integration and least squares methods, the Zhulong network facilitates highly accurate orbit determination and prediction for space objects. These methodologies leverage the wealth of data collected by the network to extrapolate trajectories with unprecedented accuracy, offering invaluable insights into the behavior and movement of celestial bodies. The results presented herein underscore the immense potential of electric optic telescopes in the realm of space surveillance. By harnessing the capabilities of the Zhulong network, researchers and astronomers can gain deeper insights into the dynamics of space objects, thereby advancing our understanding of the cosmos. Ultimately, the Zhulong telescopic observation network emerges as a pioneering tool in the quest to unravel the mysteries of the universe.

Environmental information-assisted intelligent fusion localization for vehicles in urban area

In this paper, we propose a localization methodology aimed at improving accuracy through two primary aspects: environment information assisting fusion algorithm and the elaborate localization errors prediction. First, the environment-assisted particle filter (EA-PF) is proposed by enhancing the particle filter algorithm with perceived environmental feature information, which is reflected by the rasterized map constructed based on LiDAR sensors. Then, an elaborate localization error prediction model is designed to accurately forecast Inertial Navigation System (INS) errors. The model comprises the noise characteristic feature extraction and encoding network and multi-feature error prediction network. The localization errors in the EA-PF output can be further compensated. The proposed localization methodology can be divided into two operational modes based on the availability of Global Navigation Satellite System (GNSS) signal: GNSS-available and GNSS-unavailable modes. In the GNSS-available mode, the EA-PF algorithm yields precise localization results, while in the GNSS-unavailable modes, the elaborate localization error prediction model can effectively predict INS errors and provide the compensation for the EA-PF. To validate the effectiveness of the proposed methodology, relevant test experiments were performed using the KITTI dataset. The experimental validation results demonstrate the feasibility and effectiveness of the proposed methodology. The experimental results demonstrate that the EA-PF outperforms traditional PF methods by improving the root mean square error (RMS.) by 32.70 % on average during 90 s GNSS outages. Moreover, the proposed localization methodology achieves 2.30 m RMS error on average during 90 s GNSS outage.