Neural Network Training Methods Research Articles

Compensated Neural Network Training Algorithm with Minimized Training Dataset for Modeling the Switching Transients of SiC MOSFETs

Accurate modeling of the switching transients of SiC MOSFETs is essential for overvoltage evaluation, EMI prediction, and other critical applications. Due to the fast switching speed, the switching transients of SiC MOSFETs are highly sensitive to parasitic parameters and nonlinear components, making precise modeling challenging. This paper proposes a hybrid model for SiC MOSFET, in which the analytical model is treated as the basis to provide the fundamental waveforms (knowledge-driven), while the neural network (NN) is utilized to fit the high-order and nonlinear features (data-driven). An NN training method with augmented data is proposed to minimize the training datasets. Verification results show that, even though the NN is trained with the data from a single operating condition, the model can accurately predict switching transients of other operating conditions. The proposed methodology has the potential to co-work with the “black-box” or “grey-box” models to enhance the model accuracy.

Genetic Algorithm-Based Training Method for Memristor Neural Networks

Genetic Algorithm-Based Training Method for Memristor Neural Networks

Real-Time CNN Training and Compression for Neural-Enhanced Adaptive Live Streaming.

We propose a real-time convolutional neural network (CNN) training and compression method for delivering high-quality live video even in a poor network environment. The server delivers a low-resolution video segment along with the corresponding CNN for super resolution (SR), after which the client applies the CNN to the segment in order to recover high-resolution video frames. To generate a trained CNN corresponding to a video segment in real-time, our method rapidly increases the training accuracy by promoting the overfitting property of the CNN while also using curriculum-based training. In addition, assuming that the pretrained CNN is already downloaded on the client side, we transfer only residual values between the updated and pretrained CNN parameters. These values can be quantized with low bits in real time while minimizing the amount of loss, as the distribution range is significantly narrower than that of the updated CNN. Quantitatively, our neural-enhanced adaptive live streaming pipeline (NEALS) achieves higher SR accuracy and a lower CNN compression loss rate within a constrained training time compared to the state-of-the-art CNN training and compression method. NEALS achieves 15 to 48% higher quality of the user experience compared to state-of-the-art neural-enhanced live streaming systems.

SemiH: DFT Hamiltonian neural network training with semi-supervised learning

Abstract Over the past decades, density functional theory (DFT) calculations have been utilized in various fields such as materials science and semiconductor devices. However, due to the inherent nature of DFT calculations, which rigorously consider interactions between atoms, they require significant computational cost. To address this, extensive research has recently focused on training neural networks to replace DFT calculations. However, previous methods for training neural networks necessitated an extensive number of DFT simulations to acquire the ground truth (Hamiltonians). Conversely, when dealing with a limited amount of training data, deep learning models often display increased errors in predicting Hamiltonians and band structures for testing data. This phenomenon poses the potential risk of generating inaccurate physical interpretations, including the emergence of unphysical branches within band structures. To tackle this challenge, we propose a novel deep learning-based method for calculating DFT Hamiltonians, specifically tailored to produce accurate results with limited training data. Our framework not only employs supervised learning with the calculated Hamiltonian but also generates pseudo Hamiltonians (targets for unlabeled data) and trains the neural networks on unlabeled data. Particularly, our approach, which leverages unlabeled data, is noteworthy as it marks the first attempt in the field of neural network Hamiltonians. Our framework showcases the superior performance of our framework compared to the state-of-the-art approach across various datasets, such as MoS2, Bi2Te3, HfO2, and InGaAs. Moreover, our framework demonstrates enhanced generalization performance by effectively utilizing unlabeled data, achieving noteworthy results when evaluated on data more complex than the training set, such as configurations with more atoms and temperature ranges outside the training data.

Towards Non-Region Specific Large-Scale Inundation Modelling with Machine Learning Methods

Traditional flood inundation modelling methods are computationally expensive and not suitable for near-real time inundation prediction. In this study we explore a data-driven machine learning method to complement and, in some cases, replace existing methods. Given sufficient training data and model capacity, our design enables a single neural network instance to approximate the flow characteristics of any input region, opening the possibility of applying the model to regions without available training data. To demonstrate the method we apply it to a very large >8000 km2 region of the Fitzroy river basin in Western Australia with a spatial resolution of 30 m × 30 m, placing an emphasis on efficiency and scalability. In this work we identify and address a range of practical limitations, e.g., we develop a novel water height regression method and cost function to address extreme class imbalances and by carefully constructing the input data, we introduce some natural physical constraints. Furthermore, a compact neural network design and training method was developed to enable the training problem to fit within GPU memory constraints and a novel dataset was constructed from the output of a calibrated two-dimensional hydrodynamic model. A good correlation between the predicted and groundtruth water heights was observed.

Прогнозирование помех электростатического разряда в электронном устройстве с использованием искусственной нейронной сети

Electrostatic discharge is a dangerous source of natural electromagnetic interference to the operation of modern electronic devices. Although measures have been taken to reduce and remove electrostatic discharge from the operating area of electronic devices, the absence of such discharge cannot be guaranteed. Therefore, when designing modern electronic devices, it is necessary to take into account the possibility of such electrostatic discharges in advance and take protective measures. The article proposes a method for predicting the amplitude of interference in the communication line of an electronic device when exposed to an electrostatic discharge on its metal case. The technique is based on the use of an artificial neural network. The technique includes the analysis of significant input parameters that affect the amount of interference in an electronic device; development of an experimental stand for measuring interference; choosing the structure and parameters of a neural network for predicting interference; choosing a training method for an artificial neural network; choosing a metric for assessing the quality of training; normalization of training data; training an artificial neural network using experimental noise data; predicting the amplitude of interference in the communication line of an electronic device when exposed to an electrostatic discharge on its body; where necessary, selecting and implementing electrostatic discharge protection measures. Examples of training an artificial neural network based on experimental data are given. The training was carried out for 572 epochs. For the training and testing sets, the discrepancy between the predicted data and the average measured noise values was 3.61% and 3.95%, respectively. Examples are given of predicting the amplitude of interference when exposed to electrostatic discharge. The results obtained indicate the possibility of practical use of an artificial neural network to solve problems of electromagnetic interference analysis.

Status and Prospects for the Use of Remote Sensing Data for the Detection of Wildfires

The article discusses the problem of fires as a serious threat to life, economy and ecosystems, highlighting the need for early detection and suppression of fires. The potential of the combination of Earth remote sensing and neural networks for rapid and accurate detection of natural fires is studied. The significance of applying artificial intelligence, the development of deep learning methods for neural network models, to analyze space images and detect early signs of fires is emphasized. The article also provides examples of successful projects and research in the field of wildfire detection. The final part of the paper emphasizes the need for further research and development of neural network training methods, expansion of training datasets and improvement of space imagery acquisition technologies for effective control and prevention of fires, in order to protect the environment and minimize damage to people.

Accuracy Analysis of DeepL: Breakthroughs in Machine Translation Technology

This study examines the accuracy and technological innovations of DeepL, a prominent machine translation tool. Through a comprehensive literature review, we analyze DeepL's performance compared to other translation systems, exploring its advanced neural network architecture and training methods. Key findings indicate that DeepL consistently outperforms other tools in BLEU scores and human evaluations, particularly excelling in handling context, idiomatic expressions, and specialized terminology. The research highlights DeepL's use of the Transformer model, diverse training data, and techniques like transfer learning and data augmentation. Practical applications across academic, professional, and educational sectors are discussed, with special emphasis on how DeepL benefits students and teachers by facilitating multilingual learning, enhancing comprehension of foreign texts, and assisting in accurate translation of academic materials. User feedback underscores DeepL's accuracy and user-friendly features. While demonstrating significant advancements in machine translation technology, this study also acknowledges areas for potential improvement, contributing to the ongoing development of AI-driven language solutions.

Intelligent PID Controller Based on Neural Network for AI-Driven Control Quadcopter UAV

Unmanned Aerial Vehicle (UAV), specifically a quadcopter is publicly popular which it provides services in different applications such as aerial delivery, aerial photography, military, weather forecasting and more examples to date. A Proportional-Integral-Derivative (PID) controller is one of the control techniques that can provide stabilization and reliable trajectory tracking. However, proper PID gains are needed to ensure a stable flight and it should be hybridized or improved to increase the robustness, reliability, and stabilization during flight. In this paper, an intelligent PID controller using neural network is proposed based on Levenberg-Marquardt feedforward neural network training method. The PID gains are initialized using different ranges according to the optimal gains generated by Particle Swarm Optimization, and this contributes towards a good training performance using Mean Square Error (MSE) evaluation. The trained network takes desired output and references as input data to calculate the required combination of PID gains as the output. The including of the response characteristics as the input data for the network, together with reference, error, and control input is the significance of the work. The performance of this work is presented using MSE performances, attitudes and altitude stabilization, and trajectory tracking reliability through error index performances. The simulation results graphically prove that the proposed controller provides better stability with reduced overshoot and settling times. Disturbance rejection is also enhanced by 1.7% compared to manual tuned PID controller. The reliability of the proposed controller highlights avenues for further exploration in AI-driven control strategies for quadcopter systems.

Моделирование помех в электронном устройстве при воздействии импульсного магнитного поля с использованием искусственной нейронной сети

Modern electronic means are quite sensitive to electromagnetic interference. At the same time, they must function reliably in the existing electromagnetic environment. Lightning discharges and industrial sources make a significant contribution to the formation of the electromagnetic environment around electronic equipment. In this case, pulsed magnetic fields with microsecond parameters most often arise. The most rational approach to ensuring electromagnetic compatibility of electronic means is the most complete accounting and protection from possible phenomena at the design stage. Different methods for modeling the consequences of exposure to electromagnetic interference have their own advantages and disadvantages. To develop a technique and modeling interference in electronic means based on an artificial neural network using the example of the influence of a pulsed magnetic field a the purpose of this work. A practical technique for calculation the magnitude of interference in electronic means using an artificial neural network the paper proposes. All stages of the technique are described: analysis of the main input parameters affecting the amount of interference in the electronic means; the use of a special experimental stand for measuring interference depending on significant input parameters; choosing the structure and parameters of an artificial neural network to modeling interference; choosing a training method for an artificial neural network; choosing a criterion for assessing the quality of training when solving a regression problem; normalization of training data; training an artificial neural network using measured data; modeling the amount of interference in the communication line of an electronic means when exposed to a pulsed magnetic field; assessment of consequences and selection of methods of protection against interference. As an example, we consider the problem of modeling the amount of interference in a communication line inside an electronic means when exposed to a pulsed magnetic field. The magnetic field has parameters recommended by the requirements of the regulatory document on electromagnetic compatibility of devices. In the problem under consideration, an acceptable discrepancy in results is achieved with an acceptable number of neural network training epochs.

The Comparison of the Multi-Layer Artificial Neural Network Training Methods in Terms of the Predictive Quality of the Coefficient of Friction of 1.0338 (DC04) Steel Sheet.

Friction is one of the main phenomena accompanying sheet metal forming methods, affecting the surface quality of products and the formability of the sheet metal. The most basic and cheapest way to reduce friction is to use lubricants, which should ensure the highest lubrication efficiency and at the same time be environmentally friendly. Due to the trend towards sustainable production, vegetable oils have been used in research as an alternative to petroleum-based lubricants. The analysis of friction in sheet metal forming requires an appropriate tribotester simulating the friction conditions in a specific area of the sheet metal being formed. Research has used a special strip drawing tribometer, enabling the determination the value of the coefficient of friction in the blankholder zone in the deep drawing process. Quantitative analysis of the friction phenomenon is necessary at the stage of designing the technological process and selecting technological parameters, including blankholder pressure. This article presents the results of friction testing of 1.0338 (DC04) steel sheets using a strip drawing test. The experimental tests involved pulling a strip of sheet metal between two countersamples with a rounded surface. The tests were carried out on countersamples with different levels of roughness for the range of contact pressures occurring in the blankholder zone in the deep drawing process (1.7-5 MPa). The values of the coefficient of friction determined under dry friction conditions were compared with the results for edible (corn, sunflower and rapeseed) and non-edible (Moringa, Karanja) vegetable lubricants. The tested oils are the most commonly used vegetable-based biolubricants in metal forming operations. Multi-layer artificial neural networks were used to determine the relationship between the value of the contact pressure, the roughness of the countersamples, the oil viscosity and density, and the value of the coefficient of friction. Rapeseed oil provided the best lubrication efficiency during friction testing for all of the tested samples, with an average surface roughness of Sa 0.44-1.34 μm. At the same time, as the roughness of the countersamples increased, a decrease in lubrication efficiency was observed. The lowest root mean squared error value was observed for the MLP-4-8-1 network trained with the quasi-Newton algorithm. Most of the analysed networks with different architectures trained using the various algorithms showed that the kinematic viscosity of the oil was the most important aspect in assessing the friction of the sheets tested. The influence of kinematic viscosity on the value of the coefficient of friction is strongly dependent on the surface roughness of the countersamples.

Controlled descent training

SummaryIn this work, a novel and model‐based artificial neural network (ANN) training method is developed supported by optimal control theory. The method augments training labels in order to robustly guarantee training loss convergence and improve training convergence rate. Dynamic label augmentation is proposed within the framework of gradient descent training where the convergence of training loss is controlled. First, we capture the training behavior with the help of empirical Neural Tangent Kernels (NTK) and borrow tools from systems and control theory to analyze both the local and global training dynamics (e.g., stability, reachability). Second, we propose to dynamically alter the gradient descent training mechanism via fictitious labels as control inputs and an optimal state feedback policy. In this way, we enforce locally optimal and convergent training behavior. The novel algorithm, Controlled Descent Training (CDT), guarantees local convergence. CDT unleashes new potentials in the analysis, interpretation, and design of ANN architectures. The applicability of the method is demonstrated on standard regression and classification problems.

Synthetic data generation methods for training neural networks in the task of segmenting the level of crop nitrogen status on UAV images of agricultural fields

This study is devoted to the automatization of the image masks’ construction of large sized agricultural objects in precision farming tasks for training neural network methods for crop’s nitrogen status analysis using georeferenced images. The scientific direction is extremely relevant because it allows to automate and replace the manual process of data labeling, significantly reducing the cost of preparing training samples. In the paper, four new synthetic data generation methods are proposed for training neural networks aimed at UAV image segmentation by the level of crop nitrogen supply on an agricultural field. In particular, the paper gives a description of synthetic data generation algorithms based on nitrogen covering with lines, parabolas, and areas. Experiments were carried out to test and evaluate the quality of these algorithms using eight modern image segmentation methods: two classical machine learning methods (Random Forest and XGBoost), four convolutional neural network methods based on U-Net architecture, and two transformers (TransUnet and UnetR). The results showed that two algorithms based on areas gave the best accuracy for convolutional neural networks and transformers — 98–100 %. Classical machine learning methods showed very low values for all quality metrics — 27–44 %.

РАСПОЗНАВАНИЕ АКТИВНОСТИ СТУДЕНТОВ В АУДИТОРИИ МЕТОДАМИ ГЛУБОКОГО ОБУЧЕНИЯ

Human activity recognition is one of the actively developing areas of artificial intelligence that can be implemented in learning environments to improve safety, efficiency and quality of education. Deep learning transfer is a widespread method of training neural networks and has proven its effectiveness in such complex tasks as digital image and video processing. In this paper, we propose a model for recognizing different student activities using pretrained models including VGG16, ResNet50, DenseNet201, EfficientNetB0 and Xception, which are then trained using the method proposed in the paper on a Class box image set created by the authors. The EfficientNetB0 model showed the maximum performance among the considered models and achieved an accuracy of 94,25%. The system proposed in this study aims to create a safer and more productive learning environment for students and teachers.

Polymer chemistry informed neural networks (PCINNs) for data-driven modelling of polymerization processes

A method for training neural networks to predict the outcome of polymerization processes is described that incorporates fundamental chemical knowledge. This permits generation of data-driven predictive models with limited datasets.

Cost Efficient Training Method for Artificial Neural Networks based on Engine Measurements

Cost Efficient Training Method for Artificial Neural Networks based on Engine Measurements

Azimuth estimation based on CNN and LSTM for geomagnetic and inertial sensors data

Although estimating the azimuth using a geomagnetic sensor is very useful, the estimation error may be very large due to the surrounding geomagnetic disturbance. We proposed a novel method for preprocessing appropriately for geomagnetic and inertial sensor data to be suitable for the proposed Artificial Neural Network model and training method for the model. As a result, the probability of azimuth estimation error within 1 degree is 96.4% with regression estimation. For classification estimation, when the azimuth estimation probability is 90% or more, the probability that the azimuth estimation error is within 1 degree is 100%.

Multi-source underwater DOA estimation using PSO-BP neural network based on high-order cumulant optimization

Due to the complex and changeable environment under water, the performance of traditional DOA estimation algorithms based on mathematical model, such as MUSIC, ESPRIT, etc., degrades greatly or even some mistakes can be made because of the mismatch between algorithm model and actual environment model. In addition, the neural network has the ability of generalization and mapping, it can consider the noise, transmission channel inconsistency and other factors of the objective environment. Therefore, this paper utilizes Back Propagation (BP) neural network as the basic framework of underwater DOA estimation. Furthermore, in order to improve the performance of DOA estimation of BP neural network, the following three improvements are proposed. (1) Aiming at the problem that the weight and threshold of traditional BP neural network converge slowly and easily fall into the local optimal value in the iterative process, PSO-BP-NN based on optimized particle swarm optimization (PSO) algorithm is proposed. (2) The Higher-order cumulant of the received signal is utilized to establish the training model. (3) A BP neural network training method for arbitrary number of sources is proposed. Finally, the effectiveness of the proposed algorithm is proved by comparing with the state-of-the-art algorithms and MUSIC algorithm.

Prediction of Short-Shot Defects in Injection Molding by Transfer Learning

For a long time, the traditional injection molding industry has faced challenges in improving production efficiency and product quality. With advancements in Computer-Aided Engineering (CAE) technology, many factors that could lead to product defects have been eliminated, reducing the costs associated with trial runs during the manufacturing process. However, despite the progress made in CAE simulation results, there still exists a slight deviation from actual conditions. Therefore, relying solely on CAE simulations cannot entirely prevent product defects, and businesses still need to implement real-time quality checks during the production process. In this study, we developed a Back Propagation Neural Network (BPNN) model to predict the occurrence of short-shots defects in the injection molding process using various process states as inputs. We developed a Back Propagation Neural Network (BPNN) model that takes injection molding process states as input to predict the occurrence of short-shot defects during the injection molding process. Additionally, we investigated the effectiveness of two different transfer learning methods. The first method involved training the neural network model using CAE simulation data for products with length–thickness ratios (LT) of 60 and then applying transfer learning with real process data. The second method trained the neural network model using real process data for products with LT60 and then applied transfer learning with real process data from products with LT100. From the results, we have inferred that transfer learning, as compared to conventional neural network training methods, can prevent overfitting with the same amount of training data. The short-shot prediction models trained using transfer learning achieved accuracies of 90.2% and 94.4% on the validation datasets of products with LT60 and LT100, respectively. Through integration with the injection molding machine, this enables production personnel to determine whether a product will experience a short-shot before the mold is opened, thereby increasing troubleshooting time.

A method of concrete damage detection and localization based on weakly supervised learning

AbstractAutomatic inspection of concrete surface defects based on visual elements is crucial for the timely detection of security risks in infrastructure. Moreover, accurate determination of the geographical location of the detected defects is critical for subsequent maintenance and reinforcement tasks. This study employed convolutional neural network (CNN) training methods for detection and localization. This approach employs bounding boxes to confine damaged pixels and utilizes projection loss to foster similarity learning between pixels. In addition, geotags are automatically indexed through the vectorization of invariant features in a scene, which maintains high model accuracy and reduces training costs. The proposed method can detect and classify typical concrete defects in complex scenarios and accurately locate them without the use of external sensors. In addition, the proposed model can achieve pixel‐level defect detection and geographic location determination through the cost of bounding box annotation and automatic indexing. The proposed model was evaluated using 10,691 images of four typical concrete defects in various complex environments. The results demonstrate that the proposed method achieves a pixel‐level detection accuracy of 48.75 and a location accuracy of 83.69, showing a better performance than other methods.