Minimization Of Loss Function Research Articles

Intelligent Method of Identifying the Nonlinear Dynamic Model for Helicopter Turboshaft Engines

This research focused on the helicopter turboshaft engine dynamic model, identifying task solving in unsteady and transient modes (engine starting and acceleration) based on sensor data. It is known that about 85% of helicopter turboshaft engines operate in steady-state modes, while only around 15% operate in unsteady and transient modes. Therefore, developing dynamic multi-mode models that account for engine behavior during these modes is a critical scientific and practical task. The dynamic model for starting and acceleration modes has been further developed using on-board parameters recorded by sensors (gas-generator rotor r.p.m., free turbine rotor speed, gas temperature in front of the compressor turbine, fuel consumption) to achieve a 99.88% accuracy in identifying the dynamics of these parameters. An improved Elman recurrent neural network with dynamic stack memory was introduced, enhancing the robustness and increasing the performance by 2.7 times compared to traditional Elman networks. A theorem was proposed and proven, demonstrating that the total execution time for N Push and Pop operations in the dynamic stack memory does not exceed a certain value O(N). The training algorithm for the Elman network was improved using time delay considerations and Butterworth filter preprocessing, reducing the loss function from 2.5 to 0.12% over 120 epochs. The gradient diagram showed a decrease over time, indicating the model’s approach to the minimum loss function, with optimal settings ensuring the stable training.

Image acquisition technology for unmanned aerial vehicles based on YOLO - Illustrated by the case of wind turbine blade inspection

Wind energy, as a renewable energy source, is becoming increasingly important. The maintenance and damage detection of wind turbine blades are particularly crucial. For this purpose, the study aims to optimize the You Only Look Once (YOLO) processing algorithm for drone images to improve the detection efficiency. Firstly, the damage images captured by drones are preprocessed and optimized, including deblurring, noise reduction, and image enhancement. Subsequently, the YOLOv5 model is improved in terms of structure and regression function, and a novel damage detection model is proposed. The research results indicated that the minimum loss function value of the improved model was 2.75, the average accuracy was 95 %, and the highest intersection over union was 91 %. After simulation testing, the detection effect of this model on abrasion, crackle, edge cracking, and coating peeling images was significantly better than other models in the same series. Its average time was as short as 2.43 s, reaching a maximum frame rate of 35.46. From this, the combination of drone image technology and improved image processing algorithm has a positive impact on improving the operational efficiency and safety of wind turbine blades. Compared with the traditional methods, the proposed model has significant advantages in terms of accuracy and real-time performance of damage detection, providing a new technical means for efficient maintenance of wind turbines. Meanwhile, the method shows high robustness and reliability in different types of damage detection, demonstrating the extensive potential in practical applications.

The Role of Mathematics in Artificial Intelligence and Machine Learning

Mathematics serves as the foundational backbone of artificial intelligence (AI) and machine learning (ML), providing the essential tools and frameworks for developing sophisticated algorithms and models. the pivotal role of various mathematical disciplines, including linear algebra, calculus, probability theory, and optimization, in advancing AI and ML technologies. We begin by examining how linear algebra facilitates the manipulation and transformation of high-dimensional data, which is crucial for techniques such as principal component analysis (PCA) and singular value decomposition (SVD). Next, we delve into the applications of calculus in training neural networks through gradient-based optimization methods, highlighting the importance of differentiation and integration in backpropagation and loss function minimization. the role of probability theory in handling uncertainty and making predictions, emphasizing its application in Bayesian networks, Markov decision processes, and probabilistic graphical models. Additionally, we discuss optimization techniques, both convex and non-convex, that are fundamental to finding optimal solutions in machine learning tasks, including support vector machines (SVMs) and deep learning architectures.

Practical one-shot data-driven design of fractional-order PID controller: Fictitious reference signal approach

This study proposes a one-shot data-driven tuning method for a fractional-order proportional-integral-derivative (FOPID) controller. The proposed method tunes the FOPID controller in the model-reference control formulation. A loss function is defined to evaluate the match between a given reference model and the closed-loop response while explicitly considering the closed-loop stability. A loss function value is based on the fictitious reference signal computed using the input/output data. Model matching is achieved via loss function minimization. The proposed method is simple and practical: it needs only one-shot input/output data of a plant (no plant model required), considers the bounded-input bounded-output stability of the closed-loop system from a bounded reference input to a bounded output, and automatically determines the appropriate parameter value via optimization. Numerical simulations show that the proposed approach facilitates good control performance, and destabilization can be avoided even if perfect model matching is unachievable.

Satellite image recognition using ensemble neural networks and difference gradient positive-negative momentum

The modern machine learning theory finds application in many areas of human activity. One of the most dispersed tasks is pattern recognition on satellite images. It is difficult for a person to recognize a large number of images in a short time. It made the researchers develop the automation process, such as neural network engagement. The loss function minimization and ensemble learning raise the pattern recognition accuracy. We propose the robust difference gradient positive-negative momentum optimization algorithm that achieves the global minimum of the loss function with higher accuracy and fewer iterations than known analogs. Such an optimization algorithm contains the generalized average moving estimation approach and more effective learning rate control by additional parameters. The proposed optimizer has the regret-bound rate estimation, belonging to OT, and converges to the global minimum. However, the main problems in optimization theory are vanishing and blowing gradient values, where the standard gradient-based algorithms fail to achieve the required objective function value. The vanishing and blowing gradient problems meet in Rastrigin and Rosebrock test functions, where the proposed optimization algorithm attains the global extreme in the shortest number of iterations and has a more stable convergence process than state-of-the-art methods. Afterward, there are trained deep convolutional neural networks with different optimizers on satellite images from the University of California merced dataset containing 21 object classes, where the proposed algorithm gives the highest accuracy. There is a suggested ensemble-learning model consisting of 4 networks with different optimizers. The prediction results receive weight coefficients distributed according to the majority voting and ensemble neural network retrains with the higher pattern recognition accuracy. The suggested ensemble-learning model with the developed optimizer raised the accuracy by 1 %–4 % percentage points.

Validation of 4DOF maneuvering coefficients optimization using hydrodynamic force and moment estimated from free-running model test results

The present study concerns validation of data-driven modeling for ship dynamics from free-running model test results, in terms of estimation of hydrodynamic force and moment, maneuvering coefficients, and maneuver simulation results. An estimation method for the hydrodynamic force and moment from the acceleration of a surface combatant model ship is suggested and compared with the conventional ship dynamics modeling. The hull dynamics modeling relates the motion and hydrodynamic force and moment is developed by optimizing maneuvering coefficients which leads to minimum loss function of hydrodynamic force and moment and maneuvering criteria. The maneuvering coefficients of the dynamics model is compared with a conventional system-based dynamics model. Lastly, the maneuvering simulation results are compared to the free-running model test results. The suggested model adequately predicts hydrodynamic force and moment in highly coupled sway and yaw motion.

The Role of Mathematics in Artificial Intelligence and Machine Learning

Mathematics serves as the foundational backbone of “artificial intelligence (AI) and machine learning (ML), providing the essential” tools and frameworks for developing sophisticated algorithms and models. the pivotal role of various mathematical disciplines, including linear algebra, calculus, probability theory, and optimization, in advancing AI and ML technologies. We begin by examining how linear algebra facilitates the manipulation and transformation of high-dimensional data, which is crucial for “techniques such as principal component analysis (PCA) and singular value decomposition (SVD)”. Next, we delve into the applications of calculus in training neural networks through gradient-based optimization methods, highlighting the importance of differentiation and integration in backpropagation and loss function minimization. the role of probability theory in handling uncertainty and making predictions, emphasizing its application in Bayesian networks, Markov decision processes, and probabilistic graphical models. Additionally, we discuss optimization techniques, both convex and non-convex, that are fundamental to finding optimal solutions in machine learning tasks, including support vector machines (SVMs) and deep learning architectures.

Deep Q Network Based on a Fractional Political–Smart Flower Optimization Algorithm for Real-World Object Recognition in Federated Learning

An imperative application of artificial intelligence (AI) techniques is visual object detection, and the methods of visual object detection available currently need highly equipped datasets preserved in a centralized unit. This usually results in high transmission and large storage overheads. Federated learning (FL) is an eminent machine learning technique to overcome such limitations, and this enables users to train a model together by processing the data in the local devices. In each round, each local device performs processing independently and updates the weights to the global model, which is the server. After that, the weights are aggregated and updated to the local model. In this research, an innovative framework is designed for real-world object recognition in FL using a proposed Deep Q Network (DQN) based on a Fractional Political–Smart Flower Optimization Algorithm (FP-SFOA). In the training model, object detection is performed by employing SegNet, and this classifier is effectively tuned based on the Political–Smart Flower Optimization Algorithm (PSFOA). Moreover, object recognition is performed based on the DQN, and the biases of the classifier are finely optimized based on the FP-SFOA, which is a hybridization of the Fractional Calculus (FC) concept with a Political Optimizer (PO) and a Smart Flower Optimization Algorithm (SFOA). Finally, the aggregation at the global model is accomplished using the Conditional Autoregressive Value at Risk by Regression Quantiles (CAViaRs) model. The designed FP-SFOA obtained a maximum accuracy of 0.950, minimum loss function of 0.104, minimum MSE of 0.122, minimum RMSE of 0.035, minimum FPR of 0.140, maximum average precision of 0.909, and minimum communication cost of 0.078. The proposed model obtained the highest accuracy of 0.950, which is a 14.11%, 6.42%, 7.37%, and 5.68% improvement compared to the existing methods.

A meta-learning network method for few-shot multi-class classification problems with numerical data

The support vector machine (SVM) method is an important basis of the current popular multi-class classification (MCC) methods and requires a sufficient number of samples. In the case of a limited number of samples, the problem of over-learning easily occurs, resulting in unsatisfactory classification. Therefore, this work investigates an MCC method that requires only a small number of samples. During model construction, raw data are converted into two-dimensional form via preprocessing. Via feature extraction, the learning network is measured and the loss function minimization principle is considered to better solve the problem of learning based on a small sample. Finally, three examples are provided to illustrate the feasibility and effectiveness of the proposed method.

A neural network-based enrichment of reproducing kernel approximation for modeling brittle fracture

Numerical modeling of localizations is a challenging task due to the evolving rough solution in which the localization paths are not predefined. Despite decades of efforts, there is a need for innovative discretization-independent computational methods to predict the evolution of localizations. In this work, an improved version of the neural network-enhanced Reproducing Kernel Particle Method (NN-RKPM) is proposed for modeling brittle fracture. In the proposed method, a background reproducing kernel (RK) approximation defined on a coarse and uniform discretization is enriched by a neural network (NN) approximation under a Partition of Unity framework. In the NN approximation, the deep neural network automatically locates and inserts regularized discontinuities in the function space. The NN-based enrichment functions are then patched together with RK approximation functions using RK as a Partition of Unity patching function. The optimum NN parameters defining the location, orientation, and displacement distribution across location together with RK approximation coefficients are obtained via the energy-based loss function minimization. To regularize the NN-RK approximation, a constraint on the spatial gradient of the parametric coordinates is imposed in the loss function. Analysis of the convergence properties shows that the solution convergence of the proposed method is guaranteed. The NN enrichment allows the modeling of evolving cracks by a fixed coarse RK discretization without adaptive refinement for enhanced computational efficiency. The effectiveness of the proposed method is demonstrated by a series of numerical examples involving damage propagation and branching.

Q-Learning-Aided Offloading Strategy in Edge-Assisted Federated Learning over Industrial IoT

Federated learning (FL) is a key solution to realizing a cost-efficient and intelligent Industrial Internet of Things (IIoT). To improve training efficiency and mitigate the straggler effect of FL, this paper investigates an edge-assisted FL framework over an IIoT system by combining it with a mobile edge computing (MEC) technique. In the proposed edge-assisted FL framework, each IIoT device with weak computation capacity can offload partial local data to an edge server with strong computing power for edge training. In order to obtain the optimal offloading strategy, we formulate an FL loss function minimization problem under the latency constraint in the proposed edge-assisted FL framework by optimizing the offloading data size of each device. An optimal offloading strategy is first derived in a perfect channel state information (CSI) scenario. Then, we extend the strategy into an imperfect CSI scenario and accordingly propose a Q-learning-aided offloading strategy. Finally, our simulation results show that our proposed Q-learning-based offloading strategy can improve FL test accuracy by about 4.7% compared to the conventional FL scheme. Furthermore, the proposed Q-learning-based offloading strategy can achieve similar performance to the optimal offloading strategy and always outperforms the conventional FL scheme in different system parameters, which validates the effectiveness of the proposed edge-assisted framework and Q-learning-based offloading strategy.

Optimization Methods for Convolutional Neural Networks – The LeNet-5 Algorithm

Convolutional neural networks are enhanced version of fully connected neural networks. The neural networks are used to recognize objects after training the neural network system for some datasets that can also be divided into classes at the output. These networks were a breakthrough in computer vision filed for object recognition where the system can optimize its parameters for better results with using feed forward and back propagation. The convolutional neural networks reduced the time of training and testing the dataset by replacing the full network nodes connecting to each node in the subsequent layer to some nodes or filter to each subsequent layer node. There are many algorithms for convolutional neural networks ranging from simple algorithms to complex ones. Each algorithm has different hidden layers with different hyper parameters and filters. The activation functions and number of nodes in each layer for each algorithm may be different. The applications for these convolutional neural networks cover many fields such as hand written digit recognition, alphabet handwritten recognition, and any group of objects that can be divided into classes such as cloth, X-ray imaging and many more. The LeNet-5 algorithm is one of the convolutional neural networks. With full analysis of this algorithm, I will prove that a simple module of the algorithm can provide maximum accuracy and minimum loss function than the original algorithm.

Optimum design of nonlinear structures via deep neural network-based parameterization framework

In this paper, a robust deep neural network (DNN)-based parameterization framework is proposed to directly solve the optimum design for geometrically nonlinear trusses subject to displacement constraints. The core idea is to integrate DNN into Bayesian optimization (BO) to find the best optimum structural weight. Herein, the design variables of the structure are parameterized by weights and biases of the network with the spatial coordinates of all joints as the training data. A loss function of the network is built based on the predicted cross-sectional areas and deflection constraints obtained by supporting finite element analysis (FEA) and arc-length method. Accordingly, the optimum weight corresponding to the minimum loss function is indicated as soon as the complete training process. And then it is also serving as an objective of the BO for performing the hyperparameter optimization (HPO) to find the best optimum structural weight. Several illustrative numerical examples for geometrically nonlinear space trusses are examined to determine the efficiency and reliability of the proposed approach. The obtained results demonstrate that our framework can overcome the drawbacks of applications of machine learning in computational mechanics.

Analysis of the Application of the Integrated Catastrophe Risk Model Method for the Flood Insurance Program

As the flood rises continue to grow, well-designed insurance programs are becoming an important instrument in flood risk management. One of the obstacles in the flood insurance program is the method used to calculate the premium value. This thesis refers to the Integrated Catastrophe Risk Model (ICRM) which consists of two probability events and stochastic optimization procedures with respect to observation of site-specific risk. The application of the model is illustrated in the study area simulation data. In this thesis, analysis of various aspects of trade-off, new ex-post variables, opportunity occurrence 1 and 2 and minimization of loss function. From the results of research based on these four aspects it can be concluded that the use of Integrated Catastrophe Risk Model method in the optimal flood insurance program.

A Backpropagation Learning Method for Dynamic Parameter Identification of Industrial Robots

Abstract Precise dynamic parameters are of great significance to the model-based controller of industrial robots. The least square (LS) method is widely used to identify the dynamic parameters in reality, but it is sensitive to the measurement noise and may obtain a biased solution. Besides, the nonlinear dynamics are the key and difficult point of identification, such as the nonlinear friction and the coupling effect of adjacent joint, but they are seldom solved simply and effectively. To cope with the above issues, we propose a backpropagation learning (BPL) method for parameter identification. The mean square error between the measured torque and the calculated torque from the dynamic model is taken as the loss function. The gradient of the loss function with respect to the parameters is computed, and the parameters are updated in the negative gradient direction to minimize the loss function until finding the optimal parameters with the minimum loss function. The optimal parameters will not only fit the modeled torque but also compensate for the torque caused by the unmodeled factors, thus improving the parameter accuracy. The proposed method is essentially a supervised learning method, so the impact of measurement noise is reduced by continuous training with a large amount of valid data. The proposed method is verified in an industrial robot platform, and the experimental results show that the proposed method has smaller errors than the weighted least squares (WLS) method and achieves similar accuracy to the semiparametric model (SPM) but has a better generalization ability.

Image Recognition of Male Oilseed Rape (Brassica napus) Plants Based on Convolutional Neural Network for UAAS Navigation Applications on Supplementary Pollination and Aerial Spraying

To ensure the hybrid oilseed rape (OSR, Brassica napus) seed production, two important things are necessary, the stamen sterility on the female OSR plants and the effective pollen spread onto the pistil from the OSR male plants to the OSR female plants. The unmanned agricultural aerial system (UAAS) has developed rapidly in China. It has been used on supplementary pollination and aerial spraying during the hybrid OSR seed production. This study developed a new method to rapidly recognize the male OSR plants and extract the row center line for supporting the UAAS navigation. A male OSR plant recognition model was constructed based on the convolutional neural network (CNN). The sequence images of male OSR plants were extracted, the feature regions and points were obtained from the images through morphological and boundary process methods and horizontal segmentation, respectively. The male OSR plant image recognition accuracies of different CNN structures and segmentation sizes were discussed. The male OSR plant row center lines were fitted using the least-squares method (LSM) and Hough transform. The results showed that the segmentation algorithm could segment the male OSR plants from the complex background. The highest average recognition accuracy was 93.54%, and the minimum loss function value was 0.2059 with three convolutional layers, one fully connected layer, and a segmentation size of 40 pix × 40 pix. The LSM is better for center line fitting. The average recognition model accuracies of original input images were 98% and 94%, and the average root mean square errors (RMSE) of angle were 3.22° and 1.36° under cloudy day and sunny day lighting conditions, respectively. The results demonstrate the potential of using digital imaging technology to recognize the male OSR plant row for UAAS visual navigation on the applications of hybrid OSR supplementary pollination and aerial spraying, which would be a meaningful supplement in precision agriculture.

Human psyche creation by application of natural language processing technologies

Subject of Research. The paper proposes a solution for the human psyche automatic creation based on his speech behavior analysis. It is shown that messages in social networks, instant messengers and chats can be used to form a training data set, both in the format of text messages and audio and video calls. The functions of the psychological type classifier constituents are revealed by the human speech behavior. A comparison is made between multiclass and binary classification based on the loss function minimization. Methods. The human psyche corresponded to the Myers-Briggs type indicator, which subsumes a person to one of 16 types. The technologies of Text Mining for natural language processing and a deep learning model for speech processing were used. The data set for training and testing was formed by recordings of people’s speech translated into text format. Class labels were formed by the content of a text parameters vector, which is a dictionary of frequently encountered words. A deep learning algorithm was used for the human psyche automatic creation and was based on recurrent neural networks of the Long Short-Term Memory type. The algorithm was tested both for multiclass and binary classification. The objectivity of the proposed approach to a human psyche creation was ensured by the variety of content created by a person at various time in accordance with life situations, profession, hobbies and other circumstances. Main Results. A new approach to the automatic human psyche creation is proposed, based on the binary classification and a deep learning model. The convergence of the binary classification results with the test set of the speech behavior of various people is demonstrated. The Long Short-Term Memory network application in binary classification makes it possible to achieve an accuracy equal to 83 % of the psychological type correct determination and reduce the losses to 25 %. Practical Relevance. Automatic human psyche creation based on his speech behavior enables various specialists (such as psychologists, sociologists, human resources staff members) to make decisions when working with a specific person. Analysis of a human personal qualities by his speech behavior is software-implemented.

Cross-Model Hashing Retrieval Based on Deep Residual Network

In the era of big data rich in We Media, the single mode retrieval system has been unable to meet people’s demand for information retrieval. This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes: A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network (CMHR-DRN). The model construction is divided into two stages: The first stage is the feature extraction of different modal data, including the use of Deep Residual Network (DRN) to extract the image features, using the method of combining TF-IDF with the full connection network to extract the text features, and the obtained image and text features used as the input of the second stage. In the second stage, the image and text features are mapped into Hash functions by supervised learning, and the image and text features are mapped to the common binary Hamming space. In the process of mapping, the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval. In training the model, adaptive moment estimation (Adam) is used to calculate the adaptive learning rate of each parameter, and the stochastic gradient descent (SGD) is calculated to obtain the minimum loss function. The whole training process is completed on Caffe deep learning framework. Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH, CMDN and CMSSH.

Minimizing Power Consumption of an Experimental HVAC System Based on Parallel Grid Searching

This paper proposes a parallel grid search algorithm to find an optimal operating point for minimizing the power consumption of an experimental heating, ventilating and air conditioning (HVAC) system. First, a multidimensional, nonlinear and non-convex optimization problem subject to constraints is formulated based on a semi-physical model of the experimental HVAC system. Second, the optimization problem is parallelized based on Graphics Processing Units to simultaneously compute optimization loss functions for different solutions in a searching grid, and to find the optimal solution as the one having the minimum loss function. The proposed algorithm has an advantage that the optimal solution is known with evidence as to the best one subject to current resolutions of the searching grid. Experimental studies are provided to support the proposed algorithm.

An image retrieval method of mammary cancer based on convolutional neural network

This paper designs a brand-new image retrieval method of mammary cancer based on convolution neural network. This method simulates VLAD layer in CNN network structure, designs a trainable universal VLAD layer-NET VLAD layer, reduces dimensions and optimizes VLAD descriptors, applies structure from motion algorithm to automatically label samples, and obtains the minimum loss function value by a new training program of weakly supervised ranking loss. Experiments show that this method has improved retrieval performance compared with similar retrieval methods and non-network structure retrieval methods.