Symbolic Regression Method Research Articles

Synthesis of a mobile robot spatial stabilization system based on machine learning control by symbolic regression

The spatial stabilization system synthesis problem of the robot is considered. The historical overview of methods and approaches for solving the problem of control synthesis is given. It is shown that the control synthesis problem is the most important task in the field of control, for which there are no universal numerical methods for solving it. As one of the ways to solve this problem, it is proposed to use the method of machine learning based on the application of modern symbolic regression methods. This allows you to build universal algorithms for solving control synthesis problems. Several most promising symbolic regression methods are considered for application in control tasks. The formal statement of the control synthesis problem for its numerical solution is given. Examples of solving problems of synthesis of system of spatial stabilization of mobile robot by method of network operator and variation Cartesian genetic programming are given. The problem required finding one nonlinear feedback function to move the robot from thirty initial conditions to one terminal point. Mathematical records of the obtained control functions are given. Results of simulation of control systems obtained by symbolic regression methods are given.

Indoor PM2.5 concentrations and students’ behavior in primary school classrooms

It is important to understand changes in PM2.5 concentrations to evaluate and control air pollution in the classroom. In this paper, the PM2.5 concentrations in classrooms of a primary school in North China were studied, where the particulate matter, temperature and humidity were monitored continuously from February 2018 to February 2019. Student behavior has been recognized as an important influential factor of indoor air quality; however, modeling the relationship between student behavior and PM2.5 concentrations in classrooms remains an unsolved problem. In this paper, a novel intelligent data-driven symbolic regression method is applied to model the relationships between various factors and PM2.5 concentrations. The advantage of this method is that it can automatically distill knowledge from data and discover the structures and parameters of relationship models simultaneously. By considering the annual schedules and daily behavior, a symbolic regression model is incorporated with mass conservation to describe the change in PM2.5 concentrations caused by student behaviors. The experimental results show that outdoor PM2.5 concentrations can influence the concentrations of indoor PM2.5 through meteorological variations. Moreover, student behaviors play an important role and can lead to rapid changes in indoor PM2.5 over a short timeframe. This paper provides a solid theoretical analysis of the relationship between indoor PM2.5 concentrations and student behaviors in classrooms. The numerical models proposed by this research can assist in the analysis of PM2.5 concentrations and the improvement of air quality in the classroom.

Control Synthesis as Machine Learning Control by Symbolic Regression Methods

The problem of control synthesis is considered as machine learning control. The paper proposes a mathematical formulation of machine learning control, discusses approaches of supervised and unsupervised learning by symbolic regression methods. The principle of small variation of the basic solution is presented to set up the neighbourhood of the search and to increase search efficiency of symbolic regression methods. Different symbolic regression methods such as genetic programming, network operator, Cartesian and binary genetic programming are presented in details. It is shown on the computational example the possibilities of symbolic regression methods as unsupervised machine learning control technique to the solution of MLC problem of control synthesis for obtaining the stabilization system for a mobile robot.

Multi-objective symbolic regression for physics-aware dynamic modeling

Virtually all dynamic system control methods benefit from the availability of an accurate mathematical model of the system. This includes also methods like reinforcement learning, which can be vastly sped up and made safer by using a dynamic system model. However, obtaining a sufficient amount of informative data for constructing dynamic models can be difficult. Consequently, standard data-driven model learning techniques using small data sets that do not cover all important properties of the system yield models that are partly incorrect, for instance, in terms of their steady-state characteristics or local behavior. However, often some knowledge about the desired physical properties of the model is available. Recently, several symbolic regression approaches making use of such knowledge to compensate for data insufficiency were proposed. Therefore, this knowledge should be incorporated into the model learning process to compensate for data insufficiency.In this paper, we consider a multi-objective symbolic regression method that optimizes models with respect to their training error and the measure of how well they comply with the desired physical properties. We propose an extension to the existing algorithm that helps generate a diverse set of high-quality models. Further, we propose a method for selecting a single final model out of the pool of candidate output models. We experimentally demonstrate the approach on three real systems: the TurtleBot 2 mobile robot, the Parrot Bebop 2 drone and the magnetic manipulation system. The results show that the proposed model-learning algorithm yields accurate models that are physically justified. The improvement in terms of the model’s compliance with prior knowledge over the models obtained when no prior knowledge was involved in the learning process is of several orders of magnitude.

Universal Approach to Solution of Optimization Problems by Symbolic Regression

Optimization problems and their solution by symbolic regression methods are considered. The search is performed on non-Euclidean space. In such spaces it is impossible to determine a distance between two potential solutions and, therefore, algorithms using arithmetic operations of multiplication and addition are not used there. The search of optimal solution is performed on the space of codes. It is proposed that the principle of small variations of basic solution be applied as a universal approach to create search algorithms. Small variations cause a neighborhood of a potential solution, and the solution is searched for within this neighborhood. The concept of inheritance property is introduced. It is shown that for non-Euclidean search space, the application of evolution and small variations of possible solutions is effective. Examples of using the principle of small variation of basic solution for different symbolic regression methods are presented.

Logic Guided Genetic Algorithms (Student Abstract)

We present a novel Auxiliary Truth enhanced Genetic Algorithm (GA) that uses logical or mathematical constraints as a means of data augmentation as well as to compute loss (in conjunction with the traditional MSE), with the aim of increasing both data efficiency and accuracy of symbolic regression (SR) algorithms. Our method, logic-guided genetic algorithm (LGGA), takes as input a set of labelled data points and auxiliary truths (AT) (mathematical facts known a priori about the unknown function the regressor aims to learn) and outputs a specially generated and curated dataset that can be used with any SR method. We evaluate LGGA against state-of-the-art SR tools, namely, Eureqa and TuringBot and find that using these SR tools in conjunction with LGGA results in them solving up to 30% more equations, needing only a fraction of the amount of data compared to the same tool without LGGA, i.e., resulting in up to a 61.9% improvement in data efficiency.

Machine learning and symbolic regression investigation on stability of MXene materials

Materials stability is a fundamental parameter that should be considered in almost all materials researches. In this manuscript, we employ machine learning techniques and symbolic regression to investigate material stabilities, focusing on the An+1Bn-type prototypical MXenes. Based on a small dataset, the machine learning algorithms including Random forest, KNN, Logistic regression, SVM and GaussianNB are investigated to evaluate the MXene stabilities, with the SVM algorithm achieving the best accuracy for the classification purpose. More importantly, the symbolic regression is verified to be a viable method to identify proper descriptors and construct new descriptors that correlate with the MXene material stability. This study demonstrates the viability of the machine learning and symbolic regression methods to classify materials and describe materials stability.

Vibration stability analysis of cantilever structure based on symbolic regression algorithm and modal analysis method

The vibration stability of cantilever mechanism under high-speed rotation directly affects the positioning accuracy. Modal analysis method is usually used to study the vibration stability. However, the traditional experimental modal analysis (EMA) method needs to measure the impulse excitation, while the operational modal analysis (OMA) method needs to satisfy the assumption of white noise. Therefore, the existing modal analysis methods cannot be applied to the study of vibration stability of high-frequency cantilever mechanism. In this paper, the symbolic regression (SR) algorithm is combined with the EMA method, and the robustness analysis and feasibility verification are carried out under the condition of adding noise. The validation of the new method is divided into two parts. In the first part, a three degree of freedom (DOF) linear model is constructed, and the modal parameters identified by SR method and state space method are compared. In the second part, the method is applied to identify the modal parameters of stepped bar. The results are compared with the results of LMS (Siemens’ Testlab). Based on the time-domain response signal only, the modal parameters are extracted by SR, and the main vibration frequency is extracted from the response signal. The system simulation and experimental results show the method provides a possibility to analyze the vibration stability of cantilever structure.

Multi-Objective Memetic Algorithms with Tree-Based Genetic Programming and Local Search for Symbolic Regression

Symbolic regression is to search the space of mathematical expressions to find a model that best fits a given dataset. As genetic programming (GP) with the tree representation can represent solutions as expression trees, it is popularly-used for regression. However, GP tends to evolve unnecessarily large programs (known as bloat), causing excessive use of CPU time/memory and evolving solutions with poor generalization ability. Moreover, even though the importance of local search has been proved in augmenting the search ability of GP (termed as memetic algorithms), local search is underused in GP-based methods. This work aims to handle the above problems simultaneously. To control bloat, a multi-objective (MO) technique (NSGA-II, Non-dominant Sorting Genetic Algorithm) is selected to incorporate with GP, forming a multi-objective GP (MOGP). Moreover, three mutation-based local search operators are designed and incorporated with MOGP respectively to form three multi-objective memetic algorithms (MOMA), i.e. MOMA_MR (MOMA with Mutation-based Random search), MOMA_MF (MOMA with Mutation-based Function search) and MOMA_MC (MOMA with Mutation-based Constant search). The proposed methods are tested on both benchmark functions and real-world applications, and are compared with both GP-based (i.e. GP and MOGP) and nonGP-based symbolic regression methods. Compared with GP-based methods, the proposed methods can reduce the risk of bloat with the evolved solutions significantly smaller than GP solutions, and the local search strategies introduced in the proposed methods can improve their search ability with the evolved solutions dominating MOGP solutions. In addition, among the three proposed methods, MOMA_MR performs best in RMSE for testing, yet it consumes more training time than others. Moreover, compared with six reference nonGP-based symbolic regression methods, MOMA_MR generally performs better than or similar to them consistently.

Assessing the impacts of climate change on hydrological regimes and fish EQR in two Danish catchments

Study regionTwo large-scale catchments in Denmark: Ringkøbing fiord and Mid-Zealand catchments. Study focusFuture changes in hydrological regime, extremes, groundwater levels and ecological quality ratio (EQR) for fish were assessed. Based on an ensemble of 17 bias-corrected regional climate models forced by the emission scenario RCP8.5, a dry, a medium and a wet model were identified for the future period 2071–2100. Changes in water balances for catchments, impacts on groundwater level and fish ecological quality ration (fish EQR) were evaluated. The impact changes were based on an integrated, transient, coupled groundwater and surface-water flow model, set up with a horizontal resolution of 200m × 200m for the two catchments. New hydrological insights for the regionStrengths and weaknesses of EQR assessments based on symbolic regression methods compared to expert assessments based on flow calendar and max flow changes in percent exceedance thresholds (Q95) were identified. While the modelled impact of climate change on monthly flow and flow calendar follows the same pattern at all monitoring stations, the impact on groundwater level and fish EQRs are more complex, and somewhat ambiguous across realisations and catchments.

Research of Trajectory Optimization Approaches in Synthesized Optimal Control

This article presents a study devoted to the emerging method of synthesized optimal control. This is a new type of control based on changing the position of a stable equilibrium point. The object stabilization system forces the object to move towards the equilibrium point, and by changing its position over time, it is possible to bring the object to the desired terminal state with the optimal value of the quality criterion. The implementation of such control requires the construction of two control contours. The first contour ensures the stability of the control object relative to some point in the state space. Methods of symbolic regression are applied for numerical synthesis of a stabilization system. The second contour provides optimal control of the stable equilibrium point position. The present paper provides a study of various approaches to find the optimal location of equilibrium points. A new problem statement with the search of function for optimal location of the equilibrium points in the second stage of the synthesized optimal control approach is formulated. Symbolic regression methods of solving the stated problem are discussed. In the presented numerical example, a piece-wise linear function is applied to approximate the location of equilibrium points.

Using Satellite Remote Sensing to Study the Effect of Sand Excavation on the Suspended Sediment in the Hong Kong-Zhuhai-Macau Bridge Region

The Hong Kong-Zhuhai-Macau Bridge crosses the Pearl River Estuary and is the largest bridge and tunnel project in the world. During the construction period of this project, the excessive suspended sediment was found in the construction region. The suspended sediment generated by sand excavation in the upstream was assumed to have a significant impact on the suspended sediment in the tunnel region. In this study, we assessed the impact of upstream sand excavation on the suspended sediment in the Hong Kong-Zhuhai-Macau Bridge construction area using Landsat OLI, ETM+, and TM data. Regional suspended sediment algorithms were developed for Landsat using a symbolic regression method based on data from in situ measurements in the study area from 2003 to 2014. A band shift was conducted on the remote sensing reflectance data from Landsat ETM+ and OLI to produce a time series of the suspended sediment concentrations that was internally consistent with that of the Landsat TM data. The suspended sediment distribution was extracted and used to compare under two different conditions, with and without sand excavation. The correlations of the time series of the suspended sediment concentrations in different regions in the surrounding waters, including the correlations between the construction regions and the sand excavation regions, were calculated. Our results indicated that the sand excavation north of the Pearl River Estuary had a limited impact on the surface suspended sediment concentrations in the Hong Kong-Zhuhai-Macau Bridge tunnel area.

Machine Learning Control Based on Approximation of Optimal Trajectories

The paper is devoted to an emerging trend in control—a machine learning control. Despite the popularity of the idea of machine learning, there are various interpretations of this concept, and there is an urgent need for its strict mathematical formalization. An attempt to formalize the concept of machine learning is presented in this paper. The concepts of an unknown function, work area, training set are introduced, and a mathematical formulation of the machine learning problem is presented. Based on the presented formulation, the concept of machine learning control is considered. One of the problems of machine learning control is the general synthesis of control. It implies finding a control function that depends on the state of the object, which ensures the achievement of the control goal with the optimal value of the quality criterion from any initial state of some admissible region. Supervised and unsupervised approaches to solving a problem based on symbolic regression methods are considered. As a computational example, a problem of general synthesis of optimal control for a spacecraft landing on the surface of the Moon is considered as supervised machine learning control with a training set.

A new two-step approach for solving a control system synthesis problem by symbolic regression methods

The problem of synthesizing a control system using symbolic regression methods is considered. A new approach suggested in this work considers the control system synthesis problem as the set of optimal control problems and suggests a two-step algorithm to solve it. At the first step evolutionary algorithms are used to solve the optimal control problem numerically for each initial state from a given domain. As a result of the first step a set of optimal trajectories is obtained. At the second step symbolic regression methods are considered to solve the problem of approximating the found in the first step set of trajectories optimally. The result of the second step is a solution of the control system synthesis problem as a multidimensional control function of object’s state vector. Compared to the known approach to solve the control system synthesis problem using symbolic regression methods, the suggested approach provides betters results since the search of control function is carried out based on the set of optimal trajectories. Computational experiment presents the solution of the applied problem of synthesizing the optimal control system for the spacecraft landing on the Moon. It is experimentally demonstrated that the synthesized control system provides a close to optimal landing trajectory for any initial state from a given domain.

Solution of the optimal control problem by symbolic regression method

The paper introduces an approach to solve the optimal control problem applying symbolic regression methods. All known approaches to solve the optimal control problem are based on direct or indirect methods. Indirect methods may provide analytical solution but impose restrictions on the form and dimension of control object. In direct ones the optimal control problem is reduced to the nonlinear programming problem which provides general numerical solution. Also direct methods impose no restrictions on control object and have a wide range of application to complex applied problems. The approach proposed combines the advantages of both direct and indirect methods not involving the disadvantages mentioned previously. The suggested approach provides solution to the optimal control problem as a control function of time in explicit form. This approach benefits in case of applied optimal control problem requiring a solution in explicit form while indirect methods are inapplicable due to the restrictions imposed on the control system. The computational experiment of solving the optimal control problem for a mobile robot with phase constraints is presented. The results illustrate that the proposed approach finds an optimal control as a function of time which provides a mobile robot’s motion along the optimal trajectory avoiding phase constraints.

Synthesized optimal control based on machine learning

The article discusses symbolic regression methods as a machine learning technology. The technique is tested on a complex problem of control systems synthesis. A new type of control based on changing the position of a stable equilibrium point is proposed. The implementation of such control requires the construction of a double feedback loop. The inner contour ensures the stability of the control object relative to some point in the state space. The outer contour provides optimal control of the stable equilibrium point position. To implement control, symbolic regression methods are used as machine learning technologies. It is shown that such a control is the least sensitive to external disturbances and model uncertainties.

Seismic Fragility Analysis of the Reinforced Concrete Continuous Bridge Piers Based on Machine Learning and Symbolic Regression Fusion Algorithms

In the fragility analysis, researchers mostly chose and constructed seismic intensity measures (IMs) according to past experience and personal preference, resulting in large dispersion between the sample of engineering demand parameter (EDP) and the regression function with IM as the independent variable. This problem needs to be solved urgently. Firstly, the existing 46 types of ground motion intensity measures were taken as a candidate set, and the composite intensity measures (IMs) based on machine learning methods were selected and constructed. Secondly, the modified Park–Ang damage index was taken as EDP, and the symbolic regression method was used to fit the functional relationship between the composite intensity measures (CIMs) and EDP. Finally, the probabilistic seismic demand analysis (PSDA) and seismic fragility analysis were performed by the cloud‐stripe method. Taking the pier of a three‐span continuous reinforced concrete hollow slab bridge as an example, a nonlinear finite element model was established for vulnerability analysis. And the composite IM was compared with the linear composite IM constructed by Kiani, Lu Dagang, and Liu Tingting. The functions of them were compared. The analysis results indicated that the standard deviation of the composite IM fragility curve proposed in this paper is 60% to 70% smaller than the other composite indicators which verified the efficiency, practicality, proficiency, and sufficiency of the proposed machine learning and symbolic regression fusion algorithms in constructing composite IMs.

Symbolic Regression Methods for Reinforcement Learning

Reinforcement learning algorithms can solve dynamic decision-making and optimal control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: they are black-box models offering little insight into the mappings learned, and they require extensive trial and error tuning of their hyper-parameters. In this paper, we propose a new approach to constructing smooth value functions in the form of analytic expressions by using symbolic regression. We introduce three off-line methods for finding value functions based on a state-transition model: symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions yield well-performing policies and are compact, mathematically tractable, and easy to plug into other algorithms. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with an alternative approach using neural networks shows that our method outperforms the neural network-based one.

Performing multi-target regression via gene expression programming-based ensemble models

Multi-Target Regression problem comprises the prediction of multiple continuous variables given a common set of input features, unlike traditional regression tasks, where just one output target is available. There are two major challenges when addressing this problem, namely the exploration of the inter-target dependencies and the modeling of complex input–output relationships. This work proposes a Symbolic Regression method following the basis of Gene Expression Programming paradigm to solve the multi-target regression problem, and called GEPMTR. It evolves a population of individuals, where each one represents a complete solution to the problem by using a multi-genic chromosome, and encodes a mathematical function for each target variable involving the input attributes. The proposed model can estimate the inter-target dependencies by applying some genetic operators. Furthermore, three ensemble-based methods are developed to better exploit the inter-target and input–output relationships. The effectiveness of the proposals is analyzed through an extensive experimental study on 18 datasets. The codification schema and the process followed to ensure a diverse population in GEPMTR lead to obtain an effective proposal to solve the MTR problem. Furthermore, the EGEPMTR-B ensemble method obtained the best performance across all proposed models, being the best in 8 out of 11 cases, demonstrating that more sophisticated mechanisms were not needed for ensuring that GEPMTR method would properly model the existing inter-target dependencies. Finally, the experimental results also showed that the proposed approach attains competitive results compared to state-of-the-art, showing the possibilities that can bring this research line for effectively solving the MTR problem.

Operational modal analysis of a nonlinear oscillation system under a harmonic excitation

Operational modal analysis (OMA) is a procedure that allows the modal parameters of a structure to be extracted from the measured response to an unknown excitation generated during operation. Nonlinearity is inevitably and frequently encountered in OMA. The problem: The traditional OMA method based on linear modal theory cannot be applied to a nonlinear oscillation system. The solution: This paper aims to propose a nonlinear OMA method for nonlinear oscillation systems. The new OMA method is based on the following: (1) a self-excitation phenomenon is caused by nonlinear components; (2) the nonlinear normal modes (NNMs) of the system appear under a single-frequency harmonic excitation; and (3) using forced response data, the symbolic regression method (SR) can be used to automatically search for the NNMs of the system, whose modal parameters are implicit in the expression structure expressing each NNM. The simulation result of a three-degree-of-freedom (3-DOF) nonlinear system verifies the correctness of the proposed OMA method. Then, a disc-rod rotor model is considered, and the proposed OMA method’s capability is further evaluated.