Symbolic Regression Research Articles

Extraction of physicochemical laws by symbolic regression using a Bayesian information criterion

Extraction of physicochemical laws by symbolic regression using a Bayesian information criterion

Symbolic Regression: An Alternative Method to Model the Optical Response of Photonic Biological and Bio-inspired Structures

Symbolic Regression: An Alternative Method to Model the Optical Response of Photonic Biological and Bio-inspired Structures

Accelerating graph-based tracking tasks with symbolic regression

Abstract The reconstruction of particle tracks from hits in tracking detectors is a computationally intensive task due to the large combinatorics of detector signals. Recent efforts have proven that ML techniques can be successfully applied to the tracking problem, extending and improving the conventional methods based on feature engineering. 
However, complex models can be challenging to implement on heterogeneous trigger systems, integrating architectures such as FPGAs.
Deploying the network on an FPGA is feasible but challenging and limited by its resources. An efficient alternative can employ symbolic regression (SR). We propose a novel approach that uses SR to replace a graph-based neural network. Substituting each network block with a symbolic function preserves the graph structure of the data and enables message passing. 
The technique is perfectly suitable for heterogeneous hardware, as it can be implemented more easily on FPGAs and grants faster execution times on CPU with respect to conventional methods.
While the tracking problem is the target for this work, it also provides a proof-of-principle for the method that can be applied to many use cases.

Evolving Transparent Credit Risk Models: A Symbolic Regression Approach Using Genetic Programming

Credit scoring is a cornerstone of financial risk management, enabling financial institutions to assess the likelihood of loan default. However, widely recognized contemporary credit risk metrics, like FICO (Fair Isaac Corporation) or Vantage scores, remain proprietary and inaccessible to the public. This study aims to devise an alternative credit scoring metric that mirrors the FICO score, using an extensive dataset from Lending Club. The challenge lies in the limited available insights into both the precise analytical formula and the comprehensive suite of credit-specific attributes integral to the FICO score’s calculation. Our proposed metric leverages basic information provided by potential borrowers, eliminating the need for extensive historical credit data. We aim to articulate this credit risk metric in a closed analytical form with variable complexity. To achieve this, we employ a symbolic regression method anchored in genetic programming (GP). Here, the Occam’s razor principle guides evolutionary bias toward simpler, more interpretable models. To ascertain our method’s efficacy, we juxtapose the approximation capabilities of GP-based symbolic regression with established machine learning regression models, such as Gaussian Support Vector Machines (GSVMs), Multilayer Perceptrons (MLPs), Regression Trees, and Radial Basis Function Networks (RBFNs). Our experiments indicate that GP-based symbolic regression offers accuracy comparable to these benchmark methodologies. Moreover, the resultant analytical model offers invaluable insights into credit risk evaluation mechanisms, enabling stakeholders to make informed credit risk assessments. This study contributes to the growing demand for transparent machine learning models by demonstrating the value of interpretable, data-driven credit scoring models.

Effective stress intensity factor range for fatigue cracks propagating in mixed mode I-II loading

The actual service axles are often subjected to mixed-mode loading, and predicting the mixed-mode I-II crack propagation behaviour using the mode I effective stress intensity factor (ΔKI) differs from the real service conditions. To effectively predict the fatigue crack propagation behaviour of actual service structures, the I-II stress intensity factor range (ΔKP-R) considering two closure effects was adopted to describe the fatigue crack propagation under mixed mode loading. A test database was established based on monitoring data of mode I and mixed-mode I-II (30/45/60°) crack propagation tests under different stress ratios. Combining domain knowledge and symbolic regression (SR) methods, an angle factor was proposed for constructing correlation functions between ΔKI and ΔKP-R. The results showed that the loading angle (α) only affects the initial projection of the load parallel and perpendicular to the fatigue crack growth (FCG) direction. Compared with the geometric correction factor, the correlation function acquired by the angle factor constructed by the SR method has higher accuracy, and the balance parameters (SCORE) obtained by the former are significantly higher than those obtained by the latter under the same function complexity. The SR verification results demonstrated that constructing mode I and I-II correlation functions with angle factors has a good predictive effect.

Analytical formulae for design of one-dimensional sonic crystals with smooth geometry based on symbolic regression

Even though locally periodic structures have been studied for more than three decades, the known analytical expressions relating the waveguide geometry and the acoustic transmission are limited to a few special cases. Having an access to numerical model is a great opportunity for data-driven discovery. Our choice of cubic splines to parametrize the waveguide unit cell geometry offers enough variability for waveguide design. Using Webster equation for unit cell and Floquet-Bloch theory for periodic structures, a dataset of numerical solutions was prepared. Employing the methods of physics-informed machine learning, we have extracted analytical formulae relating the waveguide geometry and the corresponding dispersion relation or directly the bandgap widths. The results contribute to the overall readability of the system and enable a deeper understanding of the underlying principles. Specifically, it allows for assessing the influence of the waveguide geometry, offering more efficient alternative to computationally demanding numerical optimization.

A generalized formula for predicting soil compression index using multi-evolutionary algorithm

Correlation between soil compression index (Cc) and state parameters is frequently referenced in studies investigating the fundamental mechanisms underlying changes in soil compressibility. However, developing an efficient formula for Cc that adequately captures the complexity of soil compressive behavior has been challenging for conventional approaches. This study utilized contemporary symbolic regression (SR) to propose a generalized formula for Cc that can represent the nonlinear relationships with state parameters across various soil types. A geological database from southern Vietnam was utilized to establish this data-driven formula. Data exploration revealed the apparent combined effects of moisture content (w), initial void ratio (e0), and moist density (ρ) on soil compressive behavior. Statistical indicators and graphical analysis were adopted to comprehensively assess the performance of the proposed formula against empirical equations found in the literature, aiming to gain a deeper understanding of the mechanism influencing changes in soil compressibility. The evaluation results demonstrated the efficiency of the proposed formula in predicting Cc, as evidenced by low error metrics and a good balance between precision and accuracy. Moreover, the applicability and limitations of the proposed formula were examined using different regional soils with specified geologic origins. Given its reliability and adequacy, the proposed formula explicitly quantified the nonlinear combined effects of e0, ρ and w on the compressibility of undisturbed soils. However, further research accounting for clay minerals, specimen preparation, and geologic origins is needed to enhance the universal applicability of our understanding of soil compressive behavior.

Circular rubber aggregate CFST stub columns under axial compression: prediction and reliability analysis

Extensive studies support using steel tubes to enhance the structural integrity of rubber aggregate concrete (RBAC), namely RBAC-filled steel tubes (RCFST). However, current design codes for assessing the axial compressive behaviour of circular stub RCFST (CS-RCFST) columns are limited. Furthermore, there is a scarcity of studies focused on ensuring the structural safety of these columns. Based on an extensive experimental database comprising 145 columns, this study explores machine learning (ML) capabilities for predicting the axial strength of CS-RCFST columns, using six typical machine-learning models, i.e., symbolic regression (SR), XGBoost, CatBoost, random forest, LightGBM, and Gaussian process regression models. The hyperparameter tuning of the introduced ML models is performed using the Bayesian Optimization technique. The comparison results show that the CatBoost model is the most reliable and accurate ML model (R2 = 0.999 and 0.993 for the training and testing sets, respectively). In addition, a simple and practical design expression for CS-RCFST columns has been developed with acceptable accuracy based on the SR model (an average test-to-prediction ratio of 0.99 and CoV of 0.132). Meanwhile, the axial strength predicted by ML models was compared with two prominent practice codes (i.e., AISC360 and EC4). The comparison results indicated that the ML models could introduce a highly reliable and accurate approach over current design standards for strength prediction. Furthermore, a reliability analysis is conducted on two different ML models to evaluate the reliability of utilising ML models in practical design applications. This assessment involves identifying the statistical properties associated with the compressive strength of RBAC, as well as introducing the required resistance design factors aligned with the target reliability recommended by code standards.

Applying Machine Learning to Elucidate Ultrafast Demagnetization Dynamics in Ni and Ni80Fe20

Understanding the correlation between fast and ultrafast demagnetization (UFD) processes is crucial for elucidating the microscopic mechanisms underlying UFD, which is pivotal for various applications in spintronics. Initial theoretical models attempt to establish this correlation but face challenges due to the complex interplay of physical phenomena. To address this, a variety of machine learning (ML) methods are employed, including supervised learning regression algorithms and symbolic regression (SR), to analyze limited experimental data and derive meaningful mathematical expressions between demagnetization time (τM) and the Gilbert damping factor (α). The results reveal that polynomial regression and K‐nearest neighbors algorithms perform best in predicting τM. Additionally, variable‐selection sure‐independence‐screening‐and‐sparsifying‐operator (VS‐SISSO) as a SR method suggests a direct correlation between τM and α for Ni and Ni80Fe20, indicating spin‐flip scattering predominantly influences the UFD mechanism. The developed models demonstrate promising predictive capabilities, validated against independent experimental data. Comparative analysis between different materials underscores the significant impact of material properties on UFD behavior. This study underscores the potential of ML in unraveling complex physical phenomena and offers valuable insights for future research in ultrafast magnetism.

STM-based symbolic regression for strength prediction of RC deep beams and corbels

This study uses symbolic regression with a strut-and-tie model to predict the shear strength of reinforced concrete deep beams (RCDBs) and corbels (RCCs). Previous studies have proposed two distinct types of models for estimating shear capacity: explainable models based on theoretical derivations and black-box models derived from machine learning (ML) methods. This study proposes a hybrid model derived from the strut-and-tie model (STM), where the performance of STM is enhanced through the ML approach using genetic programming. This model is based on a comprehensive experimental database of 810 tests for the shear strength of RC deep beams and 371 tests for RC corbels from various research papers. The developed STM-based symbolic regression (SR-STM) integrates two distinct force-transferring mechanisms: the diagonal strut mechanism utilizing concrete strength and the truss mechanism utilizing orthogonal web reinforcement. The SR-STM model is both robust and interpretable, demonstrating high prediction accuracy with mean values of the prediction-to-actual ratios of 0.999 and 1.004 and coefficient of determination values of 0.913 and 0.862 for RCDBs and RCCs, respectively, while providing explainable mathematical expressions that align with the mechanical principles of STM. The developed SR-STM model is benchmarked against several state-of-the-art models and evaluated against the CatBoost ML technique, demonstrating acceptable performance. The results highlight the SR-STM model’s effectiveness in providing reliable predictions and valuable insights for practical engineering applications. Furthermore, a SHAP (Shapley Additive Explanations) analysis was performed, and its results align with the SR-STM model, confirming the model’s effectiveness in accurately capturing the key factors influencing the shear strength of RCDBs and RCCs.

Modeling of the metal–insulator transition temperature in alio-valently doped VO2 through symbolic regression

The correlated semiconductor vanadium dioxide (VO2) exhibits an insulator–metal transition (IMT) near room temperature, which is of interest in various device applications. Precise IMT temperature control is crucial to determine the use cases across technologies such as thermochromic windows, actuators for robots or neuronal oscillators. Doping the cation or anion sites can modulate the IMT by several tens of degrees and control hysteresis. However, modeling the effects of control parameters (e.g., doping concentration, type of dopants) is challenging due to complex experimental procedures and limited data, hindering the use of traditional data-driven machine learning approaches. Symbolic regression (SR) can bridge this gap by identifying nonlinear expressions connecting key input parameters to target properties, even with small data sets. In this work, we develop SR models to capture the IMT trends in VO2 influenced by different dopant parameters. Using experimental data from the literature, our study reveals a dual nature of the IMT temperature with varying tungsten (W) doping concentrations. The symbolic model captures data trends and accounts for experimental variability, providing a complementary approach to first-principles calculations. Our feature-driven analysis across a broader class of dopants informs selectivity and provides qualitative insights into tuning phase transition properties valuable for neuromorphic computing and thermochromic windows.

A New Approximation for the Perimeter of an Ellipse

We consider the problem of approximating the perimeter of an ellipse, for which there is no known finite formula, in the context of high-precision performance. Ellipses are broadly used in many fields, like astronomy, manufacturing, medical imaging, and geophysics. They are applied on large and nanoscales, and while numerical integration can be used to obtain precision measurements, having a finite formula can be used for modeling. We propose an iterative symbolic regression approach, utilizing the pioneering work of Ramanujan’s second approximation introduced in 1914 and a known Padé approximation, leading to good results for both low and high eccentricities. Our proposed model is also compared with a very comprehensive historical collection of different approximations collated by Stanislav Sýkora. Compared with the best-known approximations in this centuries-old mathematical problem, our proposed model performs at both extremities while remaining consistent in mid-range eccentricities, whereas existing models excel only at one extremity.

Using interpretable machine learning to predict bloodstream infection and antimicrobial resistance in patients admitted to ICU: Early alert predictors based on EHR data to guide antimicrobial stewardship.

Nosocomial infections and Antimicrobial Resistance (AMR) stand as formidable healthcare challenges on a global scale. To address these issues, various infection control protocols and personalized treatment strategies, guided by laboratory tests, aim to detect bloodstream infections (BSI) and assess the potential for AMR. In this study, we introduce a machine learning (ML) approach based on Multi-Objective Symbolic Regression (MOSR), an evolutionary approach to create ML models in the form of readable mathematical equations in a multi-objective way to overcome the limitation of standard single-objective approaches. This method leverages readily available clinical data collected upon admission to intensive care units, with the goal of predicting the presence of BSI and AMR. We further assess its performance by comparing it to established ML algorithms using both naturally imbalanced real-world data and data that has been balanced through oversampling techniques. Our findings reveal that traditional ML models exhibit subpar performance across all training scenarios. In contrast, MOSR, specifically configured to minimize false negatives by optimizing also for the F1-Score, outperforms other ML algorithms and consistently delivers reliable results, irrespective of the training set balance with F1-Score.22 and.28 higher than any other alternative. This research signifies a promising path forward in enhancing Antimicrobial Stewardship (AMS) strategies. Notably, the MOSR approach can be readily implemented on a large scale, offering a new ML tool to find solutions to these critical healthcare issues affected by limited data availability.

Enhancing predictions of experimental band gap using machine learning and knowledge transfer

Density functional theory (DFT) calculations using the generalized gradient approximation (GGA) functional are known to underestimate the bandgap (Eg) compared with experimental values. Although hybrid functionals can address this issue, they are limited by their high computational costs. This paper presents a novel and efficient method to predict experimental Eg more accurately using machine learning (ML) models enhanced with knowledge transfer learning. We developed surrogate models based on the Eg calculated using the GGA functional and easily obtainable features from chemical formulas. These models exhibited superior prediction accuracy, with a coefficient of determination (R²) of 0.817 and a mean absolute error (MAE) of 0.289 eV. We further performed feature importance analysis and symbolic regression to provide a detailed interpretation of the ML models. It was verified that various features, such as the potential to form ionic compounds and the number of elements in a material, have a major influence on experimental Eg predictions. Based on these results, an interpretable formula for experimental Eg was derived, which can be used in future studies. Consequently, we developed a platform for predicting the experimental Eg values of inorganic materials, effectively narrowing the gap between theoretical calculations and experiments, which can be used to screen novel materials with promising properties.

A multi-objective reinforcement learning framework for real-time drilling optimization based on symbolic regression and perception

In the context of the global energy transition, optimizing deep-water oil and gas drilling parameters is crucial for ensuring safety while improving efficiency. Traditional methods face limitations in highly dynamic and nonlinear drilling environments, struggling to balance speed and cost-effectiveness. Furthermore, these methods rely on real-time logging-while-drilling (LWD) data for decision-making, but delays in data collection and processing hinder timely adjustments of drilling parameters, affecting decision accuracy and responsiveness. This paper proposes a multi-objective drilling parameter optimization framework, incorporating symbolic regression, time-series networks, and Markov decision processes to precisely predict ROP, formation conditions, and optimize drilling parameters in real time. Key innovations include a multi-population evolutionary symbolic regression algorithm for constructing empirical equations, the integration of variational mode decomposition (VMD) and sample entropy for data preprocessing, and multi-head self-attention time-series networks to enhance prediction accuracy. Quantile regression further estimates the range of drilling parameter adjustments. Additionally, a drilling parameter optimization deep deterministic policy gradient (DPODDPG) algorithm was developed to automate real-time parameter adjustments. Empirical analysis on the Ledong 10-1 block in the South China Sea demonstrated significant improvements: ROP increased from 54.18 m/hr to 122.17 m/hr, mechanical specific energy (MSE) decreased from 100.82 MPa to 97.78 MPa, and cost per foot reduced from 121.16 × 102 CNY/m to 51.31 × 102 CNY/m. Compared to traditional methods, the proposed framework showed clear advantages in enhancing ROP, reducing MSE, and controlling costs, further validating its superiority in complex drilling environments. This method not only significantly improves drilling efficiency and economic benefits but also adapts to complex and changing drilling conditions, showing broad application potential, particularly in challenging deep-water oil and gas drilling operations, where it can provide more efficient and reliable optimization solutions.

Advanced Model with a Trajectory Tracking Stabilisation System and Feasible Solution of the Optimal Control Problem

In this study, we consider the extended optimal control problem and search for a control function in the class of feasible functions for a real control object. Unlike the classical optimal control problem, the control function should depend on the state, not time. Therefore, the control synthesis problem for the initial-state domain should be solved, instead of the optimal control problem with one initial state. Alternatively, an optimal trajectory motion stabilisation system may be constructed. Both approaches—control and trajectory motion stabilisation system syntheses—cannot be applied to real-time control, as the task is too complex. The minimum threshold of quality criteria is searched for in the space of mathematical expression codes. Among other problems, the search space is difficult to define and the gradient is hard to determine. Therefore, the advanced control object model is used to obtain a feasible control function. The advanced model is firstly obtained before solving the optimal control problem and it already includes a trajectory motion stabilisation system; in particular, this stabilisation system is synthesised in advance at the control system design stage. When the optimal control problem appears, it is solved in real time in the classical statement, and a control function is searched for as a function of time. The advanced control object model also uses the reference model to generate the optimal trajectory. The search for the optimal control function is performed in real time and considers the synthesised stabilisation system of motion along a determined trajectory. Machine learning control via symbolic regression, namely, the network operator method, is used to directly solve the control synthesis problem. An example solution of the optimal control problem, with an advanced model moving in the environment with obstacles for a group of two mobile robots, is presented. The obtained solution is a control function for a reference model that generates a trajectory from a class of trajectories stabilised with the object’s control system.

Machine learning for locally periodic structure transmission modelling

This research aims to obtain an analytical transmission model for locally periodic structures with the approach of data-driven physics. So far, this task has been done mainly numerically for non-trivial geometries. From a dataset of Bloch phases generated for given frequency band and structure geometry parameters, it is possible to learn equations relating dispersion relation to axis-symmetric geometry. The dataset is transformed into a lower-dimensional space by applying Principal Component Analysis (PCA) to reduce the complexity of the problem. In the newly obtained coordinate system, the lower-dimensional patterns are extracted via symbolic regression. The resulting model is interpretable in terms of underlying physics and can be used, e.g., to propose an optimized design for a desired band gap width. Note that this has been so far possible only with numerical optimization repeatedly going back and forth from geometry to the dispersion relation, and hence, it significantly contributes to the overall readability of the system features.

SR-Forest: A Genetic Programming-Based Heterogeneous Ensemble Learning Method

Ensemble learning methods have been widely used in machine learning in recent years due to their high predictive performance. With the development of genetic programming-based symbolic regression methods, many papers begin to choose a popular ensemble learning method, random forests, as the baseline competitor. Instead of considering them as competitors, an alternative idea might be to consider symbolic regression as an enhancement technique for random forest. Genetic programming-based symbolic regression methods which fit a smooth function are complementary to the piecewise nature of decision trees, as the smooth variation is common in regression problems. In this article, we propose to form an ensemble model with symbolic regression-based decision trees to address this issue. Furthermore, we design a guided mutation operator to speed up the search on high-dimensional problems, a multi-fidelity evaluation strategy to reduce the computational cost and an ensemble selection mechanism to improve predictive performance. Finally, experimental results on a regression benchmark with 120 datasets show that the proposed ensemble model outperforms 25 existing symbolic regression and ensemble learning methods. Moreover, the proposed method can provide notable insights on an XGBoost hyperparameter performance prediction task, which is an important application area of ensemble learning methods.

Discovering an interpretable mathematical expression for a full wind-turbine wake with artificial intelligence enhanced symbolic regression

The rapid expansion of wind power worldwide underscores the critical significance of engineering-focused analytical wake models in both the design and operation of wind farms. These theoretically derived analytical wake models have limited predictive capabilities, particularly in the near-wake region close to the turbine rotor, due to assumptions that do not hold. Knowledge discovery methods can bridge these gaps by extracting insights, adjusting for theoretical assumptions, and developing accurate models for physical processes. In this study, we introduce a genetic symbolic regression (SR) algorithm to discover an interpretable mathematical expression for the mean velocity deficit throughout the wake, a previously unavailable insight. By incorporating a double Gaussian distribution into the SR algorithm as domain knowledge and designing a hierarchical equation structure, the search space is reduced, thus efficiently finding a concise, physically informed, and robust wake model. The proposed mathematical expression (equation) can predict the wake velocity deficit at any location in the full-wake region with high precision and stability. The model's effectiveness and practicality are validated through experimental data and high-fidelity numerical simulations.

Probabilistic framework for strain-based fatigue life prediction and uncertainty quantification using interpretable machine learning

Establishing a unified fatigue life prediction model and quantifying the uncertainty in the mechanical behavior of materials are critical to ensure the structural integrity and equipment performance. For the commonly-used strain-based fatigue methods, existing estimation methods exhibit inevitable deviations, while data-driven methods have shown poor extrapolation ability and interpretability. Therefore, this paper aims to develop a probabilistic framework for strain-based fatigue life prediction and uncertainty quantification (UQ) to provide an indication for fatigue design/assessment using interpretable machine learning (ML) techniques. Based on Shapley additive explanations (SHAP) and symbolic regression (SR), interpretable prediction models with concise expressions and outstanding prediction performance are established and optimized according to the priori physical knowledge. Moreover, accounting for the material variability, the probabilistic assessment with UQ excellently validates the prediction model, and quantifies the variability of ε-N curves. The proposed framework provides valuable reference and shows promising prospects in fatigue design for engineering components.