Impact of domain knowledge on developing pumping models for single-screw extruders using symbolic regression

Abstract

The literature provides several analytical approximation methods for predicting the flow of non‐Newtonian fluids in single‐screw extruders. While these are based on various flow conditions, they were developed mostly for extruder screws with standard geometries. We present novel analytical melt‐conveying models for predicting the flow and dissipation rates of fully developed flows of power‐law fluids within three‐dimensional screw channels. To accommodate a broad range of industrial screw designs, including both standard and high‐performance screws, the main intention of this work was to significantly extend the scope of existing theories. The flow equations were first rewritten in a dimensionless form to reduce the mathematical problem to its dimensionless influencing parameters. These were varied within wide ranges to create a set of physically independent modeling setups, the flow and dissipation rates of which were evaluated by means of a finite‐volume solver. The numerical results were then approximated analytically using symbolic regression based on genetic programming. To support the regression analysis in finding accurate solutions, we integrated domain‐specific process knowledge in the preprocessing of the dataset. We obtained three regression models for predicting the flow and dissipation rates in melt‐conveying zones and tested their accuracy successfully against an independent set of numerical solutions.Highlights Flow of power‐law fluids in three‐dimensional screw channels Identification of independent influencing parameters by dimensional analysis Numerical parametric design study for a broad range of industrial applications Integration of domain knowledge in symbolic regression Surrogate models derived from numerical simulation results

The integration of Photovoltaic (PV) systems requires the implementation of potential PV power forecasting techniques to deal with the high intermittency of weather parameters. In the PV power prediction process, Genetic Programming (GP) based on the Symbolic Regression (SR) model has a widespread deployment since it provides an effective solution for nonlinear problems. However, during the training process, SR models might miss optimal solutions due to the large search space for the leaf generations. This paper proposes a novel hybrid model that combines SR and Deep Multi-Layer Perceptron (MLP) for one-month-ahead PV power forecasting. A case study analysis using a real Australian weather dataset was conducted, where the employed input features were the solar irradiation and the historical PV power data. The main contribution of the proposed hybrid SR-MLP algorithm are as follows: (1) The training speed was significantly improved by eliminating unimportant inputs during the feature selection process performed by the Extreme Boosting and Elastic Net techniques; (2) The hyperparameters were preserved throughout the training and testing phases; (3) The proposed hybrid model made use of a reduced number of layers and neurons while guaranteeing a high forecasting accuracy; (4) The number of iterations due to the use of SR was reduced. The presented simulation results demonstrate the higher forecasting accuracy (reductions of more than 20% for Root Mean Square Error (RMSE) and 30 % for Mean Absolute Error (MAE) in addition to an improvement in the R2 evaluation metric) and robustness (preventing the SR from converging to local minima with the help of the ANN branch) of the proposed SR-MLP model as compared to individual SR and MLP models.

Discovering a meaningful symbolic expression that explains experimental data is a fundamental challenge in many scientific fields. We present a novel, open-source computational framework called Scientist-Machine Equation Detector (SciMED), which integrates scientific discipline wisdom in a scientist-in-the-loop approach, with state-of-the-art symbolic regression (SR) methods. SciMED combines a wrapper selection method, that is based on a genetic algorithm, with automatic machine learning and two levels of SR methods. We test SciMED on five configurations of a settling sphere, with and without aerodynamic non-linear drag force, and with excessive noise in the measurements. We show that SciMED is sufficiently robust to discover the correct physically meaningful symbolic expressions from the data, and demonstrate how the integration of domain knowledge enhances its performance. Our results indicate better performance on these tasks than the state-of-the-art SR software packages , even in cases where no knowledge is integrated. Moreover, we demonstrate how SciMED can alert the user about possible missing features, unlike the majority of current SR systems.

Symbolic regression methods generate expression trees that simultaneously define the functional form of a regression model and the regression parameter values. As a result, the regression problem can search many nonlinear functional forms using only the specification of simple mathematical operators such as addition, subtraction, multiplication, and division, among others. Currently, state-of-the-art symbolic regression methods leverage genetic algorithms and adaptive programming techniques. Genetic algorithms lack optimality certifications and are typically stochastic in nature. In contrast, we propose an optimization formulation for the rigorous deterministic optimization of the symbolic regression problem. We present a mixed-integer nonlinear programming (MINLP) formulation to solve the symbolic regression problem as well as several alternative models to eliminate redundancies and symmetries. We demonstrate this symbolic regression technique using an array of experiments based upon literature instances. We then use a set of 24 MINLPs from symbolic regression to compare the performance of five local and five global MINLP solvers. Finally, we use larger instances to demonstrate that a portfolio of models provides an effective solution mechanism for problems of the size typically addressed in the symbolic regression literature.

In the pursuit of optimal quantitative structure-activity relationship (QSAR) models, two key factors are paramount: the robustness of predictive ability and the interpretability of the model. Symbolic regression (SR) searches for the mathematical expressions that explain a training data set. Thus, the models provided by SR are globally interpretable. We previously proposed an SR method that can generate interpretable expressions by humans. This study introduces an enhanced symbolic regression method, termed filter-induced genetic programming 2 (FIGP2), as an extension of our previously proposed SR method. FIGP2 is designed to improve the generalizability of SR models and to be applicable to data sets in which cost-intensive descriptors are employed. The FIGP2 method incorporates two major improvements: a modified domain filter to eradicate diverging expressions based on optimal calculation and the introduction of a stability metric to penalize expressions that would lead to overfitting. Our retrospective comparative analysis using 12 structure-activity relationship data sets revealed that FIGP2 surpassed the previously proposed SR method and conventional modeling methods, such as support vector regression and multivariate linear regression in terms of predictive performance. Generated mathematical expressions by FIGP2 were relatively simple and not divergent in the domain of function. Taken together, FIGP2 can be used for making interpretable regression models with predictive ability.

Symbolic and numerical regression: experiments and applications

A mathematical model was developed for plasticating single‐screw extrusion of amorphous polymers. We considered a standard metering screw design. By introducing a ‘critical flow temperature’ (Tcf), below which an amorphous polymer may be regarded as a ‘rubber‐like’ solid, we modified the Lee‐Han melting model, which had been developed earlier for the extrusion of crystalline polymers, to model the flow of an amorphous polymer in the screw channel. Tcf is de facto a temperature equivalent to the melting point of a crystalline polymer. The introduction of Tcf was necessary for defining the interface between the solid bed and the melt pool, and between the solid bed and thin melt films surrounding the solid bed. We found from numerical simulations that (1) when the Tcf was assumed to be close to its glass transition temperature (Tg), the viscosity of the polymer became so high that no numerical solutions of the system of equations could be obtained, and (2) when the value of Tcf was assumed to be much higher than Tg, the extrusion pressure did not develop inside the screw channel. Thus, an optimum modeling value of Tcf appears to exist, enabling us to predict pressure profiles along the extruder axis. We found that for both polystyrene and polycarbonate, Tcf lies about 55°C above their respective Tgs. In carrying out the numerical simulation we employed (1) the WLF equation to describe the temperature dependence of the shear modulus of the bulk solid bed at temperatures between Tg and Tcf, (2) the WLF equation to describe the temperature dependence of the viscosity of molten polymer at temperatures between Tcf and Tg + 100°C, (3) the Arrhenius relationship to describe the temperature dependence of the viscosity of molten polymer at temperatures above Tg + 100°C, and (4) the truncated power‐law model to describe the shear‐rate dependence of the viscosity of molten polymer. We have shown that the Tg of an amorphous polymer cannot be regarded as being equal to the Tm of a crystalline polymer, because the viscosities of an amorphous polymer at or near its Tg are too large to flow like a crystalline polymer above its Tm. Also conducted was an experimental study for polystyrene and polycarbonate, using both a standard metering screw and a barrier screw design having a length‐to‐diameter ratio of 24. For the study, nine pressure transducers were mounted on the barrel along the extruder axis, and the pressure signal patterns and axial pressure profiles were measured at various screw speeds, throughputs, and head pressures. In addition to significantly higher rates, we found that the barrier screw design gives rise to much more stable pressure signals, thus minimizing surging, than the metering screw design. The experimentally measured axial pressure profiles were compared with prediction.

Surrogate-assisted optimization algorithms are a commonly used technique to solve expensive-evaluation problems, in which a regression model is built to replace an expensive function. In some acquisition functions, the only requirement for a regression model is the predictions. However, some other acquisition functions also require a regression model to estimate the “uncertainty” of the prediction, instead of merely providing predictions. Unfortunately, very few statistical modeling techniques can achieve this, such as Kriging/Gaussian processes, and recently proposed genetic programming-based (GP-based) symbolic regression with Kriging (GP2). Another method is to use a bootstrapping technique in GP-based symbolic regression to estimate prediction and its corresponding uncertainty. This paper proposes to use GP-based symbolic regression and its variants to solve multi-objective optimization problems (MOPs), which are under the framework of a surrogate-assisted multi-objective optimization algorithm (SMOA). Kriging and random forest are also compared with GP-based symbolic regression and GP2. Experiment results demonstrate that the surrogate models using the GP2 strategy can improve SMOA’s performance.

Symbolic regression for better specification

This study presents an interpretable alternative approach to power usage prediction by comparing the performance and interpretability of Genetic Programming (GP)-based symbolic regression (SR) and various neural network models (RNN, LSTM, GRU). Although conventional neural network models deliver high prediction accuracy, their black-box nature makes it difficult to clearly interpret the relationships between variables. In contrast, GP-based SR enhances interpretability by generating explicit formulas that describe these relationships. When applied to the maximum demand power prediction problem in a manufacturing plant using the same dataset, SR not only achieved higher prediction performance compared to the neural network models but also provided interpretable formulas. The experimental results empirically confirm that SR offers significant advantages over neural networks in terms of both prediction performance and interpretability.

This paper introduces a novel approach called semantic boosting regression (SBR), leveraging the principles of boosting algorithms in symbolic regression using a Memetic Semantic GP for Symbolic Regression (MSGP) algorithm as weak learners. Memetic computation facilitates the integration of domain knowledge into a population-based approach, and semantic-based algorithms enhance local improvements to achieve targeted outputs. The fusion of memetic and semantic approaches allows us to augment the exploration and exploitation capabilities inherent in Genetic Programming (GP) and identify concise symbolic expressions that maintain interpretability without compromising the expressive power of symbolic regression. Our approach echoes the boosting algorithm’s characteristic, where weak learners (e.g., MSGP) are sequentially improved upon, focusing on correcting previous errors and continuously enhancing overall performance. This iterative strategy, intrinsic to boosting methods, is adeptly adapted to our SBR model. Experimental results demonstrate that our memetic-semantic approach has equal or better performance when compared to state-of-the-art evolutionary-based techniques when addressing real-world symbolic regression challenges. This advancement helps tackle the bloating issue in GP and significantly improves generalization capabilities. However, akin to classic boosting algorithms, one limitation of our approach is the increased computational cost due to the sequential training of boosting learners.

Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general. Unfortunately, most symbolic regression methods are only applicable when the given data is complete. One common approach to handling this situation is data imputation. It works by estimating missing values based on existing data. However, which existing data should be used for imputing the missing values? The answer to this question is important when dealing with incomplete data. To address this question, this work proposes a mixed tree-vector representation for genetic programming to perform instance selection and symbolic regression on incomplete data. In this representation, each individual has two components: an expression tree and a bit vector. While the tree component constructs symbolic regression models, the vector component selects the instances that are used to impute missing values by the weighted k-nearest neighbour (WKNN) imputation method. The complete imputed instances are then used to evaluate the GP-based symbolic regression model. The obtained experimental results show the applicability of the proposed method on real-world data sets with different missingness scenarios. When compared with existing methods, the proposed method not only produces more effective symbolic regression models but also achieves more efficient imputations.

Transonic axial compressor flows exhibit complex turbulence structures that pose significant challenges for traditional turbulence models. In recent years, neural network-based turbulence models have demonstrated promising results in simulating these intricate flows. However, these models often lack interpretability, a crucial aspect of understanding the underlying physical mechanisms. Symbolic regression, capable of training highly interpretable turbulence models, offers a potential solution to elucidate the mechanisms underpinning neural network-based turbulence models. In this study, we employ evolutionary symbolic regression to interpret tensor basis neural networks (TBNNs) and develop explicit transcendental Reynolds stress models (ETRSM) for transonic axial compressor flows. Our symbolic regression turbulence models are trained on the inputs and outputs of a pre-trained TBNN. We introduce a method that independently predicts coefficients for each tensor basis, significantly reducing computational costs and enhancing the rationality of the prediction process. We develop six symbolic regression models: three transcendental and three algebraic. Through rigorous computational fluid dynamics (CFD) simulations, the transcendental models demonstrate an exceptional ability to interpret the TBNN, while the algebraic models show limited success. The symbolic regression ETRSM, characterized by high interpretability and transferability, effectively interprets the pre-trained TBNN and achieves comparable accuracy to TBNN-based turbulence models in simulating the complex turbulence flows in transonic axial compressors. These results underscore the potential of symbolic regression turbulence models for simulating industry-level CFD problems and highlight the importance of incorporating additional features in training such models. Furthermore, the method separates the prediction of individual tensor basis coefficients, significantly reducing computational costs.

Hybrid fiber-reinforced polymer (FRP) and steel reinforced concrete (hybrid FRP-steel RC) beams have gained recognition for their exceptional flexural performance, surpassing that of beams reinforced exclusively with FRP bars (FRP-RC). However, current design guidelines, such as ACI 440.11–22, fail to accurately predict the flexural strength of these hybrid systems. This study aims to enhance the predictive accuracy and interpretability of flexural strength models by applying advanced computational approaches—specifically, machine learning (ML) techniques and symbolic regression. A robust dataset of 134 experimental data points was utilized to develop predictive models. The prediction results showed that both ML and symbolic regression models significantly outperformed the ACI 440.11–22 equations, achieving lower errors (MAE, MAPE, RMSE) and higher accuracy (R2). The results demonstrate that the ML models—Gaussian process regression (GPR), NGBoost, and CatBoost—achieved high predictive accuracy, with mean R2 values approaching 1.0 and MAPE% as low as 5.19 (training) and 11.51 (testing) for GPR. Furthermore, symbolic regression yielded a transparent mathematical expression with a mean prediction ratio (µ) of 1.003, a CoV of 0.139, and a MAPE% of 11.08. These findings highlight the practical and technical advantages of symbolic regression in developing reliable, interpretable, and efficient design equations for hybrid FRP-steel RC beams.