Genetic Programming Methodology Research Articles

Optimal Dimensions of Post-Tensioned Concrete Cylindrical Walls Using Harmony Search and Ensemble Learning with SHAP

The optimal design of prestressed concrete cylindrical walls is greatly beneficial for economic and environmental impact. However, the lack of the available big enough datasets for the training of robust machine learning models is one of the factors that prevents wide adoption of machine learning techniques in structural design. The current study demonstrates the application of the well-established harmony search methodology to create a large database of optimal design configurations. The unit costs of concrete and steel used in the construction, the specific weight of the stored fluid, and the height of the cylindrical wall are the input variables whereas the optimum thicknesses of the wall with and without post-tensioning are the output variables. Based on this database, some of the most efficient ensemble learning techniques like the Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), Categorical Gradient Boosting (CatBoost) and Random Forest algorithms have been trained. An R2 score greater than 0.98 could be achieved by all of the ensemble learning models. Furthermore, the impacts of different input features on the predictions of different machine learning models have been analyzed using the SHapley Additive exPlanations (SHAP) methodology. The height of the cylindrical wall was found to have the greatest impact on the optimal wall thickness, followed by the specific weight of the stored fluid. Also, with the help of individual conditional expectation (ICE) plots the variations of predictive model outputs with respect to each input feature have been visualized. By using the genetic programming methodology, predictive equations have been obtained for the optimal wall thickness.

On the Thermal Conductivity Assessment of Oil-Based Hybrid Nanofluids using Extended Kalman Filter integrated with feed-forward neural network

Regarding their ability to enhance conventional thermal oils' thermophysical properties, oil-based hybrid nanofluids have recently been widely investigated by researchers, especially on lubrication and cooling application in the automotive industry. Thermal conductivity is one of the most crucial thermophysical properties of oil-based hybrid nanofluids, which has been studied in a minimal case of studies on the specific types of them. In this research, for the first time, a comprehensive data-intelligence analysis performed on 400 gathered data points of various types of oil-based hybrid nanofluids using a novel hybrid machine learning approach; the Extended Kalman Filter-Neural network (EKF-ANN). The genetic programming (GP) and response surface methodology (RSM) approaches were examined to appraise the main paradigm. In this research, the best subset regression analysis, as a novel feature selection scheme, was provided for finding the best input parameter among all existing predictive variables (the volume fraction, temperature, thermal conductivity of the base fluid, mean diameter, and bulk density of nanoparticles). The provided models were examined using several statistical metrics, graphical tools and trends, and sensitivity analysis. The results assessment indicated that the EKF-ANN in terms of (R = 0.9738, RMSE = 0.0071 W/m.K, and KGE = 0.9630) validation phase outperformed the RSM (R = 0.9671, RMSE = 0.0079 W/m.K, and KGE = 0.9593) and GP (R = 0.9465, RMSE = 0.010 W/m.K, and KGE = 0.9273), for accurate estimation of the thermal conductivity of oil-based hybrid nanofluids.

A new empirical model for estimation of crude oil/brine interfacial tension using genetic programming approach

Detailed understanding of the behavior of crude oils and their interactions with reservoir formations and other in-situ fluids can help the engineers to make better decisions about the future of oil reservoirs. As an important property, interfacial tension (IFT) between crude oil and brine has great impacts on the oil production efficiency in different recovery stages due to its effects on the capillary number and residual oil saturation. In the present work, a new mathematical model has been developed to estimate IFT between crude oil and brine on the basis of a number of physical properties of crude oil (i.e., specific gravity, and total acid number) and the brine (i.e., pH, NaCl equivalent salinity), temperature, and pressure. Genetic programming (GP) methodology has been implemented on a data set including 560 experimental data to develop the IFT correlation. The correlation coefficient (R2 = 0.9745), root mean square deviation (RMSD = 1.8606 mN/m), and average absolute relative deviation (AARD = 3.3932%) confirm the acceptable accuracy of the developed correlation for the prediction of IFT.

A genetic programming approach for delta hedging

In this paper, using high-frequency intra-daily data from the UK market, we employ genetic programming (GP) to uncover a hedging strategy for FTSE 100 call options, hedged using FTSE 100 futures contracts. The output from the evolved strategies is a rebalancing signal which is conditioned upon a range of dynamic non-linear factors related to market conditions including liquidity and volatility. When this signal exceeds threshold values during the trading day, the hedge position is rebalanced. The performance of the GP-evolved strategy is evaluated against a number of commonly used, time-based, deterministic hedging strategies where the hedge position is rebalanced at fixed time intervals ranging from 5 min to 1 day. Assuming the delta hedger pays the bid-ask spread on the futures contract whenever the portfolio is rebalanced, this study finds that the GP-evolved hedging strategy out-performs standard, deterministic, time-based approaches. Empirical analysis shows that the superior performance of the GP strategy is driven by its ability to account for non-linear intra-day persistence in high frequency measures of liquidity and volatility. This study is the first to apply a GP methodology for the task of delta hedging with high frequency data, a significant risk management issue for investors and market makers in financial options.

Dynamic production system identification for smart manufacturing systems

This paper presents a methodology, called production system identification, to produce a model of a manufacturing system from logs of the system's operation. The model produced is intended to aid in making production scheduling decisions. Production system identification is similar to machine-learning methods of process mining in that they both use logs of operations. However, process mining falls short of addressing important requirements; process mining does not (1) account for infrequent exceptional events that may provide insight into system capabilities and reliability, (2) offer means to validate the model relative to an understanding of causes, and (3) updated the model as the situation on the production floor changes. The paper describes a genetic programming (GP) methodology that uses Petri nets, probabilistic neural nets, and a causal model of production system dynamics to address these shortcomings. A coloured Petri net formalism appropriate to GP is developed and used to interpret the log. Interpreted logs provide a relation between Petri net states and exceptional system states that can be learned by means of novel formulation of probabilistic neural nets (PNNs). A generalized stochastic Petri net and the PNNs are used to validate the GP-generated solutions. The methodology is evaluated with an example based on an automotive assembly system.

New predictive method for estimation of natural gas hydrate formation temperature using genetic programming

Diagnosis of detailed conditions of hydrate formation, as an important issue of gas fuels, can help related industries a lot, particularly in storing, transportation and processing equipment. Hydrate formation temperature or pressure can be predicted by application of mathematical models, due to thermodynamic behavior of hydrate phenomenon. A number of thermodynamical approaches along with some mathematical techniques (analytical and numerical methods) have been used to estimate hydrate formation temperature. However, there are also a variety of other techniques which have not been investigated. Application of genetic programming in developing predictive models seems novel. In the present study, three new data-based models were produced for estimation of hydrate formation temperature of natural gas, as functions of equilibrium pressure and gas molecular weight by implementation of genetic programming methodology. A total of 891 experimental data covering large range of temperatures (10.31–89.33 °F), pressures (8.1511–10,004.7 psi) and molecular weights (16.04–58.12 g/mol) were collected from the literature and used in correlation developing. The correlation coefficient (R2 = 0.9673), root-mean-square deviation (RMSD = 2.2083 °F) and average absolute relative deviation percent (AARD = 3.0830%) show that the genetic-based new models have acceptable accuracy and efficiency.

Mining Context-Aware Association Rules Using Grammar-Based Genetic Programming.

Real-world data usually comprise features whose interpretation depends on some contextual information. Such contextual-sensitive features and patterns are of high interest to be discovered and analyzed in order to obtain the right meaning. This paper formulates the problem of mining context-aware association rules, which refers to the search for associations between itemsets such that the strength of their implication depends on a contextual feature. For the discovery of this type of associations, a model that restricts the search space and includes syntax constraints by means of a grammar-based genetic programming methodology is proposed. Grammars can be considered as a useful way of introducing subjective knowledge to the pattern mining process as they are highly related to the background knowledge of the user. The performance and usefulness of the proposed approach is examined by considering synthetically generated datasets. A posteriori analysis on different domains is also carried out to demonstrate the utility of this kind of associations. For example, in educational domains, it is essential to identify and understand contextual and context-sensitive factors that affect overall and individual student behavior and performance. The results of the experiments suggest that the approach is feasible and it automatically identifies interesting context-aware associations from real-world datasets.

Network Synthesis: Using Genetic Algorithms for Network Synthesis

Objectives: The article deals with problem of creating the new systems of the Ethernet network optimal structure, considering a possibility to adjust such system in compliance with system administrator specification. Method: The main tasks of the research have been solved using as follows: graphs theory, queuing theory; large control systems theory; simulation modeling theory; genetic algorithms theory, structural and object-oriented programming methodology. Findings: In the present article an important scientific and technical problem associated with synthesis of the Ethernet network optimal structure is being solved. The following new research findings have been obtained: the modification of a genetic algorithm for topology synthesis of the Ethernet local network has been offered, in which compared to the analogues known, a simulation system is used as a utility function which enables to consider the user traffic effect on the Ethernet network characteristics at the evaluation of this structure variant. The methods of a parametric and structural optimization of the Ethernet local network have been developed. The unique methods of optimization of a structured cabling system of the Ethernet corporate network have been developed. Improvements: The results obtained are rather universal which enables to use them in different fields of the Ethernet network application: For quality improvement, reduction of value and time required for preparation of design solutions based on the Ethernet networks; in MetroEthernet; in Industrial Ethernet; in the networks different from the Ethernet standards for example, in virtual private networks or in the networks based on multi-protocol label switching MPLS.

An Analysis of the Performance of Genetic Programming for Realised Volatility Forecasting

Abstract Traditionally, the volatility of daily returns in financial markets is modeled autoregressively using a time-series of lagged information. These autoregressive models exploit stylised empirical properties of volatility such as strong persistence, mean reversion and asymmetric dependence on lagged returns. While these methods can produce good forecasts, the approach is in essence atheoretical as it provides no insight into the nature of the causal factors and how they affect volatility. Many plausible explanatory variables relating market conditions and volatility have been identified in various studies but despite the volume of research, we lack a clear theoretical framework that links these factors together. This setting of a theory-weak environment suggests a useful role for powerful model induction methodologies such as Genetic Programming (GP). This study forecasts one-day ahead realised volatility (RV) using a GP methodology that incorporates information on market conditions including trading volume, number of transactions, bid-ask spread, average trading duration (waiting time between trades) and implied volatility. The forecasting performance from the evolved GP models is found to be significantly better than those numbers of benchmark forecasting models drawn from the finance literature, namely, the heterogeneous autoregressive (HAR) model, the generalized autoregressive conditional heteroscedasticity (GARCH) model, and a stepwise linear regression model (SR). Given the practical importance of improved forecasting performance for realised volatility this result is of significance for practitioners in financial markets.

Short-term and long-term streamflow prediction by using 'wavelet–gene expression' programming approach

Forecasting short-term and long-term streamflows is of most importance for analyzing water resources systems as well as for many other hydrological applications. In this study, a new combination method (wavelet–genetic programming) was proposed for predicting short-term and long-term streamflows. The new combination method combines the discrete wavelet transform and genetic programming methods to forecast streamflow. The results obtained showed very high agreement between the observed and modeled values. The combination models are compared with the auto regressive moving average and the classic artificial intelligences methodologies, including neuro-fuzzy system and neural networks. The benchmark results show that the new wavelet–genetic programming methodology surpasses all of the other applied models for predicting short-term and long-term streamflows.

THE NEXT-GENERATION CONSTITUTIVE CORRELATIONS FOR SIMULATION OF CYCLIC STRESS-STRAIN BEHAVIOUR OF SAND

This paper presents an innovate approach to simulate the stress-strain behaviour of sands subjected to large amplitude regular cyclic loading. New prediction correlations were derived for damping ratio (D) and shear modulus (G) of sand utilizing linear genetic programming (LGP) methodology. The correlations were developed using several cyclic torsional simple shear test results. In order to formulate D and G, new equations were developed to simulate hysteresis strain–stress curves and maximum shear stress (τmax) at different loading cycles. A genetic algorithm analysis was per­formed to optimize the parameters of the proposed formulation for stress-strain relationship. A total of 746 records were extracted from the simple shear test results to develop the τmax predictive model. Sensitivity and parametric analyses were conducted to verify the results. To investigate the applicability of the models, they were employed to simulate the stress-strain curves of portions of test results that were not included in the analysis. The LGP method precisely charac­terizes the complex hysteresis behaviour of sandy soils resulting in a very good prediction performance. The proposed design equations may be used by designers as efficient tools to determine D and G, specifically when laboratory testing is not possible.

Estimation of dynamic viscosity of natural gas based on genetic programming methodology

Investigating the behavior of natural gas can contribute to a detailed understanding of hydrocarbon reservoirs. Natural gas, alone or in association with oil in reservoirs, has a large impact on reservoir fluid properties. Thus, having knowledge about gas characteristics seems to be necessary for use in estimation and prediction purposes. In this project, dynamic viscosity of natural gas (μg), as an important quantity, was correlated with pseudo-reduced temperature (Tpr), pseudo-reduced pressure (Ppr), apparent molecular weight (Ma) and gas density (ρg) by operation of the genetic programming method on a large dataset including 1938 samples. The squared correlation coefficient (R2), average absolute relative deviation percent (AARD%) and average absolute error (AAE) are 0.999, 2.55% and 0.00084 cp, respectively. The final results show that the obtained simple-to-use model can predict viscosity of natural gases with high accuracy and confidence.

Linear genetic programming for shear strength prediction of reinforced concrete beams without stirrups

A new design equation is proposed for the prediction of shear strength of reinforced concrete (RC) beams without stirrups using an innovative linear genetic programming methodology. The shear strength was formulated in terms of several effective parameters such as shear span to depth ratio, concrete cylinder strength at date of testing, amount of longitudinal reinforcement, lever arm, and maximum specified size of coarse aggregate. A comprehensive database containing 1938 experimental test results for the RC beams was gathered from the literature to develop the model. The performance and validity of the model were further tested using several criteria. An efficient strategy was considered to guarantee the generalization of the proposed design equation. For more verification, sensitivity and parametric analysis were conducted. The results indicate that the derived model is an effective tool for the estimation of the shear capacity of members without stirrups (R=0.921). The prediction performance of the proposed model was found to be better than that of several existing buildings codes.

A NEW DESIGN EQUATION FOR PREDICTION OF ULTIMATE BEARING CAPACITY OF SHALLOW FOUNDATION ON GRANULAR SOILS

A major concern in design of structures is to provide precise estimations of ultimate bearing capacity of soil beneath their foundations. Direct determination of the bearing capacity of foundations requires performing expensive and time consuming laboratory tests. To cope with this issue, several numerical models have been presented by researchers. This paper presents the development of a new design equation for the prediction of the ultimate bearing capacity of shallow foundations on granular soils using linear genetic programming (LGP) methodology. The ultimate bearing capacity is formulated in terms of width of footing, footing geometry, depth of footing, unit weight of sand, and angle of shearing resistance. The LGP-based design equation is established using the results of several load tests on real sized foundations presented in the literature. Validity of the model is verified using a part of laboratory data that are not involved in the calibration process. The statistical measures of coefficient of determination, root mean squared error and mean absolute error are used to evaluate the performance of the model. Sensitivity and parametric analyses are conducted and discussed. The proposed model accurately characterizes the ultimate bearing capacity resulting in a very good prediction performance. The LGP model reaches a better prediction performance than the well-known prediction equations for the bearing capacity of shallow foundations.

Nonlinear modeling of soil deformation modulus through LGP-based interpretation of pressuremeter test results

Soil deformation modulus is an essential parameter for the analysis of behavior of substructures. Despite its importance, little attention is paid to developing empirical models for predicting the deformation moduli obtained from the field tests. To cope with this issue, this paper presents the development of a new prediction model for the pressuremeter soil deformation modulus utilizing a linear genetic programming (LGP) methodology. The LGP model relates the soil secant modulus obtained from the pressuremeter tests to the soil index properties. The best model was selected after developing and controlling several models with different combinations of the influencing parameters. The experimental database used for developing the models was established upon several pressuremeter tests conducted on different soil types at depths of 3–40m. To verify the applicability of the derived model, it was employed to estimate the soil moduli of portions of test results that were not included in the analysis. Further, the generalization capability of the model was verified via several statistical criteria. The sensitivity of the soil deformation modulus to the influencing variables was examined and discussed. Moisture content and soil dry unit weight were found to be efficient representatives of the initial state and consolidation history of the soil for determining its deformation modulus. The results indicate that the LGP approach accurately characterizes the soil deformation modulus leading to a very good prediction performance. The correlation coefficients between the experimental and predicted soil modulus values are equal to 0.908 and 0.901 for the calibration and testing data sets, respectively.

A combined frequency–severity approach for the analysis of rear-end crashes on urban arterials

Analysis of both the crash count and the severity of injury are required to provide the complete picture of the safety situation of any given roadway. The randomness of crashes, the one-way dependency of injury on crash occurrence and the difference in response types have typically led researchers into developing independent statistical models for crash count and severity classification. The Genetic Programming (GP) methodology adopts the concepts of evolutionary biology such as crossover and mutation in effectively giving a common heuristic approach to model the development for the two different modeling objectives. The chosen GP models have the highest hit rate for rear-end crash classification problem and the least error for function fitting (regression) problems. Higher Average Daily Traffic (ADT) is more likely to result in more crashes. Absence of on-street parking may result in diminished severity of injuries resulting from crashes as they may provide “soft” crash barrier in contrast to fixed road side objects. Graphical presentation of the frequency of crashes with varying input variables shed new light on the results and its interpretation. Higher friction coefficient of roadways result in reduced frequency of crashes during the morning peak hours, with the trend being reversed during the afternoon peak hours. Crash counts have been observed to be at a maximum at a surface width of 30 ft. Sensitivity analysis results reflect that ADT is responsible for the largest variation in crash counts on urban arterials.

Prediction of PM10 concentrations through multi–gene genetic programming

This study aims to apply a multi–gene genetic programming (MGP) methodology for predicting the daily average of PM10 concentrations on the next day. This methodology is based on the principles of the simple genetic programming (GP) algorithm. The models are also encoded in tree structures (tree expressions) that are modified following an iterative process; the model structure and parameters are optimized, simultaneously. The main differences between these two methodologies are: (i) an individual is composed by several tree structures, called genes, and not a single one; and (ii) the output value is calculated through the linear combination of the outputs of the different genes belonging to the same individual.The case study here considered was to predict the daily average of PM10 concentrations on the next day. The data were collected in an urban site with traffic influences in Oporto Metropolitan Area, Northern Portugal. The air pollutants data (daily average concentrations of SO2, CO, NO, NO2 and PM10) and the meteorological data (daily averages of temperature – T, relative humidity – RH and wind speed – WS) were used as inputs for the models. The studied period was from January 2003 to December 2005.Ten MGP runs were applied and the results showed that RH, NO2 and PM10 concentrations were the most relevant input variables, as they appeared in almost all models. The MGP runs lead to selection of different models, which presented similar results in both training and test periods. Their predictive performances were compared with ones obtained with linear statistical models. MGP models did not present better results than linear models. However, considering that the relationships between air quality and meteorological variables are nonlinear and unknown, MGP was considered as a promising technique for the prediction of the daily average PM10 concentrations.

Prediction of tropospheric ozone concentrations: Application of a methodology based on the Darwin’s Theory of Evolution

This study aims to predict the next day hourly average tropospheric ozone (O 3) concentrations using genetic programming (GP). Due to the complexity of this problem, GP is an adequate methodology as it can optimize, simultaneously, the structure of the model and its parameters. It is an artificial intelligence methodology that uses the same principles of the Darwinian Theory of Evolution. GP enables the automatic generation of mathematical expressions that are modified following an iterative process applying genetic operations. The inputs of the models were the hourly average concentrations of carbon monoxide (CO), nitrogen oxide (NO), nitrogen dioxide (NO 2) and O 3, and some meteorological variables (temperature – T; solar radiation – SR; relative humidity – RH; and wind speed – WS) measured 24 h before. GP was also applied to the principal components (PC) obtained from these variables. The analysed period was from May to July 2004 divided in training and test periods. GP was able to select the most relevant variables for prediction of O 3 concentrations. The original variables, T, RH and O 3 measured 24 h before were considered significant inputs for prediction. The selected PC had also important contributions of the same variables and of NO 2. GP models using the original variables presented better performance in training period and worse performance in test period when compared with the models obtained using PC. The results achieved using the GP methodology demonstrated that it can be very useful to solve several environmental complex problems.

Stock Selection - an Innovative Application of Genetic Programming Methodology

One of the major challenges in an information-rich financial market is how effectively to derive an optimum investment solution among vast amounts of available information. The most efficacious combination of factors or information signals can be found by evaluating millions of possibilities, which is a task well beyond the scope of manual efforts. Given the limitations of the manual approach, factor combinations are typically linear. However, the linear combination of factors might be too simple to reflect market complexities and thus fully capture the predictive power of the factors. A genetic programming process can easily explore both linear and non-linear formulae. In addition, the ease of evaluation facilitates the consideration of broader factor candidates for a stock selection model. Based upon State Street Global Advisors (SSgA)’s previous research on using genetic programming techniques to develop quantitative investment strategies, we extend our application to develop stock selection mode ls in a large investable stock universe, the S&P 500 index. Two different fitness functions are designed to derive GP models that accommodate different investment objectives. First, we demonstrate that the GP process can generate a stock selection model for a low active risk investment style. Compared to a traditional model, the GP model has significantly enhanced future stock return ranking capability. Second, to suit an active investment style, we also use the GP process to generate a model that identifies the stocks with future returns lying in the fat tails of the return distribution. A portfolio constructed based on this model aims to aggressively generate the highest returns possible compared to an index following portfolio. Our tests show that the stock selection power of the GP models is statistically significant. Historical simulation results indicate that portfolios based on GP models outperform the benchmark and the portfolio based on the traditional model. Further, we demonstrate that GP models are more robust in accommodating various market regimes and have more consistent performance than the traditional model.

Genetic programming for simultaneous feature selection and classifier design

This paper presents an online feature selection algorithm using genetic programming (GP). The proposed GP methodology simultaneously selects a good subset of features and constructs a classifier using the selected features. For a c-class problem, it provides a classifier having c trees. In this context, we introduce two new crossover operations to suit the feature selection process. As a byproduct, our algorithm produces a feature ranking scheme. We tested our method on several data sets having dimensions varying from 4 to 7129. We compared the performance of our method with results available in the literature and found that the proposed method produces consistently good results. To demonstrate the robustness of the scheme, we studied its effectiveness on data sets with known (synthetically added) redundant/bad features.