Input Variable Selection Research Articles

Input Variable Selection and Structure Optimization for LSTM-Based Soft Sensor With a Dual Nonnegative Garrote Approach

Input Variable Selection and Structure Optimization for LSTM-Based Soft Sensor With a Dual Nonnegative Garrote Approach

MODELING REGRESSION ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM (RANFIS) FOR PANEL DATA

Panel data combine cross-sectional and time-series data. Data on economic, business, social, and development issues are often presented in panel data. In constructing the panel data regression model, it is necessary to take various steps for testing the model specifications, including the Chow test and the Hausman test. This study constructed a classical panel data regression model and the regression adaptive neuro-fuzzy inference system (RANFIS). The RANFIS model is a regression model by applying fuzzy and neural network (NN) techniques expected to overcome the problem of uncertainty. One of the main problems in constructing an optimal RANFIS is selecting input variables. The input variables of RANFIS are selected on the basis of the best classical regression. These inputs are classified into optimal clusters, which depend on the degree of fuzzy membership functions. The rule bases of RANFIS are determined on the basis of optimal inputs and its clusters. The empirical study in this research is to construct a panel data regression model for the Human Development Index (HDI) in Central Java in 2017-2019. HDI depends on several variables such as the school participation rate, number of health workers, public health complaints, population growth rate, and poverty severity index as predictor variables. Based on classical regression, three variables were used as optimal inputs for RANFIS modeling. Evaluation of model performance was measured based on the RMSE and MAPE values. Based on the RANFIS, the values of RMSE and MAPE were 3.227 and 3.299, respectively. Keywords: Panel Data Regression, Human Development Index, Regression Adaptive Neuro-Fuzzy Inference System DOI： https://doi.org/10.35741/issn.0258-2724.58.3.57

Integrating data-informativity conditions in predictor models for single module identification in dynamic networks

For consistent identification of a target module in a dynamic network with the local direct method, basically two prime conditions need to be satisfied: (a) a set of structural conditions on the choice of the predictor model, i.e. a set of input and output node variables, and (b) conditions on data-informativity. While for conditions (a) constructive algorithms for node selection have been presented that appropriately guarantee that the identified object can indeed reveal the target module, the requirements for satisfying (b) have not yet been integrated fully. In this paper, we will present simplified path-based results for generic data-informativity, and show how they can be integrated in constructive algorithms for predictor model selection that provide consistent target module estimates. It is shown that data-informativity not only requires a sufficient number of external excitation signals to be present in the network, but also puts restrictions on the structure of the predictor model, i.e. the selection of input and output node variables. Some examples are presented to illustrate the new results.

Applied machine learning for prediction of waste plastic pyrolysis towards valuable fuel and chemicals production

Pyrolysis is a suitable conversion technology to address the severe ecological and environmental hurdles caused by waste plastics' ineffective pre- and/or post-user management and massive landfilling. By using machine learning (ML) algorithms, the present study developed models for predicting the products of continuous and non-catalytically processes for the pyrolysis of waste plastics. Along with different input datasets, four algorithms, including decision tree (DT), artificial neuron network (ANN), support vector machine (SVM), and Gaussian process (GP), were compared to select input variables for the most accurate models. Among these algorithms, the DT model exhibited generalisable and satisfactory accuracy (R2 > 0.99) with training data. The dataset with the elemental composition of waste plastics achieved better accuracy than that with the plastic-type for predicting liquid yields. These observations allow the predictions by the data from ultimate analysis when inaccessible to the plastic-type data in unknown plastic wastes. Besides, the combination of ultimate analysis input and the DT model also achieved excellent accuracy in liquid and gas composition predictions.

NOx Emission Prediction Based on SSA-DELM via CFD and DCS Data Fusion

The objective of this study was to obtain NOx emission prediction model at the inlet selective catalytic reduction (SCR) reactors, which was the basis of combustion optimization and denitrification treatment. A deep extreme learning machine (DELM) optimized by the sparrow optimization algorithm (SSA) was adopted to establish the NOx model based on data fusion of Computational Fluid Dynamics (CFD) simulation and Distributed Control System (DCD). The mechanism analysis and XGBoost algorithm was used to select input variables. The results show that the XGBoost-SSA-DELM-based prediction model has high prediction accuracy with mean absolute error of 2.54 mg/m3. The results of this study have important implications for research on improving combustion efficiency and reducing pollutant emissions.

Models and Forecasts of the Export Potential of the Regional Economic System

Introduction. The export potential of the regional agricultural market causes great interest from both the scientific community and government agencies. Many scientific studies are devoted to the search for methods to increase the export potential in order to ensure the sustainable development of regional economic systems. The article proposes and tests the original author’s algorithm for creating a dynamic econometric model for forecasting the volumes of production, sales and exports of the main types of livestock products at the regional level. The purpose of the article is to assess the export potential of the main products of the regional agricultural market based on the built dynamic econometric models. Materials and Methods. The research is based on a set of empirical data of result and input variables characterizing the production, sales and export of the main livestock products in the region for the period 2010–2020. Research methods include dynamic analysis, econometric modeling and forecasting. The dynamic analysis carried out made it possible to assess the change in the production, sale and export of the main agricultural products in the region, to assess the current trends. Based on the constructed econometric models, the most significant factors influencing the resulting variables were identified, the specifications and verification of the models were carried out. The method of extrapolation of the identified trends made it possible to evaluate the predicted values of the resulting variables for the medium term. Results. Based on the selected input variables, models of production, sale and export of milk, wool and eggs by agricultural producers in the region were built. From a variety of alternative models, models with the best statistical quality characteristics were selected. The high level of quality of the obtained models made it possible to use them for predictive calculations of the levels of resulting variables for the period 2021–2026. Comparison of the results of the forecasts made it possible to identify the types of livestock products that already have a high level of exportability. In addition, types of products with a low level of exportability were identified, which have a high potential for increase. Discussion and Conclusion. As a result of using econometric modeling methods, dynamic models were obtained that made it possible to obtain a forecast for the development of livestock in a region with a high export potential in the near future. The practical significance of this article lies in the possibility of influencing the production, sale and export of livestock products in the region through a change in the corresponding set of factor variables of the models. Thus, the resulting dynamic models can be used both by agricultural producers for planning economic activities, and by regional authorities when drawing up regional development programs.

Condition monitoring of wind turbine based on deep learning networks and kernel principal component analysis

Condition monitoring is frequently hampered by the abominable working conditions of wind turbines (WTs). The present study proposes a new approach for condition monitoring of WT, which involves the convolutional neural network (CNN) that cascades to the long- and short-term memory network (LSTM) with kernel principal component analysis (KPCA). First, the density-based spatial clustering of applications with noise (DBSCAN) method was used to filter data from supervisory control and data acquisition (SCADA) to improve the effectiveness of the data. Then, the KPCA algorithm was applied for performance monitoring and fault prediction of WT by selecting input variables, and the KPCA–CNN–LSTM model was established. Finally, taking a wind farm as an example, the established prediction model was used to analyze multiple components of the WT. The experimental outcomes demonstrate that the proposed model can help not only in monitoring the state of the WT but also in predicting the abnormal operation state of the WT at an early stage, thereby verifying the effectiveness of the proposed method.

Machine learning-based predictions of fatigue life for multi-principal element alloys

Compared to conventional alloys, multi-principal element alloys (MPEAs) offer attractive and tunable fatigue properties. With the advancement of machine learning (ML) in material science, it is perceived as an effective tool for predicting materials fatigue life. Therefore, in this work, we aim to predict the room temperature fatigue lives of two classes of MPEAs, the single-phase CoaCrbFecMndNie system and the multi-phase AlfCogCrhFeiMnjNik system, using four different ML algorithms. Popular ML algorithms viz., Random Forest (RF), Support Vector Machine (SVM), and boosting algorithms such as GBOOST and XGBOOST were employed. Three pairs of test and training datasets were utilized to generalize the employed ML models. Input variables selection procedure was also carried out for possible improvement of algorithms’ performance. SVM and boosting algorithms predicted fatigue lives of both classes of MPEAs within the band of the factor of two. Overall, all algorithms performed reasonably well, despite several hindering factors, such as the data's inherent scatter and limited datasets, which were acquired using different testing standards.

NOx emission predicting for coal-fired boilers based on ensemble learning methods and optimized base learners

Generally, because the actual operating data tends to be concentrated in local regions due to the habits of operators and control system design, single models can only train a prediction model from a certain data feature, but they can not fully mine and utilize the valuable information hidden in the data. In this paper, new ensemble learning methods are proposed to predict NOx concentration at the SCR inlet in coal-fired boilers. Firstly, to reduce redundant data and data dimension, mutual information(MI) feature selection method is used to select input variables. Then, all input variables are normalized and divided to prepare the data for modeling. Secondly, considering the perspective of diversity and difference, three kinds of single models, which are gate recurrent unit(GRU), convolutional neural networks(CNN) and multivariate linear regression(MLR), are constructed to serve as the base learners. To make each base learner reach their optimal state, the model structures and parameters are optimized respectively. After that, the novel ensemble learning methods are proposed and used to construct NOx emission predicting model based on the three optimized base learners, respectively. To verify the generalization of the presented ensemble models, another dataset from a different thermal power plant is selected for verification. The experimental results indicate that the presented ensemble models have excellent prediction accuracy and generalization. The RMSEs of the proposed Stacking/Blending model are only 14.115/14.017mg/Nm3 (for dataset A) and 6.842/6.447mg/Nm3 (for dataset B) respectively, which are much lower than other compared models. Additionally, the application test once again proves that the presented methods can be used in engineering practice and are able to effectively reduce the NOx emission of coal-fired boiler.

An evolutionary stacked generalization model based on deep learning and improved grasshopper optimization algorithm for predicting the remaining useful life of PEMFC

Accurate prediction of the future degradation trend (FDT) and remaining useful life (RUL) of proton exchange membrane fuel cell (PEMFC) is crucial in the prognosis and health management (PHM) of PEMFC. Therefore, this study develops a stacked generalization model (SGM) to predict the FDT and RUL of the PEMFC. Firstly, maximum information coefficient (MIC) and Akaike information criterion (AIC) are used to select input variables to obtain key information and reduce the difficulty of model training. Then use improved grasshopper optimization algorithm (GOA) to optimize the hyperparameters of the SGM to further improve the prediction performance of the model. Finally, according to the dataset after feature selection, improved GOA (IGOA) optimized SGM (M/AIC-IGOA-SGM) is established to predict FDT and RUL. Among them, IGOA is obtained by introducing Chebyshev chaotic mapping initialization, chaotic decreasing factor, and a spiral position update mechanism to GOA. The base models of SGM are support vector regression (SVR) and gated recurrent unit (GRU), and the meta-model is regularized extreme learning machine (RELM). In the comparative experiments under two different current conditions, the superiority of the proposed model is verified, and the effectiveness of M/AIC, IGOA, and SGM is discussed.

Machine learning models to predict in-hospital mortality in septic patients with diabetes.

Sepsis is a leading cause of morbidity and mortality in hospitalized patients. Up to now, there are no well-established longitudinal networks from molecular mechanisms to clinical phenotypes in sepsis. Adding to the problem, about one of the five patients presented with diabetes. For this subgroup, management is difficult, and prognosis is difficult to evaluate. From the three databases, a total of 7,001 patients were enrolled on the basis of sepsis-3 standard and diabetes diagnosis. Input variable selection is based on the result of correlation analysis in a handpicking way, and 53 variables were left. A total of 5,727 records were collected from Medical Information Mart for Intensive Care database and randomly split into a training set and an internal validation set at a ratio of 7:3. Then, logistic regression with lasso regularization, Bayes logistic regression, decision tree, random forest, and XGBoost were conducted to build the predictive model by using training set. Then, the models were tested by the internal validation set. The data from eICU Collaborative Research Database (n = 815) and dtChina critical care database (n = 459) were used to test the model performance as the external validation set. In the internal validation set, the accuracy values of logistic regression with lasso regularization, Bayes logistic regression, decision tree, random forest, and XGBoost were 0.878, 0.883, 0.865, 0.883, and 0.882, respectively. Likewise, in the external validation set 1, lasso regularization = 0.879, Bayes logistic regression = 0.877, decision tree = 0.865, random forest = 0.886, and XGBoost = 0.875. In the external validation set 2, lasso regularization= 0.715, Bayes logistic regression = 0.745, decision tree = 0.763, random forest = 0.760, and XGBoost = 0.699. The top three models for internal validation set were Bayes logistic regression, random forest, and XGBoost, whereas the top three models for external validation set 1 were random forest, logistic regression, and Bayes logistic regression. In addition, the top three models for the external validation set 2 were decision tree, random forest, and Bayes logistic regression. Random forest model performed well with the training and three validation sets. The most important features are age, albumin, and lactate.

Chaotic time series prediction based on selection of important input variables1

Chaotic systems are dynamic systems with aperiodic and pseudo-random properties, and systems in many fields exhibit chaotic time-series properties. Aiming at the fuzzy modeling problem of chaotic time series, this paper proposes a new fuzzy identification method considering the selection of important input variables. The purpose is to achieve higher model modeling and prediction accuracy by constructing a model with a simple structure. The relevant input variable was swiftly chosen in accordance with the input variable index after the Two Stage Fuzzy Curves method was used to determine the weight of the correlation between each input variable and the output from a large number of selectable input variables. The center and width of the irregular Gaussian membership function were then optimized using the fuzzy C-means clustering algorithm and the particle swarm optimization technique, which led to the determination of the fuzzy model’s underlying premise parameters. Finally, the fuzzy model’s conclusion parameters were determined using the recursive least squares method. This model is used to simulate three chaotic time series, and the outcomes of the simulation are contrasted and examined. The outcomes demonstrate that the fuzzy identification system has higher prediction accuracy based on a simpler structure, demonstrating its validity.

Deep Learning Models for SO2 Distribution in a 30 MW Boiler via Computational Fluid Dynamics Simulation Data

The distribution of SO2 in a boiler is an important factor affecting tube corrosion in a furnace. To investigate the correlation between SO2 distribution and numerous variables (e.g., temperature, O2 distribution, etc.), a hybrid deep learning model is developed via the computational fluid dynamics (CFD) simulation data. First, the combustion process under typical working conditions is simulated to output the training data set. Then, a LASSO algorithm is adopted to select input variables with a high correlation with SO2 distribution. Finally, a deep belief network combined with a restricted belief machine and a fully connected layer is developed to describe the nonlinear relationship. The proposed model is the first work to use a deep learning algorithm to obtain the correlation between SO2 distribution and other products of combustion. The results show that O2 concentration has the highest influence on SO2 distribution.

Water absorption rates and mechanical properties of material extrusion-printed continuous carbon fiber-reinforced nylon composites

Material extrusion (ME) is considered as an effective and economic method to process continuous carbon fiber (CCF)-reinforced polymer composites with complicated structures. In this investigation, water absorption tests and tensile tests were conducted on ME-printed CCF-reinforced polymer composites under hydrothermal conditions. The purpose is to meet the growing need of CCF-reinforced polymer composites for high-humidity and high-heat aerospace applications. In order to achieve optimal comprehensive performance, both the L16 (45) orthogonal array method and the fuzzy reasoning algorithm were employed to carry out multi-objective optimization of four selected input variables, including layup angle, nozzle temperature, fiber filling density, and layer thickness. Experimental results show that the comprehensive performance of composites obtained by inputting data into the fuzzy reasoning system was greater than that obtained from orthogonal test results. Specifically, the optimized input variables are 0° layup angle, 260 °C nozzle temperature, 80% fiber filling rate, and 0.6 mm layer thickness.

The Use of TensorFlow in Analyzing Air Quality Artificial Intelligence Predictions PM2.5

Artificial intelligence techniques to forecasts based on the Community Multiscale Air Quality (PM2.5) operational model can be known using TensorFlow. TensorFlow was used in this study to assess the scores of the Recurrent Neural Networks (RNN) input variables on the 6-hour forecast for July-October 2022. The relevance scores for the one- and two-day forecasts are represented by the sum of the relevance scores across the target prediction timeframe 2–5 and 4–7 previous time steps. The initial selection of input variables was based on their correlation coefficient with the measured PM2.5 concentration. Still, the order of contribution of the input variables measured by TensorFlow was different from the order of their correlation coefficients, which indicated an inconsistency between the linear and nonlinear variables of the method. It was found that the retraining of the RNN model using a subset of variables with a high relevance score resulted in a predictive ability similar to the initial set of input variables.

Global horizontal irradiance modeling from environmental inputs using machine learning with automatic model selection

Solar radiation prediction is essential in load control on electrical networks allowing a balanced energy supply in accordance with the network demand. However, stations needing specific equipment, which is typically expensive and rare, to collect the data needed for such predictions. As a result of the tension between the demand for data and the challenge of acquiring it, the adoption of novel, low-cost technological solutions is necessary to address pressing issues. Successful predictions of solar radiation at many sites across the world have been made using machine learning (ML) models. This research develops a more straightforward strategy for efficient input variable selection. The proposed framework uses the most relevant environmental information to build an efficient model to predict global horizontal irradiance for the Dar es Salaam region, Tanzania. Four different ML models are developed and compared to predict solar radiation, i.e., Multivariate Adaptive Regression Spline (MARS), Support Vector Regression (SVR), ensemble Extreme Gradient Boosting (XGB), and polynomial Ridge Regression (RR). A total of 252 different scenarios are analyzed in the search for the best combination between the ML models and variables subset. The results showed that all optimized machine learning models could predict global horizontal irradiation with small error rates. The research also indicates that the proper choice of the available environmental variables can lead to less complex modeling with improved predictions.

Practical recommendations for machine learning in underground rock engineering – On algorithm development, data balancing, and input variable selection

AbstractResearch has demonstrated that machine learning algorithms (MLAs) are a powerful addition to the rock engineering toolbox, and yet they remain a largely untapped resource in engineering practice. The reluctance to adopt MLAs as part of standard practice is often attributed to the ‘opaque’ nature of the algorithms, the complexity in developing them, and the difficulty in determining how the algorithms use the datasets. This article presents tools and processes for developing MLAs, input selection, and data balancing for practical underground rock engineering. MLAs for classification and regression – two main machine learning applications – are presented in terms of developing MLA to extract information from the dataset to obtain the desired output. Engineering verification metrics are selected based on their suitability for specific output. Methods for input selection and data balancing are discussed with a focus on selecting appropriate input data for the problem without introducing bias or excess complexity. Each tool and process for algorithm development, data preparation, and input selection is illustrated with a case study. This article demonstrates that geotechnical practitioners can extract additional value by applying MLAs to rock engineering problems. Once an understanding of the functions of MLAs is reached, the building blocks and open‐source code are available to be adapted to suit the rock mass behaviour of interest.

Unified fatigue life modelling and uncertainty estimation of Ni-based superalloy family with a supervised machine learning approach

Machine learning (ML) approaches, especially the supervised learning methods, show enormous advantages in fatigue investigation, whereas few works follow the interest in the generalization and uncertainty estimation of these data-drive approaches. Within this content, a supervised ML method based on the support vector regression (SVR) for key features identification of fatigue life is presented for the Ni-based superalloy family. The unified SVR model is effective at predicting lives for the Ni-based superalloy family under wide loading conditions and fatigue regimes compared with the classical fatigue life models. In addition, a model fusion method is employed to estimate the uncertainty and data dependency of the SVR model, with which the training strategy produces an important effect on the accuracy and stability of the predicted results. It is found that the coefficient of the uncertainty achieves the optimum where the training percentage is 70% of the modelling samples. After input variable selection by the pairwise Pearson correlation and key feature reorganization, the dimensionality reduced SVR model remains an acceptable accuracy. Predictably, the total strain range and the test frequency are recognized as the highly correlated variables with the fatigue life investigating from the perspective of datasets. The trained ML model and uncertainty estimation approach provide potential tools for fatigue investigation under complex loading conditions. Going forwards, it would be beneficial to generalization ability and uncertainty estimation of a ML model for unified fatigue life modelling.

Power Generation Prediction Method of Offshore Wind Turbines Based on Cascaded Deep Learning

Aiming at the problems of low prediction accuracy, long time, and poor results in current wind turbine generation power prediction methods, an offshore wind turbine generation power prediction method based on cascaded deep learning is proposed. Using deep belief networks, stacked autoencoding networks, and long short-term memory networks, a cascaded deep learning method is proposed to predict the power generation of offshore wind turbines. Multiple feature extractors are used to extract and fuse high-level features to form a unified feature with richer information to predict the power generation sequence of offshore wind turbines. According to the modeling strategy and port design strategy, using the stacked autoencoding networks as the basic unit, a cascaded deep learning model for generating power prediction of offshore wind turbines is established. Through the selection of input variables, the variables that have a great correlation with wind power are obtained. The layer-by-layer greedy algorithm is used for training from bottom to top, and supervised learning is used to fine-tune the network parameters from top to bottom to realize the generation power prediction of the offshore wind turbine. The experimental results show that the proposed method is effective in predicting the power generation of offshore wind turbines, which can effectively improve the prediction accuracy and shorten the prediction time.

Identifying oral disease variables associated with pneumonia emergence by application of machine learning to integrated medical and dental big data to inform eHealth approaches.

The objective of this study was to build models that define variables contributing to pneumonia risk by applying supervised Machine Learning-(ML) to medical and oral disease data to define key risk variables contributing to pneumonia emergence for any pneumonia/pneumonia subtypes. Retrospective medical and dental data were retrieved from Marshfield Clinic Health System's data warehouse and integrated electronic medical-dental health records (iEHR). Retrieved data were pre-processed prior to conducting analyses and included matching of cases to controls by (a) race/ethnicity and (b) 1:1 Case: Control ratio. Variables with >30% missing data were excluded from analysis. Datasets were divided into four subsets: (1) All Pneumonia (all cases and controls); (2) community (CAP)/healthcare associated (HCAP) pneumonias; (3) ventilator-associated (VAP)/hospital-acquired (HAP) pneumonias and (4) aspiration pneumonia (AP). Performance of five algorithms were compared across the four subsets: Naïve Bayes, Logistic Regression, Support Vector Machine (SVM), Multi-Layer Perceptron (MLP) and Random Forests. Feature (input variables) selection and ten-fold cross validation was performed on all the datasets. An evaluation set (10%) was extracted from the subsets for further validation. Model performance was evaluated in terms of total accuracy, sensitivity, specificity, F-measure, Mathews-correlation-coefficient and area under receiver operating characteristic curve (AUC). In total, 6,034 records (cases and controls) met eligibility for inclusion in the main dataset. After feature selection, the variables retained in the subsets were: All Pneumonia (n = 29 variables), CAP-HCAP (n = 26 variables); VAP-HAP (n = 40 variables) and AP (n = 37 variables), respectively. Variables retained (n = 22) were common across all four pneumonia subsets. Of these, the number of missing teeth, periodontal status, periodontal pocket depth more than 5 mm and number of restored teeth contributed to all the subsets and were retained in the model. MLP outperformed other predictive models for All Pneumonia, CAP-HCAP and AP subsets, while SVM outperformed other models in VAP-HAP subset. This study validates previously described associations between poor oral health and pneumonia. Benefits of an integrated medical-dental record and care delivery environment for modeling pneumonia risk are highlighted. Based on findings, risk score development could inform referrals and follow-up in integrated healthcare delivery environment and coordinated patient management.