Linear Model Tree Research Articles

A data-centric predictive control approach for nonlinear chemical processes

Many nonlinear chemical processes (e.g., aluminum smelter cells) are operated under simple logic or constraint-based control strategies, which often do not achieve a high level of performance. A challenge for implementing model-based process control (e.g., model predictive control) is the cost, time, and effort associated with building an accurate dynamic model of the physical system. By integrating machine learning and control theory, this work presents a novel big data-centric predictive control (DPC) approach for nonlinear chemical processes. During the offline training period, the proposed approach utilizes a learning algorithm – Random Forests with linear model trees (RF-LMT) to learn black-box models from the process operation data. For online implementation, the DPC controller uses input-output trajectories to retrieve local linear models from the linear model trees, which are then applied to predictive control for process constraints handling and performance optimization in a receding horizon. The closed-loop stability of the proposed DPC approach is ensured by an incremental dissipativity based condition. Furthermore, a systematic method is developed to derive such an offline stability constraint from the data-centric model learned from RF-LMT. The proposed DPC approach is demonstrated by a case study on control of the aluminum smelter cell, which is under logic control in current industrial practice. The study shows that the proposed approach can significantly improve the system performance by using the existing operational data.

Retraction: Using artificial neural network for forward kinematic problem of under-constrained cable robots

Cable-Driven Parallel Robot has many advantages. However, the problems of cable collision between each other and environment, the lack of proper structure and non-positive cable tension prevent the spread of them. In this work, a neural network (NN) model of under constrained cable robots is presented with external forces applied to the end-effector (EE) for computing the position of it. As in under-constrained robot’s kinematics and statics are innately coupled together, and they contemporaneously should be considered the forward kinematic problem of the robot change to an optimization problem. This approach does not require pre-knowledge of the uncertainties upper bounds and linear regression form of kinematic and dynamic models. Moreover, to ensure that all cables remain in tension, proposed control algorithm benefit the internal force concept in its structure. The main contribution of this paper has three goals. First, a method is used toward kinematic problem of the under constrained cable robot modeling using four bar linkage kinematic concept, which could be used in online control approaches for real-time purposes. Second, in order to track the position of end-effector, an online PD controller is designed by the three error criteria methods such as IAE, ISE and ITSE. Finally, as the third contribution, NN control approach is applied in order to validate the model. A model is created based on the robot’s geometry and dynamic to solve the forward kinematics problem. So, the forward kinematic problem is solved offline and used online. Moreover, an analysis of workspace is performed which discovers that the solution of the forward kinematic problem of the under-constrained cable robots is unique in this case. In addition, a modified local linear model tree algorithm for nonlinear system modelling are proposed. The results show the effectiveness of the proposed approach in modeling the under constrained cable robot.

Generalised linear model trees with global additive effects

Model-based trees are used to find subgroups in data which differ with respect to model parameters. In some applications it is natural to keep some parameters fixed globally for all observations while asking if and how other parameters vary across subgroups. Existing implementations of model-based trees can only deal with the scenario where all parameters depend on the subgroups. We propose partially additive linear model trees (PALM trees) as an extension of (generalised) linear model trees (LM and GLM trees, respectively), in which the model parameters are specified a priori to be estimated either globally from all observations or locally from the observations within the subgroups determined by the tree. Simulations show that the method has high power for detecting subgroups in the presence of global effects and reliably recovers the true parameters. Furthermore, treatment-subgroup differences are detected in an empirical application of the method to data from a mathematics exam: the PALM tree is able to detect a small subgroup of students that had a disadvantage in an exam with two versions while adjusting for overall ability effects.

Using artificial neural network for forward kinematic problem of under-constrained cable robots

Cable-Driven Parallel Robot has many advantages. However, the problems of cable collision between each other and environment, the lack of proper structure and non-positive cable tension prevent the spread of them. In this work, a neural network (NN) model of under constrained cable robots is presented with external forces applied to the end-effector (EE) for computing the position of it. As in under-constrained robot’s kinematics and statics are innately coupled together, and they contemporaneously should be considered the forward kinematic problem of the robot change to an optimization problem. This approach does not require pre-knowledge of the uncertainties upper bounds and linear regression form of kinematic and dynamic models. Moreover, to ensure that all cables remain in tension, proposed control algorithm benefit the internal force concept in its structure. The main contribution of this paper has three goals. First, a method is used toward kinematic problem of the under constrained cable robot modeling using four bar linkage kinematic concept, which could be used in online control approaches for real-time purposes. Second, in order to track the position of end-effector, an online PD controller is designed by the three error criteria methods such as IAE, ISE and ITSE. Finally, as the third contribution, NN control approach is applied in order to validate the model. A model is created based on the robot’s geometry and dynamic to solve the forward kinematics problem. So, the forward kinematic problem is solved offline and used online. Moreover, an analysis of workspace is performed which discovers that the solution of the forward kinematic problem of the under-constrained cable robots is unique in this case. In addition, a modified local linear model tree algorithm for nonlinear system modelling are proposed. The results show the effectiveness of the proposed approach in modeling the under constrained cable robot.

Validation and update of a lymph node metastasis prediction model for breast cancer

PurposeThis study aimed to validate and update a model for predicting the risk of axillary lymph node (ALN) metastasis for assisting clinical decision-making. MethodsWe included breast cancer patients diagnosed at six Dutch hospitals between 2011 and 2015 to validate the original model which includes six variables: clinical tumor size, tumor grade, estrogen receptor status, lymph node longest axis, cortical thickness and hilum status as detected by ultrasonography. Subsequently, we updated the original model using generalized linear model (GLM) tree analysis and by adjusting its intercept and slope. The area under the receiver operator characteristic curve (AUC) and calibration curve were used to assess the original and updated models. Clinical usefulness of the model was evaluated by false-negative rates (FNRs) at different cut-off points for the predictive probability. ResultsData from 1416 patients were analyzed. The AUC for the original model was 0.774. Patients were classified into four risk groups by GLM analysis, for which four updated models were created. The AUC for the updated models was 0.812. The calibration curves showed that the updated model predictions were better in agreement with actual observations than the original model predictions. FNRs of the updated models were lower than the preset 10% at all cut-off points when the predictive probability was less than 12.0%. ConclusionsThe original model showed good performance in the Dutch validation population. The updated models resulted in more accurate ALN metastasis prediction and could be useful preoperative tools in selecting low-risk patients for omission of axillary surgery.

Baseline factors associated with better response to insulin lispro low mixture or insulin glargine: A post hoc analysis of the DURABLE study

AimsTo identify baseline characteristics associated with better efficacy and safety responses to twice-daily insulin lispro low mixture (LM) or once-daily insulin glargine (IG) in insulin-naïve patients with type 2 diabetes (T2D). MethodsThis post hoc analysis of the DURABLE study used the gradient-boosting method to generate hypothetical outcomes with the alternative treatment to assigned study drug to evaluate the potential additional benefit of one insulin over the other in association with influential baseline covariates in the same patient. The magnitude of additional benefit was further quantified by the generalized linear model and recursive partitioning regression tree method. ResultsBaseline characteristics with the highest relative influence on 24-week outcomes in the overall population (LM, n = 1045; IG, n = 1046) were: for HbA1c change: fasting plasma glucose (FPG) (29.31%), age (28.57%); for reaching target HbA1c <7% (<53 mmol/mol): weight (22.41%); for weight change: weight (18.54%); for FPG: FPG (43.66%), age (20.8%); for 30-day hypoglycemia: FPG (57.09%), weight (10.1%). LM showed superiority over IG for HbA1c reduction and reaching HbA1c <7% overall, with clinically significant differences in HbA1c reduction (>0.4%) in some subpopulations. IG was superior over LM in most patients for less weight gain and hypoglycemia and lower FPG. ConclusionsDifferences in magnitude of response to twice-daily LM and once-daily IG in association with baseline characteristics of insulin-naïve patients with T2D were found. Future real-world studies using these statistical methods could help identify patients who respond better to certain insulin regimens to help guide clinicians in treatment decisions.

Detecting treatment-subgroup interactions in clustered data with generalized linear mixed-effects model trees

Identification of subgroups of patients for whom treatment A is more effective than treatment B, and vice versa, is of key importance to the development of personalized medicine. Tree-based algorithms are helpful tools for the detection of such interactions, but none of the available algorithms allow for taking into account clustered or nested dataset structures, which are particularly common in psychological research. Therefore, we propose the generalized linear mixed-effects model tree (GLMM tree) algorithm, which allows for the detection of treatment-subgroup interactions, while accounting for the clustered structure of a dataset. The algorithm uses model-based recursive partitioning to detect treatment-subgroup interactions, and a GLMM to estimate the random-effects parameters. In a simulation study, GLMM trees show higher accuracy in recovering treatment-subgroup interactions, higher predictive accuracy, and lower type II error rates than linear-model-based recursive partitioning and mixed-effects regression trees. Also, GLMM trees show somewhat higher predictive accuracy than linear mixed-effects models with pre-specified interaction effects, on average. We illustrate the application of GLMM trees on an individual patient-level data meta-analysis on treatments for depression. We conclude that GLMM trees are a promising exploratory tool for the detection of treatment-subgroup interactions in clustered datasets.

Model extension for model based MIMO control in HVAC systems

The presented work deals with the multi-input multi-output (MIMO) control of Heating Ventilating and Air Conditioning (HVAC) systems. In previous work a model based control strategy consisting of a linear model predictive control (MPC) and the Local Linear Model Tree (LoLiMoT) algorithm was presented by the authors. The underlying modeling approach is based on a detailed representation of each HVAC system component. However, the dry effect due to residual condensation after a dehumidification period was not taken into account. In literature, this effect is also not considered in sufficient detail to the best knowledge of the authors. In this contribution a model extension including a novel approach regarding the above mentioned dry effect is discussed. The obtained improvements with respect to previous work are demonstrated in simulation as well as on a real world system.

Next Check-in Location Prediction Combines with Collaborative Filtering

With the thriving of mobile location-based services, location-based social networks (LBSNs) have become a popular form of social media recently. Large numbers of “check-in” records continue to accumulate over time, which contain rich information of social and geographical context and provide a unique opportunity for researchers to study user’s social behavior, which in turn enables a variety of services including place advertisement, intelligent transportation, and Urban Computing. In this paper, we mainly study the problem of user’s location prediction. Firstly we analyzed the regular mobility patterns about the visited locations. However, on account of personality trait of neophilia, people also show propensities of novelty seeking in human mobility, such as exploring unvisited but tailored locations for them to visit. Hence, we not only propose a set of features that aim to describe the regular mobility patterns, but also use the collaborative social knowledge to analysis the users’ mobility patterns on the unvisited locations. Finally, we further extend our study combining all individual features in two supervised learning models, based on linear regression and M5 model trees, to get a higher overall prediction accuracy. And we find that the supervised methodology based on M5 model trees the offers better prediction accuracy.

Data-driven techniques for modelling the gross primary production of the páramo vegetation using climate data: Application in the Ecuadorian Andean region

As one of the main areas of carbon cycle and climate change studies, water and CO2 relations are of great significance for estimation of gross primary production (GPP). Various biogeochemical process-based models have been set up to estimate the GPP based on mathematical representation of biological, physiological and ecological processes. However, they ended up increasing the complexity and computational processing power due to the large number of physical equations that need to be solved. Computational time becomes an important matter in the simulation of multiple scenarios using models for long periods of time (e.g. climate projections). Data driven surrogate models have proven to be a useful tool for environmental modelling especially when ecological and climatic co-variates are large. The advantages of Data Driven Models (DDM) are: the possibility of adding new independent variables even if their understanding is weak, and short computational time to run. The aim is to explore the ability of DDMs to replicate a biochemical model calculating GPP. This study evaluates the performance of four surrogate DDMs, namely linear regression method (LRM), model tree (MT), instance-based learning (IBL) and artificial neural network (ANN). A simple empirical and semi-empirical relationship between GPP and climatic variables are studied. Input variable selection (IVS) methods were used to decide on the most relevant and potential environmental model inputs and then followed by a two-step approach which included a model-free and a model-based technique. Data from the highlands (páramo ecosystem) in the Ecuadorian Andean Region from 12-year time-series (2000−2011) were used to evaluate the models at various time frames and at different altitudes. The GPP time series data for the same period were derived from an earlier study using the biomodel BIOME-BGC (BioGeochemical Cycles), which is a comprehensive physical based model used in different analysis of carbon fluxes around the world. So-called IBL (nearest neighbour method) showed a great capability to reproduce the GPP when data was aggregated to monthly time frame. The computational time used to evaluate the time series with IBL as the selected DDM is shorter with enough accuracy for using it in multi-model runs.

Time Prediction Using a Neuro-Fuzzy Model for Projects in the Construction Industry

This paper presents a prediction model based on a new neuro-fuzzy algorithm for estimating time in construction projects. The output of the proposed prediction model, which is employed based on a locally linear neuro-fuzzy (LLNF) model, is useful for assessing a project status at different time horizons. Being trained by a locally linear model tree (LOLIMOT) learning algorithm, the model is intended for use by members of the project team in performing the time control of projects in the construction industry. The present paper addresses the effects of different factors on the project time and schedule by using both fuzzy sets theory (FST) and artificial neural networks (ANNs) in a construction project in Iran. The construction project is investigated to demonstrate the use and capabilities of the proposed model to see how it allows users and experts to actively interact and, consequently, make use of their own experience and knowledge in the estimation process. The proposed model is also compared to the well-known intelligent model (i.e., BPNN) to illustrate its performance in the construction industry.

Ensembles of Fuzzy Linear Model Trees for the Identification of Multioutput Systems

We address the task of discrete-time modeling of nonlinear dynamic systems with multiple outputs using measured data. In the area of control engineering, this task is typically converted into a set of classical regression problems, one for each output, which can then be solved with any nonlinear regression approach. Fuzzy models, in the Takagi–Sugeno form, are popular in this context. We use Lolimot, which is a tree learning method, to build fuzzy linear model trees. In this paper, we propose, implement, and empirically evaluate three extensions of fuzzy linear model trees. First, we consider and evaluate multioutput models. Second, we propose to use ensembles of such models. Third, we investigate the use of a search heuristic based on simulation error (as opposed to one-step-ahead prediction error), specific to the context of modeling dynamic systems. Finally, we perform an empirical evaluation and compare these approaches on six multioutput case studies, using both measured and simulated data, with noise: The case studies include modeling of the inverse dynamics of a robot arm, as well as five additional process-industry systems. Ensembles improve the performance of both single- and multioutput trees, while the heuristic specific to modeling dynamic systems only improves performance very slightly. Multioutput model trees exhibit comparable or worse predictive performance to a set of single-output models, while providing a more compact model. Overall, we can recommend the use of bagging of single-output Lolimot models, learned by using the simulation error as a search heuristic.

Neurofuzzy-Model-Based Unsteady Aerodynamic Computations Across Varying Freestream Conditions

This paper presents a reduced-order modeling approach based on recurrent local linear neurofuzzy models for predicting generalized aerodynamic forces in the time domain. Regarding aeroelastic applications, the unsteady aerodynamic loads are modeled as a nonlinear function of structural eigenmode-based disturbances. In contrast to established aerodynamic input/output model approaches trained by high-fidelity flow simulations, the Mach number is considered as an additional model input to account for varying freestream conditions. To train the relationship between the input parameters and the corresponding flow-induced forces, the local linear model tree algorithm is adopted in this work. The proposed method is tested exemplarily with respect to the AGARD 445.6 configuration in the subsonic, transonic, and supersonic flight regimes. It is shown that good conformity is obtained between the reduced-order model results and the respective full-order computational-fluid-dynamics solution. A further comparative analysis in the frequency domain in conjunction with a classical flutter analysis confirms the validity of the approach. Finally, the method’s potential for reducing the computational effort of aeroelastic analyses is demonstrated.

LoLiMoT based MPC for air handling units in HVAC systems

This paper deals with the multi-input-multi-output (MIMO) control of an air handling unit (AHU) in an industrial heating ventilating and air conditioning (HVAC) system. The temperature and the relative humidity of the supply air have to be controlled simultaneously. Due to the nonlinear behavior of such a AHU it is difficult to achieve satisfactory performance over the whole operating range using linear control. Therefore in this work a model predictive control (MPC) strategy based on a network of local linear models is proposed. A detailed mathematical model referred to as “physical model” of the AHU is derived from physical laws and is verified by measurements. By proper excitation of the physical model, the local linear models are determined using the so called LoLiMoT (Local Linear Model Tree)-algorithm. The proposed concept is compared to a conventional PI control strategy and its performance is demonstrated in simulation as well as on a real world test plant.

Non-invasive real-time prediction of inner knee temperatures during therapeutic cooling

The paper addresses the issue of non-invasive real-time prediction of hidden inner body temperature variables during therapeutic cooling or heating and proposes a solution that uses computer simulations and machine learning. The proposed approach is applied on a real-world problem in the domain of biomedicine – prediction of inner knee temperatures during therapeutic cooling (cryotherapy) after anterior cruciate ligament (ACL) reconstructive surgery. A validated simulation model of the cryotherapeutic treatment is used to generate a substantial amount of diverse data from different simulation scenarios. We apply machine learning methods on the simulated data to construct a predictive model that provides a prediction for the inner temperature variable based on other system variables whose measurement is more feasible, i.e. skin temperatures. First, we perform feature ranking using the RReliefF method. Next, based on the feature ranking results, we investigate the predictive performance and time/memory efficiency of several predictive modeling methods: linear regression, regression trees, model trees, and ensembles of regression and model trees. Results have shown that using only temperatures from skin sensors as input attributes gives excellent prediction for the temperature in the knee center. Moreover, satisfying predictive accuracy is also achieved using short history of temperatures from just two skin sensors (placed anterior and posterior to the knee) as input variables. The model trees perform the best with prediction error in the same range as the accuracy of the simulated data (0.1°C). Furthermore, they satisfy the requirements for small memory size and real-time response. We successfully validate the best performing model tree with real data from in vivo temperature measurement from a patient undergoing cryotherapy after ACL reconstruction.

Comparing success levels of different neural network structures in extracting discriminative information from the response patterns of a temperature-modulated resistive gas sensor

Performances of three neural networks, consisting of a multi-layer perceptron, a radial basis function, and a neuro-fuzzy network with local linear model tree training algorithm, in modeling and extracting discriminative features from the response patterns of a temperature-modulated resistive gas sensor are quantitatively compared. For response pattern recording, a voltage staircase containing five steps each with a 20 s plateau is applied to the micro-heater of the sensor, when 12 different target gases, each at 11 concentration levels, are present. In each test, the hidden layer neuron weights are taken as the discriminatory feature vector of the target gas. These vectors are then mapped to a 3D feature space using linear discriminant analysis. The discriminative information content of the feature vectors are determined by the calculation of the Fisher’s discriminant ratio, affording quantitative comparison among the success rates achieved by the different neural network structures. The results demonstrate a superior discrimination ratio for features extracted from local linear neuro-fuzzy and radial-basis-function networks with recognition rates of 96.27% and 90.74%, respectively.

Gaussian mixture regression and local linear network model for data‐driven estimation of air mass

The conventional approaches for modelling of charge cycle in combustion engines are based on first principles. In this approaches, it is necessary to estimate different parameters in engine, which is very difficult to implement in real-time. In addition, it requires a high parameterisation effort because of a variety of engine characteristics. The model is further used in engine control unit to improve the efficiency of the combustion. Moreover, it can be used for estimation of some physical variables in engine to increase the performance of the controller. It also avoids the installation of extra sensors and reduces the costs of production. In this study, two different data-driven modelling methods for estimation of air mass are investigated. They also offer a flexible modelling approach considering various options for equipment as well as sensors and actuators in the engines. Both data-driven methods, the Gaussian mixture regression (GMR) and the local linear model tree (LOLIMOT) algorithm, allow a flexible modelling with a high input space dimensionality, with sparsity of data, a good interpretability and also offer possibilities of adaptation and local optimisation. While the GMR uses a statistical approaches to train the model, in the LOLIMOT an incremental, heuristic approach is used. Both methods are applied to the data obtained from a spark-ignition engine and the results are compared and discussed.

Characterizing tropical tree species growth strategies: learning from inter-individual variability and scale invariance.

Understanding how tropical tree species differ in their growth strategies is critical to predict forest dynamics and assess species coexistence. Although tree growth is highly variable in tropical forests, species maximum growth is often considered as a major axis synthesizing species strategies, with fast-growing pioneer and slow-growing shade tolerant species as emblematic representatives. We used a hierarchical linear mixed model and 21-years long tree diameter increment series in a monsoon forest of the Western Ghats, India, to characterize species growth strategies and question whether maximum growth summarizes these strategies. We quantified both species responses to biotic and abiotic factors and individual tree effects unexplained by these factors. Growth responses to competition and tree size appeared highly variable among species which led to reversals in performance ranking along those two gradients. However, species-specific responses largely overlapped due to large unexplained variability resulting mostly from inter-individual growth differences consistent over time. On average one-third of the variability captured by our model was explained by covariates. This emphasizes the high dimensionality of the tree growth process, i.e. the fact that trees differ in many dimensions (genetics, life history) influencing their growth response to environmental gradients, some being unmeasured or unmeasurable. In addition, intraspecific variability increased as a power function of species maximum growth partly as a result of higher absolute responses of fast-growing species to competition and tree size. However, covariates explained on average the same proportion of intraspecific variability for slow- and fast-growing species, which showed the same range of relative responses to competition and tree size. These results reflect a scale invariance of the growth process, underlining that slow- and fast-growing species exhibit the same range of growth strategies.

Prediction of effective porosity from seismic attributes using locally linear model tree algorithm

ABSTRACTIn this study, a locally linear model tree algorithm was used to optimize a neuro‐fuzzy model for prediction of effective porosity from seismic attributes in one of Iranian oil fields located southwest of Iran. Valid identification of effective porosity distribution in fractured carbonate reservoirs is extremely essential for reservoir characterization. These high‐accuracy predictions facilitate efficient exploration and management of oil and gas resources.The multi‐attribute stepwise linear regression method was used to select five out of 26 seismic attributes one by one. These attributes introduced into the neuro‐fuzzy model to predict effective porosity. The neuro‐fuzzy model with seven locally linear models resulted in the lowest validation error. Moreover, a blind test was carried out at the location of two wells that were used neither in training nor validation. The results obtained from the validation and blind test of the model confirmed the ability of the proposed algorithm in predicting the effective porosity. In the end, the performance of this neuro‐fuzzy model was compared with two regular neural networks of a multi‐layer perceptron and a radial basis function, and the results show that a locally linear neuro‐fuzzy model trained by a locally linear model tree algorithm resulted in more accurate porosity prediction than standard neural networks, particularly in the case where irregularities increase in the data set. The production data have been also used to verify the reliability of the porosity model. The porosity sections through the two wells demonstrate that the porosity model conforms to the production rate of wells.Comparison of the locally linear neuro‐fuzzy model performance on different wells indicates that there is a distinct discrepancy in the performance of this model compared with the other techniques. This discrepancy in the performance is a function of the correlation between the model inputs and output. In the case where the strength of the relationship between seismic attributes and effective porosity decreases, the neuro‐fuzzy model results in more accurate prediction than regular neural networks, whereas the neuro‐fuzzy model has a close performance to neural networks if there is a strong relationship between seismic attributes and effective porosity.The effective porosity map, presented as the output of the method, shows a high‐porosity area in the centre of zone 2 of the Ilam reservoir. Furthermore, there is an extensive high‐porosity area in zone 4 of Sarvak that extends from the centre to the east of the reservoir.

Short-term and long-term thermal prediction of a walking beam furnace using neuro-fuzzy techniques

The walking beam furnace (WBF) is one of the most prominent process plants often met in an alloy steel production factory and characterized by high non-linearity, strong coupling, time delay, large time-constant and time variation in its parameter set and structure. From another viewpoint, the WBF is a distributed-parameter process in which the distribution of temperature is not uniform. Hence, this process plant has complicated non-linear dynamic equations that have not worked out yet. In this paper, we propose one-step non-linear predictive model for a real WBF using non-linear black-box sub-system identification based on locally linear neuro-fuzzy (LLNF) model. Furthermore, a multi-step predictive model with a precise long prediction horizon (i.e., ninety seconds ahead), developed with application of the sequential one-step predictive models, is also presented for the first time. The locally linear model tree (LOLIMOT) which is a progressive tree-based algorithm trains these models. Comparing the performance of the one-step LLNF predictive models with their associated models obtained through least squares error (LSE) solution proves that all operating zones of the WBF are of non-linear sub-systems. The recorded data from Iran Alloy Steel factory is utilized for identification and evaluation of the proposed neuro-fuzzy predictive models of the WBF process.