Linear Model Tree Research Articles

Participation in Social and Community Life Before and After Spinal Cord Injury/Disease: Factors Influencing Changes in Short-Term and Long-Term Participation.

Examining changes in participation frequency (productive, leisure, and social activities) from pre-spinal cord injury/disorder to at least 2 yrs post-spinal cord injury/disorder and identifying sociodemographic and spinal cord injury/disorder characteristics associated with significant shifts in participation frequency. The study used a longitudinal design, using data from the Swiss Spinal Cord Injury Cohort study. Pre-spinal cord injury/disorder participation frequency was assessed retrospectively 12 wks after spinal cord injury/disorder and prospectively 1 and at least 2 yrs after spinal cord injury/disorder. Linear mixed-effects model trees were used to identify subgroups with participation changes and related sociodemographic and spinal cord injury/disorder characteristics. The study involved 550 individuals (median age at spinal cord injury/disorder onset: 53 yrs, 30% female, 63.9% with traumatic etiology, and 5.6 yrs since onset). Pronounced decrease was observed prominently in productive activities. Education and age at spinal cord injury/disorder onset served as initial variables to split the tree at first level for each of the participation dimensions. This research identified participation dimensions most susceptible to changes during the initial years after spinal cord injury/disorder and pinpointed subgroups displaying clinically meaningful longitudinal variations across productive, leisure, and social activities. These findings have the potential to enhance the efficiency of rehabilitation programs, leading to improvements in long-term participation levels for individuals with spinal cord injury/disorder.

Fast linear model trees by PILOT

Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relationships, which is hard for standard decision trees. But most existing methods for fitting linear model trees are time consuming and therefore not scalable to large data sets. In addition, they are more prone to overfitting and extrapolation issues than standard regression trees. In this paper we introduce PILOT, a new algorithm for linear model trees that is fast, regularized, stable and interpretable. PILOT trains in a greedy fashion like classic regression trees, but incorporates an L2 boosting approach and a model selection rule for fitting linear models in the nodes. The abbreviation PILOT stands for PIecewise Linear Organic Tree, where ‘organic’ refers to the fact that no pruning is carried out. PILOT has the same low time and space complexity as CART without its pruning. An empirical study indicates that PILOT tends to outperform standard decision trees and other linear model trees on a variety of data sets. Moreover, we prove its consistency in an additive model setting under weak assumptions. When the data is generated by a linear model, the convergence rate is polynomial.

Caseload factors predictive of family abuse and neglect treatment outcomes

BackgroundIn child welfare, caseloads are frequently far higher than optimal. Not all cases are created equal; however, little is known about which combination and interaction of factors make caseloads more challenging and impact child and family outcomes. ObjectiveThis study aims to identify which case, provider, and organizational factors most strongly differentiate between families with favorable and less-than-positive treatment outcomes. Participants and settingParticipants were 25 family advocacy program providers and 17 supervisors at 11 Department of the Air Force installations. MethodsFollowing informed consent, participants completed demographic and caseload questionnaires, and we collected information about organizational factors. Providers were sent a weekly case update and burnout questionnaire for seven months. We used linear mixed-effects model tree (LMM tree) algorithms to determine the provider, client, and organizational characteristics that best distinguish between favorable vs. unfavorable outcomes. ResultsThe LMM tree predicting provider-rated treatment success yielded three significant partitioning variables: (a) commander involvement, (b) case complexity, and (c) % of clients in a high-risk field. The LMM predicting client-rated treatment progress yielded seven significant partitioning variables: (a) command involvement; (b) ease of reaching tenant unit command; (c) # of high-risk cases; (d) % of clients receiving Alcohol and Drug Abuse Prevention and Treatment services; (e) ease of reaching command; (f) % of clients with legal involvement; (g) provider age. ConclusionsThis study is a first step toward developing a dynamic caseload management tool. An intelligent, algorithm-informed approach to case assignment could help child welfare agencies operate in their typically resource-scarce contexts in a manner that improves outcomes.

A machine learning based prediction of reaction parameters on reaction kinetics for treatment of industrial wastewater

Industrial wastewater is a major cause of pollution of surface water bodies since it is often discharged into these water bodies without adequate treatment. Past attempts at treating industrial wastewater have led to the emergence of multiple approaches for the removal of impurities. These include conventional techniques like biological, physical, and chemical techniques, and recently advanced oxidation processes that have, under certain specific conditions, produced desired results through decomposing contaminating organic compounds into water, carbon dioxide, or less harmful molecules. This study examines the removal of phenol in industrial wastewater using a Photo-Fenton reagent on account of its cost-effectiveness and the possibility of rapid results. The effectiveness of the removal of contaminants was assessed based on the rate of reaction (k1). Data collected through laboratory experiments was then put through various Machine Learning classifiers including Neural Network, CHAID, Generalized Linear Model, Linear Classifier, and Random Trees using IBM SPSS Modeler and models trained. An accuracy of 97.2 % was observed for the Deep Neural Network. Further, it is observed that for a specific level of impurities in industrial water, the pH of solution, intensity of light, and amount of Hydrogen Peroxide (H2O2) are the most important factors in predicting the rate of reaction with concentration of [Fe3+] ions and catalyst TiO2 playing smaller roles.

Subgroup detection in linear growth curve models with generalized linear mixed model (GLMM) trees.

Growth curve models are popular tools for studying the development of a response variable within subjects over time. Heterogeneity between subjects is common in such models, and researchers are typically interested in explaining or predicting this heterogeneity. We show how generalized linear mixed-effects model (GLMM) trees can be used to identify subgroups with different trajectories in linear growth curve models. Originally developed for clustered cross-sectional data, GLMM trees are extended here to longitudinal data. The resulting extended GLMM trees are directly applicable to growth curve models as an important special case. In simulated and real-world data, we assess performance of the extensions and compare against other partitioning methods for growth curve models. Extended GLMM trees perform more accurately than the original algorithm and LongCART, and similarly accurate compared to structural equation model (SEM) trees. In addition, GLMM trees allow for modeling both discrete and continuous time series, are less sensitive to (mis-)specification of the random-effects structure and are much faster to compute.

Regional Low Flow Hydrology: Model Development and Evaluation

AbstractLow flows result from the interplay of climatic variability and catchment storage dynamics, but it is unclear which of these variables is more relevant for explaining low flow spatial patterns. Here, we develop a new conceptual model that integrates process‐based hydrological knowledge with statistics and test it for 1,400 Brazilian catchments. Through comparative hydrology, we isolate the low flow generating mechanisms and estimate their components using linear model trees. The model explains 58% of the spatial variance in 7‐day minimum annual flows (Qmin) based on climate and catchment characteristics. The primary Qmin controls depend on the spatial scale of analysis. Catchment characteristics govern Qmin up to the continental scale (107 km2), where their relative importance matches that of climate. At subcontinental scales, catchment characteristics are twice as important as climate in predicting Qmin, suggesting that low flows are governed by the catchment's capacity to attenuate the climatic variability through water storage. Geological properties are the most important catchment characteristics, particularly bedrock type, lithology and topographic slope, determining streamflow recession rates in the dry season. Soil properties, primarily soil class and depth, are half as important as geology. Climate impacts Qmin mainly through mean annual rainfall minus evaporation, representing the potential groundwater recharge, while dry‐season length has the lowest impact. These results hold mainly for highly seasonal and snow‐free climates. Low flow hydrology that combines statistics with process understanding offers a promising framework for understanding regional low flow generating mechanisms and could support other estimation models than that presented here.

Machine learning-based integrated identification of predictive combined diagnostic biomarkers for endometriosis

Background: Endometriosis (EM) is a common gynecological condition in women of reproductive age, with diverse causes and a not yet fully understood pathogenesis. Traditional diagnostics rely on single diagnostic biomarkers and does not integrate a variety of different biomarkers. This study introduces multiple machine learning techniques, enhancing the accuracy of predictive models. A novel diagnostic approach that combines various biomarkers provides a new clinical perspective for improving the diagnostic efficiency of endometriosis, holding significant potential for clinical application.Methods: In this study, GSE51981 was used as a test set, and 11 machine learning algorithms (Lasso, Stepglm, glmBoost, Support Vector Machine, Ridge, Enet, plsRglm, Random Forest, LDA, XGBoost, and NaiveBayes) were employed to construct 113 predictive models for endometriosis. The optimal model was determined based on the AUC values derived from various algorithms. These genes were then evaluated using nine machine learning algorithms (Random Forest, SVM, Gradient Boosting Machine, LASSO, XGB, NNET, Generalized Linear Model, KNN, and Decision Tree) to assess significance scores and identify diagnostic genes for each algorithm. The diagnostic value of these genes was further validated in external datasets from GSE7305, GSE11691, and GSE120103.Results: Analysis of the GSE51981 dataset revealed 62 DEGs. The Stepglm [Both] and plsRglm algorithms identified 30 genes with the most potential using the AUC evaluation. Subsequently, nine machine learning algorithms were applied to select diagnostic genes, leading to the identification of five key diagnostic genes using the LASSO algorithm. The ADAT1 gene exhibited the best single-gene predictive performance, with an AUC of 0.785. A combination of genes (FOS, EPHX1, DLGAP5, PCSK5, and ADAT1) achieves an AUC of 0.836 in the test dataset. Moreover, these genes consistently exhibited an AUC exceeding 0.78 in all validation datasets, demonstrating superior predictive performance. Furthermore, correlation analysis with immune infiltration strengthened their predictive value by demonstrating the close relationship of the diagnostic genes with immune infiltrating cells.Conclusion: A combination of biomarkers consisting of FOS, EPHX1, DLGAP5, PCSK5, and ADAT1 can serve as a diagnostic tool for endometriosis, enhancing diagnostic efficiency. The association of these genes with immune infiltrating cells reveals their potential role in the pathogenesis of endometriosis, providing new insights for early detection and treatment.

Implementing dynamic subset sensitivity analysis for early design datasets

Engaging with performance feedback in early building design often involves building a custom parametric model and generating large datasets, which is not always feasible. Alternatively, large parametric datasets of general design problems and filtering methods could be used together to explore specific design decisions. This paper investigates the generalizability of a method that dynamically assesses variable importance and likely influence on performance objectives as a precomputed design space is filtered down. The method first trains linear model trees to predict building performance objectives across a generic design space. Leaf node models are then aggregated to provide feedback on variable importance in different design space regions. This approach is tested on three design problems that vary in number of variables, samples, and design space structure to reveal advantages and potential limitations of the method. Algorithm improvements are proposed, and general recommendations are developed to apply it on future datasets.

System modeling and temperature control for a fuel cell system based on local model networks

ABSTRACT Fuel cells have been studied for use in stationary power generation and vehicle propulsion systems. The fuel cell thermal management subsystem is coupled and nonlinear, posing challenges for modeling and temperature control. This paper aims to integrate the physical models of the fuel cell stack, pump, thermostat, and other components combined with intelligent algorithms into an efficient system-level thermal management model framework and develop a model predictive controller to solve the temperature control problem. First, a physics-based nonlinear model of the fuel cell system is developed and used as a basis to identify the linearized model for different operating points. Then, the global model is obtained by fusing the local models with Gaussian validity functions using the local linear model tree method. Third, a multi-step prediction model is derived based on the local model networks, and a parameterized linear state space form is obtained and used for controller design. Furthermore, an online correction method is developed to reduce the model discrepancy. Finally, the accuracy of the system model and the performance of the proposed controller are verified by open-loop experimental data and a series of closed-loop simulation cases.

Combination of the electronic nose with microbiology as a tool for rapid detection of Salmonella

Salmonella is one of the most important foodborne pathogens and its analysis in raw and processed products is mandatory in the food industry. Although microbiological analysis is the standard practice for Salmonella determination, these assays are commonly laborious and time-consuming, thus, alternative techniques based on easy operation, few manipulation steps, low cost, and reduced time are desirable. In this paper, we demonstrate the use of an e-nose based on ionogel composites (ionic liquid + gelatine + Fe3O4 particles) as a complementary tool for the conventional microbiological detection of Salmonella. We used the proposed methodology for differentiating Salmonella from Escherichia coli, Pseudomonas fluorescens, Pseudomonas aeruginosa, and Staphylococcus aureus in nonselective medium: pre-enrichment in brain heart infusion (BHI) (incubation at 35 °C, 24 h) and enrichment in tryptone soy agar (TSA) (incubation at 35 °C, 24 h), whereas Salmonella differentiation from E. coli and P. fluorescens was also evaluated in selective media, bismuth sulfite agar (BSA), xylose lysine deoxycholate agar (XLD), and brilliant green agar (BGA) (incubation at 35 °C, 24 h). The obtained data were compared by principal component analysis (PCA) and different machine learning algorithms: multilayer perceptron (MLP), linear discriminant analysis (LDA), instance-based (IBk), and Logistic Model Trees (LMT). For the nonselective media, under optimized conditions, taking merged data of BHI + TSA (total incubation time of 48 h), an accuracy of 85% was obtained with MLP, LDA, and LMT, while five separated clusters were presented in PCA, each cluster corresponding to a bacterium. In addition, for evaluation of the e-nose for discrimination of Salmonella using selective media, considering the combination of BSA + XLD and total incubation of 72 h, the PCA showed three separated and well-defined clusters corresponding to Salmonella, E. coli, and P. fluorescens, and an accuracy of 100% was obtained for all classifiers.

Effects of accelerated biological age on depressive symptoms in a causal reasoning framework

BackgroundDepression in middle-aged and elderly individuals is multifaceted and heterogeneous, linked to biological age (BA) based on aging-related biomarkers. However, due to confounding with chronological age and the absence of subgroup analysis and causal reasoning, the association between BA and depressive symptoms (DS) might be unstable and requires further investigation. MethodsWe utilized data from the China Health and Retirement Longitudinal Study (N = 9478) to perform association analysis, causal inference, and subgroup analysis. BA acceleration (BAA) was derived using machine learning and adjusted for chronological age. A generalized linear mixed-effects model (GLMM) tree algorithm was employed to identify subgroups. The causal reasoning frame included propensity score matching and fast large-scale almost matching exactly. ResultsIn the longitudinal analysis, BAA exhibited a consistent and significant positive association with DS, even after controlling for demographic characteristics, lifestyle factors, health status, and physical functions. This association remained unchanged within the causal framework. GLMM tree analysis identified three partitioning variables (sex, satisfaction, and BMI) and five subgroups. Further subgroup analysis revealed that BAA exerted the strongest effect on DS among women with less satisfying lives. LimitationsDepressive symptoms were evaluated through scale measurements rather than clinical diagnosis. The sample was derived from the general population, not the clinically depressed population. ConclusionsThis study provided the first longitudinal evidence that biological age acceleration increases depressive symptoms under causal reasoning and subgroup analysis, particularly among less satisfied women. And the association between BAA and DS was independent of known risk factors.

Predicting fish species richness and abundance in the Lower Mekong Basin

Predictive models are widely used to investigate relationships between the distribution of fish diversity, abundance, and the environmental conditions in which they inhabit, and can guide management actions and conservation policies. Generally, the framework to model such relationships is established; however, which models perform best in predicting fish diversity and abundance remain unexplored in the Mekong River Basin. Here, we evaluated the performance of six single statistical models namely Generalized Linear Model, Classification and Regression Tree, Artificial Neural Network, k-Nearest Neighbor, Support Vector Machine and Random Forest in predicting fish species richness and abundance in the Lower Mekong Basin. We also identified key variables explaining variability and assessed the variable’s sensitivity in prediction of richness and abundance. Moreover, we explored the usefulness of an ensemble modeling approach and investigated if this approach improved model performance. Our results indicated that, overall, the six single statistical models successfully predicted the fish species richness and abundance using 14 geo-hydrological, physicochemical and climatic variables. The Random Forest model consistently out-performed all single statistical models for predicting richness (R2 = 0.85) and abundance (R2 = 0.77); whereas, Generalized Linear Model performed the worst of all models (R2 = 0.60 and 0.56 for richness and abundance). The most important predictors of variation in both richness and abundance included water level, distance from the sea and alkalinity. Additionally, dissolved oxygen, water temperature and total nitrate were important predictors of species richness, while conductivity was important for fish abundance. We found that species richness increased with increasing water level, dissolved oxygen and water temperature, but decreased with increasing distance from the sea, alkalinity and total nitrate. Fish abundance increased with conductivity, but decreased with increasing distance from the sea, water level and alkalinity. Finally, our results highlighted the usefulness of ensemble modeling (R2 = 0.90 and 0.85 for richness and abundance) for providing better predictive power than any of the six single statistical models. Our results can be used to support Mekong River management, particularly fisheries in the context of contemporary regional and global changes.

Structurally Optimized Neural Fuzzy Modeling for Model Predictive Control

This paper investigates the local linear model tree (LOLIMOT), a typical neural fuzzy model, in the multiple-input-multiple-output model predictive control (MPC). In the conventional LOLIMOT, the structural parameters including centres and variances of its Gaussian kernels are set based on equally dividing the input data space. In this paper, after the structural parameters are initially obtained from the input space partition, they are optimized by the gradient descent search, from which the space partitions are further adjusted. This makes it better for the model structure to fit the input data statistics, leading to improved modelling performance with small model size. The MPC based on the proposed structurally optimized LOLIMOT is then implemented and verified with both numerical and diesel engine plants. Validation results show that the proposed MPC has significantly better controlling performance than the MPC based on the conventional LOLIMOT, making it an attractive solution in practice.

Control oriented modeling of twin-screw granulation in the ConsiGmaTM-25 production plant

ConsiGmaTM-25 is a continuous production plant integrating a twin-screw granulation, fluid bed drying, granule conditioning, and a tableting unit. The particle size distribution (PSD), active pharmaceutical ingredient (API) content, and liquid content of wet granules after twin-screw granulation affect the quality of intermediate and final products. This paper proposes methods for real-time monitoring of these quantities and control-oriented modeling of the granulator.The PSD of wet granules is monitored via an in-line process analytical technology (PAT) probe based on the spatial velocimetry principle. The algorithm for signal processing and evaluation of PSD characteristics is developed and applied to the acquired PSD data. A dynamic process model predicting PSD characteristics from granulation parameters is trained via the local linear model tree (LoLiMoT) approach. The experimental data required for the model training are collected via systematically designed excitation runs. Finally, the performance of the identified model is examined and verified by means of a new set of validation runs. Furthermore, an in-line PAT probe based on Raman spectroscopy is developed and integrated after the granulator. The API- and liquid content of produced wet granules are evaluated from the spectral data by means of chemometric modeling, and chemometric models are validated on a separate set of experimental data. The solutions proposed in this research can be used as a reliable (and necessary) basis for the development of advanced quality-by-design control concepts (e.g., PSD process control). Such concepts would ultimately improve the ConsiGmaTM-25 process performance in terms of robustness against disturbances and quality of intermediate and final products.

Data Analysis and Research Based on Second-hand Sailboats

With the rise of sailing sports, the second-hand sailing trading market shows a great potential for development. We research a thorough research on the pricing of second-hand sailing. Firstly, the linear regression model and decision tree regression model use machine learning algorithm to accurately evaluate the sailing price; secondly, the two-factor analysis method is used to analyze the influence of manufacturer and geographical region on the market price; and finally, the accuracy of the model is evaluated by paired sample T-test and Pearson correlation analysis methods.

Prediction and Classification of User Activities Using Machine Learning Models from Location-Based Social Network Data

The current research has aimed to investigate and develop machine-learning approaches by using the data in the dataset to be applied to classify location-based social network data and predict user activities based on the nature of various locations (such as entertainment). The analysis of user activities and behavior from location-based social network data is often based on venue types, which require the input of data into various categories. This has previously been done through a tedious and time-consuming manual method. Therefore, we proposed a novel approach of using machine-learning models to extract these venue categories. In this study, we used a Weibo dataset as the main source of research and analyzed machine-learning methods for more efficient implementation. We proposed four models based on well-known machine-learning techniques, including the generalized linear model, logistic regression, deep learning, and gradient-boosted trees. We designed, tested, and evaluated these models. We then used various assessment metrics, such as the Receiver Operating Characteristic or Area Under the Curve, Accuracy, Recall, Precision, F-score, and Sensitivity, to show how well these methods performed. We discovered that the proposed machine-learning models are capable of accurately classifying the data, with deep learning outperforming the other models with 99% accuracy, followed by gradient-boosted tree with 98% and 93%, generalized linear model with 90% and 85%, and logistic regression with 86% and 91%, for multiclass distributions and single class predictions, respectively. We classified the data using our machine-learning models into the 10 classes we used in our previous study and predicted tourist destinations among the data to demonstrate the effectiveness of using machine learning for location-based social network data analysis, which is vital for the development of smart city environments in the current technological era.

Machine learning prediction of video-based learning with technology acceptance model

<span lang="EN-US">COVID-19 outbreak has significant impacts on education system as almost all countries shift to new way of teaching and learning; online learning. In this new environment, various innovative teaching methods have been created to deliver educational material in ensuring the learning outcomes such as video content. Thus, this research aims to implement machine learning prediction models for video-based learning in higher education institutions. Using survey data from 103 final year accounting students at Malaysian public university, this paper presents the fundamental frameworks of evaluating three machine learning models namely generalized linear model, random forest and decision tree. Besides demography attributes, the performance of each machine learning algorithm on the video-based learning usage has been observed based on the attributes of technology acceptance model namely perceived ease of use, perceived usefulness and attitude. The findings revealed that the perceived ease of use has given the highest weight of contributions to the generalized linear model and random forest while the major effects in decision tree has been given by the attitude variable. However, generalized linear model outperformed the two algorithms in term of the prediction accuracy.</span>

Real-Time Counterfactual Explanations For Robotic Systems With Multiple Continuous Outputs*

Although many machine learning methods, especially from the field of deep learning, have been instrumental in addressing challenges within robotic applications, we cannot take full advantage of such methods before these can provide performance and safety guarantees. The lack of trust that impedes the use of these methods mainly stems from a lack of human understanding of what exactly machine learning models have learned, and how robust their behaviour is. This is the problem the field of explainable artificial intelligence aims to solve. Based on insights from the social sciences, we know that humans prefer contrastive explanations, i.e. explanations answering the hypothetical question “what if?”. In this paper, we show that linear model trees are capable of producing answers to such questions, so-called counterfactual explanations, for robotic systems, including in the case of multiple, continuous inputs and outputs. We demonstrate the use of this method to produce counterfactual explanations for two robotic applications. Additionally, we explore the issue of infeasibility, which is of particular interest in systems governed by the laws of physics.

Retracted: Estimating Lane Change Duration for Overtaking in Nonlane‐Based Driving Behavior by Local Linear Model Trees (LOLIMOT)

Retracted: Estimating Lane Change Duration for Overtaking in Nonlane‐Based Driving Behavior by Local Linear Model Trees (LOLIMOT)

Control of a complex multistep process for the production of mesalazine

This paper presents the application of Data Driven Modelling (DDM) and Non-Linear Model Predictive Control (NMPC) for the control implementation of a continuous reactor for the production of the Active Pharmaceutical Ingredient (API), Mesalazine. The contribution of this work is to present the overall control architecture, the step-by-step controller design based on DDM, i.e. Neuro-Fuzzy/Local Linear Model Tree Models (LoLiMoT) and NMPC.We demonstrate the advantages of DDM and NMPC, in the presence of non-linear, distributed parameter, multi-variable systems as a suitable, powerful and practical way to approach complex process control challenges. Compared to conventional concepts, the inherent optimization structure of NMPC allows to obtain the desired behaviour of the multi-variable non-linear plant considering the physical constraints of operation regions. The control concept has been implemented in the real system on one central computer, XamControl (Evon),11https://evon-automation.com/. with all measurement devices and actuators centrally interconnected.