Machine Learning Regression Models Research Articles

Metabolic Response to Small Molecule Therapy in Colorectal Cancer Tracked with Raman Spectroscopy and Metabolomics

AbstractDespite numerous screening tools for colorectal cancer (CRC), 25 % of patients are diagnosed with advanced disease. Novel diagnostic technologies that are early, accurate, and rapid are imperative to assess the therapeutic efficacy of clinical drugs and identify new biomarkers of treatment response. Here Raman spectroscopy (RS) was used to track metabolic reprogramming in KRAS‐mutant HCT116 and SW837 cells, and KRAS wild‐type CC cells. RS combined with multivariate analysis methods distinguished nonresponsive, partially responsive, and responsive cells treated with cetuximab, a monoclonal antibody for EGFR inhibition, sotorasib, a clinically approved KRAS inhibitor, and various doses of trametinib, an inhibitor of the MAPK pathway. Cells treated with a combination of subtoxic doses of trametinib and BKM120, an inhibitor of the PI3K pathway, showed a synergistic response between the two pathways. Using a supervised machine learning regression model, we established a scoring methodology trained to a priori predict therapeutic response to new treatment combinations. RS metabolites were verified with mass spectrometry, and enrichment pathways were identified, including amino acid, purine, and nicotinate and nicotinamide metabolism that differentiated monotherapy from combination therapy. Our approach may ultimately be applicable to patient‐derived primary cells and cultures of patient tumors to predict effective drugs for individualized care.

Gene Age Gap Estimate (GAGE) for major depressive disorder: a penalized biological age model using gene expression.

Recent associations between Major Depressive Disorder (MDD) and measures of premature aging suggest accelerated biological aging as a potential biomarker for MDD susceptibility or MDD as a risk factor for age-related diseases. Statistical and machine learning regression models of biological age have been trained on various sources of high dimensional data to predict chronological age. Residuals or "gaps" between the predicted biological age and chronological age have been used for statistical inference, such as testing whether an increased age gap is associated with a given disease state. Recently, a gene expression-based model of biological age showed a higher age gap for individuals with MDD compared to healthy controls (HC). In the current study, we propose a machine learning approach that simplifies gene selection by using a least absolute shrinkage and selection operator (LASSO) penalty to construct an expression-based Gene Age Gap Estimate (GAGE) model. We construct the LASSO-GAGE (L-GAGE) model in an RNA-Seq study of 78 unmedicated individuals with MDD and 79 HC and then test for accelerated biological aging in MDD. When testing L-GAGE association with MDD, we account for factors such as sex and chronological age to mitigate regression to the mean effects. The L-GAGE shows higher biological aging in MDD subjects than HC, but the elevation is not statistically significant. However, when we dichotomize chronological age, the interaction between MDD status and age is significant in L-GAGE model. This effect remains statistically significant even after adjusting for chronological age and sex. We find cytomegalovirus (CMV) serostatus is associated with elevated L-GAGE. We also investigate feature selection methods Random Forest and nearest neighbor projected distance regression (NPDR) to characterize age related genes, and we find functional enrichment of infectious disease and SARS-COV pathways.

Assessment of machine learning models trained by molecular dynamics simulations results for inferring ethanol adsorption on an aluminium surface

Molecular dynamics (MD) simulations can reduce our need for experimental tests and provide detailed insight into the chemical reactions and binding kinetics. There are two challenges while dealing with MD simulations: one is the time and length scale limitations, and the latter is efficiently processing the massive amount of data resulting from the MD simulations and generating the proper reaction rates. In this work, we evaluated the use of regression machine learning (ML) methods to solve these two challenges by developing a framework for ethanol adsorption on an Aluminium (Al) slab. This framework comprises three main stages: first, an all-atom molecular dynamics model; second, ML regression models; and third, validation and testing. In stage one, the adsorption of ethanol molecules on the Al surface for various temperatures, velocities and concentrations is simulated using the large-scale atomic/molecular massively parallel simulator (LAMMPS) and ReaxFF. The outcome of stage one is utilised for training, testing, and validating the predictive models in stages two and three. We developed and evaluated 28 different ML models for predicting the number of adsorbed molecules over time, including linear regression, support vector machine (SVM), decision trees, ensemble, Gaussian process regression (GPR), neural network (NN) and Bayesian hyper-parameter optimisation models. Based on the results, the Bayesian-based GPR showed the highest accuracy and the lowest training time. The developed model can predict the number of adsorbed molecules for new cases within seconds, while MD simulations take a few weeks. This adsorption rate can then be used in macroscale simulations to tackle the time and length scale limitations. The proposed numerical framework has the potential to be generalised and, therefore, contribute to future low-cost binding reaction estimations, providing a valuable tool for industry and experimentalists.

Developing a predictive model for the maximum power conversion efficiency of inorganic perovskites: A combined approach using density functional theory and machine learning

To further improve the applicability of perovskite materials in photovoltaics, exploring perovskites with appropriate band gaps and enhanced stability is essential. Nevertheless, identifying promising perovskite materials through a perennial trial-and-error approach is both time-consuming and expensive. In this study, we introduce a method that combines machine learning (ML) and density functional theory (DFT) calculations to efficiently screen inorganic perovskite materials for photovoltaic applications. By utilizing 107 experimental data, we built a machine learning regression model capable of predicting the maximum power conversion efficiency (PCE) achieved in experiments. Light Gradient Boosting Machine (Lightgbm) exhibited superior performance with a test set R2 score of 0.89. Simultaneously, another machine learning regression model was trained using 405 data to predict the theoretical maximum PCE. The best-performing model was Extreme Gradient Boosting (Xgboost) with a test set R2 score of 0.93. By integrating these ML models with DFT calculations, we identified three potential inorganic perovskites: CsPdCl3, KGeCl3, and CsCu2Br3. These materials exhibit direct bandgaps of 1.47 eV, 1.37 eV, and 1.65 eV respectively, along with high thermal stability and favorable optical properties. This method constructs an experimental-theoretical-data driven framework for the prediction of inorganic perovskites, effectively reducing the research cycle in perovskite photovoltaics.

A generalized machine learning approach for cost-effective monitoring of irrigation suitability: A demonstration case in El Fahs aquifer (Tunisia)

This study develops and evaluates the performance of machine learning (ML) regression models, namely the Support Vector Regression (SVR), Random Forest (RF), k-Nearest Neighbors (kNN) and eXtreme Gradient Boosting (XgBoost), for estimating the Irrigation Water Quality Index (IWQI) based on groundwater samples collected from El Fahs aquifer in Tunisia. The groundwater data are used as predictors for training and testing the machine learning models. Results indicate that sodium concentration has the most influence on the IWQI estimations for all ML models. The best-performing algorithms are found to be the SVR and XgBoost. Different datasets are also collected from existing studies, and merged to generate a single dataset that is used to develop generalized machine learning models. Despite regional variations, the generalized models performed adequately when tested against the unseen data, particularly the data collected from El Fahs case site. This work highlights the potential of using ML models, together with established metrics, as screening tools for predicting and classifying the irrigation suitability of groundwater based on available literature, even when metrics are not universally applicable. The findings of this study can be used by the water authorities in data scarce environments as cost-effective monitoring tools to assess water suitability for irrigation purposes.

Retrieval performance of mangrove tree heights using multiple machine learning regression models and UAV-LiDAR point clouds

ABSTRACT Mangroves are vital coastal ecosystems that provide crucial links between land and sea. Tree height is a key indicator for assessing mangroves’ health status. Currently, there are still numerous challenges in estimating mangrove tree height. In this study, multiple deep learning and shallow machine learning regression models were developed to accurately estimate mangrove tree height using multi-dimensional Light Detection and Ranging (LiDAR) point clouds and their derivatives. We constructed a novel CNN_RepMLP model for mangrove tree height mapping. We also further verified the applicability of different types of regression models in estimating mangrove tree heights, and explored the influence of different LiDAR-derived features on the inversion accuracy for tree heights. The results indicated the following: (1) The CNN_RepMLP displayed satisfactory performance in the inversion of mangrove tree heights, and exhibited better robustness and generalization ability than the convolutional neural network (CNN) model. (2) Among the different LiDAR-derived feature combinations, combining height variables with intensity variables can not only mitigate the negative impact of height variables on inversion models, but also enhance the mangrove tree height inversion accuracy. (3) The ensemble learning framework with ExtraTrees as the meta-model can make better use of the differences and complementarities between different single base models, and exhibited better accuracy in estimating the height of mangrove trees compared with other ensemble learning models. (4) Multiple machine learning models based on multi-dimensional UAV-LiDAR point-cloud-derived features are suitable for the inversion of mangrove tree height. The CNN_RepMLP model outperformed the CNN and stacking ensemble learning models and had more detailed differentiation in terms of mangrove tree height. Its prediction results can more realistically reflect the spatial differentiation characteristics of mangrove tree height.

Tracking the Dynamics and Uncertainties of Soil Organic Carbon in Agricultural Soils Based on a Novel Robust Meta-Model Framework Using Multisource Data

Monitoring and estimating spatially resolved changes in soil organic carbon (SOC) stocks are necessary for supporting national and international policies aimed at assisting land degradation neutrality and climate change mitigation, improving soil fertility and food production, maintaining water quality, and enhancing renewable energy and ecosystem services. In this work, we report on the development and application of a data-driven, quantile regression machine learning model to estimate and predict annual SOC stocks at plow depth under the variability of climate. The model enables the analysis of SOC content levels and respective probabilities of their occurrence as a function of exogenous parameters such as monthly temperature and precipitation and endogenous, decision-dependent parameters, which can be altered by land use practices. The estimated quantiles and their trends indicate the uncertainty ranges and the respective likelihoods of plausible SOC content. The model can be used as a reduced-form scenario generator of stochastic SOC scenarios. It can be integrated as a submodel in Integrated Assessment models with detailed land use sectors such as GLOBIOM to analyze costs and find optimal land management practices to sequester SOC and fulfill food–water–energy–-environmental NEXUS security goals.

Forecasting Stock Market Performance: An Ensemble Learning-Based Approach

There is tremendous value in the ability to predict stock market trends and outcomes. The public sentiment surrounding a stock is unquestionably a vital factor contributing to the rise or fall of a stock price. This paper aims to detail how data from public sentiment can be integrated into traditional stock analyses and how these analyses can then be used to make predictions of stock price trends. Headlines from seven news publications and conversations from Yahoo! Finance’s conversations forum were processed by the Valence Aware Dictionary and sEntiment Reasoner (VADER) natural language processing package to determine numerical polarities that represent a positive, negative, or neutral public sentiment around a stock ticker. The resulting polarities were paired with popular stock-table metrics to create a dataset for a Logistic Regression machine learning model. The model was trained on approximately 4400 major stocks to determine a binary “Buy” (1) or “Not Buy” (0) recommendation for each stock. We present approaches for a variety of machine learning models – Logistic Regression, Random Forest Classifiers, and Extreme Gradient Boosting models and do a statistical comparison on the balanced accuracies of each machine learning model. This paper aims to detail our understanding of how one can leverage the public sentiment of a stock to gauge its future value.

Estimation of Effective Length of Type-A Grounding System According to IEC 62305-3 Using a Machine Learning Regression Model

Estimation of Effective Length of Type-A Grounding System According to IEC 62305-3 Using a Machine Learning Regression Model

A practical applications guide to machine learning regression models in psychology with Python

This guide presents a detailed overview of the most used machine learning (ML) techniques for psychologists who may not be familiar with advanced statistical methods, algorithms, or programming. Recognizing the growing interest in using data-driven approaches within psychological research, this guide describes applying ML techniques to investigate complex psychological phenomena. The paper covers the spectrum of algorithms, including decision trees, random forests, gradient boosting, stochastic gradient boosting, and XGBoost, highlighting their concepts and practical applications in psychology. Aiming to bridge the gap between theoretical understanding and practical performance, this paper offers step-by-step instructions on data preprocessing, correlation exploration, feature selection, and model evaluation within the Python programming environment. Readers are offered the necessary tools to apply ML in their research through explanations, examples, and visualization.

Part-scale microstructure prediction for laser powder bed fusion Ti-6Al-4V using a hybrid mechanistic and machine learning model

Laser powder bed fusion (LPBF) Ti-6Al-4V is widely studied for use in structural applications in aerospace and medical industries, but mechanical anisotropy and microstructural inhomogeneity prohibits its wider adoption. Although successful microstructure prediction models have been developed, a remaining challenge is their limited integration across length/time scales and validation by experimental studies. This work proposes a physics-augmented machine learning surrogate model to unite predictions of LPBF temperature, β phase morphology and texture, and α/α’ formation into a single framework that is calibrated and validated with experiments. First, a phase field (PF) model of the martensitic β→α’ transformation is developed and calibrated using data from in-situ synchrotron cyclic heating/cooling studies quantifying the variation of α phase fraction with time. In parallel, an established finite difference-Monte Carlo (FDMC) model predicts the part-scale temperature profile and β grain formation during solidification. A dataset is developed using LPBF cyclic temperature descriptors from the FDMC model as inputs and corresponding α/α’ phase fraction and width from the PF model as outputs. Five machine learning (ML) regression models are tested and optimized, having mean absolute error in testing ≤ 4 %, and the k-nearest neighbors (KNN) model is selected as the best performing. The KNN model is called at the nodal level during post-processing of the FDMC model to replace and downscale the response of the PF model. The combined agility and accuracy of the hybrid FDMC-ML model enables part-scale microstructure predictions that can be further used for property predictions to accelerate AM process optimization.

Quantitative CT Imaging Features Associated with Stable PRISm using Machine Learning

Rationale and ObjectivesThe structural lung features that characterize individuals with preserved ratio impaired spirometry (PRISm) that remain stable overtime are unknown. The objective of this study was to use machine learning models with computed tomography (CT) imaging to classify stable PRISm from stable controls and stable COPD and identify discriminative features. Materials and MethodsA total of 596 participants that did not transition between control, PRISm and COPD groups at baseline and 3-year follow-up were evaluated: n=274 with normal lung function (stable control), n=22 stable PRISm, and n=300 stable COPD. Investigated features included: quantitative CT (QCT) features (n=34), such as total lung volume (%TLCCT) and percentage of ground glass and reticulation (%GG+Reticulationtexture), as well as Radiomic (n=102) features, including varied intensity zone distribution grainy texture (GLDZMZDV). Logistic regression machine learning models were trained using various feature combinations (Base, Base+QCT, Base+Radiomic, Base+QCT+Radiomic). Model performances were evaluated using area under receiver operator curve (AUC) and comparisons between models were made using DeLong test; feature importance was ranked using Shapley Additive Explanations values. ResultsMachine learning models for all feature combinations achieved AUCs between 0.63-0.84 for stable PRISm vs. stable control, and 0.65-0.92 for stable PRISm vs. stable COPD classification. Models incorporating imaging features outperformed those trained solely on base features (p>0.05). Compared to stable control and COPD, those with stable PRISm exhibited decreased %TLCCT and increased %GG+Reticulationtexture and GLDZMZDV. ConclusionThese findings suggest that reduced lung volumes, and elevated high-density and ground glass/reticulation patterns on CT imaging are associated with stable PRISm.

A Computation Model to Estimate Interaction Intensity through Non-Verbal Behavioral Cues: A Case Study of Intimate Couples under the Impact of Acute Alcohol Consumption

This work introduced a novel analysis method to estimate interaction intensity, i.e., the level of positivity/negativity of interaction, for intimate couples (married and heterosexual) under the impact of alcohol, which has great influences on behavioral health. Non-verbal behaviors are critical in interpersonal interactions. However, whether computer vision-detected non-verbal behaviors can effectively estimate the interaction intensity of intimate couples is still unexplored. In this work, we proposed novel measurements and investigated their feasibility to estimate interaction intensities through machine learning regression models. Analyses were conducted based on a conflict-resolution conversation video dataset of intimate couples before and after acute alcohol consumption. Results showed the estimation error was at the lowest in the no-alcohol state but significantly increased if the model trained using no-alcohol data was applied to after-alcohol data, indicating that alcohol altered the interaction data in the feature space. While training a model using rich after-alcohol data is ideal to address the performance decrease, data collection in such a risky state is challenging in real life. Thus, we proposed a new State-Induced Domain Adaptation (SIDA) framework, which allows for improving estimation performance using only a small after-alcohol training dataset, pointing to a future direction of addressing data scarcity issues.

Predict progression free survival and overall survival using objective response rate for anti—PD1/PDL1 therapy development

In oncology anti—PD1 / PDL1 therapy development for solid tumors, objective response rate (ORR) is commonly used clinical endpoint for early phase study decision making, while progression free survival (PFS) and overall survival (OS) are widely used for late phase study decision making. Developing predictive models to late phase outcomes such as median PFS (mPFS) and median OS (mOS) based on early phase clinical outcome ORR could inform late phase study design optimization and probability of success (POS) evaluation. In existing literature, there are ORR / mPFS / mOS association and surrogacy investigations with limited number of included clinical trials. In this paper, without establishing surrogacy, we attempt to predict late phase survival (mPFS and mOS) based on early efficacy ORR and optimize late phase trial design for anti—PD1 / PDL1 therapy development. In order to include adequate number of eligible clinical trials, we built a comprehensive quantitative clinical trial landscape database (QLD) by combining information from different sources such as clinicaltrial.gov, publications, company press releases for relevant indications and therapies. We developed a generalizable algorithm to systematically extract structured data for scientific accuracy and completeness. Finally, more than 150 late phase clinical trials were identified for ORR / mPFS (ORR / mOS) predictive model development while existing literature included at most 50 trials. A tree-based machine learning regression model has been derived to account for ORR / mPFS (ORR / mOS) relationship heterogeneity across tumor type, stage, line of therapy, treatment class and borrow strength simultaneously when homogeneity persists. The proposed method ensures that the predictive model is robust and have explicit structure for clinical interpretation. Through cross validation, the average predictive mean square error of the proposed model is competitive to random forest and extreme gradient boosting methods and outperforms commonly used additive or interaction linear regression models. An example application of the proposed ORR / mPFS (ORR / mOS) predictive model on late phase trial POS evaluation for anti—PD1 / PDL1 combination therapy was illustrated.

Machine learning-assisted prediction and interpretation of electrochemical corrosion behavior in high-entropy alloys

In this study, machine learning (ML) models were successfully employed to predict the short-term electrochemical corrosion behavior of high-entropy alloys (HEAs) based on their chemical compositions. Considering the vast compositional space of HEAs, which restricts the development of corrosion-resistant HEAs, and the lack of non-destructive methods to qualitatively assess their corrosion resistance, this work represents a significant advancement in the field. The “three-step” method was applied to select the optimal feature set from 38 features, and six ML regression models were trained and compared. The eXtreme Gradient Boosting (XGBoost) and Gradient Boosting Decision Tree (GBDT) algorithms demonstrated the highest predictive accuracy (R2 = 81.02 % and 84.64 %, respectively) among the six algorithms. The model’s robust generalization capabilities were confirmed through validation on an additional dataset. Moreover, the interpretability of the model was enhanced by employing two analysis methods, which revealed that pH as an environmental factor, electronegativity difference and average electronegativity as empirical parameters, and the concentrations of Cr and Cu as compositional parameters have the most significant impact on the corrosion resistance of HEAs. The proposed methodology and framework have the potential to optimize alloy composition, facilitating the design and development of new corrosion-resistant HEAs.

Decoupling stiffness and peak moment via hierarchical snapping structures designed with machine learning

This work presents the hierarchical tape spring that uses vein topologies to enable novel structural properties that are essential for architected materials with snapping and phase transition capabilities. Such novel structural properties include bending-sense-independent snap-through with energy dissipation, and the decoupling of initial bending stiffness from peak moment. By leveraging an FEA-generated dataset of mechanical performance metrics, we employed a data-driven design approach from literature involving the use of a quadratic regression machine learning model to predict the mechanical behavior of various hierarchical tape springs. The machine learning model was then used as an objective function in a genetic algorithm that searched for hierarchical tape spring designs with user-specified combinations of peak moment and bending stiffness. Designs discovered with the genetic algorithm were validated with FEA, the results of which revealed that the decoupling of bending stiffness and peak moment was made possible by moving transverse veins along the length of hierarchical tape springs with longitudinal veins. If the transverse veins were close to the center line, the tape generally had a higher peak moment, if there were multiple transverse veins near the center line, the hierarchical tape would have a lower bending stiffness with a comparable peak moment. Finally, to prove that the hierarchical tape springs could be fabricated and tested, three designs were chosen and fabricated with a custom vacuum forming rig and tested via four-point bending. Experimental trends were similar to those observed from finite element results.

Probabilistic ductile deformation limit state prediction of monolithic exterior shear keys based on quantile regression machine learning techniques

Shear strength and ductile deformation capacity of monolithic exterior shear keys play important roles on the seismic behavior of highway bridges. Improving the ductility of the shear key through reasonable design could effectively restrict superstructure displacement and minimize damage to pier columns. However, most existing studies primarily focus on evaluating the shear strength and damage mechanisms of shear keys, with limited attention given to analytical models for assessing ductile deformation capacity. To this end, this study proposes a method that combines quantile regression and machine learning techniques to predict the ductile deformation limit states of shear keys. The proposed numerical modeling method for shear keys is validated using the experimental data on lateral load-displacement curves. On this basis, Latin hypercubic sampling and joint simulation are utilized to establish the ductile deformation limit state database. The ductile deformation limit states of shear keys are characterized by their lateral displacements in three distinct damage states. The main influencing factors of each limit states are investigated by correlation analysis. Three quantile regression machine learning models are trained and constructed to conduct point and interval prediction for the ductile deformation limit states. All three models, particularly the Gaussian process quantile regression model, are proven to demonstrate exceptional prediction accuracy and uncertainty quantification. Experimental data validation confirms that these models outperform the prediction accuracy of existing simplified models.

Machine learning mapping of lattice correlated data

We discuss a machine learning (ML) regression model to reduce the computational cost of disconnected diagrams in lattice QCD calculations. This method creates a mapping between the results of fermionic loops computed at different quark masses and flow times. The ML mapping, trained with just a small fraction of the complete data set, makes use of translational invariance and provides consistent result with comparable uncertainties over the calculation done over the whole ensemble, resulting in a significant computational gain.

Quantifying massively parallel microbial growth with spatially mediated interactions.

Quantitative understanding of microbial growth is an essential prerequisite for successful control of pathogens as well as various biotechnology applications. Even though the growth of cell populations has been extensively studied, microbial growth remains poorly characterised at the spatial level. Indeed, even isogenic populations growing at different locations on solid growth medium typically show significant location-dependent variability in growth. Here we show that this variability can be attributed to the initial physiological states of the populations, the interplay between populations interacting with their local environment and the diffusion of nutrients and energy sources coupling the environments. We further show how the causes of this variability change throughout the growth of a population. We use a dual approach, first applying machine learning regression models to discover that location dominates growth variability at specific times, and, in parallel, developing explicit population growth models to describe this spatial effect. In particular, treating nutrient and energy source concentration as a latent variable allows us to develop a mechanistic resource consumer model that captures growth variability across the shared environment. As a consequence, we are able to determine intrinsic growth parameters for each local population, removing confounders common to location-dependent variability in growth. Importantly, our explicit low-parametric model for the environment paves the way for massively parallel experimentation with configurable spatial niches for testing specific eco-evolutionary hypotheses.

Classification and regression machine learning models for predicting the combined toxicity and interactions of antibiotics and fungicides mixtures

Antibiotics and triazole fungicides coexist in varying concentrations in natural aquatic environments, resulting in complex mixtures. These mixtures can potentially affect aquatic ecosystems. Accurately distinguishing synergistic and antagonistic mixtures and predicting mixture toxicity are crucial for effective mixture risk assessment. We tested the toxicities of 75 binary mixtures of antibiotics and fungicides against Auxenochlorella pyrenoidosa. Both regression and classification models for these mixtures were developed using machine learning models: random forest (RF), k-nearest neighbors (KNN), and kernel k-nearest neighbors (KKNN). The KKNN model emerged as the best regression model with high values of determination coefficient (R2 = 0.977), explained variance in prediction leave-one-out (Q2LOO = 0.894), and explained variance in external prediction (Q2F1 = 0.929, Q2F2 = 0.929, and Q2F3 = 0.923). The RF model, the leading classifier, exhibited high accuracy (accuracy = 1 for the training set and 0.905 for the test set) in distinguishing the synergistic and antagonistic mixtures. These results provide crucial value for the risk assessment of mixtures.