Linear Machine Learning Models Research Articles

Predicting Steady-State Metabolic Power in Cerebral Palsy, Stroke, and the Elderly During Walking With and Without Assistive Devices.

Individuals with walking impairment, such as those with cerebral palsy, often face challenges in leading physically active lives due to the high energy cost of movement. Assistive devices like powered exoskeletons aim to alleviate this burden and improve mobility. Traditionally, optimizing the effectiveness of such devices has relied on time-consuming laboratory-based measurements of energy expenditure, which may not be feasible for some patient populations. To address this, our study aimed to enhance the state-of-the-art predictive model for estimating steady-state metabolic rate from 2-min walking trials to include individuals with and without walking disabilities and for a variety of terrains and wearable device conditions. Using over 200 walking trials collected from eight prior exoskeleton-related studies, we trained a simple linear machine learning model to predict metabolic power at steady state based on condition-specific factors, such as whether the trial was conducted on a treadmill (level or incline) or outdoors, as well as demographic information, such as the participant's weight or presence of walking impairment, and 2 minutes of metabolic data. We demonstrated the ability to predict steady-state metabolic rate to within an accuracy of 4.71 ± 2.7% on averageacross allwalking conditionsand patient populations, including with assistive devices andondifferent terrains. This work seeks to unlock the use of in-the-loop optimization of wearable assistive devices in individuals with limited walking capacity. A freely available MATLAB application allows other researchers to easily apply our model.

Hedging global currency risk: A dynamic machine learning approach

We propose a dynamic method to hedge foreign exchange risk of international equity portfolios. The method is based on the currency return forecasts derived from a set of alternative machine learning models, built on the main factor components of the currency return time series. To illustrate our method, we take the perspective of a US portfolio manager investing in a global equity portfolio of developing and developed economies. The analysis of several model performance indicators, back-tested on data from 27 October 2008 to 30 December 2022, allows to conclude that accurate predictions of global currency factor returns, such as those obtained with non linear machine learning models, can improve currency risk hedging of global equity portfolios.

The potential of deep learning to counter the matrix effect for assessment of honey quality and monoflorality

The complexity of biological matrices affects Near-infrared (NIR) signals. Honey, a complex food matrix, changes in chemical and physical properties over time indicating strong matrix effects when capturing NIR spectra. The varied floral sources in most honeys also contribute to this effect. A real dataset of 1656 honey samples spanning eight geographic districts of the North Island of New Zealand captured by NIR hyperspectral camera was analyzed with machine learning and deep learning models to assess mānuka honey quality and identity. One-dimensional neural network (1D-CNN) showed high capacity to accommodate the complexity of the honey matrix, better than did linear machine learning models, particularly for prediction of a trademarked continuous quality score of mānuka honey, known as Unique Mānuka Factor (UMFTM). Monofloral mānuka honey was distinguished well from multi-floral and non-mānuka when classified with 1D-CNN, achieving > 90 % class accuracy, better than conventional classifiers. Interestingly, 1D-CNN models could precisely identify important spectral regions such as Visible (666–725 nm) & NIR (991 and 1435 nm) supporting the evaluation of mānuka honey quality and determining botanical origin as Leptospermum scoparium. Both are useful for future model development and application to automated assessment of honey quality and monoflorality before lumped extraction.

An interpretable ensemble structure with a non-iterative training algorithm to improve the predictive accuracy of healthcare data analysis

The modern development of healthcare is characterized by a set of large volumes of tabular data for monitoring and diagnosing the patient's condition. In addition, modern methods of data engineering allow the synthesizing of a large number of features from an image or signals, which are presented in tabular form. The possibility of high-precision and high-speed processing of such large volumes of medical data requires the use of artificial intelligence tools. A linear machine learning model cannot accurately analyze such data, and traditional bagging, boosting, or stacking ensembles typically require significant computing power and time to implement. In this paper, the authors proposed a method for the analysis of large sets of medical data, based on a designed linear ensemble method with a non-iterative learning algorithm. The basic node of the new ensemble is an extended-input SGTM neural-like structure, which provides high-speed data processing at each level of the ensemble. Increasing prediction accuracy is ensured by dividing the large dataset into parts, the analysis of which is carried out in each node of the ensemble structure and taking into account the output signal from the previous level of the ensemble as an additional attribute on the next one. Such a design of a new ensemble structure provides both a significant increase in the prediction accuracy for large sets of medical data analysis and a significant reduction in the duration of the training procedure. Experimental studies on a large medical dataset, as well as a comparison with existing machine learning methods, confirmed the high efficiency of using the developed ensemble structure when solving the prediction task.

Machine Learning Techniques for Spatio-Temporal Air Pollution Prediction to Drive Sustainable Urban Development in the Era of Energy and Data Transformation

Sustainable urban development in the era of energy and digital transformation is crucial from a societal perspective. Utilizing modern techniques for analyzing large datasets, including machine learning and artificial intelligence, enables a deeper understanding of historical data and the efficient prediction of future events based on data from IoT sensors. This study conducted a multidimensional historical analysis of air pollution to investigate the impacts of energy transformation and environmental policy and to determine the long-term environmental implications of certain actions. Additionally, machine learning (ML) techniques were employed for air pollution prediction, taking spatial factors into account. By utilizing multiple low-cost air sensors categorized as IoT devices, this study incorporated data from various locations and assessed the influence of neighboring sensors on predictions. Different ML approaches were analyzed, including regression models, deep neural networks, and ensemble learning. The possibility of implementing such predictions in publicly accessible IT mobile systems was explored. The research was conducted in Krakow, Poland, a UNESCO-listed city that has had long struggle with air pollution. Krakow is also at the forefront of implementing policies to prohibit the use of solid fuels for heating and establishing clean transport zones. The research showed that population growth within the city does not have a negative impact on PMx concentrations, and transitioning from coal-based to sustainable energy sources emerges as the primary factor in improving air quality, especially for PMx, while the impact of transportation remains less relevant. The best results for predicting rare smog events can be achieved using linear ML models. Implementing actions based on this research can significantly contribute to building a smart city that takes into account the impact of air pollution on quality of life.

A Study on the Predictions of Wave Breaker Index in a Gravel Beach Using Linear Machine Learning Model

To date, numerous empirical formulas have been proposed through hydraulic model experiments to predict the wave breaker index, including wave height and depth of wave breaking, due to the inherent complexity of generation mechanisms. Unfortunately, research on the characteristics of wave breaking and the prediction of the wave breaker index for gravel beaches has been limited. This study aims to forecast the wave breaker index for gravel beaches using representative linear-based machine learning techniques known for their high predictive performance in regression or classification problems across various research fields. Initially, the applicability of existing empirical formulas for wave breaker indices to gravel seabeds was assessed. Various linear-based machine learning algorithms were then employed to build prediction models, aiming to overcome the limitations of existing empirical formulas in predicting wave breaker indices for gravel seabeds. Among the developed machine learning models, a new calculation formula for easily computable wave breaker indices based on the model was proposed, demonstrating high predictive performance for wave height and depth of wave breaking on gravel beaches. The study validated the predictive capabilities of the proposed wave breaker indices through hydraulic model experiments and compared them with existing empirical formulas. Despite its simplicity as a polynomial, the newly proposed empirical formula for wave breaking indices in this study exhibited exceptional predictive performance for gravel beaches.

Machine learning to predict continuous protein properties from binary cell sorting data and map unseen sequence space

Proteins are a diverse class of biomolecules responsible for wide-ranging cellular functions, from catalyzing reactions to recognizing pathogens. The ability to evolve proteins rapidly and inexpensively toward improved properties is a common objective for protein engineers. Powerful high-throughput methods like fluorescent activated cell sorting and next-generation sequencing have dramatically improved directed evolution experiments. However, it is unclear how to best leverage these data to characterize protein fitness landscapes more completely and identify lead candidates. In this work, we develop a simple yet powerful framework to improve protein optimization by predicting continuous protein properties from simple directed evolution experiments using interpretable, linear machine learning models. Importantly, we find that these models, which use data from simple but imprecise experimental estimates of protein fitness, have predictive capabilities that approach more precise but expensive data. Evaluated across five diverse protein engineering tasks, continuous properties are consistently predicted from readily available deep sequencing data, demonstrating that protein fitness space can be reasonably well modeled by linear relationships among sequence mutations. To prospectively test the utility of this approach, we generated a library of stapled peptides and applied the framework to predict affinity and specificity from simple cell sorting data. We then coupled integer linear programming, a method to optimize protein fitness from linear weights, with mutation scores from machine learning to identify variants in unseen sequence space that have improved and co-optimal properties. This approach represents a versatile tool for improved analysis and identification of protein variants across many domains of protein engineering.

Interpretable machine learning for predicting the fate and transport of pentachlorophenol in groundwater

Pentachlorophenol (PCP) is a commonly found recalcitrant and toxic groundwater contaminant that resists degradation, bioaccumulates, and has a potential for long-range environmental transport. Taking proper actions to deal with the pollutant accounting for the life cycle consequences requires a better understanding of its behavior in the subsurface. We recognize the huge potential for enhancing decision-making at contaminated groundwater sites with the arrival of machine learning (ML) techniques in environmental applications. We used ML to enhance the understanding of the dynamics of PCP transport properties in the subsurface, and to determine key hydrochemical and hydrogeological drivers affecting its transport and fate. We demonstrate how this complementary knowledge, provided by data-driven methods, may enable a more targeted planning of monitoring and remediation at two highly contaminated Swedish groundwater sites, where the method was validated. We evaluated 6 interpretable ML methods, 3 linear regressors and 3 non-linear (i.e., tree-based) regressors, to predict PCP concentration in the groundwater. The modeling results indicate that simple linear ML models were found to be useful in the prediction of observations for datasets without any missing values, while tree-based regressors were more suitable for datasets containing missing values. Considering that missing values are common in datasets collected during contaminated site investigations, this could be of significant importance for contaminated site planners and managers, ultimately reducing site investigation and monitoring costs. Furthermore, we interpreted the proposed models using the SHAP (SHapley Additive exPlanations) approach to decipher the importance of different drivers in the prediction and simulation of critical hydrogeochemical variables. Among these, sum of chlorophenols is of highest significance in the analyses. Setting that aside from the model, tetra chlorophenols, dissolved organic carbon, and conductivity found to be of highest importance. Accordingly, ML methods could potentially be used to improve the understanding of groundwater contamination transport dynamics, filling gaps in knowledge that remain when using more sophisticated deterministic modeling approaches.

Efficient generation of stable linear machine-learning force fields with uncertainty-aware active learning

Machine-learning (ML) force fields (FFs) enable an accurate and universal description of the potential energy surface of molecules and materials on the basis of a training set of ab initio data. However, large-scale applications of these methods rest on the possibility to train accurate ML models with a small number of ab initio data. In this respect, active-learning (AL) strategies, where the training set is self-generated by the model itself, combined with linear ML models are particularly promising. In this work, we explore an AL strategy based on linear regression and able to predict the model’s uncertainty on predictions for molecular configurations not sampled by the training set, thus providing a straightforward recipe for the extension of the latter. We apply this strategy to the spectral neighbor analysis potential and show that only tens of ab initio simulations of atomic forces are required to generate FFs for room-temperature molecular dynamics at or close to chemical accuracy and which stability can be systematically improved by the user at modest computational expenses. Moreover, the method does not necessitate any conformational pre-sampling, thus requiring minimal user intervention and parametrization.

Metabolomic differentiation of tumor core versus edge in glioma.

Gliomas exhibit high intratumor and interpatient heterogeneity. Recently, it has been shown that the microenvironment and phenotype differ significantly between the glioma core (inner) and edge (infiltrating) regions. This proof-of-concept study differentiates metabolic signatures associated with these regions, with the potential for prognosis and targeted therapy that could improve surgical outcomes. Paired glioma core and infiltrating edge samples were obtained from 27 patients after craniotomy. Liquid-liquid metabolite extraction was performed on the samples and metabolomic data were obtained via 2D liquid chromatography-mass spectrometry/mass spectrometry. To gauge the potential of metabolomics to identify clinically relevant predictors of survival from tumor core versus edge tissues, a boosted generalized linear machine learning model was used to predict metabolomic profiles associated with O6-methylguanine DNA methyltransferase (MGMT) promoter methylation. A panel of 66 (of 168) metabolites was found to significantly differ between glioma core and edge regions (p ≤ 0.05). Top metabolites with significantly different relative abundances included DL-alanine, creatine, cystathionine, nicotinamide, and D-pantothenic acid. Significant metabolic pathways identified by quantitative enrichment analysis included glycerophospholipid metabolism; butanoate metabolism; cysteine and methionine metabolism; glycine, serine, alanine, and threonine metabolism; purine metabolism; nicotinate and nicotinamide metabolism; and pantothenate and coenzyme A biosynthesis. The machine learning model using 4 key metabolites each within core and edge tissue specimens predicted MGMT promoter methylation status, with AUROCEdge = 0.960 and AUROCCore = 0.941. Top metabolites associated with MGMT status in the core samples included hydroxyhexanoycarnitine, spermine, succinic anhydride, and pantothenic acid, and in the edge samples metabolites included 5-cytidine monophosphate, pantothenic acid, itaconic acid, and uridine. Key metabolic differences are identified between core and edge tissue in glioma and, furthermore, demonstrate the potential for machine learning to provide insight into potential prognostic and therapeutic targets.

Prediction of hotel booking cancellations: Integration of machine learning and probability model based on interpretable feature interaction

Reliable hotel cancellation prediction can help establish appropriate operational strategies for hotel management. In this sector, personal name records (PNR) data may be the most representative information source for prediction tasks. Despite the popularity of PNR, its inherent lack of availability has been commonly disregarded in the literature. Existing studies have directly input PNR into high-dimensional machine learning (ML) models to achieve cancellation predictions. Another type of model generates cancellation prediction based on the probability modeling of samples. In this study, we propose an interpretable feature interaction method to enrich the existing PNR information. Thereafter, we empirically assess the prediction performance of the two model classes. This study specifically determines whether or not the two methods can cross-fertilize each other to improve cancellation prediction. To do so, we propose a model integrating Bayesian networks (BNs) and Lasso regression for this prediction task. This study utilizes BNs for the probability model consistent with our correlated variables and dichotomous prediction setting. Moreover, we use a linear ML model (i.e., Lasso regression), given its advantages in reducing ineffective predictors and transparency for ranking feature importance. Empirical results show that the proposed integration model has better prediction performance, and the obtained BN estimators and interactive features are the most important predictors. This study contributes to the booking cancellation literature by proposing an interpretable feature interaction and a prediction method integrating two types of effective models. The obtained accurate and interpretable cancellation prediction further contributes to offering practical implications to hoteliers in managerial decision-making.

Prediction of linear model on stunting prevalence with machine learning approach

An increase in the number of residents should be anticipated including in the health sector, especially the problem of stunting. Stunting in children disrupts height and lack of absorption of nutrients. Information and data drive change in many areas such as health, entertainment, economics, business, and other strategic areas. The stages carried out in this study are initiating, developing linear models, and making prediction results on linear machine learning models. The results of testing with the scikit-learn linear model with a minimum variable of 19 get the best test results, namely the polynomial regression with pipeline model with mean absolute percentage error (MAPE) 0.02, root mean square error (RMSE) 3.32, and coefficient of determination (R2) 1,00. Testing with the scikit-learn linear model with a maximum variable of 48 gets the best test results, namely the polynomial regression with pipeline model with MAPE 0.00, RMSE 3.79 and R2 1.00. Testing with the scikit-learn linear model with an average variable of 32 gets the best test results, namely the polynomial regression model with MAPE 0.01, RMSE 3.32, and R2 1.00. The results of testing with the scikit-learn linear model with the minimum, maximum, and average variables get the best test results, namely the polynomial regression with pipeline model.

Statistical Time Series Forecasting Models for Pandemic Prediction

COVID-19 has significantly impacted the growth prediction of a pandemic, and it is critical in determining how to battle and track the disease progression. In this case, COVID-19 data is a time-series dataset that can be projected using different methodologies. Thus, this work aims to gauge the spread of the outbreak severity over time. Furthermore, data analytics and Machine Learning (ML) techniques are employed to gain a broader understanding of virus infections. We have simulated, adjusted, and fitted several statistical time-series forecasting models, linear ML models, and nonlinear ML models. Examples of these models are Logistic Regression, Lasso, Ridge, ElasticNet, Huber Regressor, Lasso Lars, Passive Aggressive Regressor, K-Neighbors Regressor, Decision Tree Regressor, Extra Trees Regressor, Support Vector Regressions (SVR), AdaBoost Regressor, Random Forest Regressor, Bagging Regressor , AuoRegression, MovingAverage, Gradient Boosting Regressor, Autoregressive Moving Average (ARMA), Auto-Regressive Integrated Moving Averages (ARIMA), SimpleExpSmoothing, Exponential Smoothing, Holt-Winters, Simple Moving Average, Weighted Moving Average, Croston, and naive Bayes. Furthermore, our suggested methodology includes the development and evaluation of ensemble models built on top of the best-performing statistical and ML-based prediction methods. A third stage in the proposed system is to examine three different implementations to determine which model delivers the best performance. Then, this best method is used for future forecasts, and consequently, we can collect the most accurate and dependable predictions.

Bayesian model averaging to improve the yield prediction in wheat breeding trials

Accurate pre-harvest prediction of wheat yield through secondary traits helps to facilitate plant breeding and reduce costs. Machine learning (ML) algorithms are increasingly applied to grain yield with remote sensing data. However, the performance of individual ML algorithms varies for different species in different environments due to different sources of uncertainty. This study proposed a novel wheat yield prediction framework based on canopy hyperspectral reflectance (350–2500 nm) and adopted the ensemble Bayesian model averaging (EBMA) method to improve model performance. To develop the yield prediction models, important bands extracted by the Boruta feature selection method were fed into four linear ML models and four nonlinear ML models. Meanwhile, Bayesian model averaging (BMA) weights obtained based on model cross-validation performance were used to combine the predictions of individual ML models. Compared to the best-performing individual model, the EBMA models obtained a weak accuracy improvement by integrating only the linear models or the nonlinear models. Additionally, the integration of two linear models and two non-linear models simultaneously was analyzed. Results indicate that most EBMA combinations of mixed linear and non-linear models achieved higher prediction accuracy than those integrating a single type of model and the best-performing individual model. The advantage of the EBMA method is that it produces a prediction distribution that reflects the uncertainty associated with deterministic predictions. With full consideration of the model diversity of ensemble members, the EBMA modeling framework provides an alternative method for predicting grain yield in plant breeding trials.

Voltage Sag Estimation for Distribution Systems Using Linear Machine Learning Models

This paper proposes a voltage sag estimation approach based on linear machine learning models. The proposed approach estimates the voltage sag profile on distributions systems with limited monitored buses. The approach takes advantage of the priory knowledge about the linearity of the problem to train and compare three linear models: Least Squares (LS), Ridge Regression (RR), and Absolute Shrinkage and Selection Operator (LASSO). The approach is tested on the IEEE-34-bus distribution system, and the performance of models is validated through the Mean Square Error (MSE). The results show that the proposed linear machine learning models capture the internal relationships of the problem and estimate the voltage sag with high accuracy on test data.

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films

Why it is Unfortunate that "Faster," "Better" and "Less Biased" Linear Machine Learning Models "Work" so well in Electromechanical Switching of Ferroelectric Thin Films