Machine Learning Methods For Prediction Research Articles

SPOTONE: Hot Spots on Protein Complexes with Extremely Randomized Trees via Sequence-Only Features.

Protein Hot-Spots (HS) are experimentally determined amino acids, key to small ligand binding and tend to be structural landmarks on protein–protein interactions. As such, they were extensively approached by structure-based Machine Learning (ML) prediction methods. However, the availability of a much larger array of protein sequences in comparison to determined tree-dimensional structures indicates that a sequence-based HS predictor has the potential to be more useful for the scientific community. Herein, we present SPOTONE, a new ML predictor able to accurately classify protein HS via sequence-only features. This algorithm shows accuracy, AUROC, precision, recall and F1-score of 0.82, 0.83, 0.91, 0.82 and 0.85, respectively, on an independent testing set. The algorithm is deployed within a free-to-use webserver, only requiring the user to submit a FASTA file with one or more protein sequences.

New Machine Learning Approach for Detection of Injury Risk Factors in Young Team Sport Athletes.

The purpose of this article is to present how predictive machine learning methods can be utilized for detecting sport injury risk factors in a data-driven manner. The approach can be used for finding new hypotheses for risk factors and confirming the predictive power of previously recognized ones. We used three-dimensional motion analysis and physical data from 314 young basketball and floorball players (48.4% males, 15.72±1.79 yr, 173.34±9.14 cm, 64.65±10.4 kg). Both linear (L1-regularized logistic regression) and non-linear methods (random forest) were used to predict moderate and severe knee and ankle injuries (N=57) during three-year follow-up. Results were confirmed with permutation tests and predictive risk factors detected with Wilcoxon signed-rank-test (p<0.01). Random forest suggested twelve consistent injury predictors and logistic regression twenty. Ten of these were suggested in both models; sex, body mass index, hamstring flexibility, knee joint laxity, medial knee displacement, height, ankle plantar flexion at initial contact, leg press one-repetition max, and knee valgus at initial contact. Cross-validated areas under receiver operating characteristic curve were 0.65 (logistic regression) and 0.63 (random forest). The results highlight the difficulty of predicting future injuries, but also show that even with models having relatively low predictive power, certain predictive injury risk factors can be consistently detected.

Flood Prediction and Warning System using Dam Data Monitoring

Flood is one of the most devastating natural calamities affecting parts of the state from past few years. The recurring calamity necessitates an efficient early warning system since anticipation and preparedness play a key role in mitigating the impact. Though heavy and erratic rainfall has been marked as one of the main reasons for flood in several places, flood witnessed by various regions of Kerala was the result of sudden opening of reservoirs indicating poor dam management. The unforeseen flow of water often provided less time for evacuation. Prediction thus plays key role in avoiding loss of life and property, followed by such calamities. The vast benefits and potentials offered by Machine Learning makes it the most promising approach. The developed system is a model by taking Malampuzha Dam as reference. Support Vector Machine (SVM) is used as machine learning method for prediction and is programmed in python. The idea has been to create early flood prediction and warning system by monitoring different weather parameters and dam-related data. The feature vectors include current live storage, current reservoir level, rainfall and relative humidity from the period 2016-2019. Based on the analysis of these parameters, the open/closure of shutters of the dam is predicted. Release of shutters has varied impacts in the nearby regions and is measured by succeeding prediction, by mapping regions on grounds of level warning to be issued. Warning is issued through Flask-based server, by identifying vulnerable areas based on flood hazard reference for regions. The dam status prediction model delivered highest prediction accuracy of 99.14% and associated levels of warning has been generated in the development server, thus preventing unexpected release.

Complex networks of material flow in manufacturing and logistics: Modeling, analysis, and prediction using stochastic block models

Modeling complex systems as networks of interacting elements has gained increased attention in recent years. So far, network modeling in manufacturing and logistics has often focused on the description of system properties. In the data-driven world of smart manufacturing, creating material flow network models becomes a lot easier due to the ubiquitous availability of shop floor and transportation data. At the same time, these highly flexible and continuously changing smart manufacturing systems become less predictive and thus less controllable. This article investigates how the stochastic block model (SBM), a network model with a stochastic description of interconnections, can be applied to model and predict material flows in manufacturing systems. We show how to utilize its properties to forecast the dynamic development of the structure of such systems. The complete process from network modeling using material flow data to the prediction of the future development of the network is demonstrated. Different SBM variants are tested using six company data sets and evaluated in competition with classical machine learning methods for prediction. Our results show that selected SBM variants achieve the best performance in prediction in most scenarios and thus have the potential to play an important role in the management of future dynamic manufacturing systems.

Comparison of methods for predicting cow composite somatic cell counts

One of the most common and reliable ways of monitoring udder health and milk quality in dairy herds is by monthly cow composite somatic cell counts (CMSCC). However, such sampling can be time consuming, and more automated sampling tools entail extra costs. Machine learning methods for prediction have been widely investigated in mastitis detection research, and CMSCC is normally used as a predictor or gold standard in such models. Predicted CMSCC between samplings could supply important information and be used as an input for udder health decision-support tools. To our knowledge, methods to predict CMSCC are lacking. Our aim was to find a method to predict CMSCC by using regularly recorded quarter milk data such as milk flow or conductivity. The milk data were collected at the quarter level for 8 wk when milking 372 Holstein-Friesian cows, resulting in a data set of 30,734 records with information on 87 variables. The cows were milked in an automatic milking rotary and sampled once weekly to obtain CMSCC values. The machine learning methods chosen for evaluation were the generalized additive model (GAM), random forest, and multilayer perceptron (MLP). For each method, 4 models with different predictor variable setups were evaluated: models based on 7-d lagged or 3-d lagged records before the CMSCC sampling and additionally for each setup but removing cow number as a predictor variable (which captures indirect information regarding cows' overall level of CMSCC based on previous samplings). The methods were evaluated by a 5-fold cross validation and predictions on future data using models with the 4 different variable setups. The results indicated that GAM was the superior model, although MLP was equally good when fewer data were used. Information regarding the cows' level of previous CMSCC was shown to be important for prediction, lowering prediction error in both GAM and MLP. We conclude that the use of GAM or MLP for CMSCC prediction is promising.

Performance Evaluation of Machine Learning Methods for Forest Fire Modeling and Prediction

Predicting and mapping fire susceptibility is a top research priority in fire-prone forests worldwide. This study evaluates the abilities of the Bayes Network (BN), Naïve Bayes (NB), Decision Tree (DT), and Multivariate Logistic Regression (MLP) machine learning methods for the prediction and mapping fire susceptibility across the Pu Mat National Park, Nghe An Province, Vietnam. The modeling methodology was formulated based on processing the information from the 57 historical fires and a set of nine spatially explicit explanatory variables, namely elevation, slope degree, aspect, average annual temperate, drought index, river density, land cover, and distance from roads and residential areas. Using the area under the receiver operating characteristic curve (AUC) and seven other performance metrics, the models were validated in terms of their abilities to elucidate the general fire behaviors in the Pu Mat National Park and to predict future fires. Despite a few differences between the AUC values, the BN model with an AUC value of 0.96 was dominant over the other models in predicting future fires. The second best was the DT model (AUC = 0.94), followed by the NB (AUC = 0.939), and MLR (AUC = 0.937) models. Our robust analysis demonstrated that these models are sufficiently robust in response to the training and validation datasets change. Further, the results revealed that moderate to high levels of fire susceptibilities are associated with ~19% of the Pu Mat National Park where human activities are numerous. This study and the resultant susceptibility maps provide a basis for developing more efficient fire-fighting strategies and reorganizing policies in favor of sustainable management of forest resources.

Application of Machine Learning to Attribution and Prediction of Seasonal Precipitation and Temperature Trends in Canberra, Australia

Southeast Australia is frequently impacted by drought, requiring monitoring of how the various factors influencing drought change over time. Precipitation and temperature trends were analysed for Canberra, Australia, revealing decreasing autumn precipitation. However, annual precipitation remains stable as summer precipitation increased and the other seasons show no trend. Further, mean temperature increases in all seasons. These results suggest that Canberra is increasingly vulnerable to drought. Wavelet analysis suggests that the El-Niño Southern Oscillation (ENSO) influences precipitation and temperature in Canberra, although its impact on precipitation has decreased since the 2000s. Linear regression (LR) and support vector regression (SVR) were applied to attribute climate drivers of annual precipitation and mean maximum temperature (TMax). Important attributes of precipitation include ENSO, the southern annular mode (SAM), Indian Ocean Dipole (DMI) and Tasman Sea SST anomalies. Drivers of TMax included DMI and global warming attributes. The SVR models achieved high correlations of 0.737 and 0.531 on prediction of precipitation and TMax, respectively, outperforming the LR models which obtained correlations of 0.516 and 0.415 for prediction of precipitation and TMax on the testing data. This highlights the importance of continued research utilising machine learning methods for prediction of atmospheric variables and weather pattens on multiple time scales.

Immunohistochemical index prediction of breast tumor based on multi-dimension features in contrast-enhanced ultrasound.

Breast cancer is the leading killer of Chinese women. Immunohistochemistry index has great significance in the treatment strategy selection and prognosis analysis for breast cancer patients. Currently, histopathological examination of the tumor tissue through surgical biopsy is the gold standard to determine immunohistochemistry index. However, this examination is invasive and commonly causes discomfort in patients. There has been a lack of noninvasive method capable of predicting immunohistochemistry index for breast cancer patients. This paper proposes a machine learning method to predict the immunohistochemical index of breast cancer patients by using noninvasive contrast-enhanced ultrasound. A total of 119 breast cancer patients were included in this retrospective study. Each patient implemented the pathological examination of immunohistochemical expression and underwent contrast-enhanced ultrasound imaging of breast tumor. The multi-dimension features including 266 three-dimension features and 837 two-dimension dynamic features were extracted from the contrast-enhanced ultrasound sequences. Using the machine learning prediction method, 21 selected multi-dimension features were integrated to generate a model for predicting the immunohistochemistry index noninvasively. The immunohistochemical index of human epidermal growth factor receptor-2 (HER2) was predicted based on multi-dimension features in contrast-enhanced ultrasound sequence with the sensitivity of 71%, and the specificity of 79% in the testing cohort. Therefore, the noninvasive contrast-enhanced ultrasound can be used to predict the immunohistochemical index. To our best knowledge, no studies have been reported about predicting immunohistochemical index by using contrast-enhanced ultrasound sequences for breast cancer patients. Our proposed method is noninvasive and can predict immunohistochemical index by using contrast-enhanced ultrasound in several minutes, instead of relying totally on the invasive and biopsy-based histopathological examination. Graphical abstract Immunohistochemical index prediction of breast tumor based on multi-dimension features in contrast-enhanced ultrasound.

Improving customer routing in contact centers: An automated triage design based on text analytics

AbstractWe propose an automated triage design for intelligent customer routing in live‐chat contact centers and demonstrate its implementation using a real‐world data set from an S&amp;P 500 firm. The proposed design emerges as a synthesis of text analytics and predictive machine learning methods. Using numerical experiments based on the simulation of the firm's contact center, we demonstrate the service level, time, and labor cost benefits of the automated design over two other triage designs (i.e., customer choice triage and human expert triage) that are commonly employed in the real world. Through additional analyses, we explore the generalizability of the automated design for creating solutions for different types of communication channels. Our work has implications for managing customer relations under emerging communication technologies (e.g., live‐chat, e‐mail, and social media) and more broadly for demonstrating the use of text analytics and machine learning to improve Operations Management practice.

Exploration of the correlation between GPCRs and drugs based on a learning to rank algorithm

Exploring the protein - drug correlation can not only solve the problem of selecting candidate compounds but also solve related problems such as drug redirection and finding potential drug targets. Therefore, many researchers have proposed different machine learning methods for prediction of protein-drug correlations. However, many existing models simply divide the protein-drug relationship into related or irrelevant categories and do not deeply explore the most relevant target (or drug) for a given drug (or target). In order to solve this problem, this paper applies the ranking concept to the prediction of the GPCR (G Protein-Coupled Receptors)-drug correlation. This study uses two different types of data sets to explore candidate compound and potential target problems, and both sets achieved good results. In addition, this study also found that the family to which a protein belongs is not an inherent factor that affects the ranking of GPCR-drug correlations; however, if the drug affects other family members of the protein, then the protein is likely to be a potential target of the drug. This study showed that the learning to rank algorithm is a good tool for exploring protein-drug correlations.

Genetically predicted gene expression of prefrontal DRD4 gene and the differential susceptibility to childhood emotional eating in response to positive environment

Genetic differential susceptibility states that individuals may vary both by exhibiting poor responses when exposed to adverse environments, and disproportionally benefiting from positive settings. The dopamine D4 receptor gene (DRD4) may be particularly implicated in these effects, including disturbed eating behaviors that might lead to obesity. Here, we explore differential susceptibility to positive environments according to the predicted genetically regulated gene expression of prefrontal cortex DRD4 gene. Using MAVAN as the discovery cohort (Maternal Adversity, Vulnerability and Neurodevelopment) and GUSTO as the replication cohort (Growing Up in Singapore Towards Healthy Outcomes), we analyzed the interaction between a) a Positive postnatal environmental score, that accounts for positive outcomes in the postnatal period and b) the genetically regulated gene expression of prefrontal DRD4, computed using a machine learning prediction method (PrediXcan). The outcome measures were the pro-intake domains (Emotional over-eating, Food Responsiveness, Food Enjoyment and Desire to Drink) from the Child Eating Behavior Questionnaire at 48 months of age (MAVAN) and 60 months of age (GUSTO). The interaction between the positive environment and the predicted prefrontal DRD4 gene expression was significant for emotional over-eating in MAVAN (β = −0.403, p < 0.02), in which the high gene expression group had more or less emotional eating according to the exposure to lower or higher positive environment respectively, showing evidence of differential susceptibility criteria. In the replication cohort, a similar result was found with the pro-intake domain Desire to drink (β = −0.583, p < 0.05). These results provide further evidence for the genetic differential susceptibility, accounting for the benefit of positive environments.

Machine Learning Methods for Prediction of Changes in Material Flow Networks

New trends like cloud manufacturing, urban manufacturing, and additive manufacturing cause future manufacturing systems to be more decentralized and dynamic. These dynamics not only relate to the actual manufacturing process, but also to the infrastructure of networked machines, which is subject to constant change and reconfiguration. Forecasting this infrastructural change is vital to efficient mid-and long-term planning activities in such material flow networks. Therefore, we propose the application of machine learning methods for the prediction of changes in material flow networks based on readily available data about material flow, e.g., feedback data from manufacturing execution systems. State-of-the-art network embedding methods, such as Node2Vec, DeepWalk, matrix-and tensor factorization, are adapted for the manufacturing environment and subsequently evaluated using a real-world dataset.

Using Partial Combination Models to Improve Prediction Quality and Transparency in Mixed Datasets

Mixed Datasets with complex interactions between categorical and numerical attributes are common in engineering and business applications. For example, production rates in manufacturing systems are jointly influenced by several categorical and numerical attributes, such as machine and product types and their numerical attributes. This study aims to improve the prediction performance and transparency of mixed datasets with complex interactions using machine learning (ML) methods. The proposed method requires lesser data and computational effort than existing hierarchical or clustering regression methods. Multiple prediction models can be generated by partitioning a dataset into subsets with different categorical attribution combinations. One- and two-stage model selection methods are proposed to use the training and validation datasets in selecting better models among all the prediction models. Numerical results demonstrate the potential of the model selection approach in a mixed dataset from a semiconductor manufacturer. In comparison with regression models, more than 30% reduction in root mean square error is observed using the proposed model selection approach. The cross-validation test results also demonstrated a 10% improvement in accuracy against the properly tuned XGBoost models. Moreover, the proposed model selection approach is compatible with other regression or ML prediction methods and can be used to improve the model's transparency of any existing methods on mixed datasets.

Exploring the computational methods for protein-ligand binding site prediction

Proteins participate in various essential processes in vivo via interactions with other molecules. Identifying the residues participating in these interactions not only provides biological insights for protein function studies but also has great significance for drug discoveries. Therefore, predicting protein–ligand binding sites has long been under intense research in the fields of bioinformatics and computer aided drug discovery. In this review, we first introduce the research background of predicting protein–ligand binding sites and then classify the methods into four categories, namely, 3D structure-based, template similarity-based, traditional machine learning-based and deep learning-based methods. We describe representative algorithms in each category and elaborate on machine learning and deep learning-based prediction methods in more detail. Finally, we discuss the trends and challenges of the current research such as molecular dynamics simulation based cryptic binding sites prediction, and highlight prospective directions for the near future.

High-Performance Time Series Prediction With Predictive Error Compensated Wavelet Neural Networks

Machine learning (ML) algorithms have gained prominence in time series prediction problems. Depending on the nature of the time series data, it can be difficult to build an accurate ML model with the proper structure and hyperparameters. In this study, we propose a predictive error compensation wavelet neural network model (PEC-WNN) for improving the prediction accuracy of chaotic and stochastic time series data. In the proposed model, an additional network is used for the prediction of the main network error to compensate the overall prediction error. The main network takes as inputs the time series data through moving frames in multiple-scales. The same structure and hyperparameter sets are applied for quite distinct four types of problems for verification of the robustness and accuracy of the proposed model. Specifically, the Mackey-Glass, Box-Jenkins, and Lorenz Attractor benchmark problems, as well as drought forecasting are used to characterize the performance of the model for chaotic and stochastic data cases. The results show that the PEC-WNN provides significantly more accurate predictions for all compared benchmark problems with respect to conventional machine learning and time series prediction methods without changing any hyperparameter or the structure. In addition, the time and space complexity of the PEC-WNN model is less than all other compared ML methods, including long short-term memory (LSTM) and convolutional neural networks (CNNs).

Extension of pQSAR: Ensemble Model Generated by Random Forest and Partial Least Squares Regressions

Quantitative structure-activity relationship (QSAR) regression models are mathematical ones which relate the structural properties of chemicals to the potencies of the biological activities of the chemicals. In QSAR models, the physical and chemical information of the molecules is encoded into quantitative numbers called descriptors. Recently, experimental test results (profiles) have been used as descriptors of chemicals. Profile QSAR 2.0 (pQSAR) model suggested by Martin et al. , is a multitask, two step machine learning prediction method with a combination of random forest regressions (RFRs) and partial least squares regression (PLSR). In pQSAR model, one fills the profile table’s missing values with RFRs and then builds PLSR using the profile predictions. Note that in the second step of the pQSAR method, PLSR’s predictor variables are profiles; so activity values, and the response variables are also activity values. Thus we can use the PLSRs to update the profile table and then repeat the second step. In this work, we propose an extended model of pQSAR generated by RFRs and PLSRs. Experiment of updating the given full initially predicted profile table by two kinds of prediction models, RFRs and PLSRs, has been conducted iteratively for the PKIS and ChEMBL data sets. Even though prediction performance of individual combination of RFRs and PLSRs varies, the average of the all possible predicted profile tables for given iteration shows better performance. This ensemble model has better prediction performance in sense of Pearson’s $R^{2}$ compared to that of the pQSAR model.

Computer Prediction of Seawater Sensor Parameters in the Central Arctic Region Based on Hybrid Machine Learning Algorithms

In recent years, with the large-scale reduction of Arctic sea ice, the supplement of chlorophyll sensor data in seawater has become an essential part of environmental assessment. Accurately predicting the chlorophyll sensor data in seawater is of great significance to protect the Arctic marine ecological environment. A machine learning prediction method combined with wavelet transform is proposed. This process uses data from upper ocean observation buoys placed in the Arctic Ocean (A.O.) to predict the sensor analogue of chlorophyll-a (C.A.) in the upper ocean of the Arctic Ocean. Choose the best wavelet transform method and prevent the LSTM gradient from disappearing. A model combining SAE (stacked autoencoder) Bi (bidirectional) LSTM (long short-term memory) and wavelet transform is proposed. Experiments were conducted to compare the predictive performance of buoy data input as univariate at two different times and locations in the Arctic Ocean. The results show that compared with other models (such as LSTM), in the SAE Bi LSTM model, the data of the two sites have the highest prediction accuracy. The best wavelet transform methods are fourth-order four-layer and first-order four-layer. The observational data of the Chukchi Sea from 2018 to 2019 obtained the best prediction results. The root mean square error (RMSE) value is 0.02003 volts; the average absolute error (MAE) is 0.0841 volts. This research provides a new method for predicting the chlorophyll sensor parameters in the upper ocean through the sea ice buoy observed at a given point, which helps to improve the accuracy of the ocean sensor parameter prediction on the Arctic ice buoy.

Analysis of Feature Extraction Methods for Prediction of 30-Day Hospital Readmissions.

This article aims to determine possible improvements made by feature extraction methods to the machine learning prediction methods for predicting 30-day hospital readmissions. The study evaluates five feature extraction methods including principal component analysis (PCA), kernel principal component analysis (KPCA), isomap, Laplacian eigenmaps, and locality preserving projections (LPPs) for improving the accuracy of nine machine learning prediction methods in predicting 30-day hospital readmissions. The specific prediction methods considered include logistic regression, Cox regression, linear discriminant analysis, k-nearest neighbor (KNN), support vector machines (SVMs), bagged trees, boosted trees, random forest, and artificial neural networks. All models are developed in MATLAB and validated using area under the curve based on two population-based data sets from partner hospitals. Laplacian eigenmaps and isomap feature extraction provide the most improvement to the readmission predictive accuracy of KNN, SVM, bagged trees, boosted trees, and linear discriminant analysis methods. The results for artificial neural networks, random forest, Cox regression, and logistic regression show improvement for only one of the data sets. Also, PCA and LPP provided the best computation efficiency followed by KPCA, Laplacian eigenmaps, and isomap. Feature extraction methods can improve the predictive performance of machine learning methods for predicting readmissions. However, the improvement depended on the specific choice of the prediction method, feature extraction method, and the complexity of the data set features.

A novel framework for daily forecasting of ozone mass concentrations based on cycle reservoir with regular jumps neural networks

An understanding of the growth in surface concentration of ozone and its adverse health effects are important for environmental departments to make sensible decisions for future. Our hybrid model CEEMD+CRJ+MLR is the first attempt to improve CRJ in the field of air pollution prediction and ozone forecasting. For this novel framework, CEEMD has been adopted to decompose original MDA8_O3 history into several sub-series. After that, for each IMF, CRJ is used to extract time-series features. These time-series features are fed into appropriate machine learning methods for prediction. In addition to that, residual is also predicted through normal methods. A model, which is trained with data from 1 May 2014 to 31 May 2017, is validated with data from 1 June 2017 to 30 May 2018, obtained from four stations of Beijing, China. The hybrid model has input variables which are combined with related pollutants, meteorological forecasts and UV index, and predict maximum daily 8-h average ozone (MDA8_O3) concentration in different time intervals. Our experimental results show that the CEEMD+CRJ+MLR model exhibits the best performance compared with other benchmark models generally. For four stations, IA, MAE, RMSE and MAPE average of +1 (forecasting 1 day in advance) are 0.9763, 12.84, 17.81 and 18.5% respectively and of +2 are 0.9679, 15.17, 20.15 and 23.86% respectively. Especially in the case of forecasting heavy ozone concentration (Level III), a critical issue in air pollution predictions, the classification rate of our hybrid model has improved from 29.4% (for CRJ) to 83.4% in +1 and from 38% (for CRJ) to 73% in +2. For long time forecasting, the CEEMD+CRJ+MLR also shows its outstanding performance in whole levels and level III ozone concentration. Our hybrid model, with accurate and stable results, is highly effective for MDA8_O3 concentration prediction and can efficiently be applied in other regions.

110th Anniversary: Ensemble-Based Machine Learning for Industrial Fermenter Classification and Foaming Control

In the industrial sector, foaming remains an inevitable side effect of mixing, shearing, powder incorporation, and the metabolic activities of microorganisms in a bioprocess. Excessive foaming can interfere with the mixing of reactants and lead to problems such as decreased effective reactor volume, microbial contamination, product loss, and increased reaction time. Physical modeling of foaming is an arduous process as it requires estimation of foam height, which is dynamic in nature and varies for different processes. This work demonstrates a novel application of ensemble-based machine learning methods for prediction of different fermenter types in a fermentation process (to allow for successful data integration) and of the onset of foaming. Ensemble-based methods are robust nonlinear modeling techniques that aggregate a set of learners to obtain better predictive performance than a single learner. We apply two ensemble frameworks, extreme gradient boosting (XGBoost) and random forest (RF), to build clas...