Machine Learning Methods For Prediction Research Articles

A dimensionally augmented and physics-informed machine learning for quality prediction of additively manufactured high-entropy alloy

Selective laser melting (SLM) additive manufacturing (AM) is widely used due to its significant advantages in designing and manufacturing special-shaped complex components. The process parameters of SLM determine the quality of as-built parts, but it is difficult to establish an accurate and reliable mathematical model to connect process parameters with the quality of as-built parts. However, data-driven machine learning can effectively solve the analysis and prediction problem of complex process. Therefore, a machine learning (ML) prediction method based on dimensionality augmentation and physical information is proposed, which connects the process parameters (laser power, hatching space, scanning speed, and layer thickness) of SLM with the quality characteristics (top layer surface roughness and relative density) of as-built parts. The four process parameter features (4-dimensional features) are expanded to high-dimensional features through feature engineering to characterize the quality of as-built parts. In addition, the physical information of powder melting forming in SLM process is fused with ML algorithm, the theory-guided ML is used to improve the prediction accuracy of the model. In this paper, the CoCrFeNiMn high-entropy alloy as-built samples dataset is used for network training of four ML algorithms, and three assessment indexes are used to evaluate the prediction model. The results show that dimensionally augmented and physics-informed ML model has better prediction accuracy and generalization ability. The proposed method can also provide guidance for optimizing process parameters.

Applying Fuzzy Inference and Machine Learning Methods for Prediction with a Small Dataset: A Case Study for Predicting the Consequences of Oil Spills on a Ground Environment

Applying machine learning (ML) and fuzzy inference systems (FIS) requires large datasets to obtain more accurate predictions. However, in the cases of oil spills on ground environments, only small datasets are available. Therefore, this research aims to assess the suitability of ML techniques and FIS for the prediction of the consequences of oil spills on ground environments using small datasets. Consequently, we present a hybrid approach for assessing the suitability of ML (Linear Regression, Decision Trees, Support Vector Regression, Ensembles, and Gaussian Process Regression) and the adaptive neural fuzzy inference system (ANFIS) for predicting the consequences of oil spills with a small dataset. This paper proposes enlarging the initial small dataset of an oil spill on a ground environment by using the synthetic data generated by applying a mathematical model. ML techniques and ANFIS were tested with the same generated synthetic datasets to assess the proposed approach. The proposed ANFIS-based approach shows significant performance and sufficient efficiency for predicting the consequences of oil spills on ground environments with a smaller dataset than the applied ML techniques. The main finding of this paper indicates that FIS is suitable for prediction with a small dataset and provides sufficiently accurate prediction results.

A new method to forecast multi-time scale load of natural gas based on augmentation data-machine learning model

Gas load forecasting is important for the economic and reliable operation of the city gas transmission and distribution system. In this paper, a nonlinear autoregressive model (NARX) with exogenous inputs, support vector machine (SVM), Gaussian process regression (GPR) and ensemble tree model (ETREE) were used to predict and compare the gas load based on the gas load data in a certain region for past 3 years. The results showed that the prediction errors for most of days were higher than 10%. Further, simulation data were generated by considering the gas load variation trend, which was then combined with historical data to form the augmentation data set to train the model. The test results indicated that the prediction error of daily gas load in one year reduced to below 7% with a machine learning prediction method based on augmentation data. In addition, the model based on augmentation data set still performed better than original data in predicting the monthly gas load in last year as well as daily gas load in last month and week. Therefore, the method based on augmentation data proposed in this paper is a potentially good tool to forecast natural gas load.

Investigation of Machine Learning Methods for Prediction of Measured Values of Atmospheric Channel for Hybrid FSO/RF System

This research paper addresses the problems of fiberless optical communication, known as free space optics, in predicting RSSI (Received Signal Strength Indicator) parameters necessary for hard switching in a hybrid FSO/RF (Free Space Optics/Radio Frequency) system. This parameter is used to determine the intensity of the transmitted signal (in our case, a light beam) from one FSO head to another. Since we want to achieve almost 100% reliability, it is important to know the parameters of the transmission environment for the FSO and RF lines. Each of them has its limitations and, as a result, a weather monitoring station is required. The FSO is mostly affected by fog and the concentration of particles in the air, while the RF line is affected by rain and snow. It is precisely due to these influences that it is necessary (based on the mentioned RSSI parameter) to switch using the hard switching method from the primary FSO line to the backup RF line by correctly predicting this value. If the value of the RSSI parameter falls below the critical level—42 dBm—the system automatically switches to the backup RF line. There are several ways we can predict this parameter. One of them is machine learning methods such as decision trees. Our research focused on the prediction of the RSSI parameter, the methods of decision trees and decision trees using the AdaBoost regressor. Since we want to correctly predict the RSSI parameter, it is also necessary to choose the right way to predict it based on the recorded weather conditions. If we want to correctly use the hard switching method in hybrid FSO/RF systems, it is necessary to choose the correct method of predicting the RSSI parameter, which serves as an indicator for switching from the primary FSO line to the secondary RF line. Therefore, we decided to investigate methods of machine learning—the decision tree and the decision tree with the use of an AdaBoost Regressor. The main benefit of this paper is the improvement of existing machine learning methods (decision trees and decision trees using the AdaBoost regressor) for the correct prediction of the RSSI parameter for the needs of hard switching in a hybrid FSO/RF system. The method chosen in this manuscript has very good results. As can be seen in the attached graphs, over a longer period and using correctly selected training data, it is possible to achieve ideal results for the prediction of the RSSI parameter. The tables also show the effectiveness of the prediction, and the fact that it is best to train on either the first- or third-minute data. In the future, it would be appropriate to implement weather prediction or to consider other methods, such as random forests or neural networks.

Research on stock index forecasting based on ARIMA-GARCH and SVM mixed model

With the development of information technology, financial forecasting is the research direction of traditional mathematical statistics and combines emerging data mining technologies such as machine learning and deep learning to make forecasts. This paper starts from the traditional time series model , which can handle the conditional heteroscedasticity of time series data and combines the advantages of a support vector machine (SVM) that can handle the nonlinear part of financial time data . Stock market forecasts. In order to test the prediction effect of the model and compare traditional prediction methods with modern machine learning prediction methods, this paper establishes ARIMA-GARCH, SVM, and the combination ARIMA-GARCH- SVM model of these two types, respectively.

Prediction of Smoking Habits From Class-Imbalanced Saliva Microbiome Data Using Data Augmentation and Machine Learning.

Human microbiome research is moving from characterization and association studies to translational applications in medical research, clinical diagnostics, and others. One of these applications is the prediction of human traits, where machine learning (ML) methods are often employed, but face practical challenges. Class imbalance in available microbiome data is one of the major problems, which, if unaccounted for, leads to spurious prediction accuracies and limits the classifier's generalization. Here, we investigated the predictability of smoking habits from class-imbalanced saliva microbiome data by combining data augmentation techniques to account for class imbalance with ML methods for prediction. We collected publicly available saliva 16S rRNA gene sequencing data and smoking habit metadata demonstrating a serious class imbalance problem, i.e., 175 current vs. 1,070 non-current smokers. Three data augmentation techniques (synthetic minority over-sampling technique, adaptive synthetic, and tree-based associative data augmentation) were applied together with seven ML methods: logistic regression, k-nearest neighbors, support vector machine with linear and radial kernels, decision trees, random forest, and extreme gradient boosting. K-fold nested cross-validation was used with the different augmented data types and baseline non-augmented data to validate the prediction outcome. Combining data augmentation with ML generally outperformed baseline methods in our dataset. The final prediction model combined tree-based associative data augmentation and support vector machine with linear kernel, and achieved a classification performance expressed as Matthews correlation coefficient of 0.36 and AUC of 0.81. Our method successfully addresses the problem of class imbalance in microbiome data for reliable prediction of smoking habits.

Identify Bitter Peptides by Using Deep Representation Learning Features.

A bitter taste often identifies hazardous compounds and it is generally avoided by most animals and humans. Bitterness of hydrolyzed proteins is caused by the presence of bitter peptides. To improve palatability, bitter peptides need to be identified experimentally in a time-consuming and expensive process, before they can be removed or degraded. Here, we report the development of a machine learning prediction method, iBitter-DRLF, which is based on a deep learning pre-trained neural network feature extraction method. It uses three sequence embedding techniques, soft symmetric alignment (SSA), unified representation (UniRep), and bidirectional long short-term memory (BiLSTM). These were initially combined into various machine learning algorithms to build several models. After optimization, the combined features of UniRep and BiLSTM were finally selected, and the model was built in combination with a light gradient boosting machine (LGBM). The results showed that the use of deep representation learning greatly improves the ability of the model to identify bitter peptides, achieving accurate prediction based on peptide sequence data alone. By helping to identify bitter peptides, iBitter-DRLF can help research into improving the palatability of peptide therapeutics and dietary supplements in the future. A webserver is available, too.

Diagnostic Performance of a Noninvasive Breath Test for Colorectal Cancer: COBRA1 Study

Diagnostic Performance of a Noninvasive Breath Test for Colorectal Cancer: COBRA1 Study

Applying Machine Learning Prediction Methods to COVID-19 Data

The Coronavirus (COVID-19) epidemic emerged in China and has caused many problems such as loss of life, and deterioration of social and economic structure. Thus, understanding and predicting the course of the epidemic is very important. In this study, SEIR model and machine learning methods LSTM and SVM were used to predict the values of Susceptible, Exposed, Infected, and Recovered for COVID-19. For this purpose, COVID-19 data of Egypt and South Korea provided by John Hopkins University were used. The results of the methods were compared by using MAPE. Total 79% of MAPE were between 0-10. The comparisons show that although LSTM provided the better results, the results of all three methods were successful in predicting the number of cases, the number of patients who died, the peaks and dimensions of the epidemic.

Machine learning methods in prediction of basal cell skin cancer recurrence after photodynamic therapy

Objective: Verification of predictors and forecasting basal cell skin cancer recurrence (BCSC) after conducting photodynamic therapy (PDT) based on machine learning methods (ML).Methods: The prospective study of 170 patients (117 women and 53 men) was conducted. The median age was 68 years. All patients got PDT treatment on BCSC. Potential predictors of BCSC were analyzed. Primary outcome measure was the fact of tumor development recurrence.Results: During 4-year observation period the recurrence of the disease took place in 18 cases (10.6% of patients). Processing and analyzing data with the assistance of machine learning methods (ML) allowed to highlight the predictors connected with the development of BCSC recurrence development linearly and non linearly. There are such predictors as: 2nd stage of the process, its morphea-like form, localization in the thoracic cage area, the level of ESR and glucose in the blood. The most accurate forecast of BCSC recurrence was gotten using model based on multiple linear regression (LR). It was proved by high levels of quality indexes (the area under ROCcurve – 0.893, sensitivity – 0.849, specificity – 0.889). Predictive accuracy of the stochastic gradient boosting model (SGB) was less significant.Conclusions. PDT is an effective BCSC treatment method. It is proved by the results of prospective observation of patients for the period of 4 years. ML methods are an informative tool to verify predictors and forecast BCSC recurrence. Forecasting models based on multiple LR demonstrate much higher accuracy compared with SGB.

Comparative Study of Experimental and Modeling of Fly Ash-Based Concrete.

The application of supplementary cementitious materials (SCMs) in concrete has been reported as the sustainable approach toward the appropriate development. This research aims to compare the result of compressive strength (C-S) obtained from the experimental method and results estimated by employing the various modeling techniques for the fly-ash-based concrete. Although this study covers two aspects, an experimental approach and modeling techniques for predictions, the emphasis of this research is on the application of modeling methods. The physical and chemical properties of the cement and fly ash, water absorption and specific gravity of the aggregate used, surface area of the cement, and gradation of the aggregate were analyzed in the laboratory. The four predictive machine learning (PML) algorithms, such as decision tree (DT), multi-linear perceptron (MLP), random forest (RF), and bagging regressor (BR), were investigated to anticipate the C-S of concrete. Results reveal that the RF model was observed more exact in investigating the C-S of concrete containing fly ash (FA), as opposed to other employed PML techniques. The high R2 value (0.96) for the RF model indicates the high precision level for forecasting the required output as compared to DT, MLP, and BR model R2 results equal 0.88, 0.90, and 0.93, respectively. The statistical results and cross-validation (C-V) method also confirm the high predictive accuracy of the RF model. The highest contribution level of the cement towards the prediction was also reported in the sensitivity analysis and showed a 31.24% contribution. These PML methods can be effectively employed to anticipate the mechanical properties of concretes.

Machine Learning Assisted Prediction of Microstructures and Young’s Modulus of Biomedical Multi-Component β-Ti Alloys

Recently, the development of β-titanium (Ti) alloys with a low Young’s modulus as human implants has been the trend of research in biomedical materials. However, designing β-titanium alloys by conventional experimental methods is too costly and inefficient. Therefore, it is necessary to propose a method that can efficiently and reliably predict the microstructures and the mechanical properties of biomedical titanium alloys. In this study, a machine learning prediction method is proposed to accelerate the design of biomedical multi-component β-Ti alloys with low moduli. Prediction models of microstructures and Young’s moduli were built at first. The performances of the models were improved by introducing new experimental data. With the help of the models, a Ti–13Nb–12Ta–10Zr–4Sn (wt.%) alloy with a single β-phase microstructure and Young’s modulus of 69.91 GPa is successfully developed. This approach could also be used to design other advanced materials.

Machine learning methods for prediction of cancer driver genes: a survey paper.

Identifying the genes and mutations that drive the emergence of tumors is a critical step to improving our understanding of cancer and identifying new directions for disease diagnosis and treatment. Despite the large volume of genomics data, the precise detection of driver mutations and their carrying genes, known as cancer driver genes, from the millions of possible somatic mutations remains a challenge. Computational methods play an increasingly important role in discovering genomic patterns associated with cancer drivers and developing predictive models to identify these elements. Machine learning (ML), including deep learning, has been the engine behind many of these efforts and provides excellent opportunities for tackling remaining gaps in the field. Thus, this survey aims to perform a comprehensive analysis of ML-based computational approaches to identify cancer driver mutations and genes, providing an integrated, panoramic view of the broad data and algorithmic landscape within this scientific problem. We discuss how the interactions among data types and ML algorithms have been explored in previous solutions and outline current analytical limitations that deserve further attention from the scientific community. We hope that by helping readers become more familiar with significant developments in the field brought by ML, we may inspire new researchers to address open problems and advance our knowledge towards cancer driver discovery.

Dynamic optimization with side information

We develop a tractable and flexible data-driven approach for incorporating side information into multi-stage stochastic programming. The proposed framework uses predictive machine learning methods (such as k-nearest neighbors, kernel regression, and random forests) to weight the relative importance of various data-driven uncertainty sets in a robust optimization formulation. Through a novel measure concentration result for a class of supervised machine learning methods, we prove that the proposed approach is asymptotically optimal for multi-period stochastic programming with side information. We also describe a general-purpose approximation for these optimization problems, based on overlapping linear decision rules, which is computationally tractable and produces high-quality solutions for dynamic problems with many stages. Across a variety of multi-stage and single-stage examples in inventory management, finance, and shipment planning, our method achieves improvements of up to 15% over alternatives and requires less than one minute of computation time on problems with twelve stages.

The application of machine learning methods for prediction of metal immobilization remediation by biochar amendment in soil

Biochar has been used widely in heavy metal contaminated sites as a soil remediation agent. However, due to the diversity of soils, biochars, and heavy metal contamination status, the remediation efficiency is difficult to measure, owing to a variety of parameters such as soil, biochar properties, and remediation procedure. Thus, an appropriate method to predict the remediation results and to select the appropriate biochar for the remediation is required. We initially created a database on soil remediation by biochars, which has 930 datasets with 74 biochars and 43 soils in it, based on collecting and organizing data from published literatures. Then, using data from the database, we modeled the remediation of five heavy metals and metalloids (lead, cadmium, arsenic, copper, and zinc) by biochars using machine learning (ML) methods such as artificial neural network (ANN) and random forest (RF) to predict remediation efficiency based on biochar characteristics, soil physiochemical properties, incubation conditions (e.g., water holding capacity and remediation time), and the initial state of heavy metal. The ANN and RF models outperform the lineal model in terms of accuracy and predictive performance (R2 > 0.84). Meanwhile, model tolerance of the missing data and reliability of the interpolation were studied by the predicted outputs of the models. The results showed that both ANN and RF have excellent performances, with the RF model having a higher tolerance for missing data. Finally, through the interpretability of ML models, the contribution of factors used in the model were analyzed and the findings revealed that the most influential elements of remediation were the type of heavy metals, the pH value of biochar, and the dosage and remediation time. The relative importance of variables could provide the right direction for better remediation of heavy metals in soil.

Prognostic value of machine learning for acute heart failure

Abstract Funding Acknowledgements Type of funding sources: None. Background At onset of acute heart failure (AHF), various clinical fundamental parameters including vital sign, laboratory data, or initial treatment were investigated, and we can roughly estimate the prognosis. However, machine learning method for prediction of the prognosis was not studied. Purpose To elucidate prognostic value of machine learning for AHF comparing conventional statistical model. Methods We enrolled consecutive 300 patients with AHF (79.5 ± 12.1 years, 158 Males). Patients with acute coronary syndrome, mechanical circulatory support cases, and cardio-pulmonary arrest cases were excluded. The patients were randomly divided into 80% (240 cases) and 20% (60 cases), and the former was used as train data, and the latter as validation data. Objective variable was set as cardiac death in one year. First, logistic regression analysis with Akaike’s information criterion (AIC) was performed, and extracted predictive parameters. The predictive model for the cardiac prognosis was constructed by cut-off value of ROC curve analysis of propensity score was calculated. Next, machine learning (random forest method and deep learning) to build predictive model was performed with the predictors. Finally, accuracy of each predictive model was compared. Results Thirty cases showed cardiac death in one year. Logistic regression with AIC extracted 8 predictors, and the cut off-value of propensity score with the 6 parameters was 0.110. The accuracy was 0.714 and area under ROC (AUROC) was 0.836. Conversely, random forest method demonstrated the accuracy as 0.927, AUROC 0.860. On deep learning, the accuracy was 0.937 and AUROC 0.901. The top 4 high feature importance of random forest were Cl/red blood cell count/pH/Anion Gap. However, accuracy of those predictors was lower than that of machine learning. Conclusion Machine learning was a powerful tool to predict cardiac prognosis of AHF, comparing with conventional statistical model. Abstract Figure. Statistical model

Comparisons of Forecasting for Survival Outcome for Head and Neck Squamous Cell Carcinoma by using Machine Learning Models based on Multi-omics.

Background: Machine learning methods showed excellent predictive ability in a wide range of fields. For the survival of head and neck squamous cell carcinoma (HNSC), its multi-omics influence is crucial. This study attempts to establish a variety of machine learning multi-omics models to predict the survival of HNSC and find the most suitable machine learning prediction method. Methods: The HNSC clinical data and multi-omics data were downloaded from the TCGA database. The important variables were screened by the LASSO algorithm. We used a total of 12 supervised machine learning models to predict the outcome of HNSC survival and compared the results. In vitro qPCR was performed to verify core genes predicted by the random forest algorithm. Results: For omics of HNSC, the results of the twelve models showed that the performance of multi-omics was better than each single-omic alone. Results were presented, which showed that the Bayesian network(BN) model (area under the curve [AUC] 0.8250, F1 score=0.7917) and random forest(RF) model (area under the curve [AUC] 0.8002,F1 score=0.7839) played good prediction performance in HNSC multi-omics data. The results of in vitro qPCR were consistent with the RF algorithm. Conclusion: Machine learning methods could better forecast the survival outcome of HNSC. Meanwhile, this study found that the BN model and the RF model were the most superior. Moreover, the forecast result of multi-omics was better than single-omic alone in HNSC.

Laptop Price Prediction using Machine Learning

This paper presents a Laptop price prediction system by using the supervised machine learning technique. The research uses multiple linear regression as the machine learning prediction method which offered 81% prediction precision. Using multiple linear regression, there are multiple independent variables but one and only one dependent variable whose actual and predicted values are compared to find precision of results. This paper proposes a system where price is dependent variable which is predicted, and this price is derived from factors like Laptop’s model, RAM, ROM (HDD/SSD), GPU, CPU, IPS Display, and Touch Screen.

Real-time rock mass condition prediction with TBM tunneling big data using a novel rock–machine mutual feedback perception method

Real-time perception of rock mass information is of great importance to efficient tunneling and hazard prevention in tunnel boring machines (TBMs). In this study, a TBM–rock mutual feedback perception method based on data mining (DM) is proposed, which takes 10 tunneling parameters related to surrounding rock conditions as input features. For implementation, first, the database of TBM tunneling parameters was established, in which 10,807 tunneling cycles from the Songhua River water conveyance tunnel were accommodated. Then, the spectral clustering (SC) algorithm based on graph theory was introduced to cluster the TBM tunneling data. According to the clustering results and rock mass boreability index, the rock mass conditions were classified into four classes, and the reasonable distribution intervals of the main tunneling parameters corresponding to each class were presented. Meanwhile, based on the deep neural network (DNN), the real-time prediction model regarding different rock conditions was established. Finally, the rationality and adaptability of the proposed method were validated via analyzing the tunneling specific energy, feature importance, and training dataset size. The proposed TBM–rock mutual feedback perception method enables the automatic identification of rock mass conditions and the dynamic adjustment of tunneling parameters during TBM driving. Furthermore, in terms of the prediction performance, the method can predict the rock mass conditions ahead of the tunnel face in real time more accurately than the traditional machine learning prediction methods.

Abstract 9495: Prognostic Value of Machine Learning on Clinical Parameters for Cardiac Prognosis in Patinets with Acute Congestive Heart Failure

Introduction: Congestive heart failure (CHF) in acute phase, many clinical fundamental parameters including vital sign, laboratory data, or initial treatment were investigated, and we can roughly estimate the prognosis with the parameters. However, machine learning method for prediction of the prognosis is still challenging. Hypothesis: Machine learning methods have higher performance for predicting cardiac prognosis in patients with CHF than conventional statistical model. Methods: We enrolled consecutive 300 patients with CHF (79.5±12.1 years, 158 Males). Patients with acute coronary syndrome, mechanical circulatory support cases, and cardio-pulmonary arrest cases were excluded. Objective variable was set as cardiac death in one year. First, multivariate stepwise logistic regression analysis with Akaike’s information criterion (AIC) was performed, and extracted parameters which had the most versatile predictive value. Propensity score (PS) was calculated with the extracted parameters, and the prediction model constructed by cut-off value of ROC curve analysis of PS. Next, machine learning (random forest method and deep learning) to build predictive model was performed with the predictors. Finally, accuracy of each predictive model was compared. Results: Thirty cases showed cardiac death in one year. Logistic regression with AIC extracted 8 predictors, and the cut off-value of propensity score with the 8 parameters was 0.110. The accuracy was 0.714 and area under ROC (AUROC) was 0.836. Conversely, random forest method demonstrated the accuracy as 0.927, AUROC 0.860. On deep learning, the accuracy was 0.937 and AUROC 0.901.The top 4 high feature importance of random forest were Cl/red blood cell count/pH/Anion Gap. However, accuracy of those predictors was lower than that of machine learning. Conclusions: Machine learning was a powerful tool to predict cardiac prognosis of CHF, comparing with conventional statistical model.