Machine Learning-based Prediction Research Articles

Machine Learning-Based Prediction of Terrestrial Heat Flow in the Songliao Basin

Machine Learning-Based Prediction of Terrestrial Heat Flow in the Songliao Basin

Machine Learning-Based Predictions of Porosity during Cold Spray Deposition of High Entropy Alloy Coatings

Porosity poses a challenge to the mechanical properties of cold sprayed coatings, especially when it is open or surface-connected, limiting the coatings’ capabilities to act as a barrier. The porosity formation is dependent on the feedstock powder characteristics and the cold spray process parameters. We present a machine learning-based approach to predict porosity based on the above-mentioned factors. Nine different machine learning models based on linear regression (LR), decision trees, random forests, gradient boosting, support vector machine (SVM), and neural networks were explored. Considering the excellent properties of high entropy alloys, Cantor alloy was taken as the consumable. Our dataset, derived from the literature and experiments, identified SVM with a linear kernel and LR as the top-performing models based on the Pearson correlation coefficient (PCC) and root mean square error, where the PCC values exceeded 0.8. The SHapley Additive exPlanations method helped in identifying that the type of gas and powder are the top two factors in pore formation.

Machine learning-based prediction and experimental validation of heavy metal adsorption capacity of bentonite

As a natural adsorbent material, bentonite is widely used in the field of heavy metal adsorption. The heavy metal adsorption capacity of bentonite varies significantly in studies due to the differences in the properties of bentonite, solution, and heavy metal. To achieve accurate predictions of bentonite's heavy metal adsorption capacity, this study employed six machine learning (ML) regression algorithms to investigate the adsorption characteristics of bentonite. Finally, an eXtreme Gradient Boosting Regression (XGB) model with outstanding predictive performance was constructed. Explanation analysis of the XGB model further reveal the importance and influence manner of each input feature in predicting the heavy metal adsorption capacity of bentonite. The feature categories influencing heavy metal adsorption capacity were ranked in order of importance as adsorption conditions > bentonite properties > heavy metal properties. Furthermore, a web-based graphical user interface (GUI) software was developed, facilitating researchers and engineers to conveniently use the XGB model for predicting the heavy metal adsorption capacity of bentonite. This study provides new insights into the adsorption behaviors of bentonite for heavy metals, offering guidance and support for enhancing its application efficiency and addressing heavy metal pollution remediation.

Benchmarking machine learning-based real-time respiratory signal predictors in 4D SBRT.

Stereotactic body radiotherapy of thoracic and abdominal tumors has to account for respiratory intrafractional tumor motion. Commonly, an external breathing signal is continuously acquired that serves as a surrogate of the tumor motion and forms the basis of strategies like breathing-guided imaging and gated dose delivery. However, due to inherent system latencies, there exists a temporal lag between the acquired respiratory signal and the system response. Respiratory signal prediction models aim to compensate for the time delays and to improve imaging and dosedelivery. The present study explores and compares six state-of-the-art machine and deep learning-based prediction models, focusing on real-time and real-world applicability. All models and data are provided as open source and data to ensure reproducibility of the results and fosterreuse. The study was based on 2502 breathing signals ( h) acquired during clinical routine, split into independent training (50%), validation (20%), and test sets (30%). Input signal values were sampled from noisy signals, and the target signal values were selected from corresponding denoised signals. A standard linear prediction model (Linear), two state-of-the-art models in general univariate signal prediction (Dlinear, Xgboost), and three deep learning models (Lstm, Trans-Enc, Trans-TSF) were chosen. The prediction performance was evaluated for three different prediction horizons (480, 680, and 920ms). Moreover, the robustness of the different models when applied to atypical, that is, out-of-distribution (OOD) signals, was analyzed. The Lstm model achieved the lowest normalized root mean square error for all prediction horizons. The prediction errors only slightly increased for longer horizons. However, a substantial spread of the error values across the test signals was observed. Compared to typical, that is, in-distribution test signals, the prediction accuracy of all models decreased when applied to OOD signals. The more complex deep learning models Lstm and Trans-Enc showed the least performance loss, while the performance of simpler models like Linear dropped the most. Except for Trans-Enc, inference times for the different models allowed for real-time application. The application of the Lstm model achieved the lowest prediction errors. Simpler prediction filters suffer from limited signal history access, resulting in a drop in performance for OODsignals.

Corrosion prediction for preventive protection of aircraft heritage

The paper presents a study on corrosion prediction for preventive aeronautical heritage protection, considering the aeronautical heritage stored or exhibited in an aviation museum. For the purpose of the study, the hangar with exhibited historical aircraft of significant cultural and societal value is located in the Aviation Museum Kbely, Prague, Czech Republic. Until now, such a preventive approach to protecting the aircraft heritage constituted from ancient aluminum alloys, in particular, has not been presented rigorously. Monitoring the hangar meteorological, pollution, and environmental data are acquired and interrelated with measured corrosion data to find a statistical model describing atmospheric corrosion in the hangar environment. The statistical model searched represents a Gaussian process based on a likelihood approach. As a result, the Gaussian process model is regressed to predict the corrosion of aluminum alloy-based artifacts in the monitored hangar with the marginal likelihood that is compared to machine learning-based prediction. Finally, it is shown that atmospheric corrosion is accurately predicted only when, among others, a synergistic effect of airborne pollutants and wind speed is considered.

Machine learning-based prediction of tear osmolarity for contact lens practice.

This study addressed the utilisation of machine learning techniques to estimate tear osmolarity, a clinically significant yet challenging parameter to measure accurately. Elevated tear osmolarity has been observed in contact lens wearers and is associated with contact lens-induced dry eye, a common cause of discomfort leading to discontinuation of lens wear. The study explored machine learning, regression and classification techniques to predict tear osmolarity using routine clinical parameters. The data set consisted of 175 participants, primarily healthy subjects eligible for soft contact lens wear. Various clinical assessments were performed, including symptom assessment with the Ocular Surface Disease Index and 5-Item Dry Eye Questionnaire (DEQ-5), tear meniscus height (TMH), tear osmolarity, non-invasive keratometric tear film break-up time (NIKBUT), ocular redness, corneal and conjunctival fluorescein staining and Meibomian glands loss. The results revealed that simple linear regression was insufficient for accurate osmolarity prediction. Instead, more advanced regression models achieved a moderate level of predictive power, explaining approximately 32% of the osmolarity variability. Notably, key predictors for osmolarity included NIKBUT, TMH, ocular redness, Meibomian gland coverage and the DEQ-5 questionnaire. In classification tasks, distinguishing between low (<299 mOsmol/L), medium (300-307 mOsmol/L) and high osmolarity (>308 mOsmol/L) levels yielded an accuracy of approximately 80%. Key parameters for classification were similar to those in regression models, emphasising the importance of NIKBUT, TMH, ocular redness, Meibomian glands coverage and the DEQ-5 questionnaire. This study highlights the potential benefits of integrating machine learning into contact lens research and practice. It suggests the clinical utility of assessing Meibomian glands and NIKBUT in contact lens fitting and follow-up visits. Machine learning models can optimise contact lens prescriptions and aid in early detection of conditions like dry eye, ultimately enhancing ocular health and the contact lens wearing experience.

Predicting Real-World Penny Auction Durations by Integrating Game Theory and Machine Learning

Game theory and machine learning are two widely used techniques for predicting the outcomes of strategic interactions among humans. However, the game theory-based approach often relies on strong rationality and informational assumptions, while the machine learning-based approach typically requires the testing data to come from the same distribution as the training data. Our work studies how to integrate the two techniques to address these weaknesses. We focus on the interactions among real bidders in penny auctions, and develop a three-stage framework to predict the distributions of auction durations, which indicate the numbers of bids and auctioneer revenues. Specifically, we first leverage a pre-trained neural network to encode the descriptions of products in auctions into embeddings. Second, we apply game theory models to make preliminary predictions of auction durations. In particular, we tackle the challenge of accurately inferring parameters in game theory models. Third, we develop a Multi-Branch Mixture Density Network to learn the mapping from product embeddings and game-theoretic predictions to the distributions of actual auction durations. Experiments on real-world penny auction data demonstrate that our framework outperforms both game theory-based and machine learning-based prediction approaches.

Performance prediction and Bayesian optimization of screw compressors using Gaussian Process Regression

Optimizing the performance of screw compressors is critical for achieving high efficiency and reducing costs in various industrial and engineering applications. Often, the design and optimization processes are time-consuming owing to the underlying iterative complex analyses. In this context, the present research investigates the potential of Gaussian Process Regression (GPR) and Bayesian optimization for the prediction and optimization of the performance of an oil-flooded screw compressor. Specifically, the GPR-based surrogate model is developed to predict the compressor performance characteristics based on its four main geometrical design parameters such as wrap angle, relative length, tip speed of the male rotor and built-in volume ratio. The model is trained using a dataset comprising 19,200 data points relating the input design parameters with the compressor performance, obtained using physics-based multi-chamber thermodynamic models. While four different learning algorithms such as Support Vector Machine (SVM), Artificial Neural Network (ANN), Polynomial regression and GPR are explored, the GPR performed the best resulting in an R2 value of 0.99 for the test dataset after hyperparameter tuning. Further, the model is also experimentally validated on a completely unseen dataset, showing very good predictions with a maximum error of 5%. The resulting surrogate model is then used to optimize the compressor design parameters using Bayesian optimization. The results are compared with optimization using Genetic Algorithm (GA) and physics-based multi-chamber thermodynamic model. It was shown the proposed approach results in similar optimal design parameters but with a significantly less optimization time by a factor of 7. The study highlight the potential of machine learning-based prediction and optimization of screw compressors in engineering applications.

Machine learning-based prediction of rheumatoid arthritis with development of ACPA autoantibodies in the presence of non-HLA genes polymorphisms.

Machine learning (ML) algorithms can handle complex genomic data and identify predictive patterns that may not be apparent through traditional statistical methods. They become popular tools for medical applications including prediction, diagnosis or treatment of complex diseases like rheumatoid arthritis (RA). RA is an autoimmune disease in which genetic factors play a major role. Among the most important genetic factors predisposing to the development of this disease and serving as genetic markers are HLA-DRB and non-HLA genes single nucleotide polymorphisms (SNPs). Another marker of RA is the presence of anticitrullinated peptide antibodies (ACPA) which is correlated with severity of RA. We use genetic data of SNPs in four non-HLA genes (PTPN22, STAT4, TRAF1, CD40 and PADI4) to predict the occurrence of ACPA positive RA in the Polish population. This work is a comprehensive comparative analysis, wherein we assess and juxtapose various ML classifiers. Our evaluation encompasses a range of models, including logistic regression, k-nearest neighbors, naïve Bayes, decision tree, boosted trees, multilayer perceptron, and support vector machines. The top-performing models demonstrated closely matched levels of accuracy, each distinguished by its particular strengths. Among these, we highly recommend the use of a decision tree as the foremost choice, given its exceptional performance and interpretability. The sensitivity and specificity of the ML models is about 70% that are satisfying. In addition, we introduce a novel feature importance estimation method characterized by its transparent interpretability and global optimality. This method allows us to thoroughly explore all conceivable combinations of polymorphisms, enabling us to pinpoint those possessing the highest predictive power. Taken together, these findings suggest that non-HLA SNPs allow to determine the group of individuals more prone to develop RA rheumatoid arthritis and further implement more precise preventive approach.

Machine Learning-Based Prediction of Intergranular Corrosion Resistance in Austenitic Stainless Steels Exposed to Various Heat Treatments

Machine Learning-Based Prediction of Intergranular Corrosion Resistance in Austenitic Stainless Steels Exposed to Various Heat Treatments

Supervised Machine Learning-Based Prediction of Hydrogen Storage Classes Utilizing Dibenzyltoluene as an Organic Carrier.

Dibenzyltoluene (H0-DBT), a Liquid Organic Hydrogen Carrier (LOHC), presents an attractive solution for hydrogen storage due to its enhanced safety and ability to store hydrogen in a concentrated liquid form. The utilization of machine learning proves essential for accurately predicting hydrogen storage classes in H0-DBT across diverse experimental conditions. This study focuses on the classification of hydrogen storage data into three classes, low-class, medium-class and high-class, based on the hydrogen storage capacity values. We introduce Hydrogen Storage Prediction with the Support Vector Machine (HSP-SVM) model to predict the hydrogen storage classes accurately. The performance of the proposed HSP-SVM model was investigated using various techniques, which included 5-Fold Cross Validation (5-FCV), Resubstitution Validation (RV), and Holdout Validation (HV). The accuracy of the HV approach for the low, medium, and high class was 98.5%, 97%, and 98.5%, respectively. The overall accuracy of HV approach reached 97% with a miss clarification rate of 3%, whereas 5-FCV and RV possessed an overall accuracy of 93.9% with a miss clarification rate of 6.1%. The results reveal that the HV approach is optimal for predicting the hydrogen storage classes accurately.

Machine Learning-Based Prediction of NPSH, Noise, and Vibration Levels in Radial Pumps Under Cavitation Conditions

Cavitation, a physical phenomenon that detrimentally affects pump performance and reduces pump life, can cause wear on pump elements. Various engineering methods have been developed to identify the initiation and full development of the cavitation process. One such method is the determination of the net positive suction head (NPSH) through a 3% decrease in total head (Hm) at a constant flow rate. In radial pumps, commonly used in agricultural irrigation and industry, cavitation conditions result in a sudden drop in the Hm-Q curve, making it challenging to detect the 3% Hm value drop. This study differs from others in the literature by modelling NPSH, noise, and vibration levels using three machine learning models, specifically artificial neural networks (ANN), support vector machines (SVM), and decision tree regression (DTR). The best-performing model predicts NPSH, noise, and vibration levels corresponding to a 3% decrease in Hm level. The present study determined the NPSH values of a horizontal shaft centrifugal pump at different flow rates and constant operating speed, and the vibration and noise levels were measured for these NPSH values. For each of the NPSH, noise, and vibration levels, ANN, SVM and DTR models were created. The performances of these models were evaluated using criteria such as root mean squared error (RMSE), Mean Absolute Error (MAE) and mean absolute percentage error (MAPE). In addition, Taylor and error box diagrams were created. The ANN model and DTR yielded high accuracy predictions for NPSH values (R2 = 0.86 and R2 = 0.8, respectively). The ANN model provided the best prediction performance for noise and vibration levels. By entering the level of 3% drop in the Hm value of the pump as external data input to the ANN model, NPSH3, noise, and vibration levels were determined. The ANN models can be effectively employed to determine NPSH3, noise, and vibration levels, particularly in radial flow pumps, where detecting 3% reductions in manometric height value is challenging.

Multi-objective optimization and machine learning-based prediction of tensile properties of an armchair graphene sheet

Graphene is a widely used nanoparticle in different industries, especially in nanocomposite applications. Prediction of its properties is of great importance for engineers. Therefore, a comprehensive study is first performed in this study to investigate the effect of temperature, strain rate, number of layers, and dimensions on the tensile properties of graphene nanosheets including elastic modulus (E), yield strength (YS), and ultimate tensile strength (UTS) using molecular dynamics simulation. Then, E, YS, and UTS are simultaneously optimized via response surface methodology. To determine the simplest and the most accurate machine learning algorithm for the prediction of tensile properties, three algorithms of Decision Tree (DT), Random Forest (RF), and Gradient Boosted Tree (GBT) are compared and the GBT algorithm is introduced as the best one. Furthermore, the architecture of each algorithm was optimized via the Taguchi design of experiment method to enhance the prediction accuracy. The DT algorithm with a maximum depth of 8 was obtained as the most accurate one.

Machine learning-based prediction of seasonal hypoxia in eutrophic estuary using capacitive potentiometric sensor

A hypoxia occurred in eutrophic estuary was predicted using long short-term memory (LSTM) model with prediction time steps (PTSs) of 0, 1, 12, and 24 h. A capacitive potential (CP), which provides quantitative information on dissolved oxygen (DO) concentration, was used as a predictor along with precipitation, tide level, salinity, and water temperature. First, annual changes in DO concentration were clustered in three phases of annual DO trends (oversaturation, depletion, and stable) using k-means clustering. CP was the most influential variable in clustering the DO phases. The LSTM was implemented to predict the DO phases and hypoxia occurrences. In the simultaneous prediction of the depletion phase and hypoxia occurrence with a 12 h PTS, the accuracy was 92.1% using CP along with other variables; it was 3.3% higher than that achieved using variables other than CP. In the case of predicting the depletion phase and hypoxia non-occurrence using CP along with other variables, the accuracy was 61.1%, which was 5.5% higher than that when CP was not used. When using CP along with other variables, the total accuracy was highest for all PTS. Overall, the utilization of CP and machine learning techniques enables accurate predictions of both short-term and long-term hypoxia occurrences, providing us with the opportunity to proactively respond to disasters in aquaculture and environmental management due to hypoxia.

Efficient surrogate models for materials science simulations: Machine learning-based prediction of microstructure properties

Determining, understanding, and predicting the so-called structure–property relation is an important task in many scientific disciplines, such as chemistry, biology, meteorology, physics, engineering, and materials science. Structure refers to the spatial distribution of, e.g., substances, material, or matter in general, while property is a resulting characteristic that usually depends in a non-trivial way on spatial details of the structure. Traditionally, forward simulations models have been used for such tasks. Recently, several machine learning algorithms have been applied in these scientific fields to enhance and accelerate simulation models or as surrogate models. In this work, we develop and investigate the applications of six machine learning techniques based on two different datasets from the domain of materials science: data from a two-dimensional Ising model for predicting the formation of magnetic domains and data representing the evolution of dual-phase microstructures from the Cahn–Hilliard model. We analyze the accuracy and robustness of all models and elucidate the reasons for the differences in their performances. The impact of including domain knowledge through tailored features is studied, and general recommendations based on the availability and quality of training data are derived from this.

Machine learning-based prediction of proteins’ architecture using sequences of amino acids and structural alphabets

In addition to the growth of protein structures generated through wet laboratory experiments and deposited in the PDB repository, AlphaFold predictions have significantly contributed to the creation of a much larger database of protein structures. Annotating such a vast number of structures has become an increasingly challenging task. CATH is widely recognized as one the most common platforms for addressing this challenge, as it classifies proteins based on their structural and evolutionary relationships, offering the scientific community an invaluable resource for uncovering various properties, including functional annotations. While CATH annotation involves – to some extent – human intervention, keeping up with the classification of the rapidly expanding repositories of protein structures has become exceedingly difficult. Therefore, there is a pressing need for a fully automated approach. On the other hand, the abundance of protein sequences stemming from next generation sequencing technologies, lacking structural annotations, presents an additional challenge to the scientific community. Consequently, 'pre-annotating’ protein sequences with structural features, ensuring a high level of precision, could prove highly advantageous. In this paper, after a thorough investigation, we introduce a novel machine-learning model capable of classifying any protein domain, whether it has a known structure or not, into one of the 40 main CATH Architectures. We achieve an F1 Score of 0.92 using only the amino acid sequence and a score of 0.94 using both the sequence of amino acids and the sequence of structural alphabets. Communicated by Ramaswamy H. Sarma

Machine learning-based predictions of current and future susceptibility to retrogressive thaw slumps across the Northern Hemisphere

Retrogressive thaw slumps (RTSs) caused by the thawing of ground ice on permafrost slopes have dramatically increased and become a common permafrost hazard across the Northern Hemisphere during previous decades. However, a gap remains in our comprehensive understanding of the spatial controlling factors, including the climate and terrain, that are conducive to these RTSs at a global scale. Using machine learning methodologies, we mapped the current and future RTSs susceptibility distributions by incorporating a range of environmental factors and RTSs inventories. We identified freezing-degree days and maximum summer rainfall as the primary environmental factors affecting RTSs susceptibility. The final ensemble susceptibility map suggests that regions with high to very high susceptibility could constitute (11.6 ± 0.78)% of the Northern Hemisphere's permafrost region. When juxtaposed with the current (2000–2020) RTSs susceptibility map, the total area with high to very high susceptibility could witness an increase ranging from (31.7 ± 0.65)% (SSP585) to (51.9 ± 0.73)% (SSP126) by the 2041–2060. The insights gleaned from this study not only offer valuable implications for engineering applications across the Northern Hemisphere, but also provide a long-term insight into the potential change of RTSs in permafrost regions in response to climate change.

Cm-siRPred: Predicting chemically modified siRNA efficiency based on multi-view learning strategy

The rational modification of siRNA molecules is crucial for ensuring their drug-like properties. Machine learning-based prediction of chemically modified siRNA (cm-siRNA) efficiency can significantly optimize the design process of siRNA chemical modifications, saving time and cost in siRNA drug development. However, existing in-silico methods suffer from limitations such as small datasets, inadequate data representation capabilities, and lack of interpretability. Therefore, in this study, we developed the Cm-siRPred algorithm based on a multi-view learning strategy. The algorithm employs a multi-view strategy to represent the double-strand sequences, chemical modifications, and physicochemical properties of cm-siRNA. It incorporates a cross-attention model to globally correlate different representation vectors and a two-layer CNN module to learn local correlation features. The algorithm demonstrates exceptional performance in cross-validation experiments, independent dataset, and case studies on approved siRNA drugs, and showcasing its robustness and generalization ability. In addition, we developed a user-friendly webserver that enables efficient prediction of cm-siRNA efficiency and assists in the design of siRNA drug chemical modifications. In summary, Cm-siRPred is a practical tool that offers valuable technical support for siRNA chemical modification and drug efficiency research, while effectively assisting in the development of novel small nucleic acid drugs. Cm-siRPred is freely available at https://cellknowledge.com.cn/sirnapredictor/.

Machine learning-based prediction of sertraline concentration in patients with depression through therapeutic drug monitoring.

Background: Sertraline is a commonly employed antidepressant in clinical practice. In order to control the plasma concentration of sertraline within the therapeutic window to achieve the best effect and avoid adverse reactions, a personalized model to predict sertraline concentration is necessary. Aims: This study aimed to establish a personalized medication model for patients with depression receiving sertraline based on machine learning to provide a reference for clinicians to formulate drug regimens. Methods: A total of 415 patients with 496 samples of sertraline concentration from December 2019 to July 2022 at the First Hospital of Hebei Medical University were collected as the dataset. Nine different algorithms, namely, XGBoost, LightGBM, CatBoost, random forest, GBDT, SVM, lasso regression, ANN, and TabNet, were used for modeling to compare the model abilities to predict sertraline concentration. Results: XGBoost was chosen to establish the personalized medication model with the best performance (R 2 = 0.63). Five important variables, namely, sertraline dose, alanine transaminase, aspartate transaminase, uric acid, and sex, were shown to be correlated with sertraline concentration. The model prediction accuracy of sertraline concentration in the therapeutic window was 62.5%. Conclusion: In conclusion, the personalized medication model of sertraline for patients with depression based on XGBoost had good predictive ability, which provides guidance for clinicians in proposing an optimal medication regimen.

An improved machine learning-based prediction framework for early detection of events in heart failure patients using mHealth

An improved machine learning-based prediction framework for early detection of events in heart failure patients using mHealth