Gradient Boosting Regression Research Articles

Prediction of Single-Well Production Rate after Hydraulic Fracturing in Unconventional Gas Reservoirs Based on Ensemble Learning Model

To address the significant challenges in determining the single-well production of tight gas and shale gas after hydraulic fracturing, artificial intelligence (AI) methods were implemented. Machine learning (ML) algorithms such as random forest (RF), extremely randomized trees (ET), lightweight gradient boosting machines (LightGBM), gradient boosting regression (GBR), and linear regression (LR) were utilized in conjunction with reservoir geology, engineering parameters, and production data to develop several foundational models for forecasting the production of unconventional gas wells. The accuracy of these models was evaluated. Based on this, improvements in the models’ predictive accuracy and generalizability were achieved through the ensemble of machine learning models. Furthermore, this paper selected two representative tight and shale gas reservoirs to demonstrate the application of the ensemble model for well production forecasting, and a comparative analysis with actual production data was conducted. For tight gas reservoir A, the blending model achieved an MAE of 0.8419 and an MSE of 1.0930, with an R2 score of 0.8812. For shale gas reservoir B, the blending model achieved an MAE of 1.4841 and an MSE of 3.1629, with an R2 score of 0.9524. The results of the case studies indicate that the ensemble model approach employed in this study has a higher predictive accuracy and reliability than a single machine learning algorithm, and is capable of handling high-dimensional, large-scale, and imbalanced data, offering scientific validation and technical support for the assessment of the well productivity in tight and shale gas wells.

Prediction of wastewater treatment plant performance through machine learning techniques

This study investigates the use of developed machine learning techniques for modeling the performance of AlHayer, Saudi Arabia, wastewater treatment plant (ALWTP). Three physio-chemical characteristics were measured and predicted, including chemical oxygen demand (COD), biological oxygen demand (BOD), and suspended solids (SS), at ALWTP. The pre-evaluation of collected data revealed the effective capabilities of ALWTP for the removal of suspended solids, organic, and nutrient pollutants. To estimate the physio-chemical characteristics of ALWATP, four developed machine learning techniques were evaluated and compared. Logistic regression (LR), random forest (RF), gradient boosting (GB), and support vector regression (SVR) were designed. The evaluation of the proposed models showed RF outperformed other proposed models for estimating COD and SS with accuracy 91 % and 95 % in terms coefficient of determination (R2); however, GB was found the best, with accuracy 92 %, for detecting the BOD performance of ALWATP. This indicates ensemble learning models, RF and GB, can be considered a superiority soft solution for estimating physio-chemical characteristics of wastewater treatment plant.

Unveiling interactions between intervertebral disc morphologies and mechanical behavior through personalized finite element modeling.

Introduction: Intervertebral Disc (IVD) Degeneration (IDD) is a significant health concern, potentially influenced by mechanotransduction. However, the relationship between the IVD phenotypes and mechanical behavior has not been thoroughly explored in local morphologies where IDD originates. This work unveils the interplays among morphological and mechanical features potentially relevant to IDD through Abaqus UMAT simulations. Methods: A groundbreaking automated method is introduced to transform a calibrated, structured IVD finite element (FE) model into 169 patient-personalized (PP) models through a mesh morphing process. Our approach accurately replicates the real shapes of the patient's Annulus Fibrosus (AF) and Nucleus Pulposus (NP) while maintaining the same topology for all models. Using segmented magnetic resonance images from the former project MySpine, 169 models with structured hexahedral meshes were created employing the Bayesian Coherent Point Drift++ technique, generating a unique cohort of PP FE models under the Disc4All initiative. Machine learning methods, including Linear Regression, Support Vector Regression, and eXtreme Gradient Boosting Regression, were used to explore correlations between IVD morphology and mechanics. Results: We achieved PP models with AF and NP similarity scores of 92.06\% and 92.10\% compared to the segmented images. The models maintained good quality and integrity of the mesh. The cartilage endplate (CEP) shape was represented at the IVD-vertebra interfaces, ensuring personalized meshes. Validation of the constitutive model against literature data showed a minor relative error of 5.20%. Discussion: Analysis revealed the influential impact of local morphologies on indirect mechanotransduction responses, highlighting the roles of heights, sagittal areas, and volumes. While the maximum principal stress was influenced by morphologies such as heights, the disc's ellipticity influenced the minimum principal stress. Results suggest the CEPs are not influenced by their local morphologies but by those of the AF and NP. The generated free-access repository of individual disc characteristics is anticipated to be a valuable resource for the scientific community with a broad application spectrum.

Performance Analysis of Voting Regression-Based Ensemble Learning Methods for Food Demand Forecasting

Accurate demand forecasting has become very significant, especially in the food sector, since many products have a limited lifespan, and improper management can cause the organization to incur enormous waste and loss. This research focuses on the problem of analyzing accurate food demand and its prediction through the application of machine learning techniques. An ensemble technique such as voting regression is employed, leveraging Random Forest Regressor and Gradient Boosting Regressor, which were the top-performing models. By integrating these two techniques using voting regression, we can leverage their complementary strengths to enhance prediction accuracy. The ensemble aggregates the predictions of both models, typically by averaging, to produce a final prediction. This technique can assist in reducing overfitting and capturing complex relationships in the data, resulting in more robust and accurate forecasts of food demand. The outcomes of the R2-score, Root Mean Square Error (RMSE) and Mean Average Error (MAE) are 0.99, 0.01, and 0.00, respectively.

Machine learning-based comparative analysis of weather-driven rice and sugarcane yield forecasting models

This study investigates the use of various machine learning algorithms for predicting rice and sugarcane yields for Navsari district of Gujarat, India. Recognizing the critical role of weather in crop productivity, accurate forecasting becomes essential for effective resource management. In methodology, weekly averages and weighted weather indices were computed based on daily weather data to develop forecast models using machine learning algorithms such as Random Forest (RF), Support Vector Regression (SVR), K-Nearest Neighbors (KNN), XGBoost (XGB), Gradient Boost Regression (GBR), and Decision Tree (DT). Results show that RF and GBR algorithms outperform others in rice yield forecasting, while Gradient Booster and XGBoost demonstrate high accuracy in sugarcane yield prediction. However, the Mean Absolute Percentage Error (MAPE) values remained above 8%, indicating room for improvement. The study also emphasizes the importance of tuning hyperparameters for each machine learning algorithms (MLA) to achieve the most accurate predictions. Overall, the findings contribute valuable insights for stakeholders, including agricultural planners, policymakers, and researchers, emphasizing the need for continued refinement and validation of models to optimize agricultural planning and decision-making in this region. MLA highlight that features associated with temperature and relative humidity (RH) play a crucial role as the most significant contributors to the forecasting models for both rice and sugarcane yield. Introducing additional features, particularly remote sensing data, holds the potential to decrease the current error range of 8 to 10% to a more favourable and lower value.

Computed tomographic age estimation from the iliac crest and ischial tuberosity in an Indian population using supervised machine learning approaches.

Within the pelvis the iliac crest and ischial tuberosity display delayed ossification and fusion, thus, presenting as reliable maturity indicators. Amongst the different iliac crest and ischial tuberosity age estimation methods, the modified Kreitner-Kellinghaus stages constitute one of the more promising methods. The present study was directed towards establishing the applicability of the modified Kreitner-Kellinghaus method using five supervised machine learning approaches. Clinical CT scans of consenting individuals were collected and scored using the modified Kreitner-Kellinghaus method for the iliac crest and ischial tuberosity, independently. Age was subsequently estimated using different machine learning models. Cumulative scores computed from both markers were additionally employed for age estimation using machine learning. For iliac crest age estimation, Random Forest and Gradient Boosting Regression furnished lowest mean absolute error (2.42 years) and root mean square error (3.06 years). For ischial tuberosity age estimation, Gradient Boosting Regression garnered the lowest computations of mean absolute error (2.60 years) and root mean square error (3.09 years). For cumulative score based age estimation, Support Vector Regression and Gradient Boosting Regression yielded lowest mean absolute error (2.48 years) and root mean square error (3.07 years). Obtained error computations indicate that the iliac crest is a more accurate age marker in comparison to the ischial tuberosity. Additionally, cumulative score-based approaches garnered similar/ marginally more precise results in comparison to the iliac crest with all five models. This marginal improvement is not sufficient to justify employing the relatively more complicated cumulative score-based approach for age estimation. Hence, whenever available, the iliac crest should be preferred over the ischial tuberosity/ cumulative score-based approaches for age estimation.

ChatGPT Combining Machine Learning for the Prediction of Nanozyme Catalytic Types and Activities.

The design of nanozymes with superior catalytic activities is a prerequisite for broadening their biomedical applications. Previous studies have exerted significant effort in theoretical calculation and experimental trials for enhancing the catalytic activity of nanozyme. Machine learning (ML) provides a forward-looking aid in predicting nanozyme catalytic activity. However, this requires a significant amount of human effort for data collection. In addition, the prediction accuracy urgently needs to be improved. Herein, we demonstrate that ChatGPT can collaborate with humans to efficiently collect data. We establish four qualitative models (random forest (RF), decision tree (DT), adaboost random forest (adaboost-RF), and adaboost decision tree (adaboost-DT)) for predicting nanozyme catalytic types, such as peroxidase, oxidase, catalase, superoxide dismutase, and glutathione peroxidase. Furthermore, we use five quantitative models (random forest (RF), decision tree (DT), Support Vector Regression (SVR), gradient boosting regression (GBR), and fully connected deep neuron network (DNN)) to predict nanozyme catalytic activities. We find that GBR model demonstrates superior prediction performance for nanozyme catalytic activities (R2 = 0.6476 for Km and R2 = 0.95 for Kcat). Moreover, an open-access web resource, AI-ZYMES, with a ChatGPT-based nanozyme copilot is developed for predicting nanozyme catalytic types and activities and guiding the synthesis of nanozyme. The accuracy of the nanozyme copilot's responses reaches more than 90% through the retrieval augmented generation. This study provides a new potential application for ChatGPT in the field of nanozymes.

Retrieval of subsurface dissolved oxygen from surface oceanic parameters based on machine learning

Oceanic dissolved oxygen (DO) is crucial for oceanic material cycles and marine biological activities. However, obtaining subsurface DO values directly from satellite observations is limited due to the restricted observed depth. Therefore, it is essential to develop a connection between surface oceanic parameters and subsurface DO values. Machine learning (ML) methods can effectively grasp the complex relationship between input attributes and target variables, making them a valuable approach for estimating subsurface DO values based on surface oceanic parameters. In this study, the potential of ML methods for subsurface DO retrieval is analyzed. Among the selected ML methods, namely support vector regression (SVR), random forest (RF) regression, and extreme gradient boosting (XGBoosting) regression, the RF method generally demonstrates superior performance. As the depth increases, the accuracy of DO estimates tends to initially decrease, then gradually improve, with the poorest performance occurring at the depth of 600 dbar. The range of determination coefficients (R2) and root mean square error (RMSE) values based on the test dataset at different depths lies between 0.53 and 47.59 μmol/kg to 0.99 and 4.01 μmol/kg. In addition, compared to sea surface salinity (SSS) and sea surface chlorophyll-a (SCHL), sea surface temperature (SST) plays a more significant role in DO retrieval. Finally, compared to the pelagic interactions scheme for carbon and ecosystem studies (PISCES) model, the RF method achieves higher retrieval accuracies at depths above 700 dbar. In the deep ocean, the primary differences in DO values obtained from the RF method and the PISCES model-based method are noticeable in the vicinity of the equatorial region.

Novel Insights in Soil Mechanics: Integrating Experimental Investigation with Machine Learning for Unconfined Compression Parameter Prediction of Expansive Soil

This paper presents a novel application of machine learning models to clarify the intricate behaviors of expansive soils, focusing on the impact of sand content, saturation level, and dry density. Departing from conventional methods, this research utilizes a data-centric approach, employing a suite of sophisticated machine learning models to predict soil properties with remarkable precision. The inclusion of a 30% sand mixture is identified as a critical threshold for optimizing soil strength and stiffness, a finding that underscores the transformative potential of sand amendment in soil engineering. In a significant advancement, the study benchmarks the predictive power of several models including extreme gradient boosting (XGBoost), gradient boosting regression (GBR), random forest regression (RFR), decision tree regression (DTR), support vector regression (SVR), symbolic regression (SR), and artificial neural networks (ANNs and proposed ANN-GMDH). Symbolic regression equations have been developed to predict the elasticity modulus and unconfined compressive strength of the investigated expansive soil. Despite the complex behaviors of expansive soil, the trained models allow for optimally predicting the values of unconfined compressive parameters. As a result, this paper provides for the first time a reliable and simply applicable approach for estimating the unconfined compressive parameters of expansive soils. The proposed ANN-GMDH model emerges as the pre-eminent model, demonstrating exceptional accuracy with the best metrics. These results not only highlight the ANN’s superior performance but also mark this study as a groundbreaking endeavor in the application of machine learning to soil behavior prediction, setting a new benchmark in the field.

Rock Mineral Volume Inversion Using Statistical and Machine Learning Algorithms for Enhanced Geothermal Systems in Upper Rhine Graben, Eastern France

AbstractAccurately determining the mineralogical composition of rocks is essential for precise assessments of key petrophysical properties like effective porosity, water saturation, clay volume, and permeability. Mineral volume inversion is particularly critical in geological contexts characterized by heterogeneity, such as in the Upper Rhine Graben (URG), where both carbonate and siliciclastic formations are prevalent. The estimation of mineral volumes poses challenges that involve both linear and nonlinear relationships associated with geophysical data. To address this complexity, our methodology strategically integrates the robust insights from standard statistical approaches with three machine learning (ML) algorithms: multi‐layer perceptron, random forest regression, and gradient boosting regression. Furthermore, we propose a new hybrid ensemble model that incorporates a weighted average of multiple ML approaches to predict mineral composition within the Muschelkalk and Buntsandstein formations of the URG. ML techniques for mineral composition prediction in these formations exhibit robust predictive performance. The predicted mineral volumes align closely with quantitative estimates derived from X‐ray diffraction analysis. Additionally, they are in good qualitative agreement with mineral descriptions obtained from cores and cuttings of the Muschelkalk and Buntsandstein formations.

Artificial Intelligence Models for Predicting Ground Vibrations in Deep Underground Mines to Ensure the Safety of Their Surroundings

Ground vibrations induced by underground mining blasting has a significant impact on the stability and safety of surface buildings near mines. Due to the thick rock layers overlying underground mines, there is presently limited accuracy in regard to predicting ground vibrations induced by underground mine blasting. Therefore, this study aims to improve the accuracy of predicting ground vibrations induced by underground blasting by comprehensively measuring the peak particle velocity (PPV) in all three directions and independently considering on the impact of vertical distance. Random forest regression (RFR), bagging regression (BR), and gradient boosting regression (GBR) were used to regress the X-axis PPV (X-PPV), Y-axis PPV (Y-PPV), and Z-axis PPV (Z-PPV) based on blasting records measured at an iron mine. In addition, a genetic algorithm, gray wolf optimizer (GWO), and a particle swarm optimization were used to optimize the parameters of the RFR, BR, and GBR. The comparison results show that GWO-GBR is the optimal model for the prediction of the X-PPV (R2 = 0.8072), Y-PPV (R2 = 0.9147), and Z-PPV (R2 = 0.9265), respectively. Thus, the GWO-GBR model proposed in this study is considered a highly reliable model for predicting ground vibrations induced by underground mine blasting to ensure the safety of the mines’ surroundings.

The Application of Supervised Learning Algorithms in Predicting the Formation Energy of NLO Crystals

AbstractNonlinear optical crystals (NLO) are a key class of functional materials in the field of laser technology due to their excellent frequency conversion effects and physical–chemical stability. The research aims to find NLO crystals with superior stability by predicting their formation energy. In this study, only compositional information is utilized as input features and models are constructed using regression algorithms such as Random Forest Regression (RFR), Support Vector Regression (SVR), and Gradient Boosting Regression (GBR). Notably, the GBR model exhibited outstanding predictive performance, with an R2 value of 0.935 and root mean square error (RMSE) of 0.248 eV per atom. Additionally, SHapley Additive exPlanations (SHAP) analysis is employed to elucidate the fundamental principles behind the predictions by assessing the contribution of each feature to the formation energy. To validate the reliability of the models, first‐principles calculations are conducted to predict the formation energy of materials of GaP, ZnGeP2, and CdSiP2. The error range between the model predictions and the Generalized Gradient Approximation (GGA) calculated values is ≈0.1 eV per atom, confirming the accuracy of the models.

Modeling specific capacitance of carbon nanotube-based supercapacitor electrodes by machine learning algorithms

Carbon nanotubes (CNTs) have emerged as promising materials for supercapacitors (SCs) due to their unique properties and exceptional electrical conductivity. These cylindrical structures composed of carbon atoms offer several advantages for SC electrode applications. The electrochemical performance of CNT-based electrodes is strongly influenced by factors such as surface area, pore structure, and ID/IG ratio. However, the lack of a credible physical model capable of accurately predicting the performance of SCs based on these physicochemical properties of CNTs poses a challenge. In this study, we propose the utilization of a data-driven approach employing various models including a gradient boosting regression (GBR), Bayesian regression (BR), ridge regression (RR), and stochastic gradient descent (SGD) model to predict the performance of SCs with CNT electrodes based on the microstructural properties of the electrode material and electrochemical operational parameters. The developed GBR model demonstrates its feasibility by achieving a low root mean square error (RMSE) value of approximately 36.31 for the prediction of specific capacitance for test split. Additionally, a sensitivity analysis was conducted to investigate the influence of independent input parameters on a single output parameter, specifically the specific capacitance. This analysis provides insights into the relative importance and impact of various input parameters on the specific capacitance of CNT-based SCs.

Gradient boosted regression as a tool to reveal key drivers of temporal dynamics in a synthetic yeast community.

Microbial communities are vital to our lives, yet their ecological functioning and dynamics remain poorly understood. This understanding is crucial for assessing threats to these systems and leveraging their biotechnological applications. Given that temporal dynamics are linked to community functioning, this study investigated the drivers of community succession in the wine yeast community. We experimentally generated population dynamics data and used it to create an interpretable model with a gradient-boosted regression tree approach. The model was trained on temporal data of viable species populations in various combinations, including pairs, triplets, and quadruplets, and was evaluated for predictive accuracy and input feature importance. Key findings revealed that the inoculation dosage of non-Saccharomyces species significantly influences their performance in mixed cultures, while Saccharomyces cerevisiae consistently dominates regardless of initial abundance. Additionally, we observed multispecies interactions where the dynamics of Wickerhamomyces anomalus were influenced by Torulaspora delbrueckii in pairwise cultures, but this interaction was altered by the inclusion of S. cerevisiae. This study provides insights into yeast community succession and offers valuable machine learning-based analysis techniques applicable to other microbial communities, opening new avenues for harnessing microbial communities.

Application of Machine Learning Algorithms to Predict Osteoporotic Fractures in Women.

Predicting the risk of osteoporotic fractures is vital for prevention. Traditional methods such as the Fracture Risk Assessment Tool (FRAX) model use clinical factors. This study examined the predictive power of the FRAX score and machine-learning algorithms trained on FRAX parameters. We analyzed the data of 2,147 female participants from the Ansan cohort study. The FRAX parameters employed in this study included age, sex (female), height and weight, current smoking status, excessive alcohol consumption (>3 units/d of alcohol), and diagnosis of rheumatoid arthritis. Osteoporotic fracture was defined as one or more fractures of the hip, spine, or wrist during a 10-year observation period. Machine-learning algorithms, such as gradient boosting, random forest, decision tree, and logistic regression, were employed to predict osteoporotic fractures with a 70:30 training-to-test set ratio. We evaluated the area under the receiver operating characteristic curve (AUROC) scores to assess and compare the performance of these algorithms with the FRAX score. Of the 2,147 participants, 3.5% experienced osteoporotic fractures. Those with fractures were older, shorter in height, and had a higher prevalence of rheumatoid arthritis, as well as higher FRAX scores. The AUROC for the FRAX was 0.617. The machine-learning algorithms showed AUROC values of 0.662, 0.652, 0.648, and 0.637 for gradient boosting, logistic regression, decision tree, and random forest, respectively. This study highlighted the immense potential of machine-learning algorithms to improve osteoporotic fracture risk prediction in women when complete FRAX parameter information is unavailable.

A novel data-driven machine learning techniques to predict compressive strength of fly ash and recycled coarse aggregates based self-compacting concrete

Compressive strength (CS) of concrete is one of the most important factors in the construction industry and various time and effort-consuming tasks are required to measure it. To tackle such problems, the use of machine learning (ML), a branch of artificial intelligence, has recently resulted in a dramatic revolution in the construction sector, resulting in increased efficiency, accuracy, and creativity. Taking these factors into consideration, the current research was conducted on concrete manufactured with recycled coarse aggregate and fly ash generated as a byproduct of construction and demolition activities and thermal power plants. A large dataset consisting of 444 data points, along with ten input parameters, has been collected from the literature to forecast the CS of fly ash and recycled coarse aggregate-based self-compacting concrete. In this regard, ten advanced ML models, including K-Nearest Neighbors (KNN), Extra Tree Regressor (ETR), Bagging Regressor (BR), Adaboost Regressor (AR), Extreme Gradient Boosting (XGB), Linear Regression (LR), Random Forest (RF), Decision Tree Regression (DTR), Support Vector Regression (SVR) and Gradient Boosting Regression (GBR) have been considered. Furthermore, various data visualization plots and model’s performance matrices such as scatter plot, histograms, heatmaps, Shapley Additive Explanation (SHAP) Analysis, Regression Error Characteristics (REC), and errors have been utilized. In order to evaluate the most influential input parameter and depict the overall performance of ML models, sensitivity analysis and Taylor’s diagram are used. As a method of validation, the Kfold cross-validation approach has been implemented to justify the obtained output. Based on the outcome of the study, the BR model has displayed remarkable accuracy with insignificant errors and high R-squared values (R2 = 0.961), while XGB (R2 = 0.959), and DTR (R2 = 0.952) models also achieved commendable, as compared to other ML models. Additionally, water content, curing days, fly ash, and w/c ratio were found to be the most critical components that directly impact the CS of fly ash and RCA-based SCC. To cater to diverse and extensive practices, a graphical user interface has been developed to assist researchers and engineers in getting instant results of their fly ash and RCA-based SCC mixes prior to the execution of time- and resource-consuming laboratory work.

A Method Based on Artificial Intelligence That Predicts Arabica Coffee Yield by Analyzing Abiotic Factors and the Prevalence of Coffee Leaf Rust

Coffee is a perennial crop that harbors infections throughout the plant, which may worsen illness under favorable circumstances. Coffee leaf rust, a widespread coffee disease and Arabica is susceptible to leaf rust. Disease incidence and severity depend on abiotic variables. Cloudy and constant South-West monsoon weather (June–September) promotes coffee leaf rust growth. In this five-model study, an Extra Tree and Gradient Boosting regression model predicted coffee crop output in Chikamagaluru, Karnataka, with the least error utilizing biotic and abiotic factors. We investigated additional tree, gradient boosting, RF, Decision Tree, and KNN models using biotic and abiotic predictors. Used the independent testing dataset's MSE, MAE, RMSE Root mean square errors, and R-squared errors to compare model performance. The extra tree (R²=0.98 kg/ha ˉ¹ and RMSE = 7.96 kg/ha ˉ¹) and gradient boosting (R²=0.96 kg/ha ˉ¹ and RMSE = 10.96 kg/ha ˉ¹) regression models used Group 1 and 2 characteristics as predictor variables and different parameter fine tuning functions to estimate coffee yield most accurately. Compared to the less precise probabilistic models utilized in this work, such as Random Forest, decision tree, and KNN models, shown in the results section. The optimum weather parameter for coffee production forecasts and biotic-CLR incidence data outperformed random forest, decision tree, and K-Nearest Neighbor models.

Machine Learning-based Extrapolation of Crop Cultivation Cost

It is important to comprehend the relation between operational expenses such as labour, seed, irrigation, insecticides, fertilizers and manure costs necessary for the cultivation of crops. A precise cost for the cultivation of crops can offer vital information for agricultural decision-making. The main goal of the study is to compare machine learning (ML) techniques to measure relationships among operational cost characteristics for predicting crop cultivation costs before the start of the growing season using the dataset made available by the Ministry of Agriculture and Farmer Welfare of the Government of India. This paper describes various ML regression techniques, compares various learning algorithms as well as determines the most efficient regression algorithms based on the data set, the number of samples and attributes. The data set used for predicting the cost with 1680 instances includes varying costs for 14 different crops for 12 years (2010-2011 to 2021-2022). Ten different ML algorithms are considered and the crop cultivation cost is predicted. The evaluation results show that Random Forest (RF), Decision Tree (DT), Extended gradient boosting (XR) and K-Neighbours (KN) regression provide better performance in terms of coefficient of determination (R2), root mean square error (RMSE) and mean absolute error (MAE) rate while training and testing time. This study also compares different ML techniques and showed significant differences using the statistical analysis of variance (ANOVA) test. The optimal hyperparameters for the ML models are found using the gridsearchCV and randomizedsearchCV functions, which improves the model's capacity for generalisation.

Enhancing Loan Prediction Accuracy: A Comparative Analysis of Machine Learning Algorithms with XAI Integration

The contemporary financial landscape necessitates loan recommendation systems that offer both accuracy and transparency. Conventional assessment methodologies often suffer from limitations in efficiency and transparency, leading to potential risks for both lenders and borrowers. This research proposes the development of a novel loan recommendation system that leverages the power of machine learning (ML) and Explainable Artificial Intelligence (XAI). The paper delves into the processes of data collection, preprocessing, model training, evaluation, and subsequent integration into a web application using the Flask framework. The employed datasets encompass a variety of loan types, with the study aiming to identify the most effective ML algorithms from a selection that includes XGBoost, CatBoost, Random Forest, Gradient Boosting, and Logistic Regression. To enhance the system's transparency, Explainable AI methods, such as LIME, are incorporated. The culmination of this research is a web application that facilitates personalized predictions regarding loan eligibility, accompanied by clear explanations. Index terms - Loan Recommendation System, Machine Learning, Explainable AI, XGBoost, CatBoost, Random Forest, Gradient Boost, Logistic Regression,LIME.

A New Machine-Learning-Driven Grade-Point Average Prediction Approach for College Students Incorporating Psychological Evaluations in the Post-COVID-19 Era

With the rapid development of artificial intelligence in recent years, intelligent evaluation of college students’ growth by means of the monitoring data from training processes is becoming a promising technique in the field intelligent education. Current studies, however, tend to utilize course grades, which are objective, to predict students’ grade-point averages (GPAs), but usually neglect subjective factors like psychological resilience. To solve this problem, this paper takes mechanical engineering as the research object, and proposes a new machine-learning-driven GPA prediction approach to evaluate the academic performance of engineering students by incorporating psychological evaluation data into basic course scores. Specifically, this paper adopts SCL-90 psychological assessment data collected in the freshman year, including key mental health indicators such as somatization, depression, hostility, and interpersonal sensitivity indicators, as well as professional basic course scores, including mechanical principles, mechanical design, advanced mathematics, and engineering drawing. Four representative machine learning algorithms, Support Vector Machine (SVM), CNN-CBAM, Extreme Gradient Boosting (XGBoost) and Classification and Regression Tree (CART) that include deep and shallow models, respectively, are then employed to build a classification model for GPA prediction. This paper designs a validation experiment by tracking 229 students from the 2020 class from the School of Mechanical and Electrical Engineering of Henan University of Science and Technology, China. The students’ academic performance in senior grades is divided into five classes to use as the prediction labels. It is verified that psychological data and course data can be effectively integrated into GPA prediction for college students, with an accuracy rate of 83.64%. Meanwhile, this paper also reveals that anxiety indicators in the psychological assessment data have the greatest impact on college students’ academic performance, followed by interpersonal sensitivity. The experimental results also show that, for predicting junior year GPAs, psychological factors play more important role than they do in predicting sophomore GPAs. Suggestions are therefore given: the current practice in existing undergraduate teaching, i.e., only conducting psychological assessments in the initial freshman year, should be updated by introducing follow-up psychological assessments in each academic year.