Random Forest Importance Research Articles

Analysis of Molding Defection in IC Packaging and Testing Process

Molding injects a molding compound into a mold to form a protective shell around the wafer. During the injection process, overflow may occur, leading to mold flash, which reduces yield and causes significant manufacturing cost losses. This paper proposes a deep-learning-based method for detecting and predicting the occurrence of mold flash probability to address this issue. First, the paper conducts random forest importance analysis and correlation analysis to identify the key parameters that significantly impact mold flash. This paper uses these key parameters as input signals for the prediction model. The paper introduces an HLGA Transformer to construct an ensemble meta-learning model that predicts the probability of molding defects, achieving a prediction accuracy of 98.16%. The ensemble meta-learning approach proposed in this paper outperforms other methods in terms of performance. The model predictions can be communicated to the system in real time, allowing it to promptly adjust critical machine operation parameters, thereby significantly improving the molding process yield and reducing substantial manufacturing cost losses.

Predicting the success of startups using a machine learning approach

Successful investment in early-stage companies has high uncertainty. More specifically, the tools available to investors need to be more robust to reduce the risk and manage the uncertainty of startups. This research aims to use machine learning methods to design a prediction solution to identify successful startups for investors. In order to design the predicting solution and provide policies, classification, and clustering algorithms have been utilized to predict the success of startups and perform feature importance analysis based on the SHAP and permutation methods. Subsequently, the performance of four classification algorithms, such as Random Forest, Gradient Boost, Multilayer Perceptron, Logistic Regression and Support Vector Machine, are compared to predict business success. Meanwhile, Random Forest and Gradient Boosting algorithms showed the best accuracy, which was equal to 82% and 80%, respectively. Based on the feature importance of Random Forest and Gradient Boosting, which is obtained from the SHAP method, indicated that the higher values of “Number of followers on LinkedIn”, “Number of employees on LinkedIn”, “Number of followers on Twitter”, and “Last raised amount” have higher SHAP values and a more significant impact on the model output. Three clustering algorithms including hierarchy, K-means, and DBSCAN were also compared. Among them, the K-means algorithm performs best with 72% silhouette, and K-means was employed to explain each cluster’s characteristics. Finally, an effective artificial intelligence-based prediction solution has been proposed to show the way for investors to apply machine learning concepts to predict the success of startups.

Comparative Analysis of Machine Learning Models for Predicting Student Success in Online Programming Courses: A Study Based on LMS Data and External Factors

Early prediction of student performance in online programming courses is essential for implementing timely interventions to enhance academic outcomes. This study aimed to predict academic success by comparing four machine learning models: Logistic Regression, Random Forest, Support Vector Machine (SVM), and Neural Network (Multilayer Perceptron, MLP). We analyzed data from the Moodle Learning Management System (LMS) and external factors of 591 students enrolled in online object-oriented programming courses at the Universidad Estatal de Milagro (UNEMI) between 2022 and 2023. The data were preprocessed to address class imbalance using the synthetic minority oversampling technique (SMOTE), and relevant features were selected based on Random Forest importance rankings. The models were trained and optimized using Grid Search with cross-validation. Logistic Regression achieved the highest Area Under the Receiver Operating Characteristic Curve (AUC-ROC) on the test set (0.9354), indicating strong generalization capability. SVM and Neural Network models performed adequately but were slightly outperformed by the simpler models. These findings suggest that integrating LMS data with external factors enhances early prediction of student success. Logistic Regression is a practical and interpretable tool for educational institutions to identify at-risk students, and to implement personalized interventions.

Analysis on pulse rate variability for pilot workload assessment based on wearable sensor

AbstractThe workload levels of pilots directly affect their flight performance and the safety of the whole flight. To explore the real‐time workload of pilots in different flight phases (takeoff, cruise, and landing), this paper leveraged National Aeronautics and Space Administration Task Load Index (NASA‐TLX), a subjective evaluation scale, and PhotoPlethysmoGraphy (PPG) signals of 21 participants using a flight simulator and a wearable sensor. First, the workloads of pilots under different phases were explored by the NASA‐TLX scales; secondly, the pulse rate variability (PRV) features were selected by variance analysis and random forest importance evaluation; finally, the performances of the k‐nearest neighbor (KNN), random forest (RF), and support vector machine (SVM) were compared for workload levels identification. It is shown that the workloads are ranked as follows: landing > takeoff > cruise. SDNN, CVCD, CVNNI, LF, TP, SD2, and SD2/SD1 were used as selected features with significant differences in different flight phases. In addition, machine learning models can effectively identify pilot workloads, and feature selection enhances the performance of both KNN and RF classifiers. The best identification of workload was achieved using the selected PRV features as inputs to the KNN classifier, with an average accuracy of 88.9%. Our results indicate that the KNN classifier and PRV features are suitable for identifying pilot workload. The pilot workload is highest during the landing phase, which provides a reference for flight safety management. The findings from this research could contribute to developing a robust pilot workload detection system and improve current flight operation safety regulations.

Correlation between phytoplankton dynamics and hydrometeorological factors in the Three Gorges reservoir area tributary

ABSTRACT Phytoplankton blooms in reservoir areas’ rivers are a research problem of wide interest. To assess the influence of hydrometeorological factors on the occurrence and development of phytoplankton blooms in the Xiangxi River, a major tributary of the Three Gorges Reservoir. We employed the generalized additive model (GAM) and random forest importance analysis to evaluate the impact of hydrometeorological factors on phytoplankton dynamics. The findings revealed that river flow in the Xiangxi River is the most crucial factor affecting phytoplankton, with a significant decrease in phytoplankton observed when the flow exceeds 100 m3/s. At the Xiakou station within the perennial backwater area, phytoplankton exhibited correlations with river flow, temperature, solar radiation, and wind speed, with decreasing levels of importance. At the Pingyikou station and the Zhaojun station within the dynamic backwater area, phytoplankton was correlated with river flow, solar radiation, and precipitation. Furthermore, an increase in wind speed at the Xiakou station was significantly positively correlated with an increase in phytoplankton, suggesting that wind speed plays a promoting role in phytoplankton aggregation in the perennial backwater area. These results underscore the significance of hydrometeorological factors as key determinants in assessing the occurrence and development of phytoplankton blooms. This research provides valuable insights for phytoplankton blooms prediction and warning systems.

Distribution and prediction of biocrusts under the canopy of typical vascular plants on the Loess Plateau, Northwest China.

Vascular plants exert significant effects on micro-environment, thereby affecting the distribution of biological soil crusts (biocrusts). The relationship between vascular plants and the spatial distribution characteristics of biocrusts is largely unknown. We investigated the distribution characteristics of biocrusts under the canopy of vascular plants in the water-wind erosion crisscross area of the Loess Plateau, where larger areas of biocrusts had been formed since the implantation of "Grain for Green" project. We analyzed the relationship between the canopy characteristics of different vascular plants and the spatial distribution of biocrusts using correlation analysis and random forest importance ranking methods, and further constructed a predictive model for the area of biocrusts under the canopy of vascular plants. The results showed that: 1) Cyanobacteria crust was the predominant biocrusts, followed by moss crust. 2) The canopy of vascular plants affected the spatial distribution of biocrusts, with notable differences in distribution pattern across different directions under the canopy of vascular plants. Biocrusts were primarily distributed in the 270°-315° and 315°-360° directions, while was less frequent in the 90°-135° and 135°-180° directions. 3) Radially, the coverage of biocrusts gradually increased from the root-base to the edge of the canopy of vascular plants. 4) The coverage of biocrusts under canopy was significantly related to the characteristics of vascular plants, including canopy area, long crown width, short crown width, litter area and plant height. 5) The relative importance of canopy area, long crown width, and short crown width to the biocrusts under the canopy was 13.7%, 12.1%, and 11.9%, respectively, while the relative importance of plant height and species type was relatively low, being 6.7% and 4.4%, respectively. 6) Results of the random forest model demonstrated strong predictive performance for biocrusts distribution based on canopy characteristics of vascular plants, with a prediction accuracy of 0.59 (R2) and a root mean square error of 1.2 m2. This model could be applied to predict and estimate the area of biocrusts under the canopy of vascular plants. This study provided a theoretical basis for in-depth understanding of the relationship between vascular plants and biocrusts in semi-arid climate regions, as well as for predicting the spatial distribution of biocrusts.

Research on CO2-WAG in Thick Reservoirs: Geological Influencing Factors and Random Forest Importance Evaluation.

In the development process of thick reservoirs, the impact of various geological factors on the effectiveness of the CO2 water alternating gas (CO2-WAG) flooding technology remains unclear. This paper establishes multiple CO2-WAG flooding models for thick reservoirs to study the effects of sedimentary rhythm, dip angle, matrix permeability, high-permeability streaks (HPS), and barrier layers on the effectiveness of CO2-WAG flooding and then uses the random forest algorithm to rank the importance of these geological factors. The results show that different geological factors have varying degrees of impact on the distribution of water and gas migration and recovery rates during the CO2-WAG flooding process. The ranking of the importance of various factors obtained by reservoir numerical simulations and the random forest algorithm is HPS, sedimentary rhythm, dip angle, matrix permeability, and barrier layers. These research findings will provide effective guidance and a reference for the optimal selection of CO2-WAG flooding schemes for similar thick reservoirs under different geological conditions.

Added utility of temperature zone information in remote sensing-based large scale crop mapping

Accurate crop cover maps are beneficial for various aspects like water resources management, crop yield prediction, regulation insurance policies, and investigation of the effects of climate change. When making large-scale crop classification, regional harvest time and phenological growth differences occur due to varying temperatures along the study area, and using regional temperature differences while performing crop cover classification may increase the map accuracy. Therefore, in this study, we investigated for the first time the contribution of temperature information over large areas to the classification of agricultural products. Agricultural crop mapping is performed over Türkiye using Sentinel-2 Level-2A images with 10-m spatial resolution acquired between March 15, 2019, and October 15, 2019. In addition to spectral bands, Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI) are used as classification features. Twenty years of ERA5-Land 2-m temperature data is averaged to divide the study area into three temperature zones Low (LTZ), Medium (MTZ), and High-Temperature Zone (HTZ). Before the classification, feature selection using random forest importance is performed to select the most successful features. After that, a random forest classifier is created for each temperature zone. LTZ reached 89% overall accuracy (OA) with a 0.88 Kappa. MTZ reached 91% OA with 0.92 Kappa, and HTZ reached 94% OA with 0.94 Kappa, giving the best accuracy among the classifiers. Finally, test sets of all temperature zones are combined, and an OA of 92% with a Kappa of 0.93 is achieved with this combined test set. To test the advantage of temperature zoning, classification is also performed without the temperature zones, and it is observed that temperature zoning increases the OA and Kappa by 1%. A land cover classification map is then created using temperature zone classifiers with 34 crop classes and six non-agricultural classes.

Fatigue behavior analysis and life evaluation method of building steel under the influence of multiple factors

The paper aims to analyze the stress–strain response of building steel under low-cycle multiaxial fatigue and to accurately evaluate the fatigue life of specimens by selecting the more important relevant factors that affect fatigue behavior. Based on this, low-cycle multiaxial fatigue experiments of Q355D notched specimens under different multiaxial loading are carried out. Firstly, the fatigue behavior of notched specimens under different loadings and the relationship between the biaxial strain amplitude ratio and fatigue life are analyzed. Secondly, Grey correlation analysis, Pearson correlation analysis, and Random Forest importance analysis are used to screen out the important factors affecting fatigue life. Finally, the grey MGM (1, N) model is used to predict the fatigue life of notched specimens, and the results show that the predicted life based on Random Forest importance analysis is in good agreement with the experimental life. In this paper, the multiaxial fatigue problem is decoupled into two sub-problems, i.e. screening out related factors and life prediction based on the related factors, which can be used as a reference for life prediction in structural design.

Machine Learning Approach for Predicting the Hole Mobility of the Perovskite Solar Cells

AbstractTraditional computational approaches such as Monte Carlo simulation, molecular dynamics, and density functional theory (DFT) have contributed to understanding the role of hole mobility in the development of suitable hole transporting materials (HTMs) for perovskite solar cell efficiency. However, these methods often involve significant computational expenses, thereby limiting the number of feasible studies and hindering the extraction of valuable structure−property guidelines for a rational design of novel HTMs. To address these challenges, this study proposes an ultrafast predictive model that balances prediction accuracy and time efficiency, a critical aspect for predicting the hole mobility of HTMs for perovskite optimization. The model, which leverages the Random Forest learning algorithm, enables comprehensive and rapid analysis of photovoltaic features taken from previous experiments/literature and processed with the RDKit Python library. Notably, recognizing the challenges associated with high correlation in the dataset, a method to improve the calculation of feature importance in random forests is applied. The results highlight bertz_ct, chi1, and logP as key predictive features of HTM performance. However, the model underscores the need to uncover additional predictive features based on structure–property relationships to predict the optimized hole mobility. This approach significantly accelerates the discovery process, outperforming prevalent statistical methods in the prediction of hole mobility and HTM performance. This research signifies a pivotal step toward cost‐effective, accelerated stability and efficiency of HTMs, with implications for the advancement of optoelectronic devices.

Association of Performance on Multiple Cognitive Domains with Sarcopenia among Middle-Aged and Older Adults.

The relationship between cognitive function and subsequent sarcopenia remains unclear. Therefore, this study aimed to examine the associations of performance on multiple cognitive domains with sarcopenia in the middle-aged and older adults. This longitudinal analysis (wave 2011-2013) included 2,934 participants from the CHARLS study. Sarcopenia was defined by the Asian Sarcopenia Working Group 2019 criteria. Cognitive function was measured by the Chinese version of the Mini-Mental State Examination (MMSE). Three interpretable techniques, namely SHapley Additive exPlanations (SHAP) and two built-in methods (coefficients of logistic regression and Gini importance of random forest), were used to assess the relationship between MMSE, its components (orientation, attention, episodic memory, and visuospatial ability) and sarcopenia. In addition, the association of MMSE score and its components with sarcopenia was further validated using stepwise regression. All interpretable methods showed that MMSE score was important predictors of sarcopenia, especially the SHAP (MMSE score ranked top one). For its components, episodic memory, visuospatial ability, and attention showed high predictive value compared with orientation. Stepwise regression analyses showed that MMSE score and its components of episodic memory and visuospatial ability were correlated with sarcopenia, with their odds ratios of 0.93 (95% CI: 0.91-0.96, p < 0.001), 0.87 (95% CI: 0.82-0.93, p < 0.001), and 1.32 (95% CI: 1.05-1.65, p = 0.016), respectively. Better cognitive function especially episodic memory and visuospatial ability was negatively associated with incident sarcopenia among community middle-aged and older adults.

Predictions of Spartina alterniflora leaf functional traits based on hyperspectral data and machine learning models

ABSTRACT Investigating the functional traits of Spartina alterniflora can provide insights towards understanding its invasion mechanism, and developing a method leaves can improve its management in coastal wetlands. Here, we examined the relationship between 11 leaf functional traits of S. alterniflora and hyperspectral data and investigated the feature bands through importance score analysis. Using original spectral and first-order differential conversion data of feature bands, we established four prediction models: random forest (RF), support vector machine (SVM), eXtreme Gradient Boosting (XGBoost), and back propagation neural network (BPNN). The study results showed that: (1) the SVM model based on Random Forest Importance Score is well-suited for S. alterniflora leaf functional trait inversion; (2) the importance score of leaf functional traits differed, and first-order differential spectral data produced more bands with high scores compared with the original hyperspectral reflectance data; (3) first-order differential data modelling effects were slightly better than those of the original spectral data. However, the first-order differential treatment did not show a significant improvement in the validation accuracy compared with the original data, and the accuracy of some traits decreased. Our study provides a new methodological approach for improving the monitoring and management of S. alterniflora in coastal wetlands.

Exploring Hybrid Models For Short-Term Local Weather Forecasting in IoT Environment

This paper explores using and hybridizing simple prediction models to maximize the accuracy of local weather prediction while maintaining low computational effort and the need to process and acquire large volumes of data. A hybrid RF-LSTM model is proposed and evaluated in this research paper for the task of short-term local weather forecasting. The local weather stations are built within an acceptable radius of the measured area and are designed to provide a short period of forecasting - usually within one hour. The lack of local weather data might be problematic for an accurate short-term valuable prediction in sustainable applications like agriculture, transportation, energy management, and daily life. Weather forecasting is not trivial because of the non-linear nature of time series. Thus, traditional forecasting methods cannot predict the weather accurately. The advantage of the ARIMA model lies in forecasting the linear part, while the SVR model indicates the non-linear characteristic of the weather data. Both non-linear and linear approaches can represent the combined model. The hybrid ARIMA-SVR model strengthens the matched points of the ARIMA model and the SVR model in weather forecasting. The LSTM and random forest are both popular algorithms used for regression problems. LSTM is more suitable for tasks involving sequential data with long-term dependencies. Random Forest leverages the wisdom of crowds by combining multiple decision trees, providing robust predictions, and reducing overfitting. Hybrid Random forest-LSTM potentially leverages the robustness and feature importance of Random Forest along with the ability of LSTM to capture sequential dependencies. The comparison results show that the hybrid RF-LSTM model reduces the forecasting errors in metrics of MAE, R-squared, and RMSE. The proposed hybrid model can also capture the actual temperature trend in its prediction performance, which makes it even more relevant for many other possible decision-making steps in sustainable applications. Furthermore, this paper also proposes the design of a weather station based on a real-time edge IoT system. The RF-LSTM leverages the parallelized characteristics of each decision tree in the forest to accelerate the training process and faster inferences. Thus, the hybrid RF-LSTM model offers advantages in terms of faster execution speed and computational efficiency in both PC and Raspberry Pi boards. However, the RF-LSTM consumes the highest peak memory usage due to being a combination of two different models.

Multimodal deep learning for personalized renal cell carcinoma prognosis: Integrating CT imaging and clinical data

Background and Objective: Renal cell carcinoma represents a significant global health challenge with a low survival rate. The aim of this research was to devise a comprehensive deep-learning model capable of predicting survival probabilities in patients with renal cell carcinoma by integrating CT imaging and clinical data and addressing the limitations observed in prior studies. The aim is to facilitate the identification of patients requiring urgent treatment.Methods: The proposed framework comprises three modules: a 3D image feature extractor, clinical variable selection, and survival prediction. Based on the 3D CNN architecture, the feature extractor module predicts the ISUP grade of renal cell carcinoma tumors linked to mortality rates from CT images. Clinical variables are systematically selected using the Spearman score and random forest importance score as criteria. A deep learning-based network, trained with discrete LogisticHazard-based loss, performs the survival prediction. Nine distinct experiments are performed, with varying numbers of clinical variables determined by different thresholds of the Spearman and importance scores.Results: Our findings demonstrate that the proposed strategy surpasses the current literature on renal cancer prognosis based on CT scans and clinical factors. The best-performing experiment yielded a concordance index of 0.84 and an area under the curve value of 0.8 on the test cohort, which suggests strong predictive power.Conclusions: The multimodal deep-learning approach developed in this study shows promising results in estimating survival probabilities for renal cell carcinoma patients using CT imaging and clinical data. This may have potential implications in identifying patients who require urgent treatment, potentially improving patient outcomes. The code created for this project is available for the public on: GitHub

An integrated feature selection approach to high water stress yield prediction.

The timely and precise prediction of winter wheat yield plays a critical role in understanding food supply dynamics and ensuring global food security. In recent years, the application of unmanned aerial remote sensing has significantly advanced agricultural yield prediction research. This has led to the emergence of numerous vegetation indices that are sensitive to yield variations. However, not all of these vegetation indices are universally suitable for predicting yields across different environments and crop types. Consequently, the process of feature selection for vegetation index sets becomes essential to enhance the performance of yield prediction models. This study aims to develop an integrated feature selection method known as PCRF-RFE, with a focus on vegetation index feature selection. Initially, building upon prior research, we acquired multispectral images during the flowering and grain filling stages and identified 35 yield-sensitive multispectral indices. We then applied the Pearson correlation coefficient (PC) and random forest importance (RF) methods to select relevant features for the vegetation index set. Feature filtering thresholds were set at 0.53 and 1.9 for the respective methods. The union set of features selected by both methods was used for recursive feature elimination (RFE), ultimately yielding the optimal subset of features for constructing Cubist and Recurrent Neural Network (RNN) yield prediction models. The results of this study demonstrate that the Cubist model, constructed using the optimal subset of features obtained through the integrated feature selection method (PCRF-RFE), consistently outperformed the RNN model. It exhibited the highest accuracy during both the flowering and grain filling stages, surpassing models constructed using all features or subsets derived from a single feature selection method. This confirms the efficacy of the PCRF-RFE method and offers valuable insights and references for future research in the realms of feature selection and yield prediction studies.

Clinical Study of PET‐validated EEG‐machine learning algorithm predicting brain amyloid pathology in pre‐dementia Alzheimer’s disease

AbstractBackgroundCurrently positron emission tomography (PET) is used as the initial or sole biomarker of β‐amyloid (Aβ) brain pathology, which may inhibit Alzheimer’s disease (AD) drug development and clinical use due to cost, access, and tolerability. Machine learning (ML)‐based EEG biomarkers may address these challenges. Previous studies have confirmed their ability to accurately discriminate between normal, mild cognitive impairment (MCI) and Alzheimer’s dementia. However, few quantitative EEG (QEEG) biomarkers have been validated with Aβ PET. We developed a QEEG‐ML algorithm to predict brain Aβ pathology among subjective cognitive decline (SCD) and MCI patients, and validated it using Aβ PET.MethodEEG (19‐channel, eye‐closed, resting‐state) and Aβ PET data were collected from 160 human subjects with MCI (77 Aβ+, 83 Aβ‐). We excluded 45 data (23 Aβ+, 22 Aβ‐) for test verification. QEEG absolute power, relative power, power ratio, and connectivity between channels (iCoherence) comprised the input features, from which the most relevant predictive features were identified using four methods (Random Forest Importance (GBM, XGB), ElasticNet, Whitney‐Mann). We then trained six ML algorithms (SVM, Logistic, KNN, Naive Bayes, Random Forest(GBM/XGB)) using each relevant feature set, yielding 24 models (4 sets * 6 algorithms). The 45 test data sets were input into each model to compare their performance. Then, 111 MCI data (56 Aβ+, 55 Aβ‐) and 165 SCD data (31 Aβ+, 134 Aβ‐) were classified by the model for 3rd validation.ResultIn test validation, the best‐performing model showed 77.8% accuracy, 81.8% sensitivity, 73.9% specificity in discriminating Aβ+ from Aβ‐. In 3rd data validation, the best‐performing model showed 72.1% accuracy, 71.4% sensitivity, 72.7% specificity in MCI. In SCD, it showed, 69.1% accuracy, 64.5% sensitivity, 70.2% specificity.ConclusionThese findings suggest that our novel ML‐based QEEG biomarker can accurately predict the presence of brain Aβ plaque. Additional benefits of such a biomarker include reduced expense, wide availability, and high‐throughput screening and response monitoring. Future studies will assess utility as primary AD screens, adjunctive use with PET, and as a support in clinical rationales for Aβ PET and treatment choice in AD.

Determination of Geographic Origin of Turquoise by Combining Laser Ablation Inductively Coupled Plasma Mass Spectrometry Analysis and Chemometrics

Microsampling elemental analysis is widely used for gemstone and mineralogy traceability. Using laser ablation inductively coupled plasma mass spectrometry combined with chemometrics, the contents of 56 elements in turquoise samples from 5 distinct producing areas in 3 nations were measured. An origin identification model for turquoise samples from various producing areas was established through random forest importance analysis, principal component analysis, and linear discriminant analysis. When combined with random forest importance screening, the traceability efficiency of principal component analysis is significantly improved. Moreover, by taking 48 elements as characteristic variables and introducing them into the discriminant model, a Fisher discriminant model for identifying the origin of turquoise was successfully established. The effective element fingerprint information of turquoise species is closely related to the species origin, and the accuracy of cross-validation reaches 99.5%, demonstrating the feasibility of the proposed model for the identification of the origin of turquoise samples.

Predictive Analysis of Students’ Learning Performance Using Data Mining Techniques: A Comparative Study of Feature Selection Methods

The utilization of data mining techniques for the prompt prediction of academic success has gained significant importance in the current era. There is an increasing interest in utilizing these methodologies to forecast the academic performance of students, thereby facilitating educators to intervene and furnish suitable assistance when required. The purpose of this study was to determine the optimal methods for feature engineering and selection in the context of regression and classification tasks. This study compared the Boruta algorithm and Lasso regression for regression, and Recursive Feature Elimination (RFE) and Random Forest Importance (RFI) for classification. According to the findings, Gradient Boost for the regression part of this study had the least Mean Absolute Error (MAE) and Root-Mean-Square Error (RMSE) of 12.93 and 18.28, respectively, in the case of the Boruta selection method. In contrast, RFI was found to be the superior classification method, yielding an accuracy rate of 78% in the classification part. This research emphasized the significance of employing appropriate feature engineering and selection methodologies to enhance the efficacy of machine learning algorithms. Using a diverse set of machine learning techniques, this study analyzed the OULA dataset, focusing on both feature engineering and selection. Our approach was to systematically compare the performance of different models, leading to insights about the most effective strategies for predicting student success.

Exploring the variable importance in random forests under correlations: a general concept applied to donor organ quality in post-transplant survival

Random Forests are a powerful and frequently applied Machine Learning tool. The permutation variable importance (VIMP) has been proposed to improve the explainability of such a pure prediction model. It describes the expected increase in prediction error after randomly permuting a variable and disturbing its association with the outcome. However, VIMPs measure a variable’s marginal influence only, that can make its interpretation difficult or even misleading. In the present work we address the general need for improving the explainability of prediction models by exploring VIMPs in the presence of correlated variables. In particular, we propose to use a variable’s residual information for investigating if its permutation importance partially or totally originates from correlated predictors. Hypotheses tests are derived by a resampling algorithm that can further support results by providing test decisions and p-values. In simulation studies we show that the proposed test controls type I error rates. When applying the methods to a Random Forest analysis of post-transplant survival after kidney transplantation, the importance of kidney donor quality for predicting post-transplant survival is shown to be high. However, the transplant allocation policy introduces correlations with other well-known predictors, which raises the concern that the importance of kidney donor quality may simply originate from these predictors. By using the proposed method, this concern is addressed and it is demonstrated that kidney donor quality plays an important role in post-transplant survival, regardless of correlations with other predictors.

Effect and genesis of soil nitrogen loading and hydrogeological conditions on the distribution of shallow groundwater nitrogen pollution in the North China Plain.

The North China Plain (NCP) has experienced increasingly severe groundwater nitrogen (TN) pollution. However, the factors influencing TN distribution are still poorly understood. Previous studies have identified surface soil nitrogen (TSN) loading and intrinsic groundwater vulnerability (Inv) as the main factors controlling groundwater TN pollution. However, in this study, based on 3245 shallow groundwater samples in the NCP, the multiple regression analysis results(R2=0.105, p<0.001) revealed that the TN was not mainly controlled by TSN and Inv. The lower prediction accuracy indicated the large data dispersion of TN, which might be affected by nitrogen attenuation or accumulation. Thus, the NCP was divided into balance, attenuation, and accumulation zones according to the regression equation. The attenuation zone was mainly distributed in the inter-fan and fan edge parts of the pre-mountain alluvial floodplain, as well as the west and south of the runoff area, while the accumulation zone was mainly distributed in the top part of the pre-mountain alluvial floodplain and the east of discharge area. Multi-indicators comparative analysis showed that compared to the balance (Eh= 76.2mV) and accumulation (Eh=126.7mV) zones, the attenuation zone has a stronger reducing environment (Eh=30.8mV) favorable to denitrification, which can reduce the TN pollution (0.49mg/L) caused by surface nitrogen input and consume more electron donors. Conversely, the stronger oxidizing environment in the accumulation zone limited denitrification, resulting in higher TN concentrations (19.14mg/L) in the aquifers under the same TSN and Inv conditions as the other two zones. The standardized effects and significance on each path of the structural equation model (SEMs) fully confirmed the reliability of the above zonal analysis. Importantly, the feature importance (23.6%) of random forest and standardized effects (0.455, p<0.001) of SEMs showed that the Eh had the strongest influence on TN. Thus, the redox conditions of the aquifer, in addition to TSN and Inv, played a crucial role in controlling the TN pollution in the groundwater of a large region. The zoning work and the analysis of influencing factors are important to guide scientific prevention and control of groundwater nitrogen pollution.