Extremely Randomized Trees Research Articles

Utilizing Artificial Intelligence for Microbiome Decision-Making: Autism Spectrum Disorder in Children from Bosnia and Herzegovina

Background/Objectives: The study of microbiome composition shows positive indications for application in the diagnosis and treatment of many conditions and diseases. One such condition is autism spectrum disorder (ASD). We aimed to analyze gut microbiome samples from children in Bosnia and Herzegovina to identify microbial differences between neurotypical children and those with ASD. Additionally, we developed machine learning classifiers to differentiate between the two groups using microbial abundance and predicted functional pathways. Methods: A total of 60 gut microbiome samples (16S rRNA sequences) were analyzed, with 44 from children with ASD and 16 from neurotypical children. Four machine learning algorithms (Random Forest, Support Vector Classification, Gradient Boosting, and Extremely Randomized Tree Classifier) were applied to create eight classification models based on bacterial abundance at the genus level and KEGG pathways. Model accuracy was evaluated, and an external dataset was introduced to test model generalizability. Results: The highest classification accuracy (80%) was achieved with Random Forest and Extremely Randomized Tree Classifier using genus-level taxa. The Random Forest model also performed well (78%) with KEGG pathways. When tested on an independent dataset, the model maintained high accuracy (79%), confirming its generalizability. Conclusions: This study identified significant microbial differences between neurotypical children and children with ASD. Machine learning classifiers, particularly Random Forest and Extremely Randomized Tree Classifier, achieved strong accuracy. Validation with external data demonstrated that the models could generalize across different datasets, highlighting their potential use.

A novel framework for debris flow susceptibility assessment considering the uncertainty of sample selection

The uncertainty arising from random sampling of non-debris flow samples significantly impacts the accuracy of debris flow susceptibility assessments (DFSA). This study introduces a novel uncertainty elimination method, Kernel Density Estimation (KDE), and compares it with Mean and Maximum Probability Analysis (MPA) methods. Furthermore, we investigate the responses of four commonly used machine learning models to sampling uncertainty, comparing two structurally similar models (Random Forest (RF) and Extremely Randomized Trees (ERT)) with two structurally different models (Support Vector Machine (SVM) and Multilayer Perceptron (MLP)). The results indicate that the application of these uncertainty elimination methods can significantly enhance AUC values and zoning accuracy, with the KDE method outperforming the others. Specifically, the AUC values based on KDE for RF, ERT, SVM, and MLP are 0.995, 0.999, 0.999, and 0.853, respectively. The corresponding zoning accuracy for these models is 1.00, 1.00, 1.00, and 0.78, respectively. The study further reveals that the responses to sampling uncertainty vary by model architecture: RF, ERT, and SVM typically exhibit bimodal normal distributions, while the MLP model shows a unimodal distribution. Additionally, MLP is more sensitive to variations in negative samples, whereas RF and ERT are less affected due to the ensemble structure.

Developing an efficient explainable artificial intelligence approach for accurate reverse osmosis desalination plant performance prediction: application of SHAP analysis

In recent decades, securing drinkable water sources has become a pressing concern for populations in various regions worldwide. Therefore, to address the growing need for potable water, contemporary water purification technologies can be employed to convert saline sources into drinkable supplies. Therefore, the prediction of important parameters of desalination plants is a key task for designing and implementing these facilities. In this regard, artificial intelligence techniques have proven to be powerful assets in this field. These methods offer an expedited and effective means of estimating effective parameters, thus catalyzing their implementation in real-world scenarios. In this study, the predictive accuracy of six different machine learning models, including Natural Gradient-based Boosting (NGBoost), Adaptive Boosting (AdaBoost), Categorical Boosting (CatBoost), Support vector regression (SVR), Gaussian Process Regression (GPR), and Extremely Randomized Tree (ERT) was evaluated for modelling the parameter of permeate flow as a key element in system efficiency, energy consumption, and water quality using six various input combinations of feed water salt concentration, condenser inlet temperature, feed flow rate, and evaporator inlet temperature. The next phase of this research employed the SHAP interpretability method to illustrate the impact of individual variables on the model's output. Moreover, the predictive performance of the developed frameworks was evaluated using a set of five dependable statistical measures: RMSE, NS, MAE, MAPE and R 2. These indicators were utilized to provide a robust means of gauging the precision of the model's forecasts. A comparative analysis of the outcomes, as measured by the RMSE criteria, revealed that the SVR technique (RMSE = 0.125 L/(h·m2)) exhibited superior performance compared to NGBoost (RMSE = 0.163 L/(h·m2)), AdaBoost (RMSE = 0.219 L/(h·m2)), CatBoost (RMSE = 0.149 L/(h·m2)), GPR (RMSE = 0.156 L/(h·m2)), and ERT (RMSE = 0.167 L/(h·m2)) methodologies in predicting permeate flow rates. The modelling outcomes obtained during the evaluation stage demonstrated the efficacy of the SVR algorithm in enhancing the precision of permeate flow forecasts, utilizing relevant input variables.

Bayesian Optimization Driven Strategy for Detecting Credit Card Fraud with Extremely Randomized Trees

Bayesian Optimization Driven Strategy for Detecting Credit Card Fraud with Extremely Randomized Trees

Determining of gully erosion susceptibility based on UAV and machine learning in Loess Plateau

In order to determine the susceptibility of gully erosion at a small watershed scale on the Loess Plateau of China, three hybrid models were developed. These models were based on the Multi-Attributive Border Approximation Area Comparison (MABAC), frequency ratio (FR), CatBoost (CB), LightGBM (LG), and extremely-randomized tree (ET). Based on the Unmanned Aerial Vehicles (UAV) photos, a total of 83 gullies with 12,150 gully pixels and 8 conditioning variables were extracted and used to create the gully inventory database. The correlations between the conditioning parameters and the pixels of gullies were then determined using FR, and the relative importance of these conditioning factors was quantified using machine learning. Then, for gully erosion susceptibility mapping (GESM), three hybrid gully erosion susceptibility models called MABAC-FR-CB, MABAC-FR-LG, and MABAC-FR-ET were developed. The performance of three hybrid models was assessed using the receiver operating characteristic curve (ROC) and the Kappa coefficient. The results claimed that slope steepness greatly influenced the erosion of the gully. The MABAC-FR-ET performed the most precisely, with area under curvature (AUC) of 0.998 and a Kappa of 0.952. As a result, it was determined that MABAC-FR-ET is the most exact and accurate method for predicting the susceptibility to gully erosion in the study watershed.

Estimation of the Weight and Volume of Lime (Citrus aurantifolia (Christm.) Swingle) Fruit Using Computer Vision Based on Traditional Machine Learning and Deep Learning

The post-harvest process is important to increasing the market value of limes and requires focus. During this process, limes are graded and categorized based on size, weight, and volume. Therefore, identifying efficient means of estimating these properties is very important and remains an open research area. This study applies the concept of computer vision based on traditional machine learning algorithms (partial least square regression (PLS), epsilon-support vector regression (ε-SVR), decision tree (DT), random forest (RF), adaptive boosting (AB), gradient boosting (GB), Bagging meta-estimator (BME), and extremely randomized trees (ERTs)) and pre-trained deep learning (InceptionV3, MoblieNetV2, ResNet50, and VGG-16) for estimating the weight and volume of limes. Our findings showed that the BME and ResNet50 could yield the highest performance for estimating the weight and volume of limes. The BME produced Rtest2 values of 0.954 and 0.882 for weight and volume, respectively, while the Rtest2 values of ResNet50 models were between 0.951 and 0.957 for weight and volume, respectively. This study concluded that computer vision based on both traditional machine learning and deep learning could be used to estimate the weight and volume of limes. The approach proposed in this study can be adopted for applications related to computer vision in the post-harvest process.

Development of novel computational models based on artificial intelligence technique to predict liquids mixtures separation via vacuum membrane distillation

The fundamental objective of this paper is to use Machine Learning (ML) methods for building models on temperature (T) prediction using input features r and z for a membrane separation process. A hybrid model was developed based on computational fluid dynamics (CFD) to simulate the separation process and integrate the results into machine learning models. The CFD simulations were performed to estimate temperature distribution in a vacuum membrane distillation (VMD) process for separation of liquid mixtures. The evaluated ML models include Support Vector Machine (SVM), Elastic Net Regression (ENR), Extremely Randomized Trees (ERT), and Bayesian Ridge Regression (BRR). Performance was improved using Differential Evolution (DE) for hyper-parameter tuning, and model validation was performed using Monte Carlo Cross-Validation. The results clearly indicated the models’ effectiveness in temperature prediction, with SVM outperforming other models in terms of accuracy. The SVM model had a mean R2 value of 0.9969 and a standard deviation of 0.0001, indicating a strong and consistent fit to the membrane data. Furthermore, it exhibited the lowest mean squared error, mean absolute error, and mean absolute percentage error, signifying superior predictive accuracy and reliability. These outcomes highlight the importance of selecting a suitable model and optimizing hyperparameters to guarantee accurate predictions in ML tasks. It demonstrates that using SVM, optimized with DE improves accuracy and consistency for this specific predictive task in membrane separation context.

Prediction of rod-like particle mixing in rotary drums by three machine learning methods based on DEM simulation data

The mixing of non-spherical particles in rotary drums exhibits significant complexity, particularly when density segregation and size segregation occur simultaneously. Three machine learning models: artificial neural network (ANN), extremely randomized trees (ERT), and particle swarm optimized support vector regression (PSO-SVR) were developed to predict the mixing time and mixing degree at the steady mixing state of rod-like particles in rotary drums. The training, validation, and test data for the machine learning models were generated from 121 discrete element method (DEM) simulations with four independent variables: revolution frequency, particle density ratio, particle size ratio, and drum length. All three models predicted the mixing degree accurately with R2≥ 0.94. The ERT and PSO-SVR models also predicted the mixing time well with R2≥ 0.88. Building machine learning models is hundreds of times faster than running DEM simulations, making these models highly promising for predicting larger-scale simulations with more complex-shaped particles.

Application of the DRASTIC-LU/LC method combined with machine learning models to assess and predict the vulnerability of the Rmel aquifer (Northwest, Morocco)

The Rmel aquifer, located in the Tangier-Tetouan-Al Hoceima region of northwest Morocco, covers approximately 240 km2 and faces increasing pollution threats due to population growth and economic development. This study assesses aquifer vulnerability to pollution, and compares the performance of various machine learning models integrated with the DRASTIC-LU/LC method. The research used a dataset of 52 water samples analyzed for nitrate concentrations, considering eight factors influencing vulnerability: aquifer depth, net recharge, aquifer lithology, soil texture, topography, vadose zone impact, hydraulic conductivity, and land use. An information gain test was applied to evaluate the importance of these factors. Four machine learning algorithms were used with the DRASTIC-LU/LC method: multilayer perceptron (MLP), the bagging algorithm (BA), K-nearest neighbors (KNN), and extremely randomized trees (ERT). Model performance was assessed via the area under the ROC curve (ROC-AUC) to measure accuracy. The ERT model combined with DRASTIC-LU/LC achieved the highest accuracy (AUC = 0.929), followed by BA (AUC = 0.925), MLP (AUC = 0.852), and KNN (AUC = 0.787). In comparison, the original DRASTIC-LU/LC model had an AUC of 0.530. The results highlight significant vulnerability variation across the Rmel aquifer, with high to very high levels in the southern and northwestern regions, and moderate to low levels in the northeast and central areas. Vulnerability maps were validated by comparing the observed nitrate concentrations in the water samples, confirming model accuracy. Groundwater depth, net recharge, and hydraulic conductivity were identified as the most significant factors influencing vulnerability. This study demonstrates the effectiveness of integrating machine learning models with the DRASTIC-LU/LC method for accurate aquifer vulnerability assessment, offering valuable tools for public policy and groundwater management.

Screening depression among university students utilizing GHQ-12 and machine learning

The escalating incidence of depression has brought attention to the increasing concern regarding the mental well-being of university students in the current academic environment. Given the increasing mental health challenges faced by students, there is a critical need for efficient, scalable, and accurate screening methods. This study aims to address the issue by using the General Health Questionnaire-12 (GHQ-12), a well recognized tool for evaluating psychological discomfort, in combination with machine learning (ML) techniques. Firstly, for effective screening of depression, a comprehensive questionnaire has been created with the help of an expert psychiatrist. The questionnaire includes the GHQ-12, socio-demographic, and job and career-related inquiries. A total of 804 responses has been collected from various public and private universities across Bangladesh. The data has been then analyzed and preprocessed. It has been found that around 60% of the study population are suffering from depression. Lastly, 16 different ML models, including both traditional algorithms and ensemble methods has been applied to examine the data to identify trends and predictors of depression in this demographic. The models' performance has been rigorously evaluated in order to ascertain their effectiveness in precisely identifying individuals who are at risk. Among the ML models, Extremely Randomized Tree (ET) has achieved the highest accuracy of 90.26%, showcasing its classification effectiveness. A thorough investigation of the performance of the models compared, therefore clarifying their possible relevance in the early detection of depression among university students, has been presented in this paper. The findings shed light on the complex interplay among socio-demographic variables, stressors associated with one's profession, and mental well-being, which offer an original viewpoint on utilizing ML in psychological research.

Mapping high-resolution XCO2 concentrations in China from 2015 to 2020 based on spatiotemporal ensemble learning model

High-resolution column-averaged dry air mole fraction of CO2 (XCO2) data is crucial for understanding the spatiotemporal patterns of XCO2 and for mitigating carbon emissions. Due to the limited scanning range of sensors and strict inversion conditions, satellite-retrieved XCO2 data are often significantly incomplete. Machine learning models are widely used to fill these gaps in satellite XCO2 data. However, the limitations of individual machine learning models and the complexity of the spatial distribution of XCO₂ mean that the accuracy of XCO2 predictions still needs improvement. In this study, a new spatiotemporal stacked ensemble learning model (STEL) was developed by combining random forest (RF), extremely randomized trees (ERT), extreme gradient boosting (XGBoost), optical gradient boosting (LightGBM), and categorical boosting (CatBoost) using the stacking ensemble learning methodology. Considering the spatiotemporal heterogeneity of XCO2, a novel spatiotemporal weighting feature was constructed as part of the model's input parameters. Finally, the XCO2 observed by Orbiting Carbon Observatory 2 (OCO-2) was reconstructed using STEL, and a monthly mean XCO2 dataset covering China from 2015 to 2020 was generated at a spatial resolution of 0.1°. The results show that STEL exhibits superior performance and generalization capabilities compared to individual machine-learning models. R2 RMSE and MAPE were 0.9624, 1.0023 ppm, and 0.1583 % on the test set, and 0.8970, 1.4213 ppm, and 0.2475 % for R2, RMSE, and MAPE in ground validation, respectively. In 10-fold cross-validation, STEL's RMSE was reduced by 9.52 % compared to the best-performing single model (RF). The spatiotemporal trend of CO2 in China from 2015 to 2020 was analyzed using STEL XCO2 data. The results indicate that this dataset accurately reflects the spatiotemporal heterogeneity of XCO2 distribution at a fine scale. Overall, XCO2 exhibited a spatiotemporal pattern of “high in the east and low in the west” and “high in spring and low in summer.” Except in summer, high XCO₂ values were mainly distributed in the North China Plain. XCO2 trends and hotspots showed considerable spatial variation. The Pearl River Delta and Yangtze River Delta urban agglomerations have the fastest XCO2 growth rates, and the distribution of XCO2 hotspots is consistent with the distribution of population and economic centers. In the sparsely populated northwest of China, XCO2 is growing rapidly due to increased thermal power generation and coal mining. XCO2 hotspots in Northwest China are mainly located in Xinjiang, Ningxia, and Inner Mongolia. The methodology and data presented are useful for further research on carbon emissions, carbon sinks, and climate change.

Data-driven axial bearing capacity analysis of steel tubes infilled with rubberized alkali-activated concrete

This study aims to employ machine learning algorithms to analyze the axial bearing capacity of rubberized alkali-activated concrete filled steel tubes. A dataset encompassing 327 synthesized instances and seven input features is adopted for training and testing six machine learning models, including Decision Tree, Random Forest, Extremely Randomized Trees, Adaptive Boosting, Gradient Boosting Decision Trees (GBDT), and eXtreme Gradient Boosting Trees (XGBoost). The SHapley Additive exPlanation algorithm is employed to elucidate the prediction process of machine learning models and to analyze the influence of each parameter on axial bearing capacity. Comparison of evaluating metrics shows that GBDT and XGBoost models achieve highest accuracy and generalization capabilities when their Coefficient of Determination values surpassing 0.98 and Mean Absolute Percent Error remaining below 3%. Moreover, the explanation analysis of machine learning models reveals that diameter/width of the cross section, rubber content, yielding strength and thickness of steel tubes are critical variables that affect the axial bearing capacity, while compressive strength of alkali-activated concrete, specimen height, and shape of cross section show negligible impact. Besides, GBDT model overemphasizes the effect of specimen height and might lead a conservative prediction for specimens with smaller heights. Finally, compressive strength of alkali-activated concrete and diameter/width, thickness, and yielding strength of steel tubes are positively correlated with axial bearing capacity, and the increase of rubber content in alkali-activated concrete leads to the decrease of capacity.

The inversion of arid-coastal cultivated soil salinity using explainable machine learning and Sentinel-2

The escalating salinization of cultivated soil poses a significant threat to the ecological environment. It is imperative to establish a monitoring system and mitigate the spread of salinization in arid and coastal areas through remote sensing, incorporating high-precision cross-regional models for soil salt content inversion. This study focuses on typical saline-alkali soils in arid and coastal regions of China. Using Sentinel 2 data (including 6 bands and 27 spectral indices), along with soil texture, moisture content, temperature, precipitation, and digital elevation model (DEM) data to establish an arid-coastal salinity inversion model. Variable selection methods such as pearson correlation coefficient (PCC), variable importance in projection (VIP), gray relational analysis (GRA), and gradient boosting machine (GBM) were used, while using 9 models including adaptive boosting (Adaboost), extremely randomized trees (ERT), and light gradient boosting machine (LightGBM). The best model was further elucidated using the Shapley additive explanations method. Results indicate that the common sensitive characteristic variables of arid-coastal areas were spectral indices and soil properties in PCC, the spectral variable bands and indices in VIP, and all variables in GRA and GBM. The best inversion model GBM-ERT (R2 = 0.91, RMSE = 1.06) in arid areas exhibited higher accuracy than the best inversion model GBM-Adaboost (R2 = 0.77, RMSE = 1.74) in coastal areas. The arid-coastal inversion model PCC-LightGBM demonstrated the best inversion performance (R2 = 0.64, RMSE = 2.29) and simulation performance in arid (R2 = 0.67) and coastal areas (R2 = 0.63). Dead fuel index (DFI) had the most significant impact on model prediction (0.89) and the second ratio index (RI2) contributed the highest relative importance (18 %) to the model. Our analysis indicates that the arid-coastal model of PCC-LightGBM established using common characteristic variables, can effectively monitor large-scale soil salinity.

Computed tomography-based radiomics machine learning models for differentiating enchondroma and atypical cartilaginous tumor in long bones.

To explore the value of CT-based radiomics machine learning models for differentiating enchondroma from atypical cartilaginous tumor (ACT) in long bones and methods to improve model performance.59 enchondromas and 53 ACTs in long bones confirmed by pathology were collected retrospectively. The features were extracted from preoperative CT images of these patients, and least absolute shrinkage and selection operator (LASSO) regression was used for feature selection and dimensionality reduction. The selected features were used to construct classification models by thirteen machine learning algorithms. The data set was randomly divided into a training set and a test set at a proportion of 7:3 by ten-fold cross-validation to evaluate the performance of these models.A total of 1199 features were extracted, 9 features were selected, and 13 radiomics machine learning models were constructed. The area under the curve (AUC) of 11 models was more than 0.8, and that of 3 models was more than 0.9. The Extremely Randomized Trees model achieved the best performance (AUC = 0.9375 ± 0.0884), followed by the Adaptive Boosting model (AUC = 0.9188 ± 0.1010) and the Linear Discriminant Analysis model (AUC = 0.9062 ± 0.1459).CT-based radiomics machine learning models had great ability to distinguish enchondroma and ACT in long bones. By using filters to deeply mine high-order features in the original image and selecting appropriate machine learning algorithms, the performance of the model can be improved. · CT-based radiomics machine learning models can distinguish enchondroma and ACT in long bones.. · Using filters and selecting advanced machine learning algorithms can improve model performance.. · Clinical features have limited utility in distinguishing enchondroma and ACT in long bones.. · Hong R, Li Q, Ma J et al. Computed tomography-based radiomics machine learning models for differentiating enchondroma and atypical cartilaginous tumor in long bones. Fortschr Röntgenstr 2024; DOI 10.1055/a-2344-5398.

Hydrogen solubility in different chemicals: A modelling approach and review of literature data

Hydrogen (H2) solubility is a crucial parameter for industrial processes. This study utilises various Machine Learning (ML) techniques, including Decision Tree (DT), Multilayer Perceptron (MLP), Random Forest (RF), and Extremely Randomized Trees (ET), to predict H2 solubility across diverse chemical classes, encompassing n-alkanes, aliphatic and aromatic hydrocarbons, alcohols, aldehydes, diols, esters, ethers, ketones, and halides. An extensive database of 4337 samples of H2 solubility in 100 different chemicals was compiled from open literature. Problematic samples were identified through preprocessing. Three input sets were utilised for model development to determine the optimal combination of regressors. It was found that including dimensionless pressure and temperature significantly improved modelling accuracy. Higher accuracy was achieved by employing polynomial extracted features. 76 chemicals trained the model, and 20 tested it, with none overlapping. High accuracy on the test set shows applicability to a diverse chemical range beyond the study's dataset. The study's modelling results indicate RF and ET ensemble methods outperform DT and MLP in accuracy and stability, with ET being the most accurate. This study improves our understanding of H2 solubility in various chemicals, crucial for industrial processes like designing H2-based renewable energy plants. Our study advances ML in predicting chemical solubility by introducing advanced feature extraction and testing models with unseen chemicals. This not only validates our models' predictive capability but also yields deeper insights into H2 solubility. With standardized data preprocessing and a model adept at handling diverse chemicals, we demonstrate a substantial advancement in the field through a meticulous testing methodology.

IP-GCN: A deep learning model for prediction of insulin using graph convolutional network for diabetes drug design

Insulin is a kind of protein that regulates the blood sugar levels is significant to prevent complications associated with diabetes, such as cancer, neurodegenerative disorders, cardiovascular disease, and kidney damage. Insulin protein (IP) plays an active role in drug discovery, medicine, and therapeutic methods. Unlike experimental protocols, computational predictors are fast and can predict IP accurately. This work introduces a model, called IP-GCN for IP prediction. The patterns from IP are extracted by K-spaced position specific scoring matrix (KS-PSSM) and the model training is accomplished using powerful deep learning tool, called Graph Convolutional Network (GCN). Additionally, we implemented Pseudo Amino Acid Composition (PseAAC) and Dipeptide Composition (DPC) for feature encoding to assess the predictive performance of GCN. To evaluate the efficacy of our novel approach, we compare its performance with well-known deep/machine learning algorithms such as Convolutional Neural Network (CNN), Extremely Randomized Tree (ERT), and Support Vector Machine (SVM). Predictive results demonstrate that the proposed predictor (IP-GCN) secured the best performance on both training and testing datasets. The novel computational would be fruitful in diabetes drug discovery and contributes to research for therapeutic interventions in various Insulin protein associated diseases.

Investigating the Effect of Class Balancing Methods on the Performance of Machine Learning Techniques: Credit Risk Application

Credit risk arises as a result of the failure of the loans given by banks to the customers to fulfill their obligations at the end of the specified term. Technological advances allow the use of machine learning methods in various sectors. These methods aim to facilitate the identification of customers at risk with the system adapted to the creditworthiness processes of banks. For this purpose, in order to make the most appropriate evaluation in the lending process of banks, re-sampling techniques to eliminate the problem of class imbalance encountered in unbalanced data sets were made balanced and their effects on machine learning were investigated. During the implementation phase, German, Australian and HMEQ credit data sets were used. Different machine learning classification methods such as Logistic Regression (LR), K-Narest Neighbor (KNN), Naive Bayes (NB), Support Vector Machines (SVM), Multilayer Perceptron (MLP), Decision Trees (DT), Random Forests (RF), Gradient Boosting Decision Trees (GBDT), Extremely Randomized Trees, Hard and Soft Voting were used to detect risky customers. The problem of class imbalance was balanced with resampling and hybrid techniques such as Random Oversampling (ROS), Random Undersampling (RUS), Balanced Bagging Classifier (BBC), SMOTE-Tomek Links and SMOTE-ENN. In this context, the performances of three different data sets were examined in four different scenarios. As a result of the study, the hybrid method, in which oversampling and undersampling methods are used together for the class balancing problem, showed the best classification performance among machine learning techniques.

Susceptibility Modeling and Potential Risk Analysis of Thermokarst Hazard in Qinghai–Tibet Plateau Permafrost Landscapes Using a New Interpretable Ensemble Learning Method

Climate change is causing permafrost in the Qinghai–Tibet Plateau to degrade, triggering thermokarst hazards and impacting the environment. Despite their ecological importance, the distribution and risks of thermokarst lakes are not well understood due to complex influencing factors. In this study, we introduced a new interpretable ensemble learning method designed to improve the global and local interpretation of susceptibility assessments for thermokarst lakes. Our primary aim was to offer scientific support for precisely evaluating areas prone to thermokarst lake formation. In the thermokarst lake susceptibility assessment, we identified ten conditioning factors related to the formation and distribution of thermokarst lakes. In this highly accurate stacking model, the primary learning units were the random forest (RF), extremely randomized trees (EXTs), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost) algorithms. Meanwhile, gradient boosted decision trees (GBDTs) were employed as the secondary learning unit. Based on the stacking model, we assessed thermokarst lake susceptibility and validated accuracy through six evaluation indices. We examined the interpretability of the stacking model using three interpretation methods: accumulated local effects (ALE), local interpretable model-agnostic explanations (LIME), and Shapley additive explanations (SHAP). The results showed that the ensemble learning stacking model demonstrated superior performance and the highest prediction accuracy. Approximately 91.20% of the total thermokarst hazard points fell within the high and very high susceptible areas, encompassing 20.08% of the permafrost expanse in the QTP. The conclusive findings revealed that slope, elevation, the topographic wetness index (TWI), and precipitation were the primary factors influencing the assessment of thermokarst lake susceptibility. This comprehensive analysis extends to the broader impacts of thermokarst hazards, with the identified high and very high susceptibility zones affecting significant stretches of railway and highway infrastructure, substantial soil organic carbon reserves, and vast alpine grasslands. This interpretable ensemble learning model, which exhibits high accuracy, offers substantial practical significance for project route selection, construction, and operation in the QTP.

Mitigation of Adversarial Attacks in 5G Networks with a Robust Intrusion Detection System Based on Extremely Randomized Trees and Infinite Feature Selection

Intrusion Detection Systems (IDSs) are an important tool to mitigate cybersecurity threats in the ICT infrastructures. Preferable properties of the IDSs are the optimization of the attack detection accuracy and the minimization of the computing resources and time. A signification portion of IDSs presented in the research literature is based on Machine Learning (ML) and Deep Learning (DL) elements, but they may be prone to adversarial attacks, which may undermine the overall performance of the IDS algorithm. This paper proposes a novel IDS focused on the detection of cybersecurity attacks in 5G networks, which addresses in a simple but effective way two specific adversarial attacks: (1) tampering of the labeled set used to train the ML algorithm, (2) modification of the features in the training data set. The approach is based on the combination of two algorithms, which have been introduced recently in the research literature. The first algorithm is the Extremely Randomized Tree (ERT) algorithm, which enhances the capability of Decision Tree (DT) and Random Forest (RF) algorithms to perform classification in data sets, which are unbalanced and of large size as IDS data sets usually are (legitimate traffic messages are more numerous than attack related messages). The second algorithm is the recently introduced Infinite Feature Selection algorithm, which is used to optimize the choice of the hyper-parameter defined in the approach and improve the overall computing efficiency. The result of the application of the proposed approach on a recently published 5G IDS data set proves its robustness against adversarial attacks with different degrees of severity calculated as the percentage of the tampered data set samples.

Determining water and solute permeability of reverse osmosis membrane using a data-driven machine learning pipeline

Water (A) and solute permeability (B) are the key parameters characterizing the performance of reverse osmosis (RO) membranes. However, determining A and B requires multiple times of experiments that are time-consuming. Thus, developing a method to estimate A and B quickly has been encouraged. This study introduces a data-driven machine learning (ML) pipeline designed to predict A and B for RO membranes. The ML pipeline addresses all stages for estimating A and B, from data preprocessing to practical guidance on developed models. In particular, the membrane composition data, an overlooked essential component detailing A and B, was employed as features for the ML models. Sixteen ML models were tested, and categorical boost (CatBoost) and extremely randomized trees (ET) were selected as the best models for A and B, respectively. The selected ML models were retrained using the minimum yet effective features identified through feature importance analysis. Consequently, the retrained ML models exhibited high accuracy in predicting A and B (Radj,test2 = 0.925/MAEtest = 0.246 for A; Radj,test2 = 0.986/MAEtest = 0.069 for B), demonstrating the viability of a data-driven approach for parameter estimation. Moreover, this study highlights that RO membrane performance can be gauged effectively using membrane composition data and a limited set of operating conditions.