Multiple Machine Learning Models Research Articles

Using Multiple Machine Learning Models to Predict the Strength of UHPC Mixes with Various FA Percentages

Ultra High-Performance Concrete (UHPC) has shown extraordinary performance in terms of strength and durability. However, having a cost-effective and sustainable UHPC mix design is a challenge in the construction sector. This study aims on building a predictable model that can help in determining the compressive strength of UHPC. The research focuses on applying multiple machine learning (ML) models and evaluating their performance in predicting the strength prediction of UHPC. Two reliable metrics are used to evaluate the performance of the model which are the coefficient of determination (R2) and mean squared error (MSE). The parameters that are affecting the compressive strength of UHPC are fly ash percentage levels (FA%), superplasticizer content, water to binder ratio (w/b), and curing period. A total of 54 ML models were used, consisting of Linear Regression, Support Vector Machines (SVM), Neural Networks, and Random forests algorithms. Among these models, Random Forest proved to be the most effective in capturing the relationships in UHPC’s behaviour with an R squared score of 0.8857. The Random Forest ML model is also used in this paper to conduct a parametric study that will help in obtaining the compressive strength of UHPC with higher content of FA%, which is not sufficiently studied in the literature.

Evaluating Performance of Multiple Machine Learning Models for Drought Monitoring: A Case Study of Typical Grassland in Inner Mongolia

Driven by continuously evolving precipitation shifts and temperature increases, the frequency and intensity of droughts have increased. There is an obvious need to accurately monitor drought. With the popularity of machine learning, many studies have attempted to use machine learning combined with multiple indicators to construct comprehensive drought monitoring models. This study tests four machine learning model frameworks, including random forest (RF), convolutional neural network (CNN), support vector regression (SVR), and BP neural network (BP), which were used to construct four comprehensive drought monitoring models. The accuracy and drought monitoring ability of the four models when simulating a well-documented Inner Mongolian grassland site were compared. The results show that the random forest model is the best among the four models. The R2 range of the test set is 0.44–0.79, the RMSE range is 0.44–0.72, and the fitting accuracy relationship could be described as RF > CNN > SVR ≈ BP. Correlation analysis between the fitting results of the four models and SPEI found that the correlation coefficient of RF from June to September was higher than that of the other three models, though we noted the correlation coefficient of CNN in May was slightly higher than that of RF (CNN = 0.79; RF = 0.78). Our results demonstrate that comprehensive drought monitoring indices developed from RF models are accurate, have high drought monitoring ability, and can achieve the same monitoring effect as SPEI. This study can provide new technical support for comprehensive regional drought monitoring.

Investigation of the usefulness of a bile duct biopsy and bile cytology using a hyperspectral camera and machine learning.

To improve the efficiency of pathological diagnoses, the development of automatic pathological diagnostic systems using artificial intelligence (AI) is progressing; however, problems include the low interpretability of AI technology and the need for large amounts of data. We herein report the usefulness of a general-purpose method that combines a hyperspectral camera with machine learning. As a result of analyzing bile duct biopsy and bile cytology specimens, which are especially difficult to determine as benign or malignant, using multiple machine learning models, both were able to identify benign or malignant cells with an accuracy rate of more than 80% (93.3% for bile duct biopsy specimens and 83.2% for bile cytology specimens). This method has the potential to contribute to the diagnosis and treatment of bile duct cancer and is expected to be widely applied and utilized in general pathological diagnoses.

GloUTCI-M: a global monthly 1 km Universal Thermal Climate Index dataset from 2000 to 2022

Abstract. Climate change has precipitated recurrent extreme events and emerged as an imposing global challenge, exerting profound and far-reaching impacts on both the environment and human existence. The Universal Thermal Climate Index (UTCI), serving as an important approach to human comfort assessment, plays a pivotal role in gauging how humans adapt to meteorological conditions and copes with thermal and cold stress. However, the existing UTCI datasets still grapple with limitations in terms of data availability, hindering their effective application across diverse domains. We have produced GloUTCI-M, a monthly UTCI dataset boasting global coverage and an extensive time series spanning March 2000 to October 2022, with a high spatial resolution of 1 km. This dataset is the product of a comprehensive approach leveraging multiple data sources and advanced machine learning models. Our findings underscored the superior predictive capabilities of CatBoost in forecasting the UTCI (mean absolute error, MAE = 0.747 °C; root mean square error, RMSE = 0.943 °C; and coefficient of determination, R2=0.994) when compared to machine learning models such as XGBoost and LightGBM. Utilizing GloUTCI-M, the geographical boundaries of cold stress and thermal stress areas at global scale were effectively delineated. Spanning 2001–2021, the mean annual global UTCI was recorded at 17.24 °C, with a pronounced upward trend. Countries like Russia and Brazil emerged as key contributors to the mean annual global UTCI increasing, while countries like China and India exerted a more inhibitory influence on this trend. Furthermore, in contrast to existing UTCI datasets, GloUTCI-M excelled at portraying UTCI distribution at finer spatial resolutions, augmenting data accuracy. This dataset can enhance our capacity to evaluate thermal stress experienced by humans, offering substantial prospects across a wide array of applications. GloUTCI-M is publicly available at https://doi.org/10.5281/zenodo.8310513 (Yang et al., 2023).

Predictive modeling of water quality index (WQI) classes in Indian rivers: Insights from the application of multiple Machine Learning (ML) models on a decennial dataset

Predictive modeling of water quality index (WQI) classes in Indian rivers: Insights from the application of multiple Machine Learning (ML) models on a decennial dataset

Revolutionizing Cybersecurity with Machine Learning: A Comprehensive Review and Future Directions

In the realm of computing, data science has revolutionized cybersecurity operations and technologies. The key to creating automated and intelligent security systems lies in extracting patterns or insights from cybersecurity data and building data-driven models. Data science, encompassing various scientific approaches, machine learning techniques, processes, and systems, studies real-world occurrences through data analysis. Machine learning techniques, known for their flexibility, scalability, and adaptability to new and unknown challenges, have been applied across many scientific fields. Cybersecurity is rapidly expanding due to significant advancements in social networks, cloud and web technologies, online banking, mobile environments, smart grids, and more. Various machine learning techniques have effectively addressed a wide range of computer security issues. This article reviews several machine learning applications in cybersecurity, including phishing detection, network intrusion detection, keystroke dynamics authentication, cryptography, human interaction proofs, spam detection in social networks, smart meter energy consumption profiling, and security concerns associated with machine learning techniques themselves. The methodology involves collecting a large dataset of phishing and legitimate instances, extracting relevant features such as email headers, content, and URLs, and training a machine learning model using supervised learning algorithms. These models can effectively identify phishing emails and websites with high accuracy and low false positive rates. To enhance phishing detection, it is recommended to continuously update the training dataset to include new phishing techniques and employ ensemble methods that combine multiple machine learning models for improved performance

Artificial Neural Network-Based Modelling for Yield Strength Prediction of Austenitic Stainless-Steel Welds

This study aimed to develop an artificial neural network (ANN) model for predicting the yield strength of a weld metal composed of austenitic stainless steel and compare its performance with that of conventional multiple regression and machine learning models. The input parameters included the chemical composition of the nine effective elements (C, Si, Mn, P, S, Ni, Cr, Mo, and Cu) and the heat input per unit length. The ANN model (comprising five nodes in one hidden layer), which was constructed and trained using 60 data points, yielded an R2 value of 0.94 and a mean average percent error (MAPE) of 2.29%. During model verification, the ANN model exhibited superior prediction performance compared with the multiple regression and machine learning models, achieving an R2 value of 0.8644 and a MAPE of 3.06%. Consequently, the ANN model effectively predicted the variation in the yield strength and microstructure resulting from the thermal history and dilution during the welding of 3.5–9% Ni steels with stainless steel-based welding consumables. Furthermore, the application of the prediction model was demonstrated in the design of welding consumables and heat input for 9% Ni steel.

Strategic Analysis of Feature Selection Methods for Enhanced Dental Therapy Recognition in Machine Learning Applications

The popularity of smart clinics has increased significantly as a result of technical developments in fields like computer vision. At the heart of such systems is the ability to recognize objects and activities as well as perceive the environment as a whole. This is crucial for both eco-independent systems and human-machine interaction, especially in settings with constrained workspaces, like dental care. Our study delves into an extensive analysis of multiple machine learning models designed to robustly predict dental treatments. These models encompass Lazy Instance-based Learning, Sequential Minimal Optimization, Hoeffding Tree, and Random Tree. Leveraging object-oriented input sourced from gaze-guided wearable cameras, we scrutinize intricate attributes such as material properties, patients' dental conditions, and the array of instruments in use. Notably, we exploit the insight that identifying visual cues during an activity holds the potential to address the specific therapy identification challenge. Utilizing a dental data set that we gathered in the real world, we conducted our experiments and discovered that combining multiple criteria enhances accuracy in comparison to using each one alone. We did see, nevertheless, that in certain circumstances employing the symptoms alone produced superior outcomes. Additionally, symptoms demonstrated to have lesser error than combination in terms of RMS error convergence. Finally, we observed that the machine learning models' build and test durations increased as a result of the combined method. This demonstrates that adding additional parameters does not necessarily result in better outcomes in machine learning applications generally and in medical/dental applications in particular. Instead, it relies on the machine learning tool used, the settings taken into account and the input data provided. The versatility of our approach extends beyond dental contexts. It has been systematically validated across diverse domains, including the recognition of kitchen activities within smart home environments. This methodology holds relevance for various outdoor scenarios where the focal point of attention guides ongoing activities.

Improving Driving Style in Connected Vehicles via Predicting Road Surface, Traffic, and Driving Style

This paper investigates the application of ensemble learning in improving the accuracy and reliability of predictions in connected vehicle systems, focusing on driving style, road surface quality, and traffic conditions. Our study’s central methodology is the voting classifier ensemble method, which integrates predictions from multiple machine learning models to improve overall predictive performance. Specifically, the ensemble method combines insights from random forest, decision tree, and K-nearest neighbors models, leveraging their individual strengths while compensating for their weaknesses. This approach resulted in high accuracy rates of 94.67% for driving style, 99.10% for road surface, and 98.80% for traffic predictions, demonstrating the robustness of the ensemble technique. Additionally, our research emphasizes the importance of model explanation ability, employing the tree interpreter tool to provide detailed insights into how different features influence predictions. This paper proposes a model based on the algorithm GLOSA for sharing data between connected vehicles and the algorithm CTCRA for sending road information to navigation application users. Based on prediction results using ensemble learning and similarity in driving styles, road surface conditions, and traffic conditions, an ensemble learning approach is used. This not only contributes to the predictions’ transparency and trustworthiness but also highlights the practical implications of ensemble learning in improving real-time decision-making and vehicle safety in intelligent transportation systems. The findings underscore the significant potential of advanced ensemble methods for addressing complex challenges in vehicular data analysis.

A Hybrid Method Based on CRITIC Method and Machine Learning Models for Effective Fake News Detection in Thai Language

Fake news has emerged as a pervasive issue within the modern information ecosystem, leading to widespread dissemination of misinformation and erosion of trust. This paper introduces a novel hybrid approach for effectively detecting fake news in the Thai language by combining the CRITIC method with multiple machine learning models. The initial step involves collecting Thai-language fake news data from websites. Subsequently, the data undergoes a preprocessing phase. In the second step, the preprocessed data is used for validation through three basic machine learning models, namely, Naive Bayes, Decision Tree, and K-Nearest Neighbors. In the third step, the accuracy results from these three models are employed to calculate the significance weights of each model using the CRITIC method. In the final step, predictions are recalculated using the proposed method. The proposed method achieves an 83.37% accuracy, surpassing Naive Bayes (80.83%), Decision Tree (80.37%), and K-Nearest Neighbors (75.75%). This indicates a significant enhancement in performance, with the proposed method outperforming the established models by up to 7.62%. Consequently, the proposed method can enhance the performance of fake news detection in Thai language by utilizing an ensemble of the original models. A significant advantage of this approach is its simplicity coupled with high efficacy. It is postulated that this method can be adapted for detecting fake news in other languages as well.

Robust active learning framework for structural reliability analysis using uncertainty quantification and flexible meta-model

Achieving highly accurate results in assessing structural reliability with reduced time complexity is an active and challenging engineering topic, especially when working with complex real-world structures. One of the most prominent methods addressing this issue is AK-MCS, which adaptively updates the Kriging-based surrogate model for forecasting structures’ behavior. While the Kriging model’s performance may be limited compared to multiple new machine learning models, it remains dominant in reliability analysis thanks to its distinctive ability to assess uncertainty along with prediction results. This ability is necessary to determine the potential candidate sample in the active learning procedure. This study proposes a novel method, dubbed AUQ-meta, that allows us to leverage the recent powerful machine learning models while still being able to produce uncertainty quantification. The AUQ-meta method consists of three main components: a machine learning-based predictor, a quantile regression module for uncertainty assessment, and an adaptive reliability analysis process. The effectiveness and efficiency of the proposed method are consistently validated through six examples with high dimensionality, nonlinearity, and time-varying elements. In addition, comparison and sensitivity studies are carried out, providing further insights into the AUQ-meta method’s operational mechanism.

Evaluation of multiple machine learning models for ASR expansion of concrete

Machine learning algorithms (MLA) show promise in predicting alkali-silica reaction (ASR) expansion. Nevertheless, the predictive performance of different machine learning algorithms (MLAs) varies across various engineering scenarios and datasets. Given that, this study was devoted to constructing an appropriate predictive model for ASR expansion of concrete. Multiple MLAs including evolutionary-based, support vector-based, assembly-based, and neural network-based algorithms, were employed to establish predictive models. An optimization technique, the particle swarm optimization algorithm, was also utilized and integrated with the aforementioned MLA models to effectively search for optimal hyperparameters and topological structures. The predictive performance was evaluated using statistical methods, which showed that the extreme gradient boosting (XGBoost) model had the best fitting goodness among the standard algorithms, with an R2 value of 0.9695. Moreover, the Particle Swarm Optimization (PSO) model significantly improved the predictive performance of all Machine Learning Algorithm (MLA) models. The error distribution analysis was then performed. There are fewer large errors in the predicted values of the PSO-XGBoost model. Monte Carlo simulation and probability distribution functions were used to thoroughly analyze the stability of each model after multiple independent runs. The results show that all models have excellent stability. The PSO-XGBoost model performed the best and was subsequently employed to conduct feature analysis to determine the significance of each input parameter and capture the underlying evidence of ASR expansion.

Urban flood susceptibility mapping using frequency ratio and multiple decision tree-based machine learning models

Urban flood susceptibility mapping using frequency ratio and multiple decision tree-based machine learning models

TENNER: intrusion detection models for industrial networks based on ensemble learning

In the pursuit of discerning patterns within computer network attacks, the utilization of Machine Learning and Deep Learning algorithms has been prevalent for crafting detection models based on extensive network traffic datasets. Furthermore, enhancing detection efficacy is feasible by applying cluster learning techniques, wherein multiple Machine Learning models collaborate to yield detection outcomes. Nevertheless, it is imperative to discern the optimal features within the dataset for training the intrusion detection model. In the present study, we proffer a novel framework for feature selection and intrusion detection within industrial networks, employing Ensemble Learning to achieve commendable performance in terms of both high predictive accuracy and efficient learning duration. The outcomes evince that the proposed model exhibits an accuracy of 99.93%, with a mere one h and 34 min required for comprehensive training. In contrast, a model trained without the framework presented in this paper attains an accuracy of 99.94%, necessitating an extensive training period of 156 h. Notably, the detection model derived from the proposed solution demonstrates superior results in prediction time, accomplishing predictions within 0.0009 seconds, compared to the alternative model which requires 0.0076 seconds for predictions.

Predicting the risk of lung cancer using machine learning: A large study based on UK Biobank.

In response to the high incidence and poor prognosis of lung cancer, this study tends to develop a generalizable lung-cancer prediction model by using machine learning to define high-risk groups and realize the early identification and prevention of lung cancer. We included 467,888 participants from UK Biobank, using lung cancer incidence as an outcome variable, including 49 previously known high-risk factors and less studied or unstudied predictors. We developed multivariate prediction models using multiple machine learning models, namely logistic regression, naïve Bayes, random forest, and extreme gradient boosting models. The performance of the models was evaluated by calculating the areas under their receiver operating characteristic curves, Brier loss, log loss, precision, recall, and F1 scores. The Shapley additive explanations interpreter was used to visualize the models. Three were ultimately 4299 cases of lung cancer that were diagnosed in our sample. The model containing all the predictors had good predictive power, and the extreme gradient boosting model had the best performance with an area under curve of 0.998. New important predictive factors for lung cancer were also identified, namely hip circumference, waist circumference, number of cigarettes previously smoked daily, neuroticism score, age, and forced expiratory volume in 1 second. The predictive model established by incorporating novel predictive factors can be of value in the early identification of lung cancer. It may be helpful in stratifying individuals and selecting those at higher risk for inclusion in screening programs.

A combined framework for carbon emissions prediction integrating online search attention

As CO2 emissions continue to rise, the problem of global warming is becoming increasingly serious. It is important to provide a robust management decision-making basis for the reductions of carbon emissions worldwide by predicting carbon emissions accurately. However, affected by various factors, the prediction of carbon emissions is challenging due to its nonlinear and nonstationary characteristics. Thus, we propose a combination forecast model, named CEEMDAN-GWO-SVR, which incorporates multiple features to predict trends in China’s carbon emissions. First, the impact of online search attention and public health emergencies are considered in carbon emissions prediction. Since the impact of different variables on carbon emissions is lagged, the grey relational degree is used to identify the appropriate lag series. Second, irrelevant features are eliminated through RFECV. To address the issue of feature redundancy of online search attention, we propose a dimensionality reduction method based on keyword classification. Finally, to evaluate the features of the proposed framework, four evaluation indicators are tested in multiple machine learning models. The best-performed model (SVR) is optimized by CEEMDAN and GWO to enhance prediction accuracy. The empirical results indicate that the proposed framework maintains good performance in both multi-scenario and multi-step prediction.

Enhancing Crop Yield Predictions with PEnsemble 4: IoT and ML-Driven for Precision Agriculture

This research introduces the PEnsemble 4 model, a weighted ensemble prediction model that integrates multiple individual machine learning models to achieve accurate maize yield forecasting. The model incorporates unmanned aerial vehicle (UAV) imagery and Internet of Things (IoT)-based environmental data, providing a comprehensive and data-driven approach to yield prediction in maize cultivation. Considering the projected growth in global maize demand and the vulnerability of maize crops to weather conditions, improved prediction capabilities are of paramount importance. The PEnsemble 4 model addresses this need by leveraging comprehensive datasets encompassing soil attributes, nutrient composition, weather conditions, and UAV-captured vegetation imagery. By employing a combination of Huber and M estimates, the model effectively analyzes temporal patterns in vegetation indices, in particular CIre and NDRE, which serve as reliable indicators of canopy density and plant height. Notably, the PEnsemble 4 model demonstrates a remarkable accuracy rate of 91%. It advances the timeline for yield prediction from the conventional reproductive stage (R6) to the blister stage (R2), enabling earlier estimation and enhancing decision-making processes in farming operations. Moreover, the model extends its benefits beyond yield prediction, facilitating the detection of water and crop stress, as well as disease monitoring in broader agricultural contexts. By synergistically integrating IoT and machine learning technologies, the PEnsemble 4 model presents a novel and promising solution for maize yield prediction. Its application holds the potential to revolutionize crop management and protection, contributing to efficient and sustainable farming practices.

Empirical Enhancement of Intrusion Detection Systems: A Comprehensive Approach with Genetic Algorithm-based Hyperparameter Tuning and Hybrid Feature Selection

AbstractMachine learning-based IDSs have demonstrated promising outcomes in identifying and mitigating security threats within IoT networks. However, the efficacy of such systems is contingent on various hyperparameters, necessitating optimization to elevate their performance. This paper introduces a comprehensive empirical and quantitative exploration aimed at enhancing intrusion detection systems (IDSs). The study capitalizes on a genetic algorithm-based hyperparameter tuning mechanism and a pioneering hybrid feature selection approach to systematically investigate incremental performance improvements in IDS. Specifically, our work proposes a machine learning-based IDS approach tailored for detecting attacks in IoT environments. To achieve this, we introduce a hybrid feature selection method designed to identify the most salient features for the task. Additionally, we employed the genetic algorithm (GA) to fine-tune hyperparameters of multiple machine learning models, ensuring their accuracy in detecting attacks. We commence by evaluating the default hyperparameters of these models on the CICIDS2017 dataset, followed by rigorous testing of the same algorithms post-optimization through GA. Through a series of experiments, we scrutinize the impact of combining feature selection methods with hyperparameter tuning approaches. The outcomes unequivocally demonstrate the potential of hyperparameter optimization in enhancing the accuracy and efficiency of machine learning-based IDS systems for IoT networks. The empirical nature of our research method provides a meticulous analysis of the efficacy of the proposed techniques through systematic experimentation and quantitative evaluation. Consolidated in a unified manner, the results underscore the step-by-step enhancement of IDS performance, especially in terms of detection time, substantiating the efficacy of our approach in real-world scenarios.

Impact of Social Media Sentiments on Stock Market Behavior: A Machine Learning Approach to Analyzing Market Dynamics

Social media has become a valuable tool for informed decision-making. This research delves into the influence of Twitter sentiments on the stock market’s movements and price fluctuations, specifically focusing on Tesla Inc and the tweets of Elon Musk. A combination of deductive and inductive reasoning approaches is used to explore the intricate relationship between the social media platform and the stock market. Methodologically, the Twitter data undergoes rigorous processing to derive features for the machine learning predictive model, and the sentiments are extracted using the Valence Aware Dictionary and Sentiment Reasoner (VADER) tool. This study emphasizes the usefulness of social media in predictive modeling while underscoring the importance of evaluating data reliability considering challenges such as spam tweets and geographical relevance. Multiple machine learning models are tested against four distinct datasets addressing the high stock price volatility. XG Boost and Random Forest Regressor emerge as the most effective performers, particularly when moving averages are included, showing enhanced performance. This research establishes an evident correlation between social media sentiments and stock market movements, however with limited predicting power. It is also noted that integrating traditional financial metrics enriches the understanding of stock market dynamics while enhancing the model’s predictability.

Detection and real-time analysis of influenza disease using graph data science and a multi-layer perceptron model

The influenza virus can spread easily, causing significant public health concern. Despite the existence of different techniques for rapid detection and prevention of influenza, their efficiency varies significantly. Additionally, there is currently a lack of a comprehensive, interoperable, and reusable real-time model for detecting influenza infection and predicting relationships within the field of influenza analysis. This study proposed a comprehensive, real-time model for rapid and early influenza detection using symptoms. Further, new relationships in the influenza field were discovered. Multiple data sources were used for the influenza knowledge graph (KG). Throughout this study, various graph algorithms were utilized to extract significant nodes and relationship features and multiple influenza detection machine learning (ML) models were compared. Node classification and link prediction methods were employed on a multi-layer perceptron (MLP) model. Furthermore, the hyperparameters of the model were automatically tuned. The proposed MLP model demonstrated the lowest rate of loss and the highest specificity, accuracy, recall, precision, and F1-score compared to state-of-the-art ML models. Moreover, the Matthews correlation coefficient was promising. This study shows that graph data science can improve MLP model detection and assist in discovering hidden connections in influenza KG.