Soft Voting Research Articles

Industry 4.0 oriented predictive analytics of cardiovascular diseases using machine learning, hyperparameter tuning and ensemble techniques

PurposeThe world is shifting towards the fourth industrial revolution (Industry 4.0), symbolising the move to digital, fully automated habitats and cyber-physical systems. Industry 4.0 consists of innovative ideas and techniques in almost all sectors, including Smart health care, which recommends technologies and mechanisms for early prediction of life-threatening diseases. Cardiovascular disease (CVD), which includes stroke, is one of the world’s leading causes of sickness and deaths. As per the American Heart Association, CVDs are a leading cause of death globally, and it is believed that COVID-19 also influenced the health of cardiovascular and the number of patients increases as a result. Early detection of such diseases is one of the solutions for a lower mortality rate. In this work, early prediction models for CVDs are developed with the help of machine learning (ML), a form of artificial intelligence that allows computers to learn and improve on their own without requiring to be explicitly programmed.Design/methodology/approachThe proposed CVD prediction models are implemented with the help of ML techniques, namely, decision tree, random forest, k-nearest neighbours, support vector machine, logistic regression, AdaBoost and gradient boosting. To mitigate the effect of over-fitting and under-fitting problems, hyperparameter optimisation techniques are used to develop efficient disease prediction models. Furthermore, the ensemble technique using soft voting is also used to gain more insight into the data set and accurate prediction models.FindingsThe models were developed to help the health-care providers with the early diagnosis and prediction of heart disease patients, reducing the risk of developing severe diseases. The created heart disease risk evaluation model is built on the Jupyter Notebook Web application, and its performance is calculated using unbiased indicators such as true positive rate, true negative rate, accuracy, precision, misclassification rate, area under the ROC curve and cross-validation approach. The results revealed that the ensemble heart disease model outperforms the other proposed and implemented models.Originality/valueThe proposed and developed CVD prediction models aims at predicting CVDs at an early stage, thereby taking prevention and precautionary measures at a very early stage of the disease to abate the predictive maintenance as recommended in Industry 4.0. Prediction models are developed on algorithms’ default values, hyperparameter optimisations and ensemble techniques.

A Novel Integration of Face-Recognition Algorithms with a Soft Voting Scheme for Efficiently Tracking Missing Person in Challenging Large-Gathering Scenarios.

The probability of losing vulnerable companions, such as children or older ones, in large gatherings is high, and their tracking is challenging. We proposed a novel integration of face-recognition algorithms with a soft voting scheme, which was applied, on low-resolution cropped images of detected faces, in order to locate missing persons in a challenging large-crowd gathering. We considered the large-crowd gathering scenarios at Al Nabvi mosque Madinah. It is a highly uncontrolled environment with a low-resolution-images data set gathered from moving cameras. The proposed model first performs real-time face-detection from camera-captured images, and then it uses the missing person’s profile face image and applies well-known face-recognition algorithms for personal identification, and their predictions are further combined to obtain more mature prediction. The presence of a missing person is determined by a small set of consecutive frames. The novelty of this work lies in using several recognition algorithms in parallel and combining their predictions by a unique soft-voting scheme, which in return not only provides a mature prediction with spatio-temporal values but also mitigates the false results of individual recognition algorithms. The experimental results of our model showed reasonably good accuracy of missing person’s identification in an extremely challenging large-gathering scenario.

Novel ensemble intelligence methodologies for rockburst assessment in complex and variable environments

Rockburst is a severe geological hazard that restricts deep mine operations and tunnel constructions. To overcome the shortcomings of widely used algorithms in rockburst prediction, this study investigates the ensemble trees, i.e., random forest (RF), extremely randomized tree (ET), adaptive boosting machine (AdaBoost), gradient boosting machine, extreme gradient boosting machine (XGBoost), light gradient boosting machine, and category gradient boosting machine, for rockburst estimation based on 314 real rockburst cases. Additionally, Bayesian optimization is utilized to optimize these ensemble trees. To improve performance, three combination strategies, voting, bagging, and stacking, are adopted to combine multiple models according to training accuracy. ET and XGBoost receive the best capabilities (85.71% testing accuracy) in single models, and except for AdaBoost, six ensemble trees have high accuracy and can effectively foretell strong rockburst to prevent large-scale underground disasters. The combination models generated by voting, bagging, and stacking perform better than single models, and the voting 2 model that combines XGBoost, ET, and RF with simple soft voting, is the most outstanding (88.89% testing accuracy). The performed sensitivity analysis confirms that the voting 2 model has better robustness than single models and has remarkable adaptation and superiority when input parameters vary or miss, and it has more power to deal with complex and variable engineering environments. Eventually, the rockburst cases in Sanshandao Gold Mine, China, were investigated, and these data verify the practicability of voting 2 in field rockburst prediction.

Integrating LASSO Feature Selection and Soft Voting Classifier to Identify Origins of Replication Sites.

Background: DNA replication plays an indispensable role in the transmission of genetic information. It is considered to be the basis of biological inheritance and the most fundamental process in all biological life. Considering that DNA replication initiates with a special location, namely the origin of replication, a better and accurate prediction of the origins of replication sites (ORIs) is essential to gain insight into the relationship with gene expression. Objective: In this study, we have developed an efficient predictor called iORI-LAVT for ORIs identification. Methods: This work focuses on extracting feature information from three aspects, including mono-nucleotide encoding, k-mer and ring-function-hydrogen-chemical properties. Subsequently, least absolute shrinkage and selection operator (LASSO) as a feature selection is applied to select the optimal features. Comparing the different combined soft voting classifiers results, the soft voting classifier based on GaussianNB and Logistic Regression is employed as the final classifier. Results: Based on 10-fold cross-validation test, the prediction accuracies of two benchmark datasets are 90.39% and 95.96%, respectively. As for the independent dataset, our method achieves high accuracy of 91.3%. Conclusion: Compared with previous predictors, iORI-LAVT outperforms the existing methods. It is believed that iORI-LAVT predictor is a promising alternative for further research on identifying ORIs.

When CNNs meet random RNNs: Towards multi-level analysis for RGB-D object and scene recognition

Recognizing objects and scenes are two challenging but essential tasks in image understanding. In particular, the use of RGB-D sensors in handling these tasks has emerged as an important area of focus for better visual understanding. Meanwhile, deep neural networks, specifically convolutional neural networks (CNNs), have become widespread and have been applied to many visual tasks by replacing hand-crafted features with effective deep features. However, it is an open problem how to exploit deep features from a multi-layer CNN model effectively. In this paper, we propose a novel two-stage framework that extracts discriminative feature representations from multi-modal RGB-D images for object and scene recognition tasks. In the first stage, a pretrained CNN model has been employed as a backbone to extract visual features at multiple levels. The second stage maps these features into high level representations with a fully randomized structure of recursive neural networks (RNNs) efficiently. To cope with the high dimensionality of CNN activations, a random weighted pooling scheme has been proposed by extending the idea of randomness in RNNs. Multi-modal fusion has been performed through a soft voting approach by computing weights based on individual recognition confidences (i.e. SVM scores) of RGB and depth streams separately. This produces consistent class label estimation in final RGB-D classification performance. Extensive experiments verify that fully randomized structure in RNN stage encodes CNN activations to discriminative solid features successfully. Comparative experimental results on the popular Washington RGB-D Object and SUN RGB-D Scene datasets show that the proposed approach achieves superior or on-par performance compared to state-of-the-art methods both in object and scene recognition tasks. Code is available at https://github.com/acaglayan/CNN_randRNN.

Transfer inhibitory potency prediction to binary classification: A model only needs a small training set

One of the most laborious for drug discovery is to select compounds from a library for experimental evaluation. Hence, we propose a machine learning model only needs to be trained on a small dataset to predict the inhibition constant (Ki) and half maximal inhibitory concentration (IC50) for a compound. We transfer the prediction task to a simpler binary classification task based on a naive but effective idea that we only need the related rank of a compound to determine whether to take it for further examination. To achieve this, we design a data augmentation strategy to effectively leverage the relationship between the compounds in the training set. After that, we formulate a new reward function for deep reinforcement learning to balance the feature selection and the accuracy. We employ a particle swarm optimized support vector machine for the binary classification task. Finally, a soft voting mechanism is introduced to solve the contradiction from the binary classification. Sufficient experiments show that our model achieves high and reliable accuracy, and is capable of ranking compounds based on a selected set of molecular descriptors. The current results show that our model provides a potential ligand-based in silico approach for prioritizing chemicals for experimental studies.

EnANNDeep: An Ensemble-based lncRNA-protein Interaction Prediction Framework with Adaptive k-Nearest Neighbor Classifier and Deep Models.

lncRNA-protein interactions (LPIs) prediction can deepen the understanding of many important biological processes. Artificial intelligence methods have reported many possible LPIs. However, most computational techniques were evaluated mainly on one dataset, which may produce prediction bias. More importantly, they were validated only under cross validation on lncRNA-protein pairs, and did not consider the performance under cross validations on lncRNAs and proteins, thus fail to search related proteins/lncRNAs for a new lncRNA/protein. Under an ensemble learning framework (EnANNDeep) composed of adaptive k-nearest neighbor classifier and Deep models, this study focuses on systematically finding underlying linkages between lncRNAs and proteins. First, five LPI-related datasets are arranged. Second, multiple source features are integrated to depict an lncRNA-protein pair. Third, adaptive k-nearest neighbor classifier, deep neural network, and deep forest are designed to score unknown lncRNA-protein pairs, respectively. Finally, interaction probabilities from the three predictors are integrated based on a soft voting technique. In comparing to five classical LPI identification models (SFPEL, PMDKN, CatBoost, PLIPCOM, and LPI-SKF) under fivefold cross validations on lncRNAs, proteins, and LPIs, EnANNDeep computes the best average AUCs of 0.8660, 0.8775, and 0.9166, respectively, and the best average AUPRs of 0.8545, 0.8595, and 0.9054, respectively, indicating its superior LPI prediction ability. Case study analyses indicate that SNHG10 may have dense linkage with Q15717. In the ensemble framework, adaptive k-nearest neighbor classifier can separately pick the most appropriate k for each query lncRNA-protein pair. More importantly, deep models including deep neural network and deep forest can effectively learn the representative features of lncRNAs and proteins.

Sentiment Analysis on Customer Satisfaction of Digital Banking in Indonesia

Southeast Asia, including Indonesia, is seeing an increase in digital banking adoption, owing to changing customer expectations and increasing digital penetration. The pandemic Covid-19 has hastened this tendency for digital transformation. However, customer satisfaction should not be left unmanaged during this transition. This research aims to obtain customer satisfaction of digital banking in Indonesia based on sentiment analysis from Twitter. Data collected were related to three digital banks in Indonesia, namely Jenius, Jago, and Blu. Total of 34,605 tweets were collected and analyzed within the period of August 1st 2021 to October 31st 2021. Sentiment analysis was conducted using nine standalone classifiers, Naïve Bayes, Logistic Regression, K-Nearest Neighbours, Support Vector Machines, Random Forest, Decision Tree, Adaptive Boosting, eXtreme Gradient Boosting and Light Gradient Boosting Machine. Two ensemble methods were also used for this research, hard voting and soft voting. The results of this study show that SVM among other stand-alone classifiers has the best performance when used to predict sentiments with value for F1-score 73.34%. Ensemble method performed better than using stand-alone classifier, and soft voting with 5-best classifiers performed best overall with value for F1-score 74.89%. The results also show that Jago sentiments were mainly positive, Jenius sentiments mostly were negative and for Blu, most sentiments were neutral.

Predicting the Reader's English Level From Reading Fixation Patterns Using the Siamese Convolutional Neural Network.

Among numerous functions performed by the human eye, reading is a common task that best reflects an individual's understanding and cognitive patterns. Previous studies showed that text comprehension may be determined by comprehension monitoring, a metacognitive process that evaluates and regulates the pattern of comprehension. Herein, we propose a hypothesis: an individual's cognitive pattern during reading is predictive of the level of reading comprehension. According to the criteria of the College English Test Band Six (CET-6), 80 participants (sophomore and junior) were divided into a pass group (n = 40) and a non-pass group (n = 40). Heatmaps of eye fixation counts were collected by an eye-tracker while each participant executed four reading comprehension tests. Using these heatmaps as inputs, we proposed the Siamese convolutional neural network models to predict the English level of participants. Both strategies of "Trained from scratch" and "Pretrained with fine-tuning" were employed. "Soft Voting" was applied to integrate the predictions from four tests. Results showed that the Siamese network model trained by the datasets with the cluster radius of fixation equal to 25 pixels and connection layer by L1 norm distance has a satisfactory or superior performance to other comparative experiments. The AUC values of Siamese networks trained by the two strategies reach 0.941 and 0.956, respectively. This indicates that the individual reading cognitive pattern captured by the eye-tracker can predict the level of reading comprehension through advanced deep learning models.

Performance Evaluation of Multiple Classifiers for Predicting Fake News

The rise of fake news on social media has had a detrimental effect on society. Numerous performance evaluations on classifiers that can detect fake news have previously been undertaken by researchers in this area. To assess their performance, we used 14 different classifiers in this study. Secondly, we looked at how soft voting and hard voting classifiers performed in a mixture of distinct individual classifiers. Finally, heuristics are used to create 9 models of stacking classifiers. The F1 score, prediction, recall, and accuracy have all been used to assess performance. Models 6 and 7 achieved the best accuracy of 96.13 while having a larger computational complexity. For benchmarking purposes, other individual classifiers are also tested.

Multifeature Extraction and Semi-Supervised Deep Learning Scheme for State Diagnosis of Converter Transformer

The research on intelligent diagnostic models for power transmission and transformation equipment has attracted increasing interest. In the design of practical diagnostic models, the supervised learning strategy is often used. This put forward higher requirements for the number of labeled data. However, in industrial applications, the cost of collecting a large number of relevant data and effectively labeling is high. At the same time, in the case of unknown faults, the lack of prior knowledge makes accurate classification difficult to achieve. This work designs a semi-supervised hybrid framework to diagnose the state of the converter transformer in terms of the labeled data which is limited. The framework consists of three main modules: multifeature graph generation, semi-supervised training, and soft voting decision. The graph generation module encodes the vibration signal into time, frequency, and energy graphs by calculating the Gramian angular summation fields (GASF). The semi-supervised learning module integrates unsupervised learning and supervised learning strategies trained by partial label information and data reconstruction errors. The decision-making module makes multifeature decisions through soft voting. The effectiveness of the hybrid framework is verified by comparative experiments. The experimental results show that the scheme can learn and diagnose intelligently under harsh datasets.

IoT Based Railway Track Faults Detection and Localization Using Acoustic Analysis

Rail is one of the most energy efficient and economical modes of transportation amongst many. Regular inspection of railway track health is essential for ensuring robust and secure train operations. Investigations and delayed discovery pose a serious risk to the safe functioning of rail transportation. The traditional method of manually examining the rail track using a railway cart is both inefficient and susceptible to mistakes and bias. It is imperative to automate inspection in order to avert catastrophes and save countless lives particularly in zones where train accidents are numerous. The purpose of this research is to develop an Internet of Things (IoT)-based autonomous railway track fault detection system to enhance the existing railway cart system in order to address the aforementioned issues. In addition to data collection on Pakistani railway lines, this work contributes significantly to railway track fault identification and classification based on acoustic analysis, as well as fault localization. Due to their frequency of occurrences, six types of track’s faults were first targeted: wheel burnt, loose nuts and bolts, crash sleeper, creep, low joint, and point and crossing. Support vector machines, logistic regression, random forest, extra tree classifier, decision tree classifier, multilayer perceptron and ensemble with hard and soft voting were among the machine learning methods used. The results indicate that acoustic data can successfully assist in discriminating the track defects and localizing the defects in real time. The results show that MLP achieved the best results, with an accuracy of 98.4 percent.

Non-Invasive Early Diagnosis of Obstructive Lung Diseases Leveraging Machine Learning Algorithms

Lungs are a vital human body organ, and different Obstructive Lung Diseases (OLD) such as asthma, bronchitis, or lung cancer are caused by shortcomings within the lungs. Therefore, early diagnosis of OLD is crucial for such patients suffering from OLD since, after early diagnosis, breathing exercises and medical precautions can effectively improve their health state. A secure non-invasive early diagnosis of OLD is a primordial need, and in this context, digital image processing supported by Artificial Intelligence (AI) techniques is reliable and widely used in the medical field, especially for improving early disease diagnosis. Hence, this article presents an AI-based non-invasive and secured diagnosis for OLD using physiological and iris features. This research work implements different machine-learning-based techniques which classify various subjects, which are healthy and effective patients. The iris features include gray-level run-length matrix-based features, gray-level co-occurrence matrix, and statistical features. These features are extracted from iris images. Additionally, ten different classifiers and voting techniques, including hard and soft voting, are implemented and tested, and their performances are evaluated using several parameters, which are precision, accuracy, specificity, F-score, and sensitivity. Based on the statistical analysis, it is concluded that the proposed approach offers promising techniques for the non-invasive early diagnosis of OLD with an accuracy of 97.6%.

Fatality Prediction for Motor Vehicle Collisions: Mining Big Data Using Deep Learning and Ensemble Methods

Motor vehicle crashes are one of the most common causes of fatalities on the roads. Real-time severity prediction of such crashes may contribute towards reducing the rate of fatality. In this study, the fundamental goal is to develop machine learning models that predict whether the outcome of a collision will be fatal or not. A Canadian road crash dataset containing 5.8 million records is utilized in this research. In this study, ensemble models have been developed using majority and soft voting to address the class imbalance in the dataset. The prediction accuracy of approximately 75% is achieved using Convolutional Neural Networks. Moreover, a comprehensive analysis of the attributes that are important in distinguishing between fatal vs. non-fatal motor vehicle collisions has been presented in this paper. In-depth information content analysis reveals the factors that contribute the most in the prediction model. These include roadway characteristics and weather conditions at the time of the crash, vehicle type, time when the collision happen, road user class and their position, any safety device used, and the status of traffic control. With real-time data based on weather and road conditions, an automated warning system can potentially be developed utilizing the prediction model employed in this study.

Predicting miRNA-disease associations using an ensemble learning framework with resampling method.

Accumulating evidences have indicated that microRNA (miRNA) plays a crucial role in the pathogenesis and progression of various complex diseases. Inferring disease-associated miRNAs is significant to explore the etiology, diagnosis and treatment of human diseases. As the biological experiments are time-consuming and labor-intensive, developing effective computational methods has become indispensable to identify associations between miRNAs and diseases. We present an Ensemble learning framework with Resampling method for MiRNA-Disease Association (ERMDA) prediction to discover potential disease-related miRNAs. Firstly, the resampling strategy is proposed for building multiple different balanced training subsets to address the challenge of sample imbalance within the database. Then, ERMDA extracts miRNA and disease feature representations by integrating miRNA-miRNA similarities, disease-disease similarities and experimentally verified miRNA-disease association information. Next, the feature selection approach is applied to reduce the redundant information and increase the diversity among these subsets. Lastly, ERMDA constructs an individual learner on each subset to yield primitive outcomes, and the soft voting method is introduced for making the final decision based on the prediction results of individual learners. A series of experimental results demonstrates that ERMDA outperforms other state-of-the-art methods on both balanced and unbalanced testing sets. Besides, case studies conducted on the three human diseases further confirm the ERMDA's prediction capability for identifying potential disease-related miRNAs. In conclusion, these experimental results demonstrate that our method can serve as an effective and reliable tool for researchers to explore the regulatory role of miRNAs in complex diseases.

Scaling the U-net: segmentation of biodegradable bone implants in high-resolution synchrotron radiation microtomograms

Highly accurate segmentation of large 3D volumes is a demanding task. Challenging applications like the segmentation of synchrotron radiation microtomograms (SRμCT) at high-resolution, which suffer from low contrast, high spatial variability and measurement artifacts, readily exceed the capacities of conventional segmentation methods, including the manual segmentation by human experts. The quantitative characterization of the osseointegration and spatio-temporal biodegradation process of bone implants requires reliable, and very precise segmentation. We investigated the scaling of 2D U-net for high resolution grayscale volumes by three crucial model hyper-parameters (i.e., the model width, depth, and input size). To leverage the 3D information of high-resolution SRμCT, common three axes prediction fusing is extended, investigating the effect of adding more than three axes prediction. In a systematic evaluation we compare the performance of scaling the U-net by intersection over union (IoU) and quantitative measurements of osseointegration and degradation parameters. Overall, we observe that a compound scaling of the U-net and multi-axes prediction fusing with soft voting yields the highest IoU for the class “degradation layer”. Finally, the quantitative analysis showed that the parameters calculated with model segmentation deviated less from the high quality results than those obtained by a semi-automatic segmentation method.

Machine Learning Techniques Applied to Predict Tropospheric Ozone in a Semi-Arid Climate Region

In the last decade, ground-level ozone exposure has led to a significant increase in environmental and health risks. Thus, it is essential to measure and monitor atmospheric ozone concentration levels. Specifically, recent improvements in machine learning (ML) processes, based on statistical modeling, have provided a better approach to solving these risks. In this study, we compare Naive Bayes, K-Nearest Neighbors, Decision Tree, Stochastic Gradient Descent, and Extreme Gradient Boosting (XGBoost) algorithms and their ensemble technique to classify ground-level ozone concentration in the El Paso-Juarez area. As El Paso-Juarez is a non-attainment city, the concentrations of several air pollutants and meteorological parameters were analyzed. We found that the ensemble (soft voting classifier) of algorithms used in this paper provide high classification accuracy (94.55%) for the ozone dataset. Furthermore, variables that are highly responsible for the high ozone concentration such as Nitrogen Oxide (NOx), Wind Speed and Gust, and Solar radiation have been discovered.

Performance of Machine Learning-Based Multi-Model Voting Ensemble Methods for Network Threat Detection in Agriculture 4.0.

The upcoming agricultural revolution, known as Agriculture 4.0, integrates cutting-edge Information and Communication Technologies in existing operations. Various cyber threats related to the aforementioned integration have attracted increasing interest from security researchers. Network traffic analysis and classification based on Machine Learning (ML) methodologies can play a vital role in tackling such threats. Towards this direction, this research work presents and evaluates different ML classifiers for network traffic classification, i.e., K-Nearest Neighbors (KNN), Support Vector Classification (SVC), Decision Tree (DT), Random Forest (RF) and Stochastic Gradient Descent (SGD), as well as a hard voting and a soft voting ensemble model of these classifiers. In the context of this research work, three variations of the NSL-KDD dataset were utilized, i.e., initial dataset, undersampled dataset and oversampled dataset. The performance of the individual ML algorithms was evaluated in all three dataset variations and was compared to the performance of the voting ensemble methods. In most cases, both the hard and the soft voting models were found to perform better in terms of accuracy compared to the individual models.

Establishing a Credit Risk Evaluation System for SMEs Using the Soft Voting Fusion Model

In China, SMEs are facing financing difficulties, and commercial banks and financial institutions are the main financing channels for SMEs. Thus, a reasonable and efficient credit risk assessment system is important for credit markets. Based on traditional statistical methods and AI technology, a soft voting fusion model, which incorporates logistic regression, support vector machine (SVM), random forest (RF), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM), is constructed to improve the predictive accuracy of SMEs’ credit risk. To verify the feasibility and effectiveness of the proposed model, we use data from 123 SMEs nationwide that worked with a Chinese bank from 2016 to 2020, including financial information and default records. The results show that the accuracy of the soft voting fusion model is higher than that of a single machine learning (ML) algorithm, which provides a theoretical basis for the government to control credit risk in the future and offers important references for banks to make credit decisions.

A Decision-Level Fusion Method for COVID-19 Patient Health Prediction

With the continuous attempts to develop effective machine learning methods, information fusion approaches play an important role in integrating data from multiple sources and improving these methods' performance. Among the different fusion techniques, decision-level fusion has unique advantages to fuse the decisions of various classifiers and getting an effective outcome. In this paper, we propose a decision-level fusion method that combines three well-calibrated ensemble classifiers, namely, a random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGB) methods. It is used to predict the COVID-19 patient health for early monitoring and efficient treatment. A soft voting technique is used to generate the final decision result from the predictions of these calibrated classifiers. The method uses the COVID-19 patient's health information, travel demographic, and geographical data to predict the possible outcome of the COVID-19 case, recovered, or death. A different set of experiments is conducted on a public novel Corona Virus 2019 dataset using a different ratio of test sets. The experimental results show that the proposed fusion method achieved an accuracy of 97.24% and an F1-score of 0.97, which is higher than the current related work that has an accuracy of 94% and an F1-score 0.86, on 20% test set taken from the dataset.