Majority Vote Classifier Research Articles

Enhancing the accuracy of blockage detection in centrifugal pump using the majority voting classifier on an unbalanced dataset

Blockage in a centrifugal pump can adversely impact its performance. In this study, an experimental facility is developed to simulate three types of blockages (suction, discharge, and simultaneous suction and discharge). To classify these faults and detect their severity, a methodology involving the application of majority voting classifier (MVC) to the pump’s discharge pressure signals is presented. An unbalanced dataset is constructed, where the number of samples for a specific blockage condition decreases with increasing fault severity. Statistical features, entropy features, and entropies meta features are extracted from the signal and ranked using XGBoost and minimum redundancy maximum relevance (MRMR). Subsequently, the optimal features are selected based on the best performing model (Linear Discriminant Analysis) among ten different models. Best four models are selected and ensembled to form MVC. Results show that MVC achieves an accuracy of 89.90% and 88.26% for features selected by XGBoost and MRMR, respectively. Finally, the unbalanced dataset is balanced using synthetic minority oversampling and it is shown that MVC achieves an accuracy of 100% on this balanced dataset for the features selected using both approaches.

Mitigating Sociodemographic Bias in Opioid Use Disorder Prediction: Fairness-Aware Machine Learning Framework.

Opioid use disorder (OUD) is a critical public health crisis in the United States, affecting >5.5 million Americans in 2021. Machine learning has been used to predict patient risk of incident OUD. However, little is known about the fairness and bias of these predictive models. The aims of this study are two-fold: (1) to develop a machine learning bias mitigation algorithm for sociodemographic features and (2) to develop a fairness-aware weighted majority voting (WMV) classifier for OUD prediction. We used the 2020 National Survey on Drug and Health data to develop a neural network (NN) model using stochastic gradient descent (SGD; NN-SGD) and an NN model using Adam (NN-Adam) optimizers and evaluated sociodemographic bias by comparing the area under the curve values. A bias mitigation algorithm, based on equality of odds, was implemented to minimize disparities in specificity and recall. Finally, a WMV classifier was developed for fairness-aware prediction of OUD. To further analyze bias detection and mitigation, we did a 1-N matching of OUD to non-OUD cases, controlling for socioeconomic variables, and evaluated the performance of the proposed bias mitigation algorithm and WMV classifier. Our bias mitigation algorithm substantially reduced bias with NN-SGD, by 21.66% for sex, 1.48% for race, and 21.04% for income, and with NN-Adam by 16.96% for sex, 8.87% for marital status, 8.45% for working condition, and 41.62% for race. The fairness-aware WMV classifier achieved a recall of 85.37% and 92.68% and an accuracy of 58.85% and 90.21% using NN-SGD and NN-Adam, respectively. The results after matching also indicated remarkable bias reduction with NN-SGD and NN-Adam, respectively, as follows: sex (0.14% vs 0.97%), marital status (12.95% vs 10.33%), working condition (14.79% vs 15.33%), race (60.13% vs 41.71%), and income (0.35% vs 2.21%). Moreover, the fairness-aware WMV classifier achieved high performance with a recall of 100% and 85.37% and an accuracy of 73.20% and 89.38% using NN-SGD and NN-Adam, respectively. The application of the proposed bias mitigation algorithm shows promise in reducing sociodemographic bias, with the WMV classifier confirming bias reduction and high performance in OUD prediction.

ECG-based cardiac arrhythmias detection through ensemble learning and fusion of deep spatial–temporal and long-range dependency features

Cardiac arrhythmia is one of the prime reasons for death globally. Early diagnosis of heart arrhythmia is crucial to provide timely medical treatment. Heart arrhythmias are diagnosed by analyzing the electrocardiogram (ECG) of patients. Manual analysis of ECG is time-consuming and challenging. Hence, effective automated detection of heart arrhythmias is important to produce reliable results. Different deep-learning techniques to detect heart arrhythmias such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Transformer, and Hybrid CNN–LSTM were proposed. However, these techniques, when used individually, are not sufficient to effectively learn multiple features from the ECG signal. The fusion of CNN and LSTM overcomes the limitations of CNN in the existing studies as CNN–LSTM hybrids can extract spatiotemporal features. However, LSTMs suffer from long-range dependency issues due to which certain features may be ignored. Hence, to compensate for the drawbacks of the existing models, this paper proposes a more comprehensive feature fusion technique by merging CNN, LSTM, and Transformer models. The fusion of these models facilitates learning spatial, temporal, and long-range dependency features, hence, helping to capture different attributes of the ECG signal. These features are subsequently passed to a majority voting classifier equipped with three traditional base learners. The traditional learners are enriched with deep features instead of handcrafted features. Experiments are performed on the MIT-BIH arrhythmias database and the model performance is compared with that of the state-of-art models. Results reveal that the proposed model performs better than the existing models yielding an accuracy of 99.56%.

Predicting Listeria monocytogenes virulence potential using whole genome sequencing and machine learning

Contamination with food-borne pathogens, such as Listeria monocytogenes, remains a big concern for food safety. Hence, rigorous and continuous microbial surveillance is a standard procedure. At this point, however, the food industry and authorities only focus on detection of Listeria monocytogenes without characterization of individual strains into groups of more or less concern. As whole genome sequencing (WGS) gains increasing interest in the industry, this methodology presents an opportunity to obtain finer resolution of microbial traits such as virulence. Within this study, we therefore aimed to explore the use of WGS in combination with Machine Learning (ML) to predict L. monocytogenes virulence potential on a sub-species level.The WGS datasets used in this study for ML model training consisted of i) national surveillance isolates (n = 169, covering 38 MLST types) and ii) publicly available isolates acquired through the GenomeTrakr network (n = 2880, spanning 80 MLST types). We used the clinical frequency, i.e., ratio of the number of clinical isolates to total amount of isolates, as estimate for virulence potential. The predictive performance of input features from three different genomic levels (i.e., virulence genes, pan-genome genes, and single nucleotide polymorphisms (SNPs)) and six machine learning algorithms (i.e., Support Vector Machine with a linear kernel, Support Vector Machine with a radial kernel, Random Forrest, Neural Networks, LogitBoost, and Majority Voting) were compared.Our machine learning models predicted sub-species virulence potential with nested cross-validation F1-scores up to 0.88 for the majority voting classifier trained on national surveillance data and using pan-genome genes as input features. The validation of the pre-trained ML models based on 101 previously in vivo studied isolates resulted in F1-scores up to 0.76. Furthermore, we found that the more rapid and less computationally intensive raw read alignment yields comparably accurate models as de novo assembly.The results of our study suggest that a majority voting classifier trained on pan-genome genes is the best and most robust choice for the prediction of clinical frequency. Our study contributes to more rapid and precise characterization of L. monocytogenes virulence and its variation on a sub-species level. We further demonstrated a possible application of WGS data in the context of microbial hazard characterization for food safety. In the future, predictive models may assist case-specific microbial risk management in the food industry. The python code, pre-trained models, and prediction pipeline are deposited at (https://github.com/agmei/LmonoVirulenceML).

Predicting product sales based on sentiment analysis using reviews: The case of Amazon's kindle

This paper proposed a novel approach to predict sales by analyzing the sentiment of online product reviews. The contributions of this paper are: (1) examine sentiment analysis to extract emotions such as anger, fear, or participation from the product reviews; (2) identified the time lag of the interaction between reviews and sale fluctuation; (3) applied three multiclass classification methods to forecast sales and combined them using a majority vote classifier for better performance and accuracy. Online sellers can utilize the proposed models to analyze the potential influence of reviews as an addition to traditional sale forecasting methods.

Internal Leakage Detection in Hydraulic Pump Using Model-Agnostic Feature Ranking and Ensemble Classifiers

Abstract Hydraulic pumps are key drivers of fluid power-based machines and demand high reliability during operation. Internal leakage is a key performance deteriorating fault that reduces pump’s efficiency and limits its predictability and reliability. Thus, this article presents a methodology for detecting internal leakage in hydraulic pumps using an unbalanced dataset of its drive motor’s electrical power signals. Refined composite multiscale dispersion and fuzzy entropies along with three statistical indicators are extracted and followed by second-order polynomial-based features. These features are normalized and visualized using partial dependence plot (PDP) and individual conditional expectation (ICE). Subsequently, ten machine learning classifiers are trained using four features, and their statistical hypothesis test is performed using a 5 × 2 paired t-test cross-validation for p < 0.05. Subsequently, top four performing classifiers are optimized using grid and random search hyperparameter optimization techniques. Due to slight difference in their accuracies, an ensemble of three best-performing algorithms is trained using the majority voting classifiers (MaVCs) for three splitting ratios (80:20, 70:30, and 60:40). It is demonstrated that MaVC achieves the highest leakage detection accuracy of 90.91%.

Healthcare Monitoring Using Ensemble Classifiers in Fog Computing Framework

Nowadays, the cloud environment faces numerous issues like synchronizing information before the switch over the data migration. The requirement for a centralized internet of things (IoT)-based system has been restricted to some extent. Due to low scalability on security considerations, the cloud seems uninteresting. Since healthcare networks demand computer operations on large amounts of data, the sensitivity of device latency evolved among health networks is a challenging issue. In comparison to cloud domains, the new paradigms of fog computing give fresh alternatives by bringing resources closer to users by providing low latency and energy-efficient data processing solutions. Previous fog computing frameworks have various flaws, such as overvaluing response time or ignoring the accuracy of the result yet handling both at the same time compromises the network community. In this proposed work, Health Fog is integrated with the Optimized Cascaded Convolution Neural Network framework for diagnosing heart disease. Initially, the data is collected, and then pre-processing is done by Linear Discriminant Analysis. Then the features are extracted and optimized using Galactic Swarm Optimization. The optimized features are given into the Health Fog framework for diagnosing heart disease patients. It uses ensemble-based deep learning in edge computing devices, which automatically monitors real-life health networks such as heart disease analysis. Finally, the classifiers such as bagging, boosting, XGBoost, Multi-Layer Perceptron (MLP), and Partitions (PART) are used for classifying the data. Then the majority voting classifier predicts the result. This work uses FogBus architecture and evaluates the execution of power usage, bandwidth of the network, latency, execution time, and accuracy.

Detection of Distributed Denial of Service (DDoS) Attacks Using Computational Intelligence and Majority Vote-Based Ensemble Approach

The term “distributed denial of service” (DDoS) refers to one of the most common types of attacks. Sending a huge volume of data packets to the server machine is the target of a DDoS attack. This results in the majority of the consumption of network bandwidth and server, which ultimately leads to an issue with denial of service. In this paper, a majority vote-based ensemble of classifiers is utilized in the Sever technique, which results in improved accuracy and reduced computational overhead, when detecting attacks. For the experiment, the authors have used the CICDDOS2019 dataset. According to the findings of the experiment, a high level of accuracy of 99.98% was attained. In this paper, the classifiers use random forest, decision tree, and naïve bayes for majority voting classifiers, and from the results and performance, it can be seen that majority vote classifiers performed better.

Mental performance classification using fused multilevel feature generation with EEG signals

ABSTRACT Mental performance classification is a critical issue for brain-computer interfaces. Accurate and reliable classification of good or bad mental performance gives important clues for the preliminary diagnosis of some diseases and mental stress. In this work, we put forward an objective artificial intelligence model to quantify the clarity of thought during mental arithmetic tasks. The proposed model consists of: (i) multilevel feature extraction based on statistical and texture analysis methods, (ii) feature ranking and selection with a Chi2 method, (iii) classification, and (iv) weightless majority voting classifier. The novelty of the presented model comes from multilevel fused feature generation. The presented model was developed using 20 channel electroencephalography data from 36 subjects. The signals were captured while the subjects were performing mental arithmetic tasks. The individual datasets were labeled as either good or bad, based on the task results. We have obtained an accuracy of 96.77% using O2 channel with a k-nearest neighbor classifier and reached 100.0% accuracy with the majority voting classifier. Our results indicate that it is possible to determine mental performance with artificial intelligence. That might be a steppingstone to establish objective measures for the clarity of thought during a wide range of mental tasks.

Comparing Performances of Machine Learning Techniques to Forecast Dispute Resolutions

This paper compares classification performances of machine learning (ML) techniques for forecasting dispute resolutions in construction projects, thereby mitigating the impacts of potential disputes. Findings revealed that resolution cost and duration, contractor type, dispute source, and occurrence of changes were the most influential factors on dispute resolution method (DRM) preferences. The promising accuracy of the majority voting classifier (89.44%) indicates that the proposed model can provide decision-support in identification of potential resolutions. Decision-makers can avoid unsatisfactory processes using these forecasts. This paper demonstrated the effectiveness of ML techniques in classification of DRMs, and the proposed prediction model outperformed previous studies.

An Intelli MVC: An intelligent majority voting and heuristic-based hybrid classifier to predict high blood pressure

High Blood Pressure (HBP) is one of the major significant medical concerns of many people around the globe today. HBP is so common symptom many people across the globe are experiencing, irrespective of age, gender, region, and religion. HBP prediction ahead of time can help the person to avoid the consequences such as heart stroke, kidney failure, eye damage, sexual dysfunction, and even death. HBP prediction in advance is a challenging issue as it is associated with many biopsychosocial factors. Heuristic and meta-heuristic-based Machine Learning Models (MLM) exclusively supervised machine learning techniques are becoming part and parcel of medical data diagnosis. However, the reliability of outcome, usability, and understandability of such stand-alone models in processing medical data in real-time are not up to the mark. To overcome such limitations, in this paper we proposed an intelligent majority voting and heuristic-based user-friendly hybrid classifier to predict HBP (An Intelli MOC). The model considers AA-AOC (Anger level, Anxiety level-Age, Obesity level, and Cholesterol level) of a person to predict the HBP of a person. The proposed model is said to be majority vote-based and hybrid as it considers the output of three classifiers and assigns the count for each decision class. The model is said to be heuristic-based as it uses a mathematical and Fuzzy approach in obtaining the fuzzified values of each attribute in AA-AOC. The experiments are conducted on real-time data set collected from a medical diagnostic center Doctor C, Hyderabad, India. The model is executed on 1200 data records, 65% of data is used to train the model and 35% of data is used to test the model. The output of the model proved that the proposed model outperformed in terms of accuracy, precision, recall, and F-measure compared with all available state-of-the-art, existing MLM.

XAI-Fall: Explainable AI for Fall Detection on Wearable Devices Using Sequence Models and XAI Techniques

A fall detection system is vital for the safety of older people, as it contacts emergency services when it detects a person has fallen. There have been various approaches to detect falls, such as using a single tri-axial accelerometer to detect falls or fixing sensors on the walls of a room to detect falls in a particular area. These approaches have two major drawbacks: either (i) they use a single sensor, which is insufficient to detect falls, or (ii) they are attached to a wall that does not detect a person falling outside its region. Hence, to provide a robust method for detecting falls, the proposed approach uses three different sensors for fall detection, which are placed at five different locations on the subject’s body to gather the data used for training purposes. The UMAFall dataset is used to attain sensor readings to train the models for fall detection. Five models are trained corresponding to the five sensors models, and a majority voting classifier is used to determine the output. Accuracy of 93.5%, 93.5%, 97.2%, 94.6%, and 93.1% is achieved on each of the five sensors models, and 92.54% is the overall accuracy achieved by the majority voting classifier. The XAI technique called LIME is incorporated into the system in order to explain the model’s outputs and improve the model’s interpretability.

Semantic Analysis and Topic Modelling of Web-Scrapped COVID-19 Tweet Corpora through Data Mining Methodologies.

The evolution of the coronavirus (COVID-19) disease took a toll on the social, healthcare, economic, and psychological prosperity of human beings. In the past couple of months, many organizations, individuals, and governments have adopted Twitter to convey their sentiments on COVID-19, the lockdown, the pandemic, and hashtags. This paper aims to analyze the psychological reactions and discourse of Twitter users related to COVID-19. In this experiment, Latent Dirichlet Allocation (LDA) has been used for topic modeling. In addition, a Bidirectional Long Short-Term Memory (BiLSTM) model and various classification techniques such as random forest, support vector machine, logistic regression, naive Bayes, decision tree, logistic regression with stochastic gradient descent optimizer, and majority voting classifier have been adapted for analyzing the polarity of sentiment. The effectiveness of the aforesaid approaches along with LDA modeling has been tested, validated, and compared with several benchmark datasets and on a newly generated dataset for analysis. To achieve better results, a dual dataset approach has been incorporated to determine the frequency of positive and negative tweets and word clouds, which helps to identify the most effective model for analyzing the corpora. The experimental result shows that the BiLSTM approach outperforms the other approaches with an accuracy of 96.7%.

Major depressive disorder detection in polytechnic college students

Over 322 million people worldwide are affected by depression, the leading cause of disability. Major depressive disorder (MDD) is the most frequent mental disorder and contributes significantly to the global disease burden. Current depression diagnoses, on the other hand, are beset by issues such as patient denial, clinical experience, and self-report bias. Early detection of depression can assist in lessening or even eliminating its detrimental effects. To help the conventional diagnostic approaches in psychiatry, there has been much research into automated depression prediction in recent years. Automated depression identification based on machine learning techniques can help disorder analysts diagnose depression more effectively. The literature on depression has proven that the acoustic features of a depressed individual vary from a normal individual. This paper aims to identify the minimal acoustic features that can be used to detect depression accurately and propose a majority voting classifier for detecting depression (MVCDD). MVCDD is designed with the base classifiers, including Random Forest Classifier (RFC), Support Vector Classifier (SVC), and Logistic Regression (LR). The experiments are performed on the data collected from the students of Government Polytechnic, Masabtank, Hyderabad, India.

IMPROVING CORONARY HEART DISEASE PREDICTION BY OUTLIER ELIMINATION

Nowadays, heart disease is the major cause of deaths globally. According to a survey conducted by the World Health Organization, almost 18 million people die of heart diseases (or cardiovascular diseases) every day. So, there should be a system for early detection and prevention of heart disease. Detection of heart disease mostly depends on the huge pathological and clinical data that is quite complex. So, researchers and other medical professionals are showing keen interest in accurate prediction of heart disease. Heart disease is a general term for a large number of medical conditions related to heart and one of them is the coronary heart disease (CHD). Coronary heart disease is caused by the amassing of plaque on the artery walls. In this paper, various machine learning base and ensemble classifiers have been applied on heart disease dataset for efficient prediction of coronary heart disease. Various machine learning classifiers that have been employed include k-nearest neighbor, multilayer perceptron, multinomial naïve bayes, logistic regression, decision tree, random forest and support vector machine classifiers. Ensemble classifiers that have been used include majority voting, weighted average, bagging and boosting classifiers. The dataset used in this study is obtained from the Framingham Heart Study which is a long-term, ongoing cardiovascular study of people from the Framingham city in Massachusetts, USA. To evaluate the performance of the classifiers, various evaluation metrics including accuracy, precision, recall and f1 score have been used. According to our results, the best accuracy was achieved by logistic regression, random forest, majority voting, weighted average and bagging classifiers but the highest accuracy among these was achieved using weighted average ensemble classifier.

Classification of pulsar signals using ensemble gradient boosting algorithms based on asymmetric under-sampling method

The detection of pulsar signals is a highly intensive task. Numerous artificial intelligence (AI) and machine learning techniques (ML) have been proposed to classify pulsar and non-pulsar signals. While existing machine learning techniques improve classification efficiency, these methods are limited when it comes to dealing with large volumes of astronomical data, the extreme problem of class imbalance , and the polarization of high recall and precision. In this paper, to accurately classify pulsar and non-pulsar signals, extreme Gradient Boosting (XgBoost), and Light Gradient Boosting Machine (LightGBM) algorithms based on an asymmetric undersampling method are proposed. Firstly, the proposed model uses an asymmetric undersampling technique that divides the benchmark datasets into 75 subsets for the LOFAR Tied-Array All-Sky Survey (LOTAAS-1) and 9 subsets for the High Time Resolution Universe Pulsar Survey (HTRU2), eluding the class imbalance problem. Finally, XgBoost and LightGBM algorithms have been adopted for each new data subset, with a majority voting classifier being used to integrate the output of proposed models against each subset. Results of the proposed method were compared to state-of-the-art baseline models, which showed that our models significantly performed better than existing methods and gained almost 2% and 2.5% better F-score and precision results, respectively.

Novel Network for Medicinal Leaves Identification

From the ancient days, plant leaves are used to cure various infectious diseases. Even today, herbal leaves are preferred by medicinal experts for treating cancer, asthma, heart problems, etc. The recognition of these herbal plants is based on the visual perception of villagers. There are many kinds of species that seem to be very similar in color and shape. There is a high probability of human error in the identification of such plants. It is inevitable to correctly identify the species of plants to treat the patients. Therefore, a smart plant classification system is essentially required to eliminate human error. This research work develops a hybrid system that is based on deep convolutional neural networks. The system is named as AousethNet which is a modification of AlexNet by replacing its classifier namely, SoftMax with the Majority vote classifier. It is trained to predict the plant species from a huge number of leaf samples from four datasets namely Mendeley, D-Leaf, Flavia, and Folio. Typically, the performance of AousethNet with Mendeley dataset attained an accuracy of 99.89%, precision 98.61%, and very less recognition time of 0.087 s/image. This system is found to have good feature extraction and strong discrimination ability compared with the original version of AlexNet.

Radiometric Identification of Signals by Matched Whitening Transform

Radiometric identification is the problem of attributing a signal to a specific source. In this work, a radiometric identification algorithm is developed using the whitening transformation. The approach stands out from the more established methods in that it works directly on the raw IQ data and hence is featureless. As such, the commonly used dimensionality reduction algorithms do not apply. The premise of the idea is that a data set is “most white” when projected on its own whitening matrix than on any other. In practice, transformed data are never strictly white since the training and the test data differ. The Förstner-Moonen measure that quantifies the similarity of covariance matrices is used to establish the degree of whiteness. The whitening transform that produces a data set with the minimum Förstner-Moonen distance to a white noise process is the source signal. The source is determined by the output of the mode function operated on the Majority Vote Classifier decisions. Using the Förstner-Moonen measure presents a different perspective compared to maximum likelihood and Euclidean distance metrics. The whitening transform is also contrasted with the more recent deep learning approaches that are still dependent on feature vectors with large dimensions and lengthy training phases. It is shown that the proposed method is simpler to implement, requires no features vectors, needs minimal training and because of its non-iterative structure is faster than existing approaches.

Machine Learning to Improve the Prioritization and Effectiveness of Pre-Treatment Physics Chart Checks

Quality assurance (QA) is an integral part of planning for every patient prior to administering radiation therapy. The nationally regulated approach to QA involves performing multiple chart checks and often requires a high cognitive burden and significant amount of time. The purpose of this study is to explore the use of machine learning to classify the potential difficulty of physics pre-treatment chart check QA for a given radiation plan. Herein, we demonstrate a model that classifies radiation treatment plans as easy, medium, or hard with reference to QA time involved and cognitive burden. Pre-treatment data was collected for 1343 cases encompassing all cancer sites that were planned between August 2018 and October 2019. The outcome variable, a degree of difficulty on a scale of 1-10 was collected as a subjective rating by physicists who performed the pre-treatment chart checks. Scores of 1-3 were labeled as easy, 4-6 as intermediate, and 7-10 as difficult. A clinician and physicist then identified potential features, based on clinical relevance, contribution to plan complexity, and QA metrics. 1074 patient plans were used as a training set and the remaining 269 were set aside as a test set, an 80/20 split. The training data was used to train a k-nearest neighbor (KNN) classifier (k = 2) and a neural network (hidden layers = 3). Forward feature elimination was then utilized to incorporate only variables of importance prior to training decision tree (max depth = 10), random forest, and Adaboost (max depth = 0) classifiers. These five algorithms were incorporated into a majority voting classifier. Both training and test sets were equivalent in their proportion of difficult (11%), intermediate (41%), and easy (47%) cases. Accuracy on the test set was calculated for each classifier and was found to be 96% for the Adaboost and random forest classifiers, 94% for the neural network, 90% for the decision tree, and 84% for the KNN. The majority voting classifier achieved an accuracy of 95% overall, with 94% accuracy on difficult cases, 95% on medium cases, and 95% on easy cases. Precision, recall, and F-score were also calculated for each classifier and for the majority vote (see Table). We present a majority voting classifier that uses clinical, plan complexity, and QA features in order to predict the difficulty of radiation treatment plans. This will be used in our department to guide the physicist in prioritizing their activities in an effort to reduce cognitive workload and improve the effectiveness of pre-treatment chart checks. Future directions include validating this model at our community practices and addressing limitations such as class imbalance and missing data.Abstract 110; TableAdaboostDecision TreeKNNNeural NetworkRandom ForestMajority (hard vote)Accuracy96%90%84%94%96%95%Precision (PPV)96%93%86%95%97%96%Recall (Sensitivity)96%88%82%93%96%95%F-score96%90%84%94%96%95% Open table in a new tab

Fast greedy mathcal {C}-bound minimization with guarantees

The mathcal {C}-bound is a tight bound on the true risk of a majority vote classifier that relies on the individual quality and pairwise disagreement of the voters and provides PAC-Bayesian generalization guarantees. Based on this bound, MinCq is a classification algorithm that returns a dense distribution on a finite set of voters by minimizing it. Introduced later and inspired by boosting, CqBoost uses a column generation approach to build a sparse mathcal {C}-bound optimal distribution on a possibly infinite set of voters. However, both approaches have a high computational learning time because they minimize the mathcal {C}-bound by solving a quadratic program. Yet, one advantage of CqBoost is its experimental ability to provide sparse solutions. In this work, we address the problem of accelerating the mathcal {C}-bound minimization process while keeping the sparsity of the solution and without losing accuracy. We present CB-Boost, a computationally efficient classification algorithm relying on a greedy–boosting-based–mathcal {C}-bound optimization. An in-depth analysis proves the optimality of the greedy minimization process and quantifies the decrease of the mathcal {C}-bound operated by the algorithm. Generalization guarantees are then drawn based on already existing PAC-Bayesian theorems. In addition, we experimentally evaluate the relevance of CB-Boost in terms of the three main properties we expect about it: accuracy, sparsity, and computational efficiency compared to MinCq, CqBoost, Adaboost and other ensemble methods. As observed in these experiments, CB-Boost not only achieves results comparable to the state of the art, but also provides mathcal {C}-bound sub-optimal weights with very few computational demand while keeping the sparsity property of CqBoost.