Weighted Majority Algorithm Research Articles

Balancing Techniques for Advanced Financial Distress Detection Using Artificial Intelligence

Imbalanced datasets are one of the main issues encountered by artificial intelligence researchers, as machine learning (ML) algorithms can become biased toward the majority class and perform insufficiently on the minority classes. Financial distress (FD) is one of the numerous real-world applications of ML, struggling with this issue. Furthermore, the topic of financial distress holds considerable interest for both academics and practitioners due to the non-determined indicators of condition states. This research focuses on the involvement of balancing techniques according to different FD condition states. Moreover, this research was expanded by implementing ML models and dimensionality reduction techniques. During the course of this study, a Combined FD was constructed using five distinct conditions, ten distinct class balancing techniques, five distinct dimensionality reduction techniques, two features selection strategies, eleven machine learning models, and twelve weighted majority algorithms (WMAs). Results revealed that the highest area under the receiver operating characteristic (ROC) curve (AUC) score was achieved when using the extreme gradient boosting machine (XGBoost) feature selection technique, the experimental max number strategy, the undersampling methods, and the WMA 3.1 weighted majority algorithm (i.e., with categorical boosting (CatBoost), XGBoost, and random forest (RF) having equal voting weights). Moreover, this research has introduced a novel approach for setting the condition states of financial distress, including perspectives from debt and change in employment. These outcomes have been achieved utilizing authentic enterprise data from small and medium Lithuanian enterprises.

Locality sensitive hashing scheme based on online-learning

Locally Sensitive Hashing (LSH) algorithms are classical algorithms commonly used on the c-Approximate Nearest Neighbor (c-ANN) search problem. When using Euclidean distance to measure sample similarity and solve the c-ANN problem, the traditional approach is to utilize the Exact Euclidean Locality Sensitive Hashing (E2LSH) algorithm based on the p-stable distribution. However, the uncertainty of the p-stable distribution causes the hash buckets constructed by the E2LSH algorithm to vary in queries. Therefore, this paper proposes the OLLSH algorithm based on the Weighted Majority algorithm in the Online-Learning framework, which selects the hash buckets with more stable query accuracy by weighted voting on the hash buckets generated by the E2LSH algorithm. Then, we conduct simulation experiments on synthetic dataset and four real data sets and conclude that the proposed OLLSH algorithm improves the accuracy compared to the original algorithm with the same memory usage.

Deep learning-based multi-model ensemble method for classification of PQDs in a hydrogen energy-based microgrid using modified weighted majority algorithm

In this study, new multiple deep classifiers with a modified Weighted Majority Voting (WMV)-based method are proposed to identify power quality disturbances (PQDs) in a hydrogen energy-based microgrid. In the proposed approach, closed-loop deep LSTM (Long Short Time Memory), deep CNN (Convolutional Neural Network), and hybrid (CNN-LSTM) models are used for automatic feature extraction and classification. Then, a modified WMV method is employed to ensemble the outputs of the three deep learning (DL) classifier models. The enhanced WMV mechanism performs an automatically updated weighting based on the validation results of the DL classification models, unlike voting methods in the literature. The improved WMV mechanism eliminates the challenges of using multiple DL classifiers in the voting system. The mathematical data results in LabVIEW, simulation results in Matlab/Simulink, and real data results in the laboratory show that the proposed method shows superior performance in accuracy and noise immunity to state-of-the-art methods.

Arrhythmic Heartbeat Classification Using Ensemble of Random Forest and Support Vector Machine Algorithm

In this article, an automated heartbeat classification has been proposed to prevent the growing threats of cardiovascular diseases around the world. The MIT-BIH arrhythmia database has been used for the training and testing of the proposed approach. The database contains 90 095 normal (N), 2781 supraventricular (SVEB), 7008 ventricular (VEB), 802 fusions (F), and 15 unclassified (Q) beats each of 30 min duration. Total 61 features have been extracted using the time-series feature extraction library (TSFEL). Feature selection or reduction methods applied are feature scaling, removal of highly correlated and low variance features, and random forest (RF) recursive feature elimination. The methodological novelty of this study is mainly the incorporation of TSFEL during feature extractions, synthetic minority oversampling technique to create a balanced dataset, and an ensemble of RF and support vector machine (SVM) using weighted majority algorithm for heartbeat classification to improve the results. Grid search has been performed to optimize the hyperparameters of RF and SVM classifiers. A final evaluation has been carried out considering a “subject-specific” scheme. The sensitivity that our approach has achieved for the arrhythmic heartbeat classes is as follows: N: 99.50%, SVEB (S): 74.20%, VEB (V): 94.22%, F: 73.21%, and Q: 0%. The corresponding positive predictive values are N: 98.67%, SVEB (S): 90.09%, VEB (V): 95.95%, F: 88.35%, and Q: 0%. In comparison with machine learning and deep learning based state-of-the-art approaches, significant improvement in efficiency has been found. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement—</i>Automated classifications of heartbeats from ECG signals are very much required to speed up the diagnosis of cardiovascular diseases (CVDs). Our study demonstrates 98.21% accuracy compared to existing state-of-the-art approaches. This study impacts the intensive health care system by offering an easy, quick, and cost-effective diagnosis of CVDs. Besides, remote patients can also receive the treatments timely. Early intervention and prevention of CVDs can prevent almost 30% of the total death every year. The proposed approach has a societal and economic impact too, as the health, welfare, and productivity of the citizens as well as of the country will be improved significantly. Approximately 2% of gross domestic product (GDP) will be improved across low- and middle-income countries as the proposed approaches will reduce the direct (treatment cost) and indirect costs (loss in productivities at workplaces) associated with CVDs.

DCA for online prediction with expert advice

We investigate DC (Difference of Convex functions) programming and DCA (DC Algorithm) for a class of online learning techniques, namely prediction with expert advice, where the learner’s prediction is made based on the weighted average of experts’ predictions. The problem of predicting the experts’ weights is formulated as a DC program for which an online version of DCA is investigated. The two so-called approximate/complete variants of online DCA based schemes are designed, and their regrets are proved to be logarithmic/sublinear. The four proposed algorithms for online prediction with expert advice are furthermore applied to online binary classification. Experimental results tested on various benchmark datasets showed their performance and their superiority over three standard online prediction with expert advice algorithms—the well-known weighted majority algorithm and two online convex optimization algorithms.

A heterogeneous online learning ensemble for non-stationary environments

Learning in non-stationary environments is a challenging task which requires the updating of predictive models to deal with changes in the underlying probability distribution of the problem, i.e., dealing with concept drift. Most work in this area is concerned with updating the learning system so that it can quickly recover from concept drift, while little work has been dedicated to investigating what type of predictive model is most suitable at any given time. This paper aims to investigate the benefits of online model selection for predictive modelling in non-stationary environments. A novel heterogeneous ensemble approach is proposed to intelligently switch between different types of base models in an ensemble to increase the predictive performance of online learning in non-stationary environments. This approach is Heterogeneous Dynamic Weighted Majority (HDWM). It makes use of “seed” learners of different types to maintain ensemble diversity, overcoming problems of existing dynamic ensembles that may undergo loss of diversity due to the exclusion of base learners. The algorithm has been evaluated on artificial and real-world data streams against existing well-known approaches such as a heterogeneous Weighted Majority Algorithm (WMA) and a homogeneous Dynamic Weighted Majority (DWM). The results show that HDWM performed significantly better than WMA in non-stationary environments. Also, when recurring concept drifts were present, the predictive performance of HDWM showed an improvement over DWM.

A fully distributed algorithm for user-base station association in HetNets

The dense deployment of small-cell base stations in heterogeneous networks (HetNets) requires efficient resource allocation and interference management techniques. Especially, the problem of associating users to base stations (BSs) and allocating frequency channels must be revised and carefully studied. Finding the optimal solution of such problem is NP-hard. Further, it requires huge amount of information exchange between the BSs. In order to efficiently solve this problem in a distributed manner, we model it using non-cooperative game theory. The proposed game model is proved to not always admit pure Nash equilibria (PNEs), even though simulations show that, for slow fading channels, a PNE exists for most instances of the game. It is shown that, when the game admits PNEs, its prices of anarchy and stability are close to one. By modifying the players’ actions set and hence obtaining a new game model, we guarantee the existence of PNEs at the expense of performance degradation. Next, a fully distributed algorithm, based on a learning mechanism, is proposed. It requires no communication between the BSs and needs only one bit of feedback. Simulations show that the fully distributed algorithm has tight-to-optimal performance and solves efficiently the trade-off between complexity, information exchange and performance. We benchmark the proposed algorithm against the centralized optimal algorithm, the maximum signal to interference-plus-noise ratio algorithm, the best response dynamics algorithm and the randomized weighted majority algorithm.

Cascading randomized weighted majority: A new online ensemble learning algorithm

With the increasing volume of data in the world, the best approach for learning from this data is to exploit an online learning algorithm. Online ensemble methods are online algorithms which take advantage of an ensemble of classifiers to predict labels of data. Prediction with expert advice is a well-studied problem in the online ensemble learning literature. The Weighted Majority algorithm and the randomized weighted majority (RWM) are the most well-known solutions to this problem, aiming to converge to the best expert. Since among some expert, The best one does not necessarily have the minimum error in all regions of data space, defining specific regions and converging to the best expert in each of these regions will lead to a better result. In this paper, we aim to resolve this defect of RWM algorithms by proposing a novel online ensemble algorithm to the problem of prediction with expert advice. We propose a cascading version of RWM to achieve not only better experimental results but also a better error bound for sufficiently large datasets.

On Solving the Problem of Identifying Unreliable Sensors Without a Knowledge of the Ground Truth: The Case of Stochastic Environments.

The purpose of this paper is to propose a solution to an extremely pertinent problem, namely, that of identifying unreliable sensors (in a domain of reliable and unreliable ones) without any knowledge of the ground truth. This fascinating paradox can be formulated in simple terms as trying to identify stochastic liars without any additional information about the truth. Though apparently impossible, we will show that it is feasible to solve the problem, a claim that is counter-intuitive in and of itself. One aspect of our contribution is to show how redundancy can be introduced, and how it can be effectively utilized in resolving this paradox. Legacy work and the reported literature (for example, in the so-called weighted majority algorithm) have merely addressed assessing the reliability of a sensor by comparing its reading to the ground truth either in an online or an offline manner. Unfortunately, the fundamental assumption of revealing the ground truth cannot be always guaranteed (or even expected) in many real life scenarios. While some extensions of the Condorcet jury theorem [9] can lead to a probabilistic guarantee on the quality of the fused process, they do not provide a solution to the unreliable sensor identification problem. The essence of our approach involves studying the agreement of each sensor with the rest of the sensors, and not comparing the reading of the individual sensors with the ground truth-as advocated in the literature. Under some mild conditions on the reliability of the sensors, we can prove that we can, indeed, filter out the unreliable ones. Our approach leverages the power of the theory of learning automata (LA) so as to gradually learn the identity of the reliable and unreliable sensors. To achieve this, we resort to a team of LA, where a distinct automaton is associated with each sensor. The solution provided here has been subjected to rigorous experimental tests, and the results presented are, in our opinion, both novel and conclusive.

A novel SVM-kNN-PSO ensemble method for intrusion detection system

In machine learning, a combination of classifiers, known as an ensemble classifier, often outperforms individual ones. While many ensemble approaches exist, it remains, however, a difficult task to find a suitable ensemble configuration for a particular dataset. This paper proposes a novel ensemble construction method that uses PSO generated weights to create ensemble of classifiers with better accuracy for intrusion detection. Local unimodal sampling (LUS) method is used as a meta-optimizer to find better behavioral parameters for PSO. For our empirical study, we took five random subsets from the well-known KDD99 dataset. Ensemble classifiers are created using the new approaches as well as the weighted majority algorithm (WMA) approach. Our experimental results suggest that the new approach can generate ensembles that outperform WMA in terms of classification accuracy.

Qualitative trust model with a configurable method to aggregate ordinal data

Trust models are mechanisms that allow agents to build trust without relying on a trusted central authority. Our goal was to develop a trust model that would operate with values that humans easily understand and manipulate: qualitative and ordinal values. The result is a trust model that computes trust from experiences created in interactions and from opinions obtained from third-party agents. The trust model, termed qualitative trust model (QTM), uses qualitative and ordinal values for assessing experiences, expressing opinions and estimating trust. We treat such values appropriately; we never convert them to numbers, but merely use their relative order. To aggregate a collection of such values, we propose an aggregation method that is based on comparing distributions and show some of its properties; the method can be used in other domains and can be seen as an alternative to median and similar methods. To cope with lying agents, QTM estimates trustworthiness in opinion providers with a modified version of the weighted majority algorithm, and additionally combines trustworthiness with social links between agents; such links are obtained implicitly by observing how agents provide opinions about each other. Finally, we compare QTM against a set of well-known trust models and demonstrate that it consistently performs well and on par with other quantitative models, and in many cases even outperforms them, particularly when the number of direct experiences is low.

Committee, Expert Advice, and the Weighted Majority Algorithm: An Application to the Pricing Decision of a Monopolist

We consider a repeated pricing decision problem of a monopolist (the decision-maker) who does not know the demand function of some new product, and hence the profit function. To decide, she is helped by a committee of N experts. Each expert has an estimation of the unknown demand function and use it to advise the decision-maker on how she should modify the current price. Decisions are taken with a weighted majority rule, where the weight of each expert, which may be interpreted as her decision power, evolves as a function of its accuracy. When a perfect exists, i.e., who always gives the correct advice, we show that she ends up with all the decision power in the long-run so that the decision-maker finds the optimal price. When such a perfect does not exist, the decision-maker is actually unable to consistently select an expert over time so that the sequences of prices and weights describe a limit cycle. Interestingly enough, if the decision-maker takes a large sample of the stationary behavior of prices, the empirical mean turns out to be arbitrarily close to the optimal price, independently of the "quality" of the experts, as long as there experts are "diverse" enough. This result gives thus support to the thesis developed in the books of Surowiecki (The wisdom of the crowd. Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations, 2005) and Page (The difference: How the power of diversity creates better groups, firms, schools, and societies, 2007).

A Trustworthiness Based Ad-Hoc Secure Interoperation Method

A trustworthiness-based ad-hoc secure interoperation method is proposed, in which the concept of trustworthiness is introduced to describe the probability of proper collaboration. The trustworthiness of an autonomic domain on a user is decided jointly by direct experiences of interactions and other domains' assessments on the user. Only the users who satisfy the requirements of target domains' trust policies have the privileges to access entry roles. Records of a user's malicious actions will decrease his trustworthiness and then accordingly reduce his privileges. Target domain uses weighted majority algorithm to update recommender's trustworthiness, which is reduced by unfair ratings. Experimental results show that this method can effectively resist cheating and malicious actions.

DNA-based algorithms for learning Boolean formulae

We apply a DNA-based massively parallel exhaustive search to solving the computational learning problems of DNF (disjunctive normal form) Boolean formulae. Learning DNF formulae from examples is one of the most important open problems in computational learning theory and the problem of learning 3-term DNF formulae is known as intractable if RP ≠ NP. We propose new methods to encode any k-term DNF formula to a DNA strand, evaluate the encoded DNF formula for a truth-value assignment by using hybridization and primer extension with DNA polymerase, and find a consistent DNF formula with the given examples. By employing these methods, we show that the class of k-term DNF formulae (for any constant k) and the class of general DNF formulae are efficiently learnable on DNA computer. Second, in order for the DNA-based learning algorithm to be robust for errors in the data, we implement the weighted majority algorithm on DNA computers, called DNA-based majority algorithm via amplification (DNAMA), which take a strategy of ``amplifying'' the consistent (correct) DNA strands. We show a theoretical analysis for the mistake bound of the DNA-based majority algorithm via amplification, and imply that the amplification to ``double the volumes'' of the correct DNA strands in the test tube works well.

Worst-case analysis of the perception and exponentiated update algorithms

The absolute loss is the absolute difference between the desired and predicted outcome. This paper demonstrates worst-case upper bounds on the absolute loss for the Perception learning algorithm and the Exponentiated Update learning algorithm, which is related to the Weighted Majority algorithm. The bounds characterize the behavior of the algorithms over any sequence of trials, where each trial consists of an example and a desired outcome interval (any value in the interval is an acceptable outcome). The worst-case absolute loss of both algorithms is bounded by: the absolute loss of the best linear function in a comparison class, plus a constant dependent on the initial weight vector, plus a per-trial loss. The per-trial loss can be eliminated if the learning algorithm is allowed a tolerance from the desired outcome. For concept learning, the worst-case bounds lead to mistake bounds that are comparable to past results.

The Weighted Majority Algorithm

We study the construction of prediction algorithms in a situation in which a learner faces a sequence of trials, with a prediction to be made in each, and the goal of the learner is to make few mistakes. We are interested in the case where the learner has reason to believe that one of some pool of known algorithms will perform well, but the learner does not know which one. A simple and effective method, based on weighted voting, is introduced for constructing a compound algorithm in such a circumstance. We call this method the Weighted Majority Algorithm. We show that this algorithm is robust in the presence of errors in the data. We discuss various versions of the Weighted Majority Algorithm and prove mistake bounds for them that are closely related to the mistake bounds of the best algorithms of the pool. For example, given a sequence of trials, if there is an algorithm in the pool A that makes at most m mistakes then the Weighted Majority Algorithm will make at most c (log | A | + m ) mistakes on that sequence, where c is fixed constant.