Passive-Aggressive Algorithm Research Articles

Online imbalance learning with unpredictable feature evolution and label scarcity

Recently, online learning with imbalanced data streams has aroused wide concern, which reflects an uneven distribution of different classes in data streams. Existing approaches have conventionally been conducted on stationary feature space and they assume that we can obtain the entire labels of data streams in the case of supervised learning. However, in many real scenarios, e.g., the environment monitoring task, new features flood in, and old features are partially lost during the changing environment as the different lifespans of different sensors. Besides, each instance needs to be labeled by experts, resulting in expensive costs and scarcity of labels. To address the above problems, this paper proposes a novel Online Imbalance learning with unpredictable Feature evolution and Label scarcity (OIFL) algorithm. First, we utilize margin-based online active learning to selectively label valuable instances. After obtaining the labels, we handle imbalanced class distribution by optimizing F-measure and transforming F-measure optimization into a weighted surrogate loss minimization. When data streams arrive with augmented features, we combine the online passive-aggressive algorithm and structural risk minimization to update the classifier in the divided feature space. When data streams arrive with incomplete features, we leverage variance to identify the most informative features following the empirical risk minimization principle and continue to update the existing classifier as before. Finally, we obtain a sparse but reliable learner by the strategy of projecting truncation. We derive theoretical analyses of OIFL. Also, experiments on the synthetic datasets and real-world data streams to validate the effectiveness of our method.

Research on Digital Business English Teaching Mode under the Background of Big Data Technology

Abstract This paper proposes an automatic grammar error correction model based on an encoder-decoder structure, which is combined with a new business English teaching model that can improve the teaching quality of teachers and enable students to get feedback quickly. The model employs a dual encoder structure and extracts source sentence information through an encoder that is based on the attention mechanism. Meanwhile, a fluency-based data augmentation method is also proposed in order to be able to select sentences with different fluency levels based on the fluency ratio between the target sentence and the source sentence. Finally, a Passive-Aggressive algorithm for candidate word selection has been designed, which can avoid generating repetitive and short-biased English sentences. It has been verified that the total grade of the class that incorporates this paper’s model in business English teaching has increased by 9.3 points on average, with the most obvious improvement in business writing by 3 points, followed by translation by 2.8 points. The proposed teaching model in this paper can enhance students’ comprehensive use of business English.

An Intelligent Fusion Model with Portfolio Selection and Machine Learning for Stock Market Prediction.

Developing reliable equity market models allows investors to make more informed decisions. A trading model can reduce the risks associated with investment and allow traders to choose the best-paying stocks. However, stock market analysis is complicated with batch processing techniques since stock prices are highly correlated. In recent years, advances in machine learning have given us a lot of chances to use forecasting theory and risk optimization together. The study postulates a unique two-stage framework. First, the mean-variance approach is utilized to select probable stocks (portfolio construction), thereby minimizing investment risk. Second, we present an online machine learning technique, a combination of “perceptron” and “passive-aggressive algorithm,” to predict future stock price movements for the upcoming period. We have calculated the classification reports, AUC score, accuracy, and Hamming loss for the proposed framework in the real-world datasets of 20 health sector indices for four different geographical reasons for the performance evaluation. Lastly, we conduct a numerical comparison of our method's outcomes to those generated via conventional solutions by previous studies. Our aftermath reveals that learning-based ensemble strategies with portfolio selection are effective in comparison.

Real and Fake News Detection Smart System Using Passive Aggressive Algorithm (Supervised Machine Learning)

Today where the internet is ubiquitous, everyone intake news from various online platforms . Along with the increase in the use of social media platforms like Facebook, Twitter, etc. News spread rapidly among millions of users within a very short spam of time. The spread of fake news has far-reaching consequences like the creation of biased opinions to swaying election outcomes for the benefit of certain candidates. Moreover, spammers use appealing news headlines to generate revenue using advertisements via clickbaits. In this paper, we aim to perform binary classification of various news articles available online with the help of concepts pertaining to Artificial Intelligence, Natural Language Processing and Machine Learning(Supervised ML). Our aim is to provide the news is authentic or fake.

Adaptive one-pass passive-aggressive radial basis function for classification problems

This paper presents a novel adaptive one-pass Passive-Aggressive Radial Basis Function (APARBF) for classification problems. The APARBF uses elliptic Gaussian neurons followed by a Softmax layer and cross-entropy loss function. This network tries to overcome the elasticity-plasticity dilemma by using the Passive-Aggressive (PA) algorithm and adapting the hidden layer structure. The weight updates have to be plastic to acquire the most information from each sample and at the same time need to be elastic to retrain the information from the past instances. Inspired by PA, a novel update formula for cross-entropy loss minimization has been derived. The adaptive design of the network lets it start with zero hidden neurons and grow or shrink according to the data. For kernel parameters, the adaptive structure determines the correct number of hidden neurons and updates recursively each neuron’s center and covariance matrix. To evaluate our network, we perform two series of experiments. The first experiments compare the proposed APARBF with other recently developed one-pass algorithms (i.e., OVIG, OBHT, SCW, AROW, OGD, and PA). The subsequent experiments include comparing the proposed algorithm with some online adaptive structures such as FGAP-RBF, C-Mantec, McNN, and PBL-McNN, based on their mean classification error and the number of hidden neurons. Wilcoxon sign rank test and Friedman test clearly show the superiority of the proposed network’s results compared with its competitors in one-pass classification problems.

A Taxonomy of Fake News Classification Techniques: Survey and Implementation Aspects

In the present era, social media platforms such as Facebook, WhatsApp, Twitter, and Telegram are significant sources of information distribution, and people believe it without knowing their origin and genuineness. Social media has fascinated people worldwide in spreading fake news due to its easy availability, cost-effectiveness, and ease of information sharing. Fake news can be generated to mislead the community for personal or commercial gains. It can also be used for other personal benefits such as defaming eminent personalities, amendment of government policies, etc. Thus, to mitigate the awful consequences of fake news, several research types have been conducted for its detection with high accuracy to prevent its fatal outcome. Motivated by the aforementioned concerns, we present a comprehensive survey of the existing fake news identification techniques in this paper. Then, we select Machine Learning (ML) models such as Long-Short Term Memory (LSTM), Passive Aggressive Algorithm, Random Forest (RF), and Naive Bayes (NB) and train them to detect fake news articles on the self-aggregated dataset. Later, we implemented these models by hyper tuning various parameters such as smoothing, drop out factor, and batch size, which has shown promising results in accuracy and other evaluation metrics such as F1-score, recall, precision, and Area under the ROC Curve (AUC) score. The model is trained on 6335 news articles, with LSTM showing the highest accuracy of 92.34% in predicting fake news and NB were showing the highest recall. Based on these results, we propose a hybrid fake news detection technique using NB and LSTM. At last, challenges and open issues along with future research directions are discussed to facilitate the research in this domain further.

Performance Comparison of Machine Learning Algorithms for Albanian News articles

This paper presented a methodology to classify news articles in Albanian languages with machine learning algorithms and NLP. In this proposed work Multinomial Naïve Bayes, Logistic Regression, k-nearest neighbors, Bernoulli, Centroid, SVM, SGD, Perceptron, Passive Aggressive, Decision Tree, and Random Forest machine learning Classification algorithms are trained to classify Albanian news articles and compared in term of accuracy, training time and testing time. We have used 70% of the dataset for training and 30% for tests. The execution time of training and testing algorithms is recorded and presented for different input sizes of data. For experimental we used different numbers of inputs, starting from 8000 articles, and continuing in increasing order by 8000 until the level of 80000 in 8 categories. Experimental results indicate that the Passive Aggressive algorithm shows the best performance in terms of accuracy for Albanian news articles. While the classifier with the lowest accuracy was Random Forest. Logistic Regression, SVM, and Decision Tree have a high delay in training data. SVM also has the longest testing time compared to other classifiers.

RT to Win! Predicting Message Propagation in Twitter

Twitter is a very popular way for people to share information on a bewildering multitude of topics. Tweets are propagated using a variety of channels: by following users or lists, by searching or by retweeting. Of these vectors, retweeting is arguably the most effective, as it can potentially reach the most people, given its viral nature. A key task is predicting if a tweet will be retweeted, and solving this problem furthers our understanding of message propagation within large user communities. We carry out a human experiment on the task of deciding whether a tweet will be retweeted which shows that the task is possible, as human performance levels are much above chance. Using a machine learning approach based on the passive-aggressive algorithm, we are able to automatically predict retweets as well as humans. Analyzing the learned model, we find that performance is dominated by social features, but that tweet features add a substantial boost.

An Efficient Spam SMS Analysis Model based on Multinomial Naïve Bayes model Using Passive Aggressive Algorithm

The social media can be a platform for information consumption nowadays. On the one hand, it’s free of cost, easy access, and different data dissemination lead people to hunt out and consume social media news. On the contrary, it allows for the broad spread of “spams,” i.e., inferiority news with deliberately false information. The widespread spread of spams has the potential for very negative impacts on people and society. Consequently, the detection of spam on social media has recently become an important research that draws tremendous attention. NLP, an artificial intelligence (AI) division, uses computers and human natural language to produce useful data. In text classification activities, such as spam detection and sentiment analysis, text generation, language translations and document classification, NLP is widely used.

Online Learning and Active Learning: A Comparative Study of Passive-Aggressive Algorithm With Support Vector Machine (SVM)

Passive aggressive online learning is an extension of Support Vector Machine (SVM) to the context of online learning for binary classification. In this paper we consider the application of the algorithm on anomaly labeling for IJCNN 2001 Neural Network Competition dataset from LibSVM dataset repository1 from Ford Research Laboratory. We also work on an improved version of the online learning algorithm called Active learning and we compare both algorithms to that of SVM (from LibSVM library). We propose different experimental setups for comparing the algorithms.

PA-PseU: An incremental passive-aggressive based method for identifying RNA pseudouridine sites via Chou’s 5-steps rule

Pseudouridine (Ψ) is one of the most abundant RNA modifications existing in ubiquitous organisms and participates a bunch of biological processes. Identification of pseudouridine sites has significant meanings for the study of biological process and drug developments related to RNA. Wet experiments for pseudouridine sites are expensive, time-consuming and easily influenced by the environment although they can detect Ψ sites in whole transcriptome. While some computational methods have been presented, their performances are still unsatisfactory and their computation costs are usually large. In this article, we propose an incremental identification method called PA-PseU which is based on Passive-Aggressive algorithm as Ψ sites classifier. The combination of chi-square test and logistic regression is used to select optimal feature subsets. Its effectiveness has been demonstrated via 10-fold cross-validation, jack-knife test and independent test. The average leave-one-out accuracies of PA-PseU among different species are respectively 87.80% in cross validation sets and 86.70% in independent testing sets, which improve 16.40% and 12.00% respectively, contrasting with 71.40% and 74.70% of the best previous predictor RF-PseU. All results demonstrate that our model outperforms the state-of-art methods by a big margin and improves the previous models substantially. Moreover, PA-PseU is an incremental method which takes low computation cost with rapid computation speed. PA-PseU can be a useful tool for identifying the Ψ sites in transcriptome analysis. All the data, code and related materials are accessible at https://github.com/Jensen-Wang/PA-PseU.

Anomaly Detection using Machine Learning Techniques in Wireless Sensor Networks

The number of Wireless sensor network (WSN) deployments have been growing so exponentially over the recent years. Due to their small size and cost-effective, WSN are attracting many industries to use them in various applications. Environmental monitoring, security of buildings and precision agriculture are few example among several other fields. However, WSN faces high security threats considering most of them are deployed in unattended nature and hostile environment. In the aim of providing secure data processing in the WSN, many techniques are proposed to protect the data privacy while being transferred from the sensors to the base station. This work is focusing on attack detection which is an essential task to secure the network and the data. Anomaly detection is a key challenge in order to ensure the security and prevent malicious attacks in wireless sensor networks. Various machine learning techniques have been used by researchers these days to detect anomalies using offline learning algorithms. On the other hand online learning classifiers have not been thoroughly addressed in the literature. Our aim is to provide an intrusion detection model compatible with the characteristics of WSN. This model is built based on information gain ratio and the online Passive aggressive classifier. Firstly, the information gain ratio is used to select the relevant features of the sensor data. Secondly, the online Passive aggressive algorithm is trained to detect and classify different type of Deny of Service attacks. The experiment was conducted on a wireless sensor network-detection system (WSN-DS) dataset. The proposed model ID-GOPA results detection rate of 96% determining whether the network is in its normal mode or exposed to any type of attack. The detection accuracy is 86%, 68%, 63%, and 46% for scheduling, grayhole, flooding and blackhole attacks, respectively, in addition to 99% for normal traffic. These results shows that our model based on offline learning can be providing good anomaly detection to the WSN and replace online learning in some cases.

A Novel Online Portfolio Selection Strategy with Multiperiodical Asymmetric Mean Reversion

Mean reversion is an important property when constructing efficient contrarian strategies. Researchers observe that mean reversion has multiperiodical and asymmetric nature simultaneously in real market. To better utilize mean reversion and improve the existing online portfolio selection strategies, we propose a new online strategy named multiperiodical asymmetric mean reversion (MAMR). The MAMR strategy incorporates a multipiecewise loss function with the moving average method and then imitates the passive-aggressive algorithm. We further provide a solution via convex optimization. This strategy runs in linear time and thus is suitable for large-scale trading applications. Our empirical results testing six real market datasets show that this strategy can achieve better results in bearing higher transaction cost.

Budgeted Passive-Aggressive Learning for Online Multiclass Classification

Online multiclass classification is a specific problem of online learning that performs a sequence of multiclass classification tasks given the knowledge of previous tasks. The goal is to make correct predictions for this sequence. It is generally considered a more complicated problem than its binary counterpart, online binary classification. A popular algorithm, called the passive-aggressive algorithm, was primarily proposed for binary problems and later extended as the multiclass passive-aggressive (MPA) algorithm for multiclass problems. The nature of MPA allows itself to implement the kernel trick, which enables us to make better predictions with a kernel-based model. However, this approach suffers from the curse of kernelization that causes unbounded growth of the model in memory usage and runtime. To solve the growth problem, we first introduce a resource perspective that gives an alternative and equivalent interpretation of the kernel-based MPA algorithm. Based on the resource perspective, we propose the budgeted MPA (BMPA) algorithm, which approximates the kernel-based MPA algorithm. BMPA limits the maximum number of available resources by removal and fully exploits them through a constrained optimization. We study three removal strategies and give a relative mistake bound that provides a unified analysis. Simulation experiments on various datasets are conducted to demonstrate that BMPA is effective and competitive with state-of-the-art budgeted online algorithms.

Online learning using projections onto shrinkage closed balls for adaptive brain-computer interface

Wearable/portable brain-computer interfaces (BCIs) for the long-term end use are a focus of recent BCI research. A challenge is how to update the BCI to meet changes in electroencephalography (EEG) signals, since the resource are so limited that retraining of traditional well-performed models, such as a support vector machine, is nearly impossible. To cope with this challenge, less-demanding adaptive online learning can be considered. We investigated an adaptive projected sub-gradient method (APSM) that is originated from the set theoretic estimation formulation and the projections onto convex sets theory. APSM provides a unifying framework for both adaptive classification and regression tasks. Coefficients of APSM are adjusted online as data arrive sequentially, with a regularization constraint made by projections onto a fixed closed ball. We extended the general APSM to a shrinkage form, where shrinkage closed balls were used instead of the original fixed one, expecting a more controllable fading effect and better adaptability. The convergence of shrinkage APSM was proved. It was also demonstrated that as shrinkage factor approached to 1, the limit point of shrinkage APSM would approach to the optimal solution with the least norm, which could be especially beneficial for generalization of the classifier. The performance of the proposed method was evaluated, and compared with those of the general APSM, the incremental support vector machine, and the passive aggressive algorithm, through an event-related potential-based BCI experiment. Results showed the advantage of the proposed method over the others on both the online classification performance and the easiness of tuning. Our study revealed the effectiveness of the proposed method for adaptive EEG classification, making it a promising tool for on-device training and updating of wearable/portable BCIs, as well as for application in other related fields, such as EEG-based biometrics.

Online Feature Selection of Class Imbalance via PA Algorithm

Imbalance classification techniques have been frequently applied in many machine learning application domains where the number of the majority (or positive) class of a dataset is much larger than that of the minority (or negative) class. Meanwhile, feature selection (FS) is one of the key techniques for the high-dimensional classification task in a manner which greatly improves the classification performance and the computational efficiency. However, most studies of feature selection and imbalance classification are restricted to off-line batch learning, which is not well adapted to some practical scenarios. In this paper, we aim to solve high-dimensional imbalanced classification problem accurately and efficiently with only a small number of active features in an online fashion, and we propose two novel online learning algorithms for this purpose. In our approach, a classifier which involves only a small and fixed number of features is constructed to classify a sequence of imbalanced data received in an online manner. We formulate the construction of such online learner into an optimization problem and use an iterative approach to solve the problem based on the passive-aggressive (PA) algorithm as well as a truncated gradient (TG) method. We evaluate the performance of the proposed algorithms based on several real-world datasets, and our experimental results have demonstrated the effectiveness of the proposed algorithms in comparison with the baselines.

Online Passive-Aggressive Active learning

We investigate online active learning techniques for online classification tasks. Unlike traditional supervised learning approaches, either batch or online learning, which often require to request class labels of each incoming instance, online active learning queries only a subset of informative incoming instances to update the classification model, aiming to maximize classification performance with minimal human labelling effort during the entire online learning task. In this paper, we present a new family of online active learning algorithms called Passive-Aggressive Active (PAA) learning algorithms by adapting the Passive-Aggressive algorithms in online active learning settings. Unlike conventional Perceptron-based approaches that employ only the misclassified instances for updating the model, the proposed PAA learning algorithms not only use the misclassified instances to update the classifier, but also exploit correctly classified examples with low prediction confidence. Specifically, we propose several variants of PAA algorithms to tackle three types of online learning tasks: binary classification, multi-class classification, and cost-sensitive classification. We give the mistake bounds of the proposed algorithms in theory, and conduct extensive experiments to evaluate the empirical performance of our techniques on both standard and large-scale datasets, in which the encouraging results validate the empirical effectiveness of the proposed algorithms.

Robust online algorithm for adaptive linear regression parameter estimation and prediction

In this article, we focus on adaptive linear regression methods and propose a new technique. The article begins with a review of the online passive aggressive algorithm (OPAA), an adaptive linear regression algorithm from the machine learning literature. We highlight the strengths and weaknesses of OPAA and compare it with other popular adaptive regression techniques such as moving window and recursive least squares, recursive partial least squares, and just‐in‐time or locally weighted regression. Modifications to OPAA are proposed to make it more robust and better suited for industrial soft‐sensor applications. The new algorithm is called smoothed passive aggressive algorithm (SPAA), and like OPAA, it follows a cautious parameter update strategy but is more robust. The trade‐off between SPAA's computation complexity and accuracy can be easily controlled by manipulating just two tuning parameters. We also demonstrate that the SPAA framework is quite flexible and a number of variants are easily formulated. Application of SPAA to estimate the time‐varying parameters of a numerically simulated autoregressive with exogenous terms (ARX) model and to predict the Reid vapor pressure of the bottoms flow from an industrial column demonstrates its superior performance over OPAA and comparable performance with the other popular algorithms. Copyright © 2016 John Wiley & Sons, Ltd.

One-pass online learning: A local approach

Online learning is very important for processing sequential data and helps alleviate the computation burden on large scale data as well. Especially, one-pass online learning is to predict a new coming sample׳s label and update the model based on the prediction, where each coming sample is used only once and never stored. So far, existing one-pass online learning methods are globally modeled and do not take the local structure of the data distribution into consideration, which is a significant factor of handling the nonlinear data separation case. In this work, we propose a local online learning (LOL) method, a multiple hyperplane Passive Aggressive algorithm integrated with online clustering, so that all local hyperplanes are learned jointly and working cooperatively. This is achieved by formulating a common component as information traffic among multiple hyperplanes in LOL. A joint optimization algorithm is proposed and theoretical analysis on the cumulative error is also provided. Extensive experiments on 11 datasets show that LOL can learn a nonlinear decision boundary, overall achieving notably better performance without using any kernel modeling and second order modeling.

A Self Training Approach to Automatic Modulation Classification Based on Semi-supervised Online Passive Aggressive Algorithm

Automatic classification of modulation type in detected signals is an intermediate step between channel equalization and signal demodulation. This subsystem is an essential task for an intelligent receiver in various civil and military applications. Most of automatic modulation classification algorithms have been evaluated in SNR aware case which is obviously an unrealistic case. In this paper, we propose a semi-supervised online passive-aggressive classifier that uses a self-training approach for additive white Gaussian noise channels with unknown or variable signal to noise ratios to classify the modulated signals. An appropriate set of selected feature helps the general system work for a set of initial samples of each class. In the adaptation phase, unlabeled high confidence samples are used to adapt to system. An appropriate confidence measure is proposed to collect confident samples. Simulation results shows that adding unlabeled input samples to the training set improve the generalization capacity of the presented classifier in the target SNR. In addition to online characteristics of the algorithm that make the approach suitable for cognitive radio systems, this algorithm converges with a few number of signal samples. Simulation results in SNR unaware conditions show that employing this learning method results in high classification rate close to SNR aware conditions.