Unbalanced Binary Classification Research Articles

High-Dimensional Unbalanced Binary Classification by Genetic Programming with Multi-Criterion Fitness Evaluation and Selection.

High-dimensional unbalanced classification is challenging because of the joint effects of high dimensionality and class imbalance. Genetic programming (GP) has the potential benefits for use in high-dimensional classification due to its built-in capability to select informative features. However, once data are not evenly distributed, GP tends to develop biased classifiers which achieve a high accuracy on the majority class but a low accuracy on the minority class. Unfortunately, the minority class is often at least as important as the majority class. It is of importance to investigate how GP can be effectively utilized for high-dimensional unbalanced classification. In this article, to address the performance bias issue of GP, a new two-criterion fitness function is developed, which considers two criteria, that is, the approximation of area under the curve (AUC) and the classification clarity (i.e., how well a program can separate two classes). The obtained values on the two criteria are combined in pairs, instead of summing them together. Furthermore, this article designs a three-criterion tournament selection to effectively identify and select good programs to be used by genetic operators for generating offspring during the evolutionary learning process. The experimental results show that the proposed method achieves better classification performance than other compared methods.

Semantics in Multi-objective Genetic Programming

Semantics has become a key topic of research in Genetic Programming (GP). Semantics refers to the outputs (behaviour) of a GP individual when this is run on a dataset. The majority of works that focus on semantic diversity in single-objective GP indicates that it is highly beneficial in evolutionary search. Surprisingly, there is minuscule research conducted in semantics in Multi-objective GP (MOGP). In this work we make a leap beyond our understanding of semantics in MOGP and propose SDO: Semantic-based Distance as an additional criteriOn. This naturally encourages semantic diversity in MOGP. To do so, we find a pivot in the less dense region of the first Pareto front (most promising front). This is then used to compute a distance between the pivot and every individual in the population. The resulting distance is then used as an additional criterion to be optimised to favour semantic diversity. We also use two other semantic-based methods as baselines, called Semantic Similarity-based Crossover and Semantic-based Crowding Distance. Furthermore, we also use the Non-dominated Sorting Genetic Algorithm II and the Strength Pareto Evolutionary Algorithm 2 for comparison too. We use highly unbalanced binary classification problems and consistently show how our proposed SDO approach produces more non-dominated solutions and better diversity, leading to better statistically significant results, using the hypervolume results as evaluation measure, compared to the rest of the other four methods.

EPX: An R package for the ensemble of subsets of variables for highly unbalanced binary classification.

In binary classification problems with a rare class of interest, there is relatively little information available for the rare class to build a model. On the other hand, the number of useful variables to develop a model for classification can be high-dimensional. For example, in drug discovery, there are usually a very few bioactive compounds in a large chemical library, whereas thousands of potentially useful explanatory variables characterize a compound's chemical structure. The sparsity of information for the rare class of interest makes it difficult for the standard classification models to exploit the richness of the useful feature variables. Thus, the objective of this paper is to develop an R package which clusters the feature variables into diverse subsets to be aggregated into a powerful ensemble for the detection of a rare class object. The ensemble of phalanxes (EPX) builds a classifier by exploiting the richness of feature variables using several diverse subsets of variables, called phalanxes, and outperforms many competitive state-of-the-art classification methods in terms of predictive ranking of the rare class of interest. We present an R package EPX which implements the algorithm to form the ensemble of phalanxes as well as its associated functions. We further show how the ensemble of phalanxes can be constructed using parallel computing to lower the computational burden given high-dimensional data. The R package EPX shows a flexible way of clustering feature variable space into smaller and diverse subsets of variables to develop an ensemble of phalanxes which better ranks a rare class object in a highly unbalanced two class classification problem. The ensemble EPX will be useful to detect the rare drug-like active biomolecules for development in drug discovery (Tomal et al., Mar. 2016) [1] and homologous proteins using similarity scores of amino acid sequences in protein homology (Tomal et al., 2019) [2]. The package EPX is freely available to download from CRAN (https://CRAN.R-project.org/package=EPX).

Learning Structurally Incoherent Background and Target Dictionaries for Hyperspectral Target Detection

Existing sparsity-based hyperspectral image (HSI) target detection methods have two key problems. 1) The background dictionary is locally constructed by the pixels between the inner and outer windows, surrounding and enclosing the central test pixel. The dual-window strategy is intricate and might result in impure background dictionary deteriorating the detection performance. 2) For an unbalanced binary classification problem, the target dictionary atoms are generally inadequate compared with the background dictionary, which might yield unstable performance. For the issues, this article proposes a novel structurally incoherent background and target dictionaries (SIBTD) learning model for HSI target detection. Specifically, with the concept that the observed HSI data is composed of low-rank background, sparsely distributed targets, and dense Gaussian noise, the background and target dictionaries can be jointly derived from the observed HSI data. Additionally, the introduction of structural incoherence can enhances the discrimination between the target and background dictionaries. Thus, the developed model can not only lead to a pure and unified background dictionary but also augment the target dictionary for improved detection performance. Besides, an efficient optimization algorithm is devised to solve SIBTD model, and the performance of SIBTD is verified on three benchmark HSI datasets in comparison with several state-of-the-art detectors.

Feature selection for speaker verification using genetic programming

We present a study examining feature selection from high performing models evolved using genetic programming (GP) on the problem of automatic speaker verification (ASV). ASV is a highly unbalanced binary classification problem in which a given speaker must be verified against everyone else. We evolve classification models for 10 individual speakers using a variety of fitness functions and data sampling techniques and examine the generalisation of each model on a 1:9 unbalanced set. A significant difference between train and test performance is found which may indicate overfitting in the models. Using only the best generalising models, we examine two methods for selecting the most important features. We compare the performance of a number of tuned machine learning classifiers using the full 275 features and a reduced set of 20 features from both feature selection methods. Results show that using only the top 20 features found in high performing GP programs led to test classifications that are as good as, or better than, those obtained using all data in the majority of experiments undertaken. The classification accuracy between speakers varies considerably across all experiments showing that some speakers are easier to classify than others. This indicates that in such real-world classification problems, the content and quality of the original data has a very high influence on the quality of results obtainable.

A semi-supervised online sequential extreme learning machine method

This paper proposes a learning algorithm called Semi-supervised Online Sequential ELM, denoted as SOS-ELM. It aims to provide a solution for streaming data applications by learning from just the newly arrived observations, called a chunk. In addition, SOS-ELM can utilize both labeled and unlabeled training data by combining the advantages of two existing algorithms: Online Sequential ELM (OS-ELM) and Semi-Supervised ELM (SS-ELM). The rationale behind our algorithm exploits an optimal condition to alleviate empirical risk and structure risk used by SS-ELM, in combination with block calculation of matrices similar to OS-ELM. Efficient implementation of the SOS-ELM algorithm is made viable by an additional assumption that there is negligible structural relationship between chunks from different times. Experiments have been performed on standard benchmark problems for regression, balanced binary classification, unbalanced binary classification and multi-class classification by comparing the performance of the proposed SOS-ELM with OS-ELM and SS-ELM. The experimental results show that the SOS-ELM outperforms OS-ELM in generalization performance with similar training speed, and in addition outperforms SS-ELM with much lower supervision overheads.

Speaker state classification based on fusion of asymmetric simple partial least squares (SIMPLS) and support vector machines

This paper presents our studies of the effects of acoustic features, speaker normalization methods, and statistical modeling techniques on speaker state classification. We focus on the investigation of the effect of simple partial least squares (SIMPLS) in unbalanced binary classification. Beyond dimension reduction and low computational complexity, SIMPLS classifier (SIMPLSC) shows, especially, higher prediction accuracy to the class with the smaller data number. Therefore, an asymmetric SIMPLS classifier (ASIMPLSC) is proposed to enhance the performance of SIMPLSC to the class with the larger data number. Furthermore, we combine multiple system outputs (ASIMPLS classifier and Support Vector Machines) by score-level fusion to exploit the complementary information in diverse systems. The proposed speaker state classification system is evaluated with several experiments on unbalanced data sets. Within the Interspeech 2011 Speaker State Challenge, we could achieve the best results for the 2-class task of the Sleepiness Sub-Challenge with an unweighted average recall of 71.7%. Further experimental results on the SEMAINE data sets show that the ASIMPLSC achieves an absolute improvement of 6.1%, 6.1%, 24.5%, and 1.3% on the weighted average recall value, over the AVEC 2011 baseline system on the emotional speech binary classification tasks of four dimensions, namely, activation, expectation, power, and valence, respectively.