Classifier Design Research Articles

Submodular Asymmetric Feature Selection in Cascade Object Detection

A cascade classifier has turned out to be effective insliding-window based real-time object detection. In acascade classifier, node learning is the key process,which includes feature selection and classifier design. Previous algorithms fail to effectively tackle the asymmetry and intersection problems existing in cascade classification, thereby limiting the performance of object detection. In this paper, we improve current feature selection algorithm by addressing both asymmetry and intersection problems. We formulate asymmetric feature selection as a submodular function maximization problem. We then propose a new algorithm SAFS with formal performance guarantee to solve this problem.We use face detection as a case study and perform experiments on two real-world face detection datasets. The experimental results demonstrate that our algorithm SAFS outperforms the state-of-art feature selection algorithms in cascade object detection, such as FFS and LACBoost.

Classifying continuous, real-time e-nose sensor data using a bio-inspired spiking network modelled on the insect olfactory system

In many application domains, conventional e-noses are frequently outperformed in both speed and accuracy by their biological counterparts. Exploring potential bio-inspired improvements, we note a number of neuronal network models have demonstrated some success in classifying static datasets by abstracting the insect olfactory system. However, these designs remain largely unproven in practical settings, where sensor data is real-time, continuous, potentially noisy, lacks a precise onset signal and accurate classification requires the inclusion of temporal aspects into the feature set. This investigation therefore seeks to inform and develop the potential and suitability of biomimetic classifiers for use with typical real-world sensor data. Taking a generic classifier design inspired by the inhibition and competition in the insect antennal lobe, we apply it to identifying 20 individual chemical odours from the timeseries of responses of metal oxide sensors. We show that four out of twelve available sensors and the first 30 s (10%) of the sensors’ continuous response are sufficient to deliver 92% accurate classification without access to an odour onset signal. In contrast to previous approaches, once training is complete, sensor signals can be fed continuously into the classifier without requiring discretization. We conclude that for continuous data there may be a conceptual advantage in using spiking networks, in particular where time is an essential component of computation. Classification was achieved in real time using a GPU-accelerated spiking neural network simulator developed in our group.

Robust Boosted Parameter Based Combined Classifier for Rotation Invariant Texture Classification

ABSTRACTTexture analysis and classification remain as one of the biggest challenges for the field of computer vision and pattern recognition. This article presents a robust hybrid combination technique to build a combined classifier that is able to tackle the problem of classification of rotation-invariant 2D textures. Diversity in the components of the combined classifier is enforced through variation of the parameters related to both architecture design and training stages of a neural network classifier. The boosting algorithm is used to make perturbation of the training set using Multi-Layer Perceptron (MLP) as the base classifier. The final decision of the proposed combined classifier is based on the majority voting. Experiments’ results on a standard benchmark database of rotated textures show that the proposed hybrid combination method is very robust, and it presents an excellent texture discrimination for all considered classes, overcoming traditional texture modification methods.

Design and optimization of neural classifier to identify around shoulder motions

The objective of the work is to investigate the classification of different movements based on the SEMG pattern recognition method. The testing was conducted for four arm movements using several experiments with artificial neural network classification scheme. Six time domain features were extracted and consequently classification was implemented using back propagation neural classifier (BPNC). Further, the realization of projected network was verified using cross validation (CV) process; hence ANOVA algorithm was carried out. Performance of the network is analyzed by considering mean square error value. A comparison was performed between the extracted features and back propagation network results reported in the literature. The concurrent result indicates the significance of proposed network with classification accuracy (CA) of 100% recorded from two channels, while analysis of variance technique helps in investigating the effectiveness of classified signal for recognition tasks.

Classifier design based on the mechanism of capture-suppress of PCNN

针对医学图像识别的重要性, 本文提出了一种结合彩色图像特征场和基于脉冲耦合神经网络捕获-抑制机制的图像分类器设计方法. 首先, 构造符合人类视觉的彩色图像特征场; 然后, 利用脉冲耦合神经网络设计分类器实现对图像的识别. 最后, 应用于JAFFE人脸表情数据库和中医舌象的分类识别中, 并与ADABOOST方法和SVM方法进行比较. 实验结果表明, 本文方法可以获得更好的医学图像识别效果.

Evaluation of Error Rate Estimators in Discriminant Analysis with Multivariate Binary Variables

Classification problems often suffers from small samples in conjunction with large number of features, which makes error estimation problematic. When a sample is small, there is insufficient data to split the sample and the same data are used for both classifier design and error estimation. Error estimation can suffer from high variance, bias or both. The problem of choosing a suitable error estimator is exacerbated by the fact that estimation performance depends on the rule used to design the classifier, the feature-label distribution to which the classifier is to be applied and the sample size. This paper is concerned with evaluation of error rate estimators in two group discriminant analysis with multivariate binary variables. Behaviour of eight most commonly used estimators are compared and contrasted by mean of Monte Carlo Simulation. The criterion used for comparing those error rate estimators is sum squared error rate (SSE). Four experimental factors are considered for the simulation namely: the number of variables, the sample size relative to number of variables, the prior probability and the correlation between the variables in the populations. From the analysis carried out the estimators can be ranked as follows: DS, O, OS, U, R, JK, P and D.

Mental Workload Recognition by Combining Wavelet Packet Transform and Kernel Spectral Regression Techniques

A Mental Workload (MWL) recognition system was developed based on psychophysiological data to assess temporal variations in MWL levels. Salient EEG features were first extracted by using fuzzy mutual-information-based wavelet-packet transform (FMI-WPT). Then we adopted the kernel spectral regression linear discriminant analysis (KSRDA) to reduce the EEG feature dimensionality and to simultaneously enhance the inter-class discrimination capacity of the MWL classifiers. By combining FMIWPT and KSRDA techniques, we designed, evaluated and compared different types of MWL classifiers. The results demonstrated a improvement of the MWL classification accuracy by the proposed feature reduction method and classifier design framework. Particularly, it was shown by extensive comparative studies that the KNN and SVM outperform other classifiers.

HEp-2 cell classification: The role of Gaussian Scale Space Theory as a pre-processing approach

Indirect Immunofluorescence Imaging of Human Epithelial Type 2 (HEp-2) cells is an effective way to identify the presence of Anti-Nuclear Antibody (ANA). Most existing works on HEp-2 cell classification mainly focus on feature extraction, feature encoding and classifier design. Very few efforts have been devoted to study the importance of the pre-processing techniques. In this paper, we analyze the importance of the pre-processing, and investigate the role of Gaussian Scale Space (GSS) theory as a pre-processing approach for the HEp-2 cell classification task. We validate the GSS pre-processing under the Local Binary Pattern (LBP) and the Bag-of-Words (BoW) frameworks. Under the BoW framework, the introduced pre-processing approach, using only one Local Orientation Adaptive Descriptor (LOAD), achieved superior performance on the Executable Thematic on Pattern Recognition Techniques for Indirect Immunofluorescence (ET-PRT-IIF) image analysis. Our system, using only one feature, outperformed the winner of the ICPR 2014 contest that combined four types of features. Meanwhile, the proposed pre-processing method is not restricted to this work; it can be generalized to many existing works.

Granular classifiers and their design through refinement of information granules

In this study, we focus on the design and refinements of granular pattern classifiers, namely classifiers, which deal with a collection of information granules formed in a certain feature space. The development of this category of classifiers is realized as a two-phase design process. First, information granules occupying some regions of the feature space are formed through invoking mechanisms of clustering or fuzzy clustering. As a result, regions in the feature space are built, which are densely occupied by the patterns predominantly belonging to the same class. We offer a detailed way of assessing the character and quality of information granules and their information (classification-oriented) content. The resulting description is utilized in the realization of the classification mechanism being considered at the second phase of the design of the granular classifier. The mapping from the collection of information granules to class assignment (classification) involves matching of a pattern to be classified to individual information granules and aggregating them by considering the information content of the corresponding granules. In the study, a number of descriptors capturing information content and aggregation functions are analyzed. To improve the performance of the granular classifier, a refinement of information granules is carried out, in which highly heterogeneous information granules (viz. those containing patterns belonging to various classes) are refined (split, specialized), and their refined versions are afterwards used in the buildup of the classifier. A series of experiments involving both synthetic data as well as those publicly available is reported and analyzed, illustrating the main advantages of granular classifiers and their design procedure.

Design of radial basis function neural network classifier realized with the aid of data preprocessing techniques: design and analysis

In this paper, we introduce a new architecture of optimized Radial Basis Function neural network classifier developed with the aid of fuzzy clustering and data preprocessing techniques and discuss its comprehensive design methodology. In the preprocessing part, the Linear Discriminant Analysis (LDA) or Principal Component Analysis (PCA) algorithm forms a front end of the network. The transformed data produced here are used as the inputs of the network. In the premise part, the Fuzzy C-Means (FCM) algorithm determines the receptive field associated with the condition part of the rules. The connection weights of the classifier are of functional nature and come as polynomial functions forming the consequent part. The Particle Swarm Optimization algorithm optimizes a number of essential parameters needed to improve the accuracy of the classifier. Those optimized parameters include the type of data preprocessing, the dimensionality of the feature vectors produced by the LDA (or PCA), the number of clusters (rules), the fuzzification coefficient used in the FCM algorithm and the orders of the polynomials of networks. The performance of the proposed classifier is reported for several benchmarking data-sets and is compared with the performance of other classifiers reported in the previous studies.

Multi-objective evolutionary design of granular rule-based classifiers

In the last years, rule-based systems have been widely employed in several different application domains. The performance of these systems is strongly affected by the process of information granulation, which defines in terms of specific information granules such as sets, fuzzy sets and rough sets, the labels used in the rules. Generally, information granules are either provided by an expert, when possible, or extracted from the available data. In the framework of rule-based classifiers, we investigate the importance of determining an effective information granulation from data, preserving the comprehensibility of the granules. We show how the accuracies of rule-based classifiers can be increased by learning number and parameters of the granules, which partition the involved variables. To perform this analysis, we exploit a multi-objective evolutionary approach to the classifier generation we have recently proposed. We discuss different levels of information granulation optimization employing both the learning of the number of granules per variable and the tuning of each granule during the evolutionary process. We show and discuss the results obtained on several classification benchmark datasets using fuzzy sets and intervals as types of information granules.

An Optimization-Based Framework for the Transformation of Incomplete Biological Knowledge into a Probabilistic Structure and Its Application to the Utilization of Gene/Protein Signaling Pathways in Discrete Phenotype Classification.

Phenotype classification via genomic data is hampered by small sample sizes that negatively impact classifier design. Utilization of prior biological knowledge in conjunction with training data can improve both classifier design and error estimation via the construction of the optimal Bayesian classifier. In the genomic setting, gene/protein signaling pathways provide a key source of biological knowledge. Although these pathways are neither complete, nor regulatory, with no timing associated with them, they are capable of constraining the set of possible models representing the underlying interaction between molecules. The aim of this paper is to provide a framework and the mathematical tools to transform signaling pathways to prior probabilities governing uncertainty classes of feature-label distributions used in classifier design. Structural motifs extracted from the signaling pathways are mapped to a set of constraints on a prior probability on a Multinomial distribution. Being the conjugate prior for the Multinomial distribution, we propose optimization paradigms to estimate the parameters of a Dirichlet distribution in the Bayesian setting. The performance of the proposed methods is tested on two widely studied pathways: mammalian cell cycle and a p53 pathway model.

Diabetes Detection using Genetic Programming

Diabetes is a flopping of the body caused due to the absence of insulin and has gained popularity, globally. Physicians analyze diabetes using a blood glucose test; we cannot visibly categorize the person as diabetic or not based on these indicators. A pre-diabetic stage can aware the doctors and the patient about the denigrating health and can conscious the patient about the concerned measures. So proposed work intend a multi-class genetic programming (GP) based classifier design that will help the medical practitioner to confirm his/her diagnosis towards pre-diabetic, diabetic and non-diabetic patients. This system will design in two phases, first phase consist generation of a single feature from available features using Genetic Programming from the training data. The second phase consists of use the test data for checking of the classifier. Analysis of diabetes can be complemented by this GP based classifier.

Supraventricular Tachycardia Classification in the 12-Lead ECG Using Atrial Waves Detection and a Clinically Based Tree Scheme.

Specific supraventricular tachycardia (SVT) classification using surface ECG is considered a challenging task, since the atrial electrical activity (AEA) waves, which are a crucial element for obtaining diagnosis, are frequently hidden. In this paper, we present a fully automated SVT classification method that embeds our recently developed hidden AEA detector in a clinically based tree scheme. The process begins with initial noise removal and QRS detection. Then, ventricular features are extracted. According to these features, an initial AEA-wave search window is defined and a single AEA-wave is detected. Using a synthetic Gaussian signal and a linear combination of 12-lead ECG signals, all AEA-waves are detected. In accord with the atrial and ventricular information found, classification to atrial fibrillation, atrial flutter, atrioventricular nodal reentry tachycardia, atrioventricular reentry tachycardia, or sinus rhythm is performed in the framework of a clinically oriented decision tree. A study was performed to evaluate the classification from 68 patients (26 were used for the classifier's design, 42 were used for its validation). Average sensitivity of 83.21% [95% confidence interval (CI): 79.33-86.49%], average specificity of 95.80% (95% CI: 94.73-96.67%), and average accuracy of 93.29% (95% CI: 92.13-94.28%) were achieved compared to the definite diagnosis. In conclusion, the presented method may serve as a valuable decision support tool, allowing accurate detection of SVTs using noninvasive means.

Improving the Accuracy of Computer-aided Diagnosis for Breast MR Imaging by Differentiating between Mass and Nonmass Lesions.

To determine suitable features and optimal classifier design for a computer-aided diagnosis (CAD) system to differentiate among mass and nonmass enhancements during dynamic contrast material-enhanced magnetic resonance (MR) imaging of the breast. Two hundred eighty histologically proved mass lesions and 129 histologically proved nonmass lesions from MR imaging studies were retrospectively collected. The institutional research ethics board approved this study and waived informed consent. Breast Imaging Reporting and Data System classification of mass and nonmass enhancement was obtained from radiologic reports. Image data from dynamic contrast-enhanced MR imaging were extracted and analyzed by using feature selection techniques and binary, multiclass, and cascade classifiers. Performance was assessed by measuring the area under the receiver operating characteristics curve (AUC), sensitivity, and specificity. Bootstrap cross validation was used to predict the best classifier for the classification task of mass and nonmass benign and malignant breast lesions. A total of 176 features were extracted. Feature relevance ranking indicated unequal importance of kinetic, texture, and morphologic features for mass and nonmass lesions. The best classifier performance was a two-stage cascade classifier (mass vs nonmass followed by malignant vs benign classification) (AUC, 0.91; 95% confidence interval (CI): 0.88, 0.94) compared with one-shot classifier (ie, all benign vs malignant classification) (AUC, 0.89; 95% CI: 0.85, 0.92). The AUC was 2% higher for cascade (median percent difference obtained by using paired bootstrapped samples) and was significant (P = .0027). Our proposed two-stage cascade classifier decreases the overall misclassification rate by 12%, with 72 of 409 missed diagnoses with cascade versus 82 of 409 missed diagnoses with one-shot classifier. Separately optimizing feature selection and training classifiers for mass and nonmass lesions improves the accuracy of CAD for breast MR imaging. By cascading classifiers, we achieved a significant improvement in performance with respect to the use of a one-shot classifier. Our cascaded classifier may provide an advantage for screening women at high risk for breast cancer, in whom the ability to diagnose cancers at an early stage is of primary importance.

Representative Vector Machines: A Unified Framework for Classical Classifiers.

Classifier design is a fundamental problem in pattern recognition. A variety of pattern classification methods such as the nearest neighbor (NN) classifier, support vector machine (SVM), and sparse representation-based classification (SRC) have been proposed in the literature. These typical and widely used classifiers were originally developed from different theory or application motivations and they are conventionally treated as independent and specific solutions for pattern classification. This paper proposes a novel pattern classification framework, namely, representative vector machines (or RVMs for short). The basic idea of RVMs is to assign the class label of a test example according to its nearest representative vector. The contributions of RVMs are twofold. On one hand, the proposed RVMs establish a unified framework of classical classifiers because NN, SVM, and SRC can be interpreted as the special cases of RVMs with different definitions of representative vectors. Thus, the underlying relationship among a number of classical classifiers is revealed for better understanding of pattern classification. On the other hand, novel and advanced classifiers are inspired in the framework of RVMs. For example, a robust pattern classification method called discriminant vector machine (DVM) is motivated from RVMs. Given a test example, DVM first finds its k -NNs and then performs classification based on the robust M-estimator and manifold regularization. Extensive experimental evaluations on a variety of visual recognition tasks such as face recognition (Yale and face recognition grand challenge databases), object categorization (Caltech-101 dataset), and action recognition (Action Similarity LAbeliNg) demonstrate the advantages of DVM over other classifiers.

Energy-aware embedded classifier design for real-time emotion analysis.

Detection and classification of human emotions from multiple bio-signals has a wide variety of applications. Though electronic devices are available in the market today that acquire multiple body signals, the classification of human emotions in real-time, adapted to the tight energy budgets of wearable embedded systems is a big challenge. In this paper we present an embedded classifier for real-time emotion classification. We propose a system that operates at different energy budgeted modes, depending on the available energy, where each mode is constrained by an operating energy bound. The classifier has an offline training phase where feature selection is performed for each operating mode, with an energy-budget aware algorithm that we propose. Across the different operating modes, the classification accuracy ranges from 95% - 75% and 89% - 70% for arousal and valence respectively. The accuracy is traded off for less power consumption, which results in an increased battery life of up to 7.7 times (from 146.1 to 1126.9 hours).

Fuzzy <inline-formula><tex-math>$(c+p)$</tex-math></inline-formula>-Means Clustering and Its Application to a Fuzzy Rule-Based Classifier: Toward Good Generalization and Good Interpretability

This paper introduces a new classifier design method based on a modification of the classical fuzzy c-means clustering. First, a new fuzzy c-means clustering with p constant prototypes is proposed. This method can be considered a generalization of the concept of the conditional fuzzy clustering with some prototypes a priori known. A special initialization of the prototypes is introduced. Next, the proposed clustering method is used to construct the premises of an IF-THEN rule-based classifier. The conclusions of these rules are obtained by minimization of a criterion function with various approximations of a misclassification error (e.g., based on the quadratic, the linear, the sigmoidal or the Huber's loss function). The conjugate gradient algorithm is used to minimize the proposed criterion function. Each IF-THEN rule is represented in the Mamdani-Assilan form, which has good interpretability. Finally, an extensive experimental analysis on 14 benchmark datasets is performed to demonstrate the validity of the classifier introduced. Its competitiveness to the state-of-the-art classifiers, with respect to both performance and interpretability, is also shown.

Integrating new classes on the fly in evolving fuzzy classifier designs and their application in visual inspection

In this paper, we address the problem of integrating new classes on the fly into on-line classification systems. The main focus is on visual inspection tasks, although the concepts proposed in this paper can easily be applied to any other on-line classification systems. We use evolving fuzzy classifiers (EFCs), which can adapt their structure and update their parameters in incrementally due to embedded on-line adaptable classifier learning engines. We consider two different model architectures — classical single model and an all-pairs approach that uses class information to decompose the classification problem into several smaller sub-problems. The latter technique is essential for establishing new classes quickly and efficiently in the classifier, and for reducing class imbalance. Methodological novelties are (i) making appropriate structural changes in the EFC whenever a new class appears while operating in a single-pass incremental manner and (ii) estimating the expected change in classifier accuracy on the older classes. The estimation is based on an analysis of the impact of new classes on the established decision boundaries. This is important for operators, who are already familiar with an established classifier, the accuracy of which is known. The new concepts are evaluated in a real-world visual inspection scenario, where the main task is to classify event types which may occur dynamically on micro-fluidic chips and may reduce their quality. The results show stable performance of established classifiers and efficient (low number of samples requested) as well as fast integration (steeply rising accuracy curves) of new event types (classes).

An application of feature selection for the fuzzy rule based classifier design with the order based sematics of linguistic terms for high-dimensional datasets

An application of feature selection for the fuzzy rule based classifier design with the order based sematics of linguistic terms for high-dimensional datasets