Pattern Classification Systems Research Articles

A Generalized Deep Learning Model for Denoising Image Datasets

Advance in technology world has lots of contributions from artificial intelligence which is a highly growing area. The failure of traditional algorithms has led to the employment of deep learning algorithms in various fields like pattern recognition, recommendation systems and classification systems. Removal of noise from images can be done using traditional noise removal filters. These filters can either remove more noise that wanted or leave unwanted noise than what is needed in the data. Utilization of Convolutional neural networks designed based on the dataset requirements along with the noise removal filter can yield better results. In this work, evaluation of the performance of convolutional neural network (CNN) against existing image denoising algorithms has been successfully executed . The proposed model is a generalized CNN model which can recognize and classify any type of noisy image given. Two types of model were compared where one model 1 uses the Adam optimizer and model 2 uses the Stochastic Gradient Descent (SGD) optimizer. The image dataset used here is MNIST handwritten dataset, which is trained, tested and validated with both the models by adding three different types of noise viz, Poisson, Salt and Pepper as well as Gaussian Noise. More accuracy and better results were given by the model 2 which uses the SGD optimizer.

Feature Selection Method Based on Differential Correlation Information Entropy

Feature selection is one of the major aspects of pattern classification systems. In previous studies, Ding and Peng recognized the importance of feature selection and proposed a minimum redundancy feature selection method to minimize redundant features for sequential selection in microarray gene expression data. However, since the minimum redundancy feature selection method is used mainly to measure the dependency between random variables of mutual information, the results cannot be optimal without consideration of global feature selection. Therefore, based on the framework of minimum redundancy-maximum correlation, this paper introduces entropy to measure global feature selection and proposes a new feature subset evaluation method, differential correlation information entropy. In our function, different bivariate correlation metrics are selected. Then, the feature selection is completed through sequence forward search. Two different classification models are used on eleven standard data sets of the UCI machine learning knowledge base to compare various comparison algorithms, such as mRMR, reliefF and feature selection method with joint maximal information entropy, with our method. The experimental results show that feature selection based on our proposed method is obviously superior to that of other models.

Advanced approaches for information processing in multimedia, decision‐making, and security systems

Advanced approaches for information processing in multimedia, decision‐making, and security systems

Corrections to “An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System” [Apr 12 224-234

The potential of type-2 fuzzy sets to manage high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system (FLS) is how to estimate the parameters of the type-2 fuzzy membership function (T2MF) and the footprint of uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach to learn and tune Gaussian interval type-2 membership functions (IT2MFs) with application to multidimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and cross-validation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods, and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung computer-aided detection system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

An adaptive model for the autonomous monitoring and management of water end use

Most pattern classification systems are usually developed based on the training of historical data, and as a result, the performance of these models relies heavily on the amount of collected information. However, in many cases, such data collection process is relatively costly, which eventually limits the efficiency as well as the widespread implementation of the final developed model. In this context, the paper focuses on presenting an advanced universal water management system, which could interface with both water consumers and utilities via smart phone and web application. Originally, Autoflow©, a prototype tool that is used to disaggregate total water consumption into each end-use category was developed, which achieved an accuracy ranging from 74 to 94%. However, a drawback of this model was that it was trained with data collected from only Australia; therefore, accuracy reductions would likely be observed when this system is implemented in different countries having very different water using appliances and behaviour patterns. To avoid the costly data collection process for model calibration when operating in new regions, this research study introduces an enhanced model, namely AutoflowU (i.e. U stands for Universal). This new tool can be applied in residential properties globally to autonomously disaggregate water consumption into the seven main water end-use categories, namely: shower, toilet, tap, clothes washer, dishwasher, evaporative air cooler and irrigation, without the need for collecting new regional end-use data for model calibration. In order to develop this new tool, Decision Trees, Dynamic Time Warping (DTW), Self Organising Map (SOM) and Hidden Markov Model (HMM) techniques were utilised. The test results obtained from 230 properties in both Australia and the US showed that the AutoflowU achieved 72–93% accuracy.

The Semi-Variogram and Spectral Distortion Measures for Image Texture Retrieval.

Semi-variogram estimators and distortion measures of signal spectra are utilized in this paper for image texture retrieval. On the use of the complete Brodatz database, most high retrieval rates are reportedly based on multiple features and the combinations of multiple algorithms, while the classification using single features is still a challenge to the retrieval of diverse texture images. The semi-variogram, which is theoretically sound and the cornerstone of spatial statistics, has the characteristics shared between true randomness and complete determinism and, therefore, can be used as a useful tool for both the structural and statistical analysis of texture images. Meanwhile, spectral distortion measures derived from the theory of linear predictive coding provide a rigorously mathematical model for signal-based similarity matching and have been proven useful for many practical pattern classification systems. Experimental results obtained from testing the proposed approach using the complete Brodatz database, and the the University of Illinois at Urbana-Champaign texture database suggests the effectiveness of the proposed approach as a single-feature-based dissimilarity measure for real-time texture retrieval.

Identifying Security Evaluation of PatternClassifiers Under attack

Pattern classification is a branch of machine learning that focuses on recognition of patterns and regularities in data. In adversarial applications like biometric authentication, spam filtering, network intrusion detection the pattern classification systems are used. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. We propose a framework for evaluation of pattern security,model of adversary for defining any attack scenario. Reported results show that security evaluation can provide a more complete understanding of the classifier’s behavior in adversarial environments, and lead to better design choices

Feature selection based on sparse representation with the measures of classification error rate and complexity of boundary

Feature subset selection plays an important role in pattern recognition, classification systems, and data mining. We study how to select good features by optimizing multivariate performance measures based on sparse representation. In this paper, we first propose a novel feature evaluation measure, called counting region covering (CRC), for estimating classification complexity in different feature subspaces. Then, we present a unified feature selection framework by optimizing the classification error rate and complexity of boundary simultaneously. This allows us to select a compact set of superior features at high classification accuracy. Finally, we discuss the influence of weighting factors in feature selection framework. We perform experimental comparison of our algorithm and other methods using a support vector machine classifier and five different data sets (iris, wine, sonar, iono, and waveform). Experimental results on five real-world datasets demonstrate the effectiveness of our algorithm.

A Comprehensive Survey on Pattern Classifier's Security Evaluation under Attack

many years, various tools based on pattern classification system have been used in security related applications like, spam filtering, biometric authentication system, network intrusion detection system, pattern classification systems are used, in which brilliant and adaptive adversary may changes data to make the classifier produce false negatives (regular). Measurement of pattern classifier security performance is very important part for making decisions, finding product viability, for differentiate various classifiers. Pattern classification systems may exhibit obligations, exploitation affect their performance, produce limitations to practical utility, if adversarial scenario is not taken into account. At design phase, the system evaluates the classifiers security. The classifiers security means performance degradation for related attacks may acquire when operation runs. A phenomenon is used for classifier security evaluation, adversary model for defining attack scenarios that generates training and testing sets.

An optimized classification algorithm by BP neural network based on PLS and HCA

Due to some correlative or repetitive factors between features or samples with high dimension and large amount of sample data, when traditional back-propagation (BP) neural network is used to solve this classification problem, it will present a series of problems such as network structural redundancy, low learning efficiency, occupation of storage space, consumption of computing time, and so on. All of these problems will restrict the operating efficiency and classification precision of neural network. To avoid them, partial least squares (PLS) algorithm is used to reduce the feature dimension of original data into low-dimensional data as the input of BP neural network, so that it can simplify the structure and accelerate convergence, thus improving the training speed and operating efficiency. In order to improve the classification precision of BP neural network by using hierarchical cluster analysis (HCA), similar samples are put into a sub-class, and some different sub-classes can be obtained. For each sub-class, a different training session can be conducted to find a corresponding precision BP neural network model, and the simulation samples of different sub-classes can be recognized by the corresponding network model. In this paper, the theories of PLS and HCA are combined together with the property of BP neural network, and an optimized classification algorithm by BP neural network based on PLS and HCA (PLS-HCA-BP algorithm) is proposed. The new algorithm is aimed at improving the operating efficiency and classification precision so as to provide a more reliable and more convenient tool for complex pattern classification systems. Three experiments and comparisons with four other algorithms are carried out to verify the superiority of the proposed algorithm, and the results indicate a good picture of the PLS-HCA-BP algorithm, which is worthy of further promotion.

An advanced ACO algorithm for feature subset selection

Abstract Feature selection is an important task for data analysis and information retrieval processing, pattern classification systems, and data mining applications. It reduces the number of features by removing noisy, irrelevant and redundant data. In this paper, a novel feature selection algorithm based on Ant Colony Optimization (ACO), called Advanced Binary ACO (ABACO), is presented. Features are treated as graph nodes to construct a graph model and are fully connected to each other. In this graph, each node has two sub-nodes, one for selecting and the other for deselecting the feature. Ant colony algorithm is used to select nodes while ants should visit all features. The use of several statistical measures is examined as the heuristic function for visibility of the edges in the graph. At the end of a tour, each ant has a binary vector with the same length as the number of features, where 1 implies selecting and 0 implies deselecting the corresponding feature. The performance of proposed algorithm is compared to the performance of Binary Genetic Algorithm (BGA), Binary Particle Swarm Optimization (BPSO), CatfishBPSO, Improved Binary Gravitational Search Algorithm (IBGSA), and some prominent ACO-based algorithms on the task of feature selection on 12 well-known UCI datasets. Simulation results verify that the algorithm provides a suitable feature subset with good classification accuracy using a smaller feature set than competing feature selection methods.

Security Evaluation of Pattern Classifiers under Attack

Pattern classification systems are commonly used in adversarial applications, like biometric authentication, network intrusion detection, and spam filtering, in which data can be purposely manipulated by humans to undermine their operation. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. In this paper, we address one of the main open issues: evaluating at design phase the security of pattern classifiers, namely, the performance degradation under potential attacks they may incur during operation. We propose a framework for empirical evaluation of classifier security that formalizes and generalizes the main ideas proposed in the literature, and give examples of its use in three real applications. Reported results show that security evaluation can provide a more complete understanding of the classifier's behavior in adversarial environments, and lead to better design choices.

Feature selection based on cluster and variability analyses for ordinal multi-class classification problems

Feature selection is an essential problem for pattern classification systems. This paper studies how to provide systems with the most characterizing features for ordinal multi-class classification task. The integration of cluster analyses and variability analyses advances a novel feature selection scheme with efficiency. The Huang-index method using fuzzy c-means is employed to enhance cluster validity and optimizes a consistent number of clusters among the features. A new entropy-based feature evaluation method is formulated for the authentication of relevant features. Then, multivariate statistical analyses are utilized to solve the redundancy between relevant features. Experimental results show that our new feature selection scheme sifts successfully a compact subset of characterizing features for classification problems with multiple classes.

An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System

The potential of type-2 fuzzy sets to manage high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system (FLS) is how to estimate the parameters of the type-2 fuzzy membership function (T2MF) and the footprint of uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach to learn and tune Gaussian interval type-2 membership functions (IT2MFs) with application to multidimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and cross-validation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods, and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung computer-aided detection system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

A novel hypothesis-margin based approach for feature selection with side pairwise constraints

Feature selection is an important problem for pattern classification systems. As compared to unsupervised feature selection methods, the supervised ones have better performance. However, almost all existing supervised ones use class labels as supervised information, very less work has been done for other forms of supervision information such as pairwise constraints, which specifies whether a pair of data samples belongs to the same class ( must-link constraints) or different classes ( cannot-link constraints). In reality, pairwise constraints can be easily obtained by specifying whether some pairs of examples belong to the same class or not. Therefore, a new filter method for feature selection with pairwise constraints, called Constraint Score, was proposed. Unfortunately, Constraint Score does not consider the case where only cannot-link constraints are given. Also, the conclusion ‘ must-link constraints are more important than cannot-link constraints’ given by Constraint Score algorithm needs to be further verified, since ‘ cannot-link constraints’ seems more important than ‘ must-link constraints’ from the viewpoint of hypothesis-margin or margin. In addition, like the existing supervised feature selection methods, the currently proposed hypothesis-margin based approach for feature selection, called Simba, also utilizes class labels as supervision information. In this paper, to further study the feature selection problem aiming at pairwise constraints, we introduce a novel hypothesis-margin based approach for feature selection with side pairwise constraints, called Simba - sc , which only uses cannot-link constraints as supervision information. We compare our algorithm with the well-known Constraint Score, Fisher Score and Laplacian Score algorithms. Experiments are carried out on 6 UCI data sets using three different classifiers. Experimental results show that, with a few cannot-link constraints, Simba - sc achieves similar or even higher performance than Fisher Score with full class labels on all training data, and has better or comparable performance than Constraint Score.

Hierarchical classification of dynamically varying radar pulse repetition interval modulation patterns

The central purpose of passive signal intercept receivers is to perform automatic categorization of unknown radar signals. Currently, there is an urgent need to develop intelligent classification algorithms for these devices due to emerging complexity of radar waveforms. Especially multifunction radars (MFRs) capable of performing several simultaneous tasks by utilizing complex, dynamically varying scheduled waveforms are a major challenge for automatic pattern classification systems. To assist recognition of complex radar emissions in modern intercept receivers, we have developed a novel method to recognize dynamically varying pulse repetition interval (PRI) modulation patterns emitted by MFRs. We use robust feature extraction and classifier design techniques to assist recognition in unpredictable real-world signal environments. We classify received pulse trains hierarchically which allows unambiguous detection of the subpatterns using a sliding window. Accuracy, robustness and reliability of the technique are demonstrated with extensive simulations using both static and dynamically varying PRI modulation patterns.

Minimum classification error training in example based speech and pattern recognition using sparse weight matrices

The Minimum Classification Error (MCE) criterion is a well-known criterion in pattern classification systems. The aim of MCE training is to minimize the resulting classification error when trying to classify a new data set. Usually, these classification systems use some form of statistical model to describe the data. These systems usually do not work very well when this underlying model is incorrect. Speech recognition systems traditionally use Hidden Markov Models (HMM) with Gaussian (or Gaussian mixture) probability density functions as their basic model. It is well known that these models make some assumptions that are not correct. In example based approaches, these statistical models are absent and are replaced by the pure data. The absence of statistical models has created the need for parameters to model the data space accurately. For this work, we use the MCE criterion to create a system that is able to work together with this example based approach. Moreover, we extend the locally scaled distance measure with sparse, block diagonal weight matrices resulting in a better model for the data space and avoiding the computational load caused by using full matrices. We illustrate the approach with some example experiments on databases from pattern recognition and with speech recognition.

SOLUTION OVER-FIT CONTROL IN EVOLUTIONARY MULTIOBJECTIVE OPTIMIZATION OF PATTERN CLASSIFICATION SYSTEMS

The optimization of many engineering systems is challenged by the solution over-fit to the data set used to evaluate potential solutions during the evolutionary process. The solution over-fit phenomenon is hard to detect and is especially prevalent in problems involving example-based training, such as pattern feature selection and pattern classifier design. For these applications, uncontrolled over-fit can lead to biased features being extracted and degraded classifier generalization abilities. This paper details the performance of a solution over-fit control strategy used in the multiobjective evolutionary optimization of a multileveled classification system. This control, embedded within a solution validation procedure, minimizes the over-fit effects without modifying the dominance relation used in the processing of candidate solutions. Extensive experimental analysis using multiobjective genetic and memetic algorithms demonstrates both the need and the efficiency of the proposed over-fit control for pattern classification systems optimization.

The Random Neural Network: A Survey

The random neural network (RNN) is a recurrent neural network model inspired by the spiking behaviour of biological neuronal networks. Contrary to most artificial neural network models, neurons in the RNN interact by probabilistically exchanging excitatory and inhibitory spiking signals. The model is described by analytical equations, has a low complexity supervised learning algorithm and is a universal approximator for bounded continuous functions. The RNN has been applied in a variety of areas including pattern recognition, classification, image processing, combinatorial optimization and communication systems. It has also inspired research activity in modelling interacting entities in various systems such as queueing and gene regulatory networks. This paper presents a review of the theory, extension models, learning algorithms and applications of the RNN.

Processing fuzzy set membership functionals as vectors

A framework is presented for processing fuzzy sets for which the universe of discourse X = {x} is a separable Hilbert Space, which, in particular, may be a Euclidian Space. In a given application, X would constitute a feature space. The membership functions of sets in such X are then "membership functionals", that is, mappings from a vector space to the real line. This paper considers the class ? of fuzzy sets A, the membership functionals μ A of which belong to a Reproducing Kernel Hilbert Space (RKHS) F(X) of bounded analytic functionals on X, and satisfy the constraint $$0\,\leqslant \mu (x) \leqslant 1,x \in X$$ . These functionals can be expanded in abstract power series in x, commonly known as Volterra functional series in x. Because of the one-to-one relationship between the fuzzy sets A and their respective μ A , one can process the sets A as objects using their μ A as intermediaries. Thus the structure of the uncertainty present in the fuzzy sets can be processed in a vector space without descending to the level of processing of vectors in the feature space as usually done in the literature in the field. Also, the framework allows one to integrate human and machine judgments in the definition of fuzzy sets; and to use concepts analogous to probabilistic concepts in assigning membership values to combinations of fuzzy sets. Some analytical and interpretive consequences of this approach are presented and discussed. A result of particular interest is the best approximation $$\hat{\mu}_{A}$$ of a membership functional μ A in F(X) based on interpolation $$\hat {\mu}_{A} (v^{i}) = u^{i}$$ on a training set {(v i , u i ),i = 1, . . . , q} and under the positivity constraint. The optimal analytical solution comes out in the form of an Optimal Interpolative Neural Network (OINN) proposed by the author in 1990 for best approximation of pattern classification systems in a F(X) space setting. An example is briefly described of an application of this approach to the diagnosis of Alzheimer's disease.