Statistical Pattern Classification Research Articles

Law of large numbers for a two-dimensional class cover problem

We prove a Law of Large Numbers (LLN) for the domination number of class cover catch digraphs (CCCD) generated by random points in two (or higher) dimensions. DeVinney and Wierman (2002) proved the Strong Law of Large Numbers (SLLN) for the uniform distribution in one dimension, and Wierman and Xiang (2008) extended the SLLN to the case of general distributions in one dimension. In this article, using subadditive processes, we prove a SLLN result for the domination number generated by Poisson points in ℝ2. From this we obtain a Weak Law of Large Numbers (WLLN) for the domination number generated by random points in [0, 1]2 from uniform distribution first, and then extend these result to the case of bounded continuous distributions. We also extend the results to higher dimensions. The domination number of CCCDs and related digraphs have applications in statistical pattern classification and spatial data analysis.

Development of a Prototype Smart City System for Refuse Disposal Management

The future of modern cities largely depends on how well they can tackle problems that confront them by embracing the next era of digital revolution. A vital element of such revolution is the creation of smart cities. Smart city is an evolving paradigm that involves the deployment of information communication technology wares into public or private infrastructure to provide intelligent data gathering and analysis. To align concretely with the smart city revolution in the area of environmental cleanliness, this paper involves the development of a smart city system for refuse disposal management. The architecture of the proposed system is an adaptation of the Jalali reference smart city architecture. It features four essential layers, which are signal sensing and processing, network, intelligent user application and Internet of Things (IoT) web application layers. A proof of concept prototype was implemented based on the designed architecture of the proposed system. The signal sensing and processing layer was implemented to produce a smart refuse bin that contains the Arduino microcontroller board, Wi-Fi/GSM transceiver, proximity sensor, gas sensor, temperature sensor and other relevant electronic components. The network layer provides interconnectivity among the layers via the internet. The intelligent user application layer was realized with non-browser client application, statistical feature extraction method and pattern classifiers. Whereas the IoT web application layer was realised with ThingSpeak, which is an online web application for IoT based projects. The sensors in the smart refuse bin generate multivariate dataset that corresponds to the status of refuse in the bin. Training and testing features were extracted from the dataset using first order statistical feature extraction method. Afterward, multilayer perceptron artificial neural network and support vector machine were trained and compared experimentally. The multilayer perceptron artificial neural network model gave the overall best accuracy of 98.0% and the least mean square error of 0.0036.

A Study on the Measurement of Respiratory Rate Using Image Alignment and Statistical Pattern Classification

A Study on the Measurement of Respiratory Rate Using Image Alignment and Statistical Pattern Classification

Automatic Analysis of Microaneurysms Turnover to Diagnose the Progression of Diabetic Retinopathy

Diabetic retinopathy (DR) is one of the most common microvascular complications and its early detection is critical for the prevention of vision loss. Recent studies have indicated that microaneurysms (MAs) are the hallmark of DR. However, the detection of MAs relies on trained clinicians and relatively expensive software. Moreover, manual errors often lower the accuracy of this detection. Therefore, an automatic analysis technique is highly demanded in the detection of DR progression. In this paper, we present a novel and complete methodology involving two different ways from the view of MAs turnover and pathological risk factors to diagnose the progression of DR. Specifically, one approach follows the traditional image analysis-based roadmap to obtain MAs turnover. The other investigates seven pathological features, related with MAs turnover, to classify the unchanged, new, and resolved MAs by means of statistical analysis and pattern classification techniques. The evaluations on Grampian diabetes database show that the proposed image analysis method could achieve a sensitivity of 94% and a specificity of 93%, while the classification model could achieve 89% sensitivity and 88% specificity, respectively. We also analyzed the potential weight of pathological risk factors leading to the MAs turnover, which could provide an alternative guidance for the progression of DR than traditional detection methods. In conclusion, this study provides a novel and noninvasive detection technique for early diagnosis of diabetic retinopathy with a competitive accuracy.

Identifying dynamic knowledge flow patterns of business method patents with a hidden Markov model

As an alternative to the conventional R&D innovation, business models are becoming an important locus of lucrative innovation. Due to the rise of the internet economy, business model innovation today often involves technological innovation, and this can be evidenced by business method (BM) patents. Of several mechanisms that stimulate business model innovation, the role of BM patents is probably most noteworthy. To understand how BM patents play their roles in business model innovation, we need to observe the long-term knowledge flow process. Therefore, we aim to identify dynamic patterns of knowledge flows driven by BM patents using a hidden Markov model (HMM) and patent citation data as an input. An HMM is a popular statistical tool for modelling a wide range of time series data. Since it does not have any general theoretical limit in regard to statistical pattern classification, an HMM is capable of characterizing various temporal patterns. A case study is conducted with the BM patents in 16 USPTO subclasses related to secure transactions. After patterns of the individual subclasses are generated, they are grouped into four major patterns through clustering analysis and their characteristics are closely examined. Our analysis reveals that the BM patents for secure transaction in general play increasingly important roles in advancement of business models, facilitating the transfer of knowledge, and thus can provide valuable insights in formulating more effective strategies or policies for business model innovation.

An investigative study into the sensitivity of different partial discharge φ-q-n pattern resolution sizes on statistical neural network pattern classification

This paper investigates the sensitivity of statistical fingerprints to different phase resolution (PR) and amplitude bins (AB) sizes of partial discharge (PD) φ-q-n (phase-amplitude-number) patterns. In particular, this paper compares the capability of the ensemble neural network (ENN) and the single neural network (SNN) in recognizing and distinguishing different resolution sizes of φ-q-n discharge patterns. The training fingerprints for both the SNN and ENN comprise statistical fingerprints from different φ-q-n measurements. The result shows that there exists statistical distinction for different PR and AB sizes on some of the statistical fingerprints. Additionally, the ENN and SNN outputs change depending on training and testing with different PR and AB sizes. Furthermore, the ENN appears to be more sensitive in recognizing and discriminating the resolution changes when compared with the SNN. Finally, the results are assessed for practical implementation in the power industry and benefits to practitioners in the field are highlighted.

Improper Complex-Valued Bhattacharyya Distance

Motivated by application of complex-valued signal processing techniques in statistical pattern recognition, classification, and Gaussian mixture (GM) modeling, this paper derives analytical expressions for computing the Bhattacharyya coefficient/distance (BC/BD) between two improper complex-valued Gaussian distributions. The BC/BD is one of the most widely used statistical measures for evaluating class separability in classification problems, feature extraction in pattern recognition, and for GM reduction (GMR) purposes. The BC provides an upper bound on the Bayes error, which is commonly known as the best criterion to evaluate feature sets. Although the computation of the BC/BD between real-valued signals is a well-known result, it has not yet been extended to the case of improper complex-valued Gaussian densities. This paper addresses this gap. We analyze the role of the pseudocovariance matrix, which characterizes the noncircularity of the signal, and show that it carries critical second-order statistical information for computing the BC/BD. We derive upper and lower bounds on the BD in terms of the eigenvalues of the covariance and pseudocovariance matrices of the underlying densities. The theoretical bounds are then used to introduce the concept of β -dominance in the context of statistical distance measures. The BC is pseudometric, since it fails to satisfy the triangle inequality. Using the Matusita distance (a full-metric variant of the BC), we propose an intuitively pleasing indirect distance measure for comparing two general GMs. Finally, we investigate the application of the proposed BC/BD measures for GMR purposes and develop two BC-based GMR algorithms.

Detection of Heart Muscle Damage from Automated Analysis of Echocardiogram Video

ABSTRACTIn this work, an approach for heart muscle damage detection from echocardiography sequences is proposed. To exemplify the approach, a system is presented which involves image denoising and enhancement and segmentation of the left ventricle (LV) for extracting the heart wall boundaries. Using the heart wall boundaries global LV parameters are calculated followed by statistical pattern recognition and classification to identify the heart muscle damage or myocardial ischemia (MI). The performance of this algorithm is assessed in 60 real patient data with both normal and abnormal conditions. The experimental results reveal that the proposed method can be used as an effective tool for detection of heart muscle damage or MI automatically.

Automated classification of oncoming ground vehicles using acoustic echolocation and supervised machine learning

In order to perform useful tasks, robots must have the ability to notice, recognize, and respond to objects in their environment. This requires the acquisition and synthesis of information from a variety of sensors. Here we focus on acoustic echolocation measurements of approaching vehicles, where an acoustic parametric array propagates an audible signal to the oncoming target and the reflected backscattered signal is recorded using the Microsoft Kinect microphone array. Although useful information about the target is hidden inside the noisy time domain measurements, the Dynamic Wavelet Fingerprint process (DWFP) is used to create a time-frequency representation of the data. Intelligent feature selection allows the creation of a small-dimensional feature vector that best differentiates between vehicle types for use in statistical pattern classification routines. Using experimentally measured data from real vehicles at 50 m, this process is able to correctly classify vehicles into one of five known classes with 94% accuracy. Fully three-dimensional simulations allow us to study the nonlinear beam propagation and interaction with real-world targets to improve classification results.

Segmented Mixture-of-Gaussian Classification for Hyperspectral Image Analysis

The same high dimensionality of hyperspectral imagery that facilitates detection of subtle differences in spectral response due to differing chemical composition also hinders the deployment of traditional statistical pattern-classification procedures, particularly when relatively few training samples are available. Traditional approaches to addressing this issue, which typically employ dimensionality reduction based on either projection or feature selection, are at best suboptimal for hyperspectral classification tasks. A divide-and-conquer algorithm is proposed to exploit the high correlation between successive spectral bands and the resulting block-diagonal correlation structure to partition the hyperspectral space into approximately independent subspaces. Subsequently, dimensionality reduction based on a graph-theoretic locality-preserving discriminant analysis is combined with classification driven by Gaussian mixture models independently in each subspace. The locality-preserving discriminant analysis preserves the potentially multimodal statistical structure of the data, which the Gaussian mixture model classifier learns in the reduced-dimensional subspace. Experimental results demonstrate that the proposed system significantly outperforms traditional classification approaches, even when few training samples are employed.

Discrimination of individual tigers (Panthera tigris) from long distance roars

This paper investigates the extent of tiger (Panthera tigris) vocal individuality through both qualitative and quantitative approaches using long distance roars from six individual tigers at Omaha's Henry Doorly Zoo in Omaha, NE. The framework for comparison across individuals includes statistical and discriminant function analysis across whole vocalization measures and statistical pattern classification using a hidden Markov model (HMM) with frame-based spectral features comprised of Greenwood frequency cepstral coefficients. Individual discrimination accuracy is evaluated as a function of spectral model complexity, represented by the number of mixtures in the underlying Gaussian mixture model (GMM), and temporal model complexity, represented by the number of sequential states in the HMM. Results indicate that the temporal pattern of the vocalization is the most significant factor in accurate discrimination. Overall baseline discrimination accuracy for this data set is about 70% using high level features without complex spectral or temporal models. Accuracy increases to about 80% when more complex spectral models (multiple mixture GMMs) are incorporated, and increases to a final accuracy of 90% when more detailed temporal models (10-state HMMs) are used. Classification accuracy is stable across a relatively wide range of configurations in terms of spectral and temporal model resolution.

Locality-Preserving Dimensionality Reduction and Classification for Hyperspectral Image Analysis

Hyperspectral imagery typically provides a wealth of information captured in a wide range of the electromagnetic spectrum for each pixel in the image; however, when used in statistical pattern-classification tasks, the resulting high-dimensional feature spaces often tend to result in ill-conditioned formulations. Popular dimensionality-reduction techniques such as principal component analysis, linear discriminant analysis, and their variants typically assume a Gaussian distribution. The quadratic maximum-likelihood classifier commonly employed for hyperspectral analysis also assumes single-Gaussian class-conditional distributions. Departing from this single-Gaussian assumption, a classification paradigm designed to exploit the rich statistical structure of the data is proposed. The proposed framework employs local Fisher's discriminant analysis to reduce the dimensionality of the data while preserving its multimodal structure, while a subsequent Gaussian mixture model or support vector machine provides effective classification of the reduced-dimension multimodal data. Experimental results on several different multiple-class hyperspectral-classification tasks demonstrate that the proposed approach significantly outperforms several traditional alternatives.

Gait based Gender Identification using Statistical Pattern Classifiers

Gait based gender identification has received a great attention from researchers in the last decade due to its potential in different applications. This will help a human identification system to focus only on the identified gender related features, which can improve search speed and efficiency of the retrieval system by limiting the subsequent searching space into either a male database or female database. In this paper after preprocessing, four binary moment features and two spatial features are extracted from human silhouette. Then the extracted features are used for training and testing two different pattern classifiers kNearest Neighbor (kNN) and Support Vector Machine(SVM). Experimental results show superior performance of our approach to the existing gender classifiers. To evaluate the performance of the proposed algorithm experiments have been conducted on NLPR database.

Protocol oblivious classification of multimedia traffic

AbstractVoice and video over IP are becoming increasingly popular and represent the largest source of profits as consumer interest in online voice and video services increases, and as broadband deployments proliferate. In order to tap the potential profits that VoIP and IPTV offer, carrier networks have to efficiently and accurately manage and track the delivery of IP services. The traditional approach of using port numbers to classify traffic is infeasible due to the usage of dynamic port number. In this paper, we focus on a statistical pattern classification technique to identify multimedia traffic. Based on the intuitions that voice and video data streams show strong regularities in the packet inter‐arrival times (IATs) and the associated packet sizes when combined together in one single stochastic process, we propose a system, called VOVClassifier, for voice and video traffic classification. VOVClassifier is an automated self‐learning system that classifies traffic data by extracting features from frequency domain using Power Spectral Density (PSD) analysis and grouping features using Subspace Decomposition. We applied VOVClassifier to real packet traces collected from different network scenarios. Results demonstrate the effectiveness and robustness of our approach that is capable of achieving a detection rate of up to 100% for voice and 96.5% for video while keeping the false positive rate close to 0%. Copyright © 2009 John Wiley & Sons, Ltd.

A Novel Weighted Voting for K-Nearest Neighbor Rule

K-nearest neighbor rule (KNN) is the well-known non-parametric technique in the statistical pattern classification, owing to its simplicity, intuitiveness and effectiveness. In this paper, we firstly review the related works in brief and detailedly analyze the sensitivity issue on the choice of the neighborhood size k , existed in the KNN rule. Motivated by the problem, a novel dual weighted voting scheme for KNN is developed. With the goal of overcoming the sensitivity of the choice of the neighborhood size k and improving the classification performance, the proposed classifier mainly employs the dual weighted voting function to reduce the effect of the outliers in the k nearest neighbors of each query object. To verify the superiority of the proposed classifier, the experiments are conducted on one artificial data set and twelve real data sets, in comparison with the other classifiers. Experimental results suggest that our proposed classifier is an effective algorithm for the classification tasks in many practical situations, owing to its satisfactory classification performance and robustness over a wide range of k .

Statistical pattern classification: A review and some current problems/paradigms.

Unsupervised clustering is the ability to automatically partition a set of data patterns into meaningful groups without prior knowledge of the groups or their number. Supervised classification is the ability to automatically recognize a data pattern as an instance from one of a known set of classes. These problems are fundamental to a variety of application domains, including scientific (e.g., bioinformatics), engineering (e.g., speech recognition), business (e.g., marketing and document clustering), and military (target detection). In this talk, we first review the basic supervised and unsupervised classification problems, standard solution methodologies, and how to characterize their performance. We then discuss some recent problem variants and associated paradigms, including semisupervised learning, unsupervised clustering in high-dimensional feature spaces, and decision fusion techniques.

Joint discriminative–generative modelling based on statistical tests for classification

In statistical pattern classification, generative approaches, such as linear discriminant analysis (LDA), assume a data-generating process (DGP), whereas discriminative approaches, such as linear logistic regression (LLR), do not model the DGP. In general, a generative classifier performs better than its discriminative counterpart if the DGP is well-specified and worse than the latter if the DGP is clearly mis-specified. In view of this, this paper presents a joint discriminative–generative modelling (JoDiG) approach, by partitioning predictor variables X into two sub-vectors, namely X G , to which a generative approach is applied, and X D , to be treated by a discriminative approach. This partitioning of X is based on statistical tests of the assumed DGP: the variables that clearly fail the tests are grouped as X D and the rest as X G . Then the generative and discriminative approaches are combined in a probabilistic rather than a heuristic way. The principle of the JoDiG approach is quite generic, but for illustrative purposes numerical studies of the paper focus on a widely-used case, in which the DGP assumes a multivariate normal distribution for each class. In this case, the JoDiG approach uses LDA for X G and LLR for X D . Numerical experiments on real and simulated data demonstrate that the performance of this new approach to classification is similar to or better than that of its discriminative and generative counterparts, in particular when the size of the training-set is comparable to the dimension of the data.

Fast Algorithm and Efficient Implementation of GMM-Based Pattern Classifiers

This paper proposes a fast decision algorithm in pattern classification based on Gaussian mixture models (GMM). Statistical pattern classification problems often meet a situation that comparison between probabilities is obvious and involve redundant computations. When GMM is adopted for the probability model, the exponential function should be evaluated. This work firstly reduces the exponential computations to simple and rough interval calculations. The exponential function is realized by scaling and multiplication with powers of two so that the decision is efficiently realized. For finer decision, a refinement process is also proposed. In order to verify the significance, experimental results on TI DM6437 EVM board and TED TB-3S-3400DSP-IMG board are shown through the application to a color extraction problem. It is verified that the classification was almost completed without any refinement process and the refinement process can proceed the residual decisions.

Three-Dimensional Object Reconstruction and Recognition Using Computational Integral Imaging and Statistical Pattern Analysis

In this paper, we discuss computational reconstruction and statistical pattern classification using integral imaging. Three-dimensional object information is numerically reconstructed at arbitrary depth-levels by averaging the corresponding pixels. The longitudinal distance and object boundary are estimated where the standard deviation of the intensity is minimized. Fisher linear discriminant analysis combined with principal component analysis is adopted for the classification of out-of-plane rotated objects. The Fisher linear discriminant analysis maximizes the class-discrimination while the principal component analysis minimizes the error between the original and the restored images. The presented method provides promising results for the distortion-tolerant pattern classification.

A CLT for a one-dimensional class cover problem

The central limit theorem is proved for the domination number of random class cover catch digraphs generated by uniform data in one dimension. The class cover problem is motivated by applications in statistical pattern classification.