Modified Quadratic Discriminant Function Research Articles

Compact MQDF classifiers using sparse coding for handwritten Chinese character recognition

The modified quadratic discriminant function (MQDF) is an effective classifier for handwritten Chinese character recognition (HCCR). However, it suffers from high memory requirement for the storage of its parameters, which makes it impractical to be embedded in memory limited hand-held devices. In this paper, we explore the applicability of sparse coding to build compact MQDF classifiers. To be specific, we use sparse coding to compact the parameters of MQDF. Two methods of sparse coding, viz., the maximum likelihood-based method and the K-SVD method, are adopted to build two compact MQDF classifiers, namely, MQDF-ML classifier and MQDF-KSVD classifier. Furthermore, we learn multiple dictionaries rather than single dictionary for sparse coding, because the multiple dictionary learning is capable of not only greatly reducing the computational complexity, but also alleviating the degradation of recognition accuracy, compared to the single dictionary learning. Experiments and comparison with the existing method have demonstrated the effectiveness of our proposed method for the issue of unconstrained handwritten Chinese character recognition.

Discriminative quadratic feature learning for handwritten Chinese character recognition

In this paper, we propose a feature learning method for handwritten Chinese character recognition (HCCR), called discriminative quadratic feature learning (DQFL). Based on original gradient direction feature representation, quadratic correlation between features is used to promote the feature dimensionality, then discriminative feature extraction (DFE) is used for dimensionality reduction. By combining dimensionality promotion and reduction, we can learn a much more discriminative and nonlinear feature representation, which can then boost the classification accuracy significantly. For dimensionality promotion, two types of correlation are exploited, namely, statistical correlation and spatial correlation. Statistical correlation is computed on multiple local feature vectors in different regions of the character image; while spatial correlation encodes the dependency between features of two positions. Feature correlation increases the dimensionality by over 40,000. DFE then reduces the dimensionality to less than 300 without losing discriminability. Classification is performed using nearest prototype classifier (NPC), modified quadratic discriminant function (MQDF) and discriminative learning quadratic discriminant function (DLQDF). In experiments on the CASIA-HWDB1.1 standard dataset, the proposed DQFL method improves the test accuracies of NPC, MQDF and DLQDF by 4.94%, 1.83%, and 1.82%, respectively. The test accuracy is further improved by training set expansion. On the ICDAR 2013 Chinese handwriting recognition competition dataset, the proposed DQFL+DLQDF classifier outperforms the best participating system based on deep convolutional neural network (CNN), while the test speed is much faster.

Importance sampling based discriminative learning for large scale offline handwritten Chinese character recognition

The development of a discriminative learning framework based on importance sampling for large-scale classification tasks is reported in this paper. The framework involves the assignment of samples with different weights according to the sample importance weight function derived from the Bayesian classification rule. Three methods are used to calculate the sample importance weights for learning the modified quadratic discriminant function (MQDF). (1) Rejection sampling method. The method selects important samples as a training subset and trains different levels of MQDFs by focusing on different types of samples. (2) Boosting algorithm. The algorithm modifies the sample importance weights iteratively according to the recognition performance. (3) Minimum classification error (MCE) rule. The parameter of the importance weight function is estimated using the MCE rule. In general, the cursive samples are usually misclassified or prone to be misclassified by the MQDF learned under the maximum likelihood estimation (MLE) rule. The proposed importance sampling framework thereby makes the MQDF classifier focus more on cursive samples than on normal samples. Such a strategy allows the MQDF to achieve higher accuracy while maintaining lower computational complexity. Comprehensive experiments on three Chinese handwritten character datasets demonstrated that the proposed framework exhibits promising character recognition accuracy.

Principal component analysis‐based learning for preceding vehicle classification

This study presents a new scheme for cluster generation and classification of preceding vehicles from images. The proposed clustering algorithm models the distribution of vehicle images using ‘vehicle’ clusters. ‘Non-vehicle’ clusters are generated by modelling the distribution of non-vehicle images. The clusters are created using K-means clustering algorithm. Hierarchically related nested eigenspaces are acquired to reassign the patterns of each cluster. An appropriate classifier is obtained to classify the vehicles based on the ‘distance-from-feature-space’ measurement. The eigenspaces of vehicle clusters together with non-vehicle clusters are used for classification. The approach of modelling the distribution of vehicle and non-vehicle images and the choice of the classifier used are investigated through experiments thoroughly. Comparison on the performance of the proposed scheme is made with that of MultiClustered Modified Quadratic Discriminant Function approach of categorising the preceding vehicles. The superior performance of the proposed scheme is clearly illustrated through the classification results.

Writer Adaptation with Style Transfer Mapping

Adapting a writer-independent classifier toward the unique handwriting style of a particular writer has the potential to significantly increase accuracy for personalized handwriting recognition. This paper proposes a novel framework of style transfer mapping (STM) for writer adaptation. The STM is a writer-specific class-independent feature transformation which has a closed-form solution. After style transfer mapping, the data of different writers are projected onto a style-free space, where the writer-independent classifier needs no change to classify the transformed data and can achieve significantly higher accuracy. The framework of STM can be combined with different types of classifiers for supervised, unsupervised, and semi-supervised adaptation, where writer-specific data can be either labeled or unlabeled and need not cover all classes. In this paper, we combine STM with the state-of-the-art classifiers for large-category Chinese handwriting recognition: learning vector quantization (LVQ) and modified quadratic discriminant function (MQDF). Experiments on the online Chinese handwriting database CASIA-OLHWDB demonstrate that STM-based adaptation is very efficient and effective in improving classification accuracy. Semi-supervised adaptation achieves the best performance, while unsupervised adaptation is even better than supervised adaptation. On handwritten text data, semi-supervised adaptation achieves error reduction rates 31.95 and 25.00 percent by LVQ and MQDF, respectively.

Online and offline handwritten Chinese character recognition: Benchmarking on new databases

Recently, the Institute of Automation of Chinese Academy of Sciences (CASIA) released the unconstrained online and offline Chinese handwriting databases CASIA-OLHWDB and CASIA-HWDB, which contain isolated character samples and handwritten texts produced by 1020 writers. This paper presents our benchmarking results using state-of-the-art methods on the isolated character datasets OLHWDB1.0 and HWDB1.0 (called DB1.0 in general), OLHWDB1.1 and HWDB1.1 (called DB1.1 in general). The DB1.1 covers 3755 Chinese character classes as in the level-1 set of GB2312-80. The evaluated methods include 1D and pseudo 2D normalization methods, gradient direction feature extraction from binary images and from gray-scale images, online stroke direction feature extraction from pen-down trajectory and from pen lifts, classification using the modified quadratic discriminant function (MQDF), discriminative feature extraction (DFE), and discriminative learning quadratic discriminant function (DLQDF). Our experiments reported the highest test accuracies 89.55% and 93.22% on the HWDB1.1 (offline) and OLHWDB1.1 (online), respectively, when using the MQDF classifier trained with DB1.1. When training with both the DB1.0 and DB1.1, the test accuracies on HWDB1.1 and OLHWDB are improved to 90.71% and 93.95%, respectively. Using DFE and DLQDF, the best results on HWDB1.1 and OLHWDB1.1 are 92.08% and 94.85%, respectively. Our results are comparable to the best results of the ICDAR2011 Chinese Handwriting Recognition Competition though we used less training samples.

Building compact recognizers of handwritten Chinese characters using precision constrained Gaussian model, minimum classification error training and parameter compression

In our previous work, a so-called precision constrained Gaussian model (PCGM) was proposed for character modeling to design compact recognizers of handwritten Chinese characters. A maximum likelihood training procedure was developed to estimate model parameters from training data. In this paper, we extend the above-mentioned work by using minimum classification error (MCE) training to improve recognition accuracy and using both split vector quantization and scalar quantization techniques to further compress model parameters. Experimental results on a handwritten character recognition task with a vocabulary of 2,965 Kanji characters demonstrate that MCE-trained and compressed PCGM-based classifiers can achieve much higher recognition accuracies than their counterparts based on traditional modified quadratic discriminant function (MQDF) when the footprint of the classifiers has to be made very small, e.g., less than 2 MB.

Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship model

Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship modelThis paper presents novel feature extraction and classification methods for online handwritten Chinese character recognition (HCCR). TheX-graph andY-graph transformation is proposed for deriving a feature, which shows useful properties such as invariance to different writing styles. Central to the proposed method is the idea of capturing the geometrical and topological information from the trajectory of the handwritten character using theX-graph and theY-graph. For feature size reduction, the Haar wavelet transformation was applied on the graphs. For classification, the coefficient of determination (R2p) from the two-dimensional unreplicated linear functional relationship model is proposed as a similarity measure. The proposed methods show strong discrimination power when handling problems related to size, position and slant variation, stroke shape deformation, close resemblance of characters, and non-normalization. The proposed recognition system is applied to a database with 3000 frequently used Chinese characters, yielding a high recognition rate of 97.4% with reduced processing time of 75.31%, 73.05%, 58.27% and 40.69% when compared with recognition systems using the city block distance with deviation (CBDD), the minimum distance (MD), the compound Mahalanobis function (CMF) and the modified quadratic discriminant function (MQDF), respectively. High precision rates were also achieved.

Chinese minority script identification method based on wavelet feature and MQDF

In order to classify the type of the Chinese minority scripts,the method of identifying the kinds of Chinese minority scripts based on wavelet analysis and Modified Quadratic Discriminant Function (MQDF) was presented. Using wavelet energy and wavelet energy distribution proportion as features by wavelet multi-resolution transform,multivariate classifier in MQDF was constructed. A sample data set was built which contained six common Chinese minority scripts: Tibetan,Tai Lue,Naxi Pictographs,Uighur,Tai Le,Yi and Chinese and English in total,some samples were used for training,others were for testing,and the proportions of the training samples in dataset were variant. Obviously,the experimental result shows that,in multi-level decomposition,the method is better than the traditional Bayes and K-Nearest Neighbor (KNN) classification in recognition rate.

A Lexicon-Driven Handwritten City-Name Recognition Scheme for Indian Postal Automation

A lexicon-driven segmentation-recognition scheme on Bangla handwritten city-name recognition is proposed for Indian postal automation. In the proposed scheme, at first, binarization of the input document is done and then to take care of slanted handwriting of different individuals a slant correction technique is performed. Next, due to the script characteristics of Bangla, a water reservoir concept is applied to pre-segment the slant corrected city-names into possible primitive components(characters or its parts). Pre-segmented components of a city-name are then merged into possible characters to get the best city-name using the lexicon information. In order to merge these primitive components into characters and to find optimum character segmentation, dynamic programming (DP) is applied using total likelihood of the characters of a city-name as an objective function. To compute the likelihood of a character, Modified Quadratic Discriminant Function (MQDF) is used. The features used in the MQDF are mainly based on the directional features of the contour points of the components. We tested our system on 84 different Bangla city-names and 94.08% accuracy was obtained from the proposed system.

Modeling inverse covariance matrices by expansion of tied basis matrices for online handwritten Chinese character recognition

The state-of-the-art modified quadratic discriminant function (MQDF) based approach for online handwritten Chinese character recognition (HCCR) assumes that the feature vectors of each character class can be modeled by a Gaussian distribution with a mean vector and a full covariance matrix. In order to achieve a high recognition accuracy, enough number of leading eigenvectors of the covariance matrix have to be retained in MQDF. This paper presents a new approach to modeling each inverse covariance matrix by basis expansion, where expansion coefficients are character-dependent while a common set of basis matrices are shared by all the character classes. Consequently, our approach can achieve a much better accuracy–memory tradeoff. The usefulness of the proposed approach to designing compact HCCR systems has been confirmed and demonstrated by comparative experiments on popular Nakayosi and Kuchibue Japanese character databases.

High accuracy handwritten Chinese character recognition using LDA-based compound distances

To improve the accuracy of handwritten Chinese character recognition (HCCR), we propose linear discriminant analysis (LDA)-based compound distances for discriminating similar characters. The LDA-based method is an extension of previous compound Mahalanobis function (CMF), which calculates a complementary distance on a one-dimensional subspace (discriminant vector) for discriminating two classes and combines this complementary distance with a baseline quadratic classifier. We use LDA to estimate the discriminant vector for better discriminability and show that under restrictive assumptions, the CMF is a special case of our LDA-based method. Further improvements can be obtained when the discriminant vector is estimated from higher-dimensional feature spaces. We evaluated the methods in experiments on the ETL9B and CASIA databases using the modified quadratic discriminant function (MQDF) as baseline classifier. The results demonstrate the superiority of LDA-based method over the CMF and the superiority of discriminant vector learning from high-dimensional feature spaces. Compared to the MQDF, the proposed method reduces the error rates by factors of over 26%.

Building compact MQDF classifier for large character set recognition by subspace distribution sharing

Quadratic classifier with modified quadratic discriminant function (MQDF) has been successfully applied to recognition of handwritten characters to achieve very good performance. However, for large category classification problem such as Chinese character recognition, the storage of the parameters for the MQDF classifier is usually too large to make it practical to be embedded in the memory limited hand-held devices. In this paper, we aim at building a compact and high accuracy MQDF classifier for these embedded systems. A method by combining linear discriminant analysis and subspace distribution sharing is proposed to greatly compress the storage of the MQDF classifier from 76.4 to 2.06 MB, while the recognition accuracy still remains above 97%, with only 0.88% accuracy loss. Furthermore, a two-level minimum distance classifier is employed to accelerate the recognition process. Fast recognition speed and compact dictionary size make the high accuracy quadratic classifier become practical for hand-held devices.

DISCRIMINATIVE TRAINING BASED QUADRATIC CLASSIFIER FOR HANDWRITTEN CHARACTER RECOGNITION

In offline handwritten character recognition, the classifier with modified quadratic discriminant function (MQDF) has achieved good performance. The parameters of MQDF classifier are commonly estimated by the maximum likelihood (ML) estimator, which maximizes the within-class likelihood instead of directly minimizing the classification errors. To improve the performance of MQDF classifier, in this paper, the MQDF parameters are revised by discriminative training using a minimum classification error (MCE) criterion. The proposed algorithm is applied to recognizing handwritten numerals and handwritten Chinese characters, the recognition rates obtained are among the highest that have ever been reported.

Discriminative learning quadratic discriminant function for handwriting recognition.

In character string recognition integrating segmentation and classification, high classification accuracy and resistance to noncharacters are desired to the underlying classifier. In a previous evaluation study, the modified quadratic discriminant function (MQDF) proposed by Kimura et al. was shown to be superior in noncharacter resistance but inferior in classification accuracy to neural networks. This paper proposes a discriminative learning algorithm to optimize the parameters of MQDF with aim to improve the classification accuracy while preserving the superior noncharacter resistance. We refer to the resulting classifier as discriminative learning QDF (DLQDF). The parameters of DLQDF adhere to the structure of MQDF under the Gaussian density assumption and are optimized under the minimum classification error (MCE) criterion. The promise of DLQDF is justified in handwritten digit recognition and numeral string recognition, where the performance of DLQDF is comparable to or superior to that of neural classifiers. The results are also competitive to the best ones reported in the literature.