Large Number Of Training Examples Research Articles

Pruned fuzzy K-nearest neighbor classifier for beat classification

Arrhythmia beat classification is an active area of research in ECG based clinical decision support systems. In this paper, Pruned Fuzzy K-nearest neighbor (PFKNN) classifier is proposed to classify six types of beats present in the MIT-BIH Arrhythmia database. We have tested our classifier on ~ 103100 beats for six beat types present in the database. Fuzzy KNN (FKNN) can be implemented very easily but large number of training examples used for classification can be very time consuming and requires large storage space. Hence, we have proposed a time efficient Arif-Fayyaz pruning algorithm especially suitable for FKNN which can maintain good classification accuracy with appropriate retained ratio of training data. By using Arif-Fayyaz pruning algorithm with Fuzzy KNN, we have achieved a beat classification accuracy of 97% and geometric mean of sensitivity of 94.5% with only 19% of the total training examples. The accuracy and sensitivity is comparable to FKNN when all the training data is used. Principal Component Analysis is used to further reduce the dimension of feature space from eleven to six without compromising the accuracy and sensitivity. PFKNN was found to robust against noise present in the ECG data.

A Cooperative Learning Scheme for Interactive Video Search

The main idea of an interactive search is to gradually improve search quality of retrieval system via user interaction. While a large amount of work has been made in the past, most of the existing approaches typically require labeling effort for updating the query model. Unfortunately, it is time-consuming and tedious to label a large number of training examples. We aim to develop a novel text-driven cooperative learning scheme, which can offer users a quite natural query fashion and alleviate significantly the burden on users without compromising search performance. Starting with an advanced text-driven video search engine, a multi-view cooperative training strategy is proposed for learning from feedback data a refined ranking function. The main merit of proposed framework is its ability in mining training samples automatically from previous answer set and implicitly combining multiple modalities for effectively learning users' query intent. Evaluation on TRECVID' 06 video corpus shows that the proposed scheme with few training seeds achieves a comparable performance with classic interactive schemes.

Constructing Multilayer Feedforward Neural Networks to Approximate Nonlinear Functions in Engineering Mechanics Applications

Abstract This paper presents a major step in the development and validation of a systematic prototype-based methodology for designing multilayer feedforward neural networks to model nonlinearities common in engineering mechanics. The applications of this work include (but are not limited to) system identification of nonlinear dynamic systems and neural-network-based damage detection. In this and previous studies (Pei, J. S., 2001, “Parametric and Nonparametric Identification of Nonlinear Systems,” Ph.D. thesis, Columbia University; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part I: Formulation,” J. Eng. Mech., 132(12), pp. 1290–1300; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part II: Applications,” J. Eng. Mech., 132(12), pp. 1301–1312; Pei, J. S., Wright, J. P., and Smyth, A. W., 2005, “Mapping Polynomial Fitting Into Feedforward Neural Networks for Modeling Nonlinear Dynamic Systems and Beyond,” Comput. Methods Appl. Mech. Eng., 194(42–44), pp. 4481–4505), the authors do not presume to provide a universal method to approximate any arbitrary function. Rather the focus is given to the development of a procedure which will consistently lead to successful approximations of nonlinear functions within the specified field. This is done by examining the dominant features of the function to be approximated and exploiting the strength of the sigmoidal basis function. As a result, a greater efficiency and understanding of both neural network architecture (e.g., the number of hidden nodes) as well as weight and bias values is achieved. Through the use of illuminating mathematical insights and a large number of training examples, this study demonstrates the simplicity, power, and versatility of the proposed prototype-based initialization methodology. A clear procedure for initializing neural networks to model various nonlinear functions commonly seen in engineering mechanics is provided. The proposed methodology is compared with the widely used Nguyen–Widrow initialization to demonstrate its robustness and efficiency in the specified applications. Future work is also identified.

Protein–protein docking benchmark version 3.0

We present version 3.0 of our publicly available protein-protein docking benchmark. This update includes 40 new test cases, representing a 48% increase from Benchmark 2.0. For all of the new cases, the crystal structures of both binding partners are available. As with Benchmark 2.0, Structural Classification of Proteins (Murzin et al., J Mol Biol 1995;247:536-540) was used to remove redundant test cases. The 124 unbound-unbound test cases in Benchmark 3.0 are classified into 88 rigid-body cases, 19 medium-difficulty cases, and 17 difficult cases, based on the degree of conformational change at the interface upon complex formation. In addition to providing the community with more test cases for evaluating docking methods, the expansion of Benchmark 3.0 will facilitate the development of new algorithms that require a large number of training examples. Benchmark 3.0 is available to the public at http://zlab.bu.edu/benchmark.

Iterative kernel principal component analysis for image modeling

In recent years, Kernel Principal Component Analysis (KPCA) has been suggested for various image processing tasks requiring an image model such as, e.g., denoising or compression. The original form of KPCA, however, can be only applied to strongly restricted image classes due to the limited number of training examples that can be processed. We therefore propose a new iterative method for performing KPCA, the Kernel Hebbian Algorithm which iteratively estimates the Kernel Principal Components with only linear order memory complexity. In our experiments, we compute models for complex image classes such as faces and natural images which require a large number of training examples. The resulting image models are tested in single-frame super-resolution and denoising applications. The KPCA model is not specifically tailored to these tasks; in fact, the same model can be used in super-resolution with variable input resolution, or denoising with unknown noise characteristics. In spite of this, both super-resolution and denoising performance are comparable to existing methods.

Asymptotic statistical theory of overtraining and cross-validation

A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Based on the cross-validation stopping we consider the ratio the examples should be divided into training and cross-validation sets in order to obtain the optimum performance. Although cross-validated early stopping is useless in the asymptotic region, it surely decreases the generalization error in the nonasymptotic region. Our large scale simulations done on a CM5 are in good agreement with our analytical findings.

Explanation-based generalization: A unifying view

The problem of formulating general concepts from specific training examples has long been a major focus of machine learning research. While most previous research has focused on empirical methods for generalizing from a large number of training examples using no domain-specific knowledge, in the past few years new methods have been developed for applying domain-specific knowledge to for-mulate valid generalizations from single training examples. The characteristic common to these methods is that their ability to generalize from a single example follows from their ability to explain why the training example is a member of the concept being learned. This paper proposes a general, domain-independent mechanism, called EBG, that unifies previous approaches to explanation-based generalization. The EBG method is illustrated in the context of several example problems, and used to contrast several existing systems for explanation-based generalization. The perspective on explanation-based generalization afforded by this general method is also used to identify open research problems in this area.

Explanation-based generalization: a unifying view

The problem of formulating general concepts from specific training examples has long been a major focus of machine learning research. While most previous research has focused on empirical methods for generalizing from a large number of training examples using no domain-specific knowledge, in the past few years new methods have been developed for applying domain-specific knowledge to formulate valid generalizations from single training examples. The characteristic common to these methods is that their ability to generalize from a single example follows from their ability to explain why the training example is a member of the concept being learned. This paper proposes a general, domain-independent mechanism, called EBG, that unifies previous approaches to explanation-based generalization. The EBG method is illustrated in the context of several example problems, and used to contrast several existing systems for explanation-based generalization. The perspective on explanation-based generalization afforded by this general method is also used to identify open research problems in this area.