Number Of Training Instances Research Articles

Improving English verb sense disambiguation performance with linguistically motivated features and clear sense distinction boundaries

This paper presents a high-performance broad-coverage supervised word sense disambiguation (WSD) system for English verbs that uses linguistically motivated features and a smoothed maximum entropy machine learning model. We describe three specific enhancements to our system’s treatment of linguistically motivated features which resulted in the best published results on SENSEVAL-2 verbs. We then present the results of training our system on OntoNotes data, both the SemEval-2007 task and additional data. OntoNotes data is designed to provide clear sense distinctions, based on using explicit syntactic and semantic criteria to group WordNet senses, with sufficient examples to constitute high quality, broad coverage training data. Using similar syntactic and semantic features for WSD, we achieve performance comparable to that of human taggers, and competitive with the top results for the SemEval-2007 task. Empirical analysis of our results suggests that clarifying sense boundaries and/or increasing the number of training instances for certain verbs could further improve system performance.

Discretization for naive-Bayes learning: managing discretization bias and variance

Quantitative attributes are usually discretized in Naive-Bayes learning. We establish simple conditions under which discretization is equivalent to use of the true probability density function during naive-Bayes learning. The use of different discretization techniques can be expected to affect the classification bias and variance of generated naive-Bayes classifiers, effects we name discretization bias and variance. We argue that by properly managing discretization bias and variance, we can effectively reduce naive-Bayes classification error. In particular, we supply insights into managing discretization bias and variance by adjusting the number of intervals and the number of training instances contained in each interval. We accordingly propose proportional discretization and fixed frequency discretization, two efficient unsupervised discretization methods that are able to effectively manage discretization bias and variance. We evaluate our new techniques against four key discretization methods for naive-Bayes classifiers. The experimental results support our theoretical analyses by showing that with statistically significant frequency, naive-Bayes classifiers trained on data discretized by our new methods are able to achieve lower classification error than those trained on data discretized by current established discretization methods.

A novel gray-based reduced NN classification method

In pattern recognition, instance-based learning (also known as nearest neighbor rule) has become increasingly popular and can yield excellent performance. In instance-based learning, however, the storage of training set rises along with the number of training instances. Moreover, in such a case, a new, unseen instance takes a long time to classify because all training instances have to be considered when determining the ‘nearness’ or ‘similarity’ among instances. This study presents a novel reduced classification method for instance-based learning based on the gray relational structure. Here, only some training instances in the original training set are adopted for the pattern classification tasks. The relationships among instances are first determined according to the gray relational structure. In the relational structure, the inward edges of each training instance, indicating how many times each instance is considered as the nearest neighbor or neighbors in determining the class labels of other instances can be obtained. This method excludes training instances with no or few inward edges for the pattern classification tasks. By using the proposed instance pruning approach, new instances can be classified with a few training instances. Nine data sets are adopted to demonstrate the performance of the proposed learning approach. Experimental results indicate that the classification accuracy can be maintained when most of the training instances are pruned before learning. Additionally, the number of remained training instances in the proposal presented here is comparable to that of other existing instance pruning techniques.

PARALLEL NEURAL LEARNING FOR CONTROL PROBLEMS ON A BUS-BASED ARCHITECTURE

In [6], we distributed training instances over a single-channel broadcast communication model to speed up execution of the back-propagation learning algorithm for classification problems. In this paper, we extend this concept to control problems, where the output is not necessarily 0 or 1, but ranges over an interval. We first propose a modified back-propagation learning algorithm that incrementally decreases the error threshold by half in order to process training instances with large weight changes as quickly as possible. This modified back-propagation learning algorithm is then parallelized using the single-channel broadcast communication model to n processors, where n is the number of training instances. Finally, the parallel back-propagation learning algorithm is modified for execution on a bounded number of processors to cope with real-world conditions.

A Nearly Optimal Back-Propagation Learning Algorithm on a Bus-Based Architecture

The back-propagation model is applied mainly to multiple layers of neurons with feed-forward connections. It applies the generalized delta learning rule to derive appropriate weights on connections from a set of training instances. In the past, a parallel back-propagation learning algorithm on a bus-based architecture with an infinite number of processors was proposed to speed up the learning of weights. In this paper, the parallel back-propagation learning algorithm is modified, so that it is applicable to any number of processors. The given training instances are equally partitioned and put on these processors. When learning starts, each processor searches its own training subset to update the weight matrix and to broadcast the new result. Both analytical and experimental results show that the average speed-up can reach nearly O(r) by r processors if r is much less than the number of training instances.

IGTree: using trees for compression and classification in lazy learning algorithms

We describe the IGTree learning algorithm, which compresses an instance base into a tree structure. The concept of information gain is used as a heuristic function for performing this compression. IGTree produces trees that, compared to other lazy learning approaches, reduce storage requirements and the time required to compute classifications. Furthermore, we obtained similar or better generalization accuracy with IGTree when trained on two complex linguistic tasks, viz. letter–phoneme transliteration and part-of-speech-tagging, when compared to alternative lazy learning and decision tree approaches (viz., IB1, information-gain-weighted IB1, and C4.5). A third experiment, with the task of word hyphenation, demonstrates that when the mutual differences in information gain of features is too small, IGTree as well as information-gain-weighted IB1 perform worse than IB1. These results indicate that IGTree is a useful algorithm for problems characterized by the availability of a large number of training instances described by symbolic features with sufficiently differing information gain values.

An efficient inductive learning method for object-oriented database using attribute entropy

The data-driven characteristic of the Version Space rule-learning method works efficiently in memory even if the training set is enormous. However, the concept hierarchy of each attribute used to generalize/specialize the hypothesis of a specific/general (S/G) set is processed sequentially and instance by instance, which degrades its performance. As for ID3, the decision tree is generated from the order of attributes according to their entropies to reduce the number of attributes in some of the tree paths. Unlike Version Space, ID3 generates an extremely complex decision tree when the training set is enormous. Therefore, we propose a method called AGE (A_RCH+OG_L+ASE_, where ARCH=Automatic geneRation of Concept Hierarchies, OGL=Optimal Generalization Level, and ASE=Attribute Selection by Entropy), taking advantages of Version Space and ID3 to learn rules from object-oriented databases (OODBs) with the least number of learning features according to the entropy. By simulations, we found the performance of our learning algorithm is better than both Version Space and ID3. Furthermore, AGE's time complexity and space complexity are both linear with the number of training instances.

Learning capacity and sample complexity on expert networks

A major development in knowledge-based neural networks is the integration of symbolic expert rule-based knowledge into neural networks, resulting in so-called rule-based neural (or connectionist) networks. An expert network here refers to a particular construct in which the uncertainty management model of symbolic expert systems is mapped into the activation function of the neural network. This paper addresses a yet-to-be-answered question: Why can expert networks generalize more effectively from a finite number of training instances than multilayered perceptrons? It formally shows that expert networks reduce generalization dimensionality and require relatively small sample sizes for correct generalization.

Learning concepts in parallel based upon the strategy of version space

Applies the technique of parallel processing to concept learning. A parallel version-space learning algorithm based upon the principle of divide-and-conquer is proposed. Its time complexity is analyzed to be O(k log/sub 2/n) with n processors, where n is the number of given training instances and k is a coefficient depending on the application domains. For a bounded number of processors in real situations, a modified parallel learning algorithm is then proposed. Experimental results are then performed on a real learning problem, showing that our parallel learning algorithm works, and being quite consistent with the results of theoretical analysis. We conclude that when the number of training instances is large, it is worth learning in parallel because of its faster execution.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Parallel perceptron learning on a single-channel broadcast communication model

A parallel perceptron learning algorithm based upon a single-channel broadcast communication model has been proposed here. Since it can process training instances in parallel, instead of one by one in the conventional algorithm, large speedup can be expected. Theoretical analysis shows: with n processors, the average speedup ranges from O(log n) to O( n) under a variety of assumptions (where n is the number of training instances). Experimental results further show the actual average speedup is approximately being O( n 0.91/log n). Extension to a bounded number of processors and to the backpropagtion learning have also been discussed.