Volume Of Training Data Research Articles

Online Non-Negative Convolutive Pattern Learning for Speech Signals

The unsupervised learning of spectro-temporal patterns within speech signals is of interest in a broad range of applications. Where patterns are non-negative and convolutive in nature, relevant learning algorithms include convolutive non-negative matrix factorization (CNMF) and its sparse alternative, convolutive non-negative sparse coding (CNSC). Both algorithms, however, place unrealistic demands on computing power and memory which prohibit their application in large scale tasks. This paper proposes a new online implementation of CNMF and CNSC which processes input data piece-by-piece and updates learned patterns gradually with accumulated statistics. The proposed approach facilitates pattern learning with huge volumes of training data that are beyond the capability of existing alternatives. We show that, with unlimited data and computing resources, the new online learning algorithm almost surely converges to a local minimum of the objective cost function. In more realistic situations, where the amount of data is large and computing power is limited, online learning tends to obtain lower empirical cost than conventional batch learning.

A toolbox for animal call recognition

Monitoring the natural environment is increasingly important as habit degradation and climate change reduce the world's biodiversity. We have developed software tools and applications to assist ecologists with the collection and analysis of acoustic data at large spatial and temporal scales. One of our key objectives is automated animal call recognition, and our approach has three novel attributes. First, we work with raw environmental audio, contaminated by noise and artefacts and containing calls that vary greatly in volume depending on the animal's proximity to the microphone. Second, initial experimentation suggested that no single recognizer could deal with the enormous variety of calls. Therefore, we developed a toolbox of generic recognizers to extract invariant features for each call type. Third, many species are cryptic and offer little data with which to train a recognizer. Many popular machine learning methods require large volumes of training and validation data and considerable time and expertise to prepare. Consequently we adopt bootstrap techniques that can be initiated with little data and refined subsequently. In this paper, we describe our recognition tools and present results for real ecological problems.

Artificial intelligence solution to transmission loss allocation problem

The problem of transmission loss allocation of deregulated power system has been solved through the application of artificial neural network (ANN). Two network structures namely Levenberg–Marquardt back propagation (LMBP) and Bayesian regularization back propagation (BRBP) have been trained and their performance compared. It has been found that LMBP network gives faster solution for same accuracy level. As the working range of power flow transaction is quite vast, a huge volume of data need to be stored and processed for the training of neural network. The time needed for training of neural network against such huge data is prohibitive for real time application of the ANN based solution tool where raw data are used for training. A simple filtering technique has been found to be very effective to improve the solution time and training data volume requirement and make the proposed technique suitable for real time applications. With the use of filtered data for training both the training network have shown comparable performance.

3D inversion of DC data using artificial neural networks

In this paper, we investigate the applicability of artificial neural networks in inverting three-dimensional DC resistivity imaging data. The model used to produce synthetic data for training the artificial neural network (ANN) system was a homogeneous medium of resistivity 100 Ωm with an embedded anomalous body of resistivity 1000 Ωm. The different sizes for anomalous body were selected and their location was changed to different positions within the homogeneous model mesh elements. The 3D data set was generated using a finite element forward modeling code through standard 3D modeling software. We investigated different learning paradigms in the training process of the neural network. Resilient propagation was more efficient than any other paradigm. We studied the effect of the data type used on neural network inversion and found that the use of location and the apparent resistivity of data points as the input and corresponding true resistivity as the output of networks produces satisfactory results. We also investigated the effect of the training data pool volume on the inversion properties. We created several synthetic data sets to study the interpolation and extrapolation properties of the ANN. The range of 100–1000 Ωm was divided into six resistivity values as the background resistivity and different resistivity values were also used for the anomalous body. Results from numerous neural network tests indicate that the neural network possesses sufficient interpolation and extrapolation abilities with the selected volume of training data. The trained network was also applied on a real field dataset, collected by a pole-pole array using a square grid (8 ×8) with a 2-m electrode spacing. The inversion results demonstrate that the trained network was able to invert three-dimensional electrical resistivity imaging data. The interpreted results of neural network also agree with the known information about the investigation area.

A Matrix Extension Built Under-Sampled Likelihood Ratio Test with Application to MUSIC Breakdown Prediction and Cure

Likelihood ratio tests are used in a range of detection-estimation problems, but normally cannot be extended to cases where training data volume T is smaller than the dimension M of the observations. We propose a non-degenerate normalized LR test that can be used for detection-estimation in such under-sampled training conditions. The LR is formed based on non-degenerate band extension of the original degenerate sample covariance matrix. This LR is then applied within a generalized likeli- hood ratio test framework to an array processing problem where the presence of closely spaced signal can be robustly detected, but their individual directions of arrival cannot be fully resolved by subspace-based DOA techniques such as MUSIC. In that case, MUSIC produces direction of arrival estimates for some sources with very large errors (outliers). We use the under-sampled likelihood ratio to detect the presence of such MUSIC outliers and provide corrected DOA estimates.

Bayesian Network Learning with Parameter Constraints

The task of learning models for many real-world problems requires incorporating domain knowledge into learning algorithms, to enable accurate learning from a realistic volume of training data. This paper considers a variety of types of domain knowledge for constraining parameter estimates when learning Bayesian networks. In particular, we consider domain knowledge that constrains the values or relationships among subsets of parameters in a Bayesian network with known structure. We incorporate a wide variety of parameter constraints into learning procedures for Bayesian networks, by formulating this task as a constrained optimization problem. The assumptions made in module networks, dynamic Bayes nets and context specific independence models can be viewed as particular cases of such parameter constraints. We present closed form solutions or fast iterative algorithms for estimating parameters subject to several specific classes of parameter constraints, including equalities and inequalities among parameters, constraints on individual parameters, and constraints on sums and ratios of parameters, for discrete and continuous variables. Our methods cover learning from both frequentist and Bayesian points of view, from both complete and incomplete data. We present formal guarantees for our estimators, as well as methods for automatically learning useful parameter constraints from data. To validate our approach, we apply it to the domain of fMRI brain image analysis. Here we demonstrate the ability of our system to first learn useful relationships among parameters, and then to use them to constrain the training of the Bayesian network, resulting in improved cross-validated accuracy of the learned model. Experiments on synthetic data are also presented.

Practical issues in the application of speech technology to network and customer service applications

This paper proposes a simple model to characterise the different stages of short telephone transactions. It also discusses the impact of the context of the caller when entering an automated service. Three different styles of service were then identified, namely, large vocabulary information gathering, spoken language command and natural language task identification for helpdesks. By considering human dialogue equivalents, the requirements for each style are considered. Consequently, it is shown that each style pushes different technological limits. Three case studies, selected from current project from BT laboratories, are presented to highlight the practical design issues in these different styles. The styles and case studies presented are: • Information gathering – UK name and address recognition. • Spoken language command – network service configuration. • Natural language helpdesks – BT operator services. It is shown that large vocabulary information gathering systems require high accuracy, careful data modelling and well-designed strategies to boost confidence and accuracy. Spoken language command requires dialogue and grammar design and test complexity to be managed. Natural language task identification requires large volumes of training data, good learning algorithms and good data generalisation techniques. These styles can be mixed into a single interaction meaning that design frameworks of the future will have to address all of the aspects of the different interaction styles.

Speaker adaptation of acoustic models using correlations of training transfer vectors

The authors proposed for acoustic models based on the hidden Markov model a method that involves applying constraints to the model structure and tying the model's parameters in order to improve the training efficiency. Conventionally, the tied structure of an acoustic model is, mostly, defined by tying several adjacent parameters and expressing them with a single representative parameter. This method can be regarded as a tying method based on the parameters' values, under the assumption that adjacent parameters, usually, exhibit similar behavior. As opposed to this concept, the current study proposes a tied structure with consideration of transfer (movement) of parameters. A large volume of training data was used to measure transfer of each parameter during training, and tying relationships regarding the transfer vectors were organized between parameters performing statistically similar movements. In particular, in the current study, the authors concentrated on mean vectors of fundamental distributions and followed movements of these mean vectors during training of initial models (speaker-independent models) by acoustic data from different speakers. The structure was defined by identifying the mean vectors characterized by strong correlation of movements during training, and tying their corresponding transfer vectors. Speaker adaptation tests confirmed high training efficiency of the model obtained as a result of this tying. © 2000 Scripta Technica, Syst Comp Jpn, 31(14): 74–82, 2000

Abstracting Uncertain Knowledge: Case for Neural Nets Application

ABSTRACTNeural networks have been used successfully in practical applications where human expertise exists but no clear rules are known. There is a more difficult case when the acquisition of labelled data points is very expensive, such as in labelling of ground data to match satellite images in geographic information systems.In fact, the dependence of neural networks on large volumes of training data result in the neural solution, producing more inconsistent results over a number of trials using the same data, but different initialisations of the weights.We present our method of generating IF-THEN rules expressing the trained neural network's behaviour. By using the causal index on characteristic input patterns, we produce a list of inputs which were significant in reaching the decision made and a well-ordered sequence of rules governing this decision. This method correctly produced rules for 94% of the decisions made by a sample network.The principle of selecting the next most likely decision (that the ...

Neural networks for the classification of image texture

In this paper, an extensive study of the efficiency of secondary features extracted from co-occurrence matrices and then direct use of co-occurrence matrices for the classification of texture images is carried out. First, the classification capabilities of individual, and then several combinations of, secondary features extracted from the co-occurrence matrices are examined. An adequate combination consisting of four of those features is hence established. A feed-forward artificial neural network is used as a classifier. The classification performance of the network as a function of the training data volume, its size and the training strategy are investigated. Next, an investigation is made into the direct use of co-occurrence matrix entries for classification. A simple technique, where the gray levels in the original images are grouped into a small number of equal bands, is employed in order to reduce the size of the matrix. It is shown that results obtained here compare favorably with those obtained earlier using the set of secondary features. The efficiency of the last technique is further demonstrated by means of considering a practical application. This involves the surface texture classification of wear particles found in the lubricating oil of moving machinery and engines. Again, it is shown that high recognition rates can be achieved when dealing with such classification tasks.