Welch Algorithm Research Articles

Blind equalization of multilevel turbo coded‐continuous phase frequency shift keying over MIMO channels

AbstractIn this paper, multilevel turbo coded‐continuous phase frequency shift keying (MLTC‐CPFSK) is introduced and its bit error performance in multiple input multiple output (MIMO) fading channels are investigated considering a blind maximum likelihood channel estimation. Multilevel turbo coded signals with continuous phase modulation (CPM) provides low spectral occupancy and is suitable for power and bandwidth‐limited channels. Besides, antenna diversity is one of the best method to combat with multipath fading effects. The performance of 2LTC for 4‐ary CPFSK over AWGN, Rician (for Rician channel parameter K=10 dB) and Rayleigh channels are given for 1Tx–1Rx, 2Tx–1Rx and 2Tx–2Rx antenna configurations. Channel capacities of 2LTC‐4CPFSK signals are obtained as −5.26, −7.65 and −7.14 dB for 1Tx–1Rx, 2Tx–1Rx and 2Tx–2Rx antenna configurations, respectively. Baum–Welch (BW) algorithm is used to estimate the channel parameters. Bit error probabilities of 2 level turbo coded 4 CPFSK (2LTC‐4CPFSK) are drawn in the cases of no channel state information (CSI), BW estimation, and perfect CSI. Approximately 0.1 and 0.75 dB gains in Es/N0 are obtained using BW channel estimator for Rician and Rayleigh channels, respectively. Therefore, MLTC‐CPFSK with BW channel estimator has excellent performance in MIMO fading channels. Copyright © 2006 John Wiley & Sons, Ltd.

Lloyd clustering of Gauss mixture models for image compression and classification

Gauss mixtures have gained popularity in statistics and statistical signal processing applications for a variety of reasons, including their ability to well approximate a large class of interesting densities and the availability of algorithms such as the Baum–Welch or expectation-maximization (EM) algorithm for constructing the models based on observed data. We here consider a quantization approach to Gauss mixture design based on the information theoretic view of Gaussian sources as a “worst case” for robust signal compression. Results in high-rate quantization theory suggest distortion measures suitable for Lloyd clustering of Gaussian components based on a training set of data. The approach provides a Gauss mixture model and an associated Gauss mixture vector quantizer which is locally robust. We describe the quantizer mismatch distortion and its relation to other distortion measures including the traditional squared error, the Kullback–Leibler (relative entropy) and minimum discrimination information, and the log-likehood distortions. The resulting Lloyd clustering algorithm is demonstrated by applications to image vector quantization, texture classification, and North Atlantic pipeline image classification.

Effects of RR segment duration on HRV spectrum estimation

Although patterns of heart rate variability (HRV) hold considerable promise for clarifying issues in clinical applications, the inappropriate quantification and interpretation of these patterns may obscure critical issues or relationships and may impede rather than foster the development of clinical applications. The duration of the RR interval series is not a matter of convenience but a fine balance between two important issues: acceptable variance and stationarity of the time series on one hand, and acceptable resolution of the spectral estimate and reduced spectral leakage on the other. Further, in the standard short-term HRV analysis, it has been observed that the previous studies in HRV spectral analysis use a wide range of RR interval segment duration for spectral estimation by Welch's algorithm. The standardization of RR interval segment duration is also important for comparisons among studies and is essential for within-study experimental contrasts. In the present study, a comparative analysis for RR interval segment durations has been made to propose an optimal RR interval segment duration. Firstly a simulated signal was analyzed with Hann window and zero padding for the segment lengths of 1024, 512, 256 and 128 samples resampled at 4 Hz with 50% overlapping. Again, the above procedure was applied to RR interval series and it was concluded that segment length of 256 samples with 50% overlapping provides a smoothed spectral estimate with clearly outlined peaks in low- and high-frequency bands. This easily understandable and interpretable spectral estimate leads to a better visual and automated analysis, which is not only desirable in basic physiology studies, but also a prerequisite for a widespread utilization of frequency domain techniques in clinical studies, where simplicity and effectiveness of information are of primary importance.

Simplified training algorithm for hierarchical hidden Markov models

AbstractThis paper considers an extension of the hidden Markov model, called the hierarchical hidden Markov model, and proposes an EM algorithm and an approximate algorithm. The EM algorithm proposed in this paper differs from the existing training algorithm, which is called the Baum–Welch algorithm, and guarantees that the likelihood is always increased by updating the parameters. The approximate algorithm has the advantage that it can be applied to problems in which observation of the training sentence and training of the parameters proceed in parallel. These algorithms and their derivations are simplified, compared to the existing training algorithm, by using stochastic context‐free grammar. © 2004 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 87(5): 59–69, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10172

Improved Hidden Markov Model training for multiple sequence alignment by a particle swarm optimization—evolutionary algorithm hybrid

Multiple sequence alignment (MSA) is one of the basic problems in computational biology. Realistic problem instances of MSA are computationally intractable for exact algorithms. One way to tackle MSA is to use Hidden Markov Models (HMMs), which are known to be very powerful in the related problem domain of speech recognition. However, the training of HMMs is computationally hard and there is no known exact method that can guarantee optimal training within reasonable computing time. Perhaps the most powerful training method is the Baum–Welch algorithm, which is fast, but bears the problem of stagnation at local optima. In the study reported in this paper, we used a hybrid algorithm combining particle swarm optimization with evolutionary algorithms to train HMMs for the alignment of protein sequences. Our experiments show that our approach yields better alignments for a set of benchmark protein sequences than the most commonly applied HMM training methods, such as Baum–Welch and Simulated Annealing.

Overall risk criterion estimation of hidden Markov model parameters

In this paper, we propose a novel discriminative objective function for the estimation of hidden Markov model (HMM) parameters, based on the calculation of overall risk. For continuous speech recognition, the algorithm minimises the risk of misclassification on the training database and thus maximises recognition accuracy. The calculation of the risk is based on the measurement of Levenshtein distance between the correct transcription and the N-best recognised transcriptions, which counts the number of recognition errors – deleted, substituted and inserted words. The minimisation of the proposed criterion is implemented by using the extended Baum–Welch algorithm for the estimation of the discrete HMM parameters and the Normandin’s extension of the algorithm for the estimation of the continuous densities. We tested the performance of the proposed algorithm on two tasks: phoneme recognition on the TIMIT database and continuous speech recognition on the Resource Management (RM) database. Results show consistent decrease of the recognition error rate when compared to the standard maximum likelihood estimation training. The highest achieved decrease of word error rate in our experiments is 20.8% on the TIMIT phoneme recognition task, 39.6% on the RM task with context-independent HMMs and 18.22% with context-dependent models.

A constrained HMM-based approach to the estimation of perceptual switching dynamics in pigeons

We describe a method for the analysis of behavioural data from experiments on perception of bistable stimuli in pigeons. The approach is based on a Hidden Markov Model (HMM) with additional linear factorial constraints for which a modified Baum–Welch algorithm is derived. It allows the estimation of the perceptual switching events, which might directly relate to transitions between states of activation of corresponding neural populations. From the resulting time series, characteristics of the underlying perceptual dynamics can be estimated. We also demonstrate that—inspite of the Markov assumption—the method can reveal certain non-Markovian contributions to the dynamics.

Bayesian Restoration of Ion Channel Records using Hidden Markov Models

Hidden Markov models have been used to restore recorded signals of single ion channels buried in background noise. Parameter estimation and signal restoration are usually carried out through likelihood maximization by using variants of the Baum–Welch forward–backward procedures. This paper presents an alternative approach for dealing with this inferential task. The inferences are made by using a combination of the framework provided by Bayesian statistics and numerical methods based on Markov chain Monte Carlo stochastic simulation. The reliability of this approach is tested by using synthetic signals of known characteristics. The expectations of the model parameters estimated here are close to those calculated using the Baum–Welch algorithm, but the present methods also yield estimates of their errors. Comparisons of the results of the Bayesian Markov Chain Monte Carlo approach with those obtained by filtering and thresholding demonstrate clearly the superiority of the new methods.

Optimisation of HMM topology and its model parameters by genetic algorithms

Hidden Markov model (HMM) is currently the most popular approach to speech recognition. However, the problems of finding a good HMM model and its optimised model parameters are still of great interest to the researchers in this area. In our previous work, we have successfully applied the genetic algorithm (GA) to the HMM training process to obtain the optimised model parameters (Chau et al. Proc. ICASSP (1997) 1727) of the HMM models. In this paper, we further extend our work and propose a new training method based on GA and Baum–Welch algorithms to obtain an HMM model with optimised number of states in the HMM models and its model parameters. In this work, we are not only able to overcome the shortcomings of the slow convergence speed of the simple GA-HMM approach. In addition, this method also finds better number of states in the HMM topology as well as its model parameters. From our experiments with the 100 words extracted from the TIMIT corpus, our method is able to find the optimal topology in all cases. In addition, the HMMs trained by our GA HMM training have a better recognition capability than the HMMs trained by the Baum–Welch algorithm. In addition, 290 words are randomly selected from the TMIIT database for testing the recognition performances of both approaches, it is found that the GA-HMM approach has a recognition rate of 95.86% while the Baum–Welch method has a recognition rate of 93.1%. This implies that the HMMs trained by our GA-HMM method are more optimised than the HMMs trained by the Baum–Welch method.

Unconstrained Farsi handwritten word recognition using fuzzy vector quantization and hidden Markov models

An unconstrained Farsi handwritten word recognition system based on fuzzy vector quantization (FVQ) and hidden Markov model (HMM) for reading city names in postal addresses is presented. Preprocessing techniques including binarization, noise removal, slope correction and baseline estimation are described. Each word image is represented by its contour information. The histogram of chain code slopes of the image strips (frames), scanned from right to left by a sliding window, is used as feature vectors. Fuzzy c-means (FCM) clustering is used for generating a fuzzy codebook. A separate HMM is trained by modified Baum–Welch algorithm for each city name. A test image is recognized by finding the best match (likelihood) between the image and all of the HMM word models using forward algorithm. Experimental results show the advantages of using FVQ/HMM recognizer engine instead of conventional discrete HMMs.

Penalized maximum-likelihood estimation, the Baum–Welch algorithm, diagonal balancing of symmetric matrices and applications to training acoustic data

We study penalized maximum-likelihood estimation methods for nonparametric density estimation and propose their use in training acoustic models for speech recognition. Several algorithms for the numerical solution of the optimization problems that we encounter are proposed and analyzed.

Hidden Markov Model Based Word Recognition and Its Application to Legal Amount Reading on French Checks

A hidden Markov model (HMM) based word recognition algorithm for the recognition of legal amounts from French bank checks is presented. This algorithm is part of the A2iA INTERCHEQUE recognition system. The algorithm starts from images of handwritten words which have been automatically segmented from binary check images. After finding the lower-case zone on the complete amount, words are slant corrected and then segmented into graphemes. Then, features are extracted from the graphemes, and the feature vectors are vector quantized resulting in a sequence of symbols for each word. Likelihoods of all word classes are computed by a set of HMMs, which have been previously trained using either the Viterbi algorithm or the Baum–Welch algorithm. The various parameters of the system have been identified and their importance evaluated. Results have been obtained on large real-life data bases of French handwritten checks. The HMM-based system has been shown to outperform a holistic word recognizer and another HMM-type word recognizer from the A2iA INTERCHEQUE recognition system. Word recognition rates of about 89% for the 26-word vocabulary relevant for legal amount recognition on French bank checks have been obtained. More recently, a Neural Network–HMM hybrid has been designed, which produces even better recognition rates.

Speech enhancement based on neural predictive hidden Markov model

In this paper, we describe a new approach to speech enhancement by modeling directly the statistical characteristics of the speech waveform. To represent the nonlinear and nonstationary nature of speech, it is assumed that speech is the output of a neural predictive hidden Markov model (NPHMM). The NPHMM is a nonlinear autoregressive process whose time-varying parameters are controlled by a Markov chain. Given some speech data, the parameter of NPHMM is estimated by a learning algorithm based on the combination of Baum–Welch algorithm and a neural network learning algorithm using the well known back propagation technique. Given the parameters of NPHMM, a recursive estimation method using multiple Kalman filters, governed by a Markov state chain according to the transition probabilities is developed for enhancing speech signals degraded by statistically independent additive noise characteristics assumed to be white and Gaussian. Under various input signal-to-noise ratios (SNRs), the proposed recursive speech enhancement method achieves an improvement over the method based on hidden filter model (Lee and Shirai, 1996) of about 0.8–1.2 dB in terms of the measured output SNR.

A Cycle Generation Algorithm for Finite Undirected Linear Graphs

When the algorithms of J. T. Welch, Jr. were implemented it was discovered that they did not perform as described. The generation of all cycles from a basis is faulty. The generation of the basis is apparently correct. A modified version of Welch's Algorithm 3 is presented. The reasons for modifying Welch's algorithms are presented with examples.

Algorithms for finding a fundamental set of cycles for an undirected linear graph

Given the adjacency matrix of the graph, the algorithm presented in this paper finds a spanning tree and then constructs the set of fundamental cycles. Our algorithm is slower than an algorithm presented by Welch by a ratio of N /3 ( N is the number of nodes) but requires less storage. For graphs with a large number of nodes and edges, when storage is limited our algorithm is superior to Welch's; however, when the graphs are small, or machine storage is very large, Welch's algorithm is superior. Timing estimates and storage requirements for both methods are presented.