Character Error Rate Research Articles

A Dialectal Chinese Speech Recognition Framework

A framework for dialectal Chinese speech recognition is proposed and studied, in which a relatively small dialectal Chinese (or in other words Chinese influenced by the native dialect) speech corpus and dialect-related knowledge are adopted to transform a standard Chinese (or Putonghua, abbreviated as PTH) speech recognizer into a dialectal Chinese speech recognizer. Two kinds of knowledge sources are explored: one is expert knowledge and the other is a small dialectal Chinese corpus. These knowledge sources provide information at four levels: phonetic level, lexicon level, language level, and acoustic decoder level. This paper takes Wu dialectal Chinese (WDC) as an example target language. The goal is to establish a WDC speech recognizer from an existing PTH speech recognizer based on the Initial-Final structure of the Chinese language and a study of how dialectal Chinese speakers speak Putonghua. The authors propose to use context-independent PTH-IF mappings (where IF means either a Chinese Initial or a Chinese Final), context-independent WDC-IF mappings, and syllable-dependent WDC-IF mappings (obtained from either experts or data), and combine them with the supervised maximum likelihood linear regression (MLLR) acoustic model adaptation method. To reduce the size of the multi-pronunciation lexicon introduced by the IF mappings, which might also enlarge the lexicon confusion and hence lead to the performance degradation, a Multi-Pronunciation Expansion (MPE) method based on the accumulated uni-gram probability (AUP) is proposed. In addition, some commonly used WDC words are selected and added to the lexicon. Compared with the original PTH speech recognizer, the resulting WDC speech recognizer achieves 10–18% absolute Character Error Rate (CER) reduction when recognizing WDC, with only a 0.62% CER increase when recognizing PTH. The proposed framework and methods are expected to work not only for Wu dialectal Chinese but also for other dialectal Chinese languages and even other languages.

An Innovative Prosody Modeling Method for Chinese Speech Recognition

This paper presents an innovative method for prosody modeling in Chinese speech recognition. Our method first evaluated the reliability of the prosodic information by which the recognition system dynamically tunes the balance between the spectral scores and prosodic scores. The basic idea of this method is to use prosodic knowledge based on its reliability. The higher the reliability, the more the prosodic information contributes to recognition. Thus, this method will not introduce extra errors but will incorporate more knowledge into the recognition system. Experimental results showed that this method reduced the relative word error rate by as much as 52.9% and 46.0% for Mandarin and Cantonese digit string recognition tasks, respectively. When incorporating tone information into Cantonese Large Vocabulary Continuous Speech Recognition (LVCSR) via the proposed method, a 20.16% relative character error rate reduction was obtained.

Tone Modeling for Continuous Mandarin Speech Recognition

Tone study is very important for Mandarin speech recognition. In this paper, a Mixture Stochastic Polynomial Tone Model (MSPTM) is proposed for tone modeling in continuous Mandarin speech. In this model the pitch contour, main representative of tone pattern, is described as a mixed stochastic trajectory. The mean trajectory is represented by a polynomial function of normalized time while the variance is time varying. Effective training and tone recognition algorithms were developed. The experimental results based on the proposed MSPTM showed 40.7% tone recognition error rate reduction relative to the traditional Hidden Markov Model (HMM) tone model. We also present a decision tree based approach to learning the tone pattern variation in continuous speech. The phonetic and linguistic factors that may affect the tone patterns were taken into consideration while constructing the tree. After the tree was established, 28 different tone patterns were obtained. We found that in addition to the tone of the neighboring syllable, Consonant/Vowel type of the syllable and the position of the syllable in the utterance also made important contributions to tone pattern variations in continuous speech. Finally, a new approach of integrating tone information into the search process at word level is discussed. Experiments on continuous Mandarin speech recognition showed that the new tone model and tone information integration method were efficient, achieving a 16.2% relative character error rate reduction.

Improving the syllable-synchronous network search algorithm for word decoding in continuous Chinese speech recognition

The previously proposed syllable-synchronous network search (SSNS) algorithm plays a very important role in the word decoding of the continuous Chinese speech recognition and achieves satisfying performance. Several related key factors that may affect the overall word decoding effect are carefully studied in this paper, including the perfecting of the vocabulary, the big-discount Turing re-estimating of theN-Gram probabilities, and the managing of the searching path buffers. Based on these discussions, corresponding approaches to improving the SSNS algorithm are proposed. Compared with the previous version of SSNS algorithm, the new version decreases the Chinese character error rate (CCER) in the word decoding by 42.1% across a database consisting of a large number of testing sentences (syllable strings).

Productivity prediction by extrapolation: Using workload memory as a predictor of target performance

In order to assess if productivity based on extrapolated data is a good predictor of longer texts, an experimental study was conducted. Two full-sized text input devices for touch typing and two miniaturized for tapping were used, all featuing QWERTY layout, in a repeated measurement design. Twenty subjects were exposed to both a task within the limit of working memory (nine words) and four running memory tasks (approx. 275 words). For miniaturized tapping keyboards, extrapolated data significantly underestimated both entry speed (uncorrected wpm, up to 17%) and character error rate (up to 61%) whereas it significantly overestimated ratio of correct words (up to 62%) of running memory tasks. Further, error-corrected entry speed was significantly overestimated up to a factor of 2.7. Results based on extrapolated productivity metrics must therefore be interpreted with caution. Running memory tasks with text length of more than 32 words is needed to assess productivity of text input devices if tapping is used.

The TREC-5 Confusion Track: Comparing Retrieval Methods for Scanned Text

A known-item search is a particular information retrieval task in which the system is asked to find a single target document in a large document set. The TREC-5 confusion track used a set of 49 known-item tasks to study the impact of data corruption on retrieval system performance. Two corrupted versions of a 55,600 document corpus whose true content was known were created by applying OCR techniques to page images. The first version of the corpus used the page images as scanned, resulting in an estimated character error rate of approximately 5%. The second version used page images that had been down-sampled, resulting in an estimated character error rate of approximately 20%. The true text and each of the corrupted versions were then searched using the same set of 49 questions. In general, retrieval methods that attempted a probabilistic reconstruction of the original clean text fared better than methods that simply accepted corrupted versions of the query text.

Quality assessment and restoration of typewritten document images

We present a useful method for assessing the quality of a typewritten document image and automatically selecting an optimal restoration method based on that assessment. We use five quality measures that assess the severity of background speckle, touching characters, and broken characters. A linear classifier uses these measures to select a restoration method. On a 139-document corpus, our methodology reduced the corpus OCR character error rate from 20.27% to 12.60%.

An omnifont open-vocabulary OCR system for English and Arabic

We present an omnifont, unlimited-vocabulary OCR system for English and Arabic. The system is based on hidden Markov models (HMM), an approach that has proven to be very successful in the area of automatic speech recognition. We focus on two aspects of the OCR system. First, we address the issue of how to perform OCR on omnifont and multi-style data, such as plain and italic, without the need to have a separate model for each style. The amount of training data from each style, which is used to train a single model, becomes an important issue in the face of the conditional independence assumption inherent in the use of HMMs. We demonstrate mathematically and empirically how to allocate training data among the different styles to alleviate this problem. Second, we show how to use a word-based HMM system to perform character recognition with unlimited vocabulary. The method includes the use of a trigram language model on character sequences. Using all these techniques, we have achieved character error rates of 1.1 percent on data from the University of Washington English Document Image Database and 3.3 percent on data from the DARPA Arabic OCR Corpus.

What size test set gives good error rate estimates?

We address the problem of determining what size test set guarantees statistically significant results in a character recognition task, as a function of the expected error rate. We provide a statistical analysis showing that if, for example, the expected character error rate is around 1 percent, then, with a test set of at least 10,000 statistically independent handwritten characters (which could be obtained by taking 100 characters from each of 100 different writers), we guarantee, with 95 percent confidence, that: (1) the expected value of the character error rate is not worse than 1.25 E, where E is the empirical character error rate of the best recognizer, calculated on the test set; and (2) a difference of 0.3 E between the error rates of two recognizers is significant. We developed this framework with character recognition applications in mind, but it applies as well to speech recognition and to other pattern recognition problems.

Error Rate of QPRS Evaluated in Amplitude-Phase Space

A method for evaluating the character error rate for QPRS in the signal space using the classical Bayes hypothesis-testing technique is presented. Since the decision regions are rectangular, analytical expressions for the error rates can be found. This method may be extended to account for the effect of carrier tracking error and, in principle, phase jitter.

On the computation of the probability of post-decoding error events for block codes (Corresp.)

The computation of the probability of certain post-decoding error events is formulated for block codes whose weight distributions are known. A bounded-distance decoder for group codes is assumed. Numerical examples included show post-decoding character error rates for the Golay code and for some Reed-Solomon codes and the distribution of post-decoding errors for the same Reed-Solomon codes. An extension to errors and erasures decoding is also given.

Context in word recognition

Relatively low character error rates can often lead to prohibitive levels of word error rates. This paper examines several techniques for integrating an independent contextual postprocessor (CPP) into a full classification system. Using positional binary n-grams the CPP can correct many errors directly. In those cases where the correction process leads to ambiguity, the CPP can direct additional processing. Experimental results demonstrate that almost all of the derived improvement results from CPP-directed reclassification. This only requires that the CPP have the classifier likelihood fed forward to it. Therefore, a standardized CPP can be built independently of the rest of the classification system. An initial 45% word error rate is reduced to about a 2% word error rate and a 1% reject rate. Presence of a dictionary allows these figures to be reduced even further.

1969-70 Connection Survey: Low-Speed Data Transmission Performance on the Switched Telecommunications Network

Error statistics of low-speed, start-stop data transmission over the Bell System switched telecommunications network are reported in this article. During the 1969–70 Toll Connection Survey, measurements were made on 534 connections with over 21 million characters transmitted, to give an overall average character error rate of 1.46 × 10−4, Over 90 percent of the low-speed test calls contained about 36,000 or 54,000 characters. A character error rate of 10−4 or less is indicated for 77.6 percent of all calls, while 95.0 percent of calls have a lost character rate of 10−4 or less. Error-free performance is shown for 48.3 percent of all calls, and 89.3 percent have no lost characters. Statistics of bursts, error-free intervals, block errors, and carrier failure durations are also presented.

Error Rate Considerations for Coherent Phase-Shift Keyed Systems with Co-Channel Interference

In this paper we present a theoretical analysis of the performance of an m-phase coherent phase-shift keyed system in the presence of random gaussian noise and interference. An explicit expression is given for the probability of error of the phase angle of the received signal; we show that this probability of error can be expressed as a converging power series. We show that the coefficients of this series are expressible in terms of well-known and well-tabulated functions, and we give methods of evaluating the character error rates of the systems. We also show that this error rate is minimum when all the interference power is concentrated in a single interferer, and that it attains its maximum [P <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</inf> ] <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">max</inf> when the total interference power is equally distributed amongst the K interferers. The limiting case when K goes to infinity is considered. The cases of K = 1, and m = 2, 4, 8, and 16 are treated in some detail, and the results are given graphically. The usefulness of the results presented in this paper is that the designer can have at his disposal very simple expressions with which to evaluate the performance of any given Coherent Phase-Shift Keyed system when the received signal is corrupted by both interference and random gaussian noise.

Error-Rate Considerations for Digital Phase-Modulation Systems

The performance of digital phase-modulation systems has been investigated by many authors, and we first present a simple and alternate method of evaluating the character error rates of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</tex> -phase coherent and differentially coherent phase-shift keyed (DCPSK) systems when the receiver is ideal. An ideal receiver is assumed to operate with zero-width decision thresholds, but in any practical receiver, the voltage (or current) in question must differ from the threshold value by a finite amount in order for the device to respond properly and reliably. This finite-width decision threshold for any practical receiver can be expressed in terms of a certain parameter γ, and we give methods of evaluating the character error rates of coherent phase-shift keyed (CPSK) and differentially coherent phase-shift keyed systems when the receiver has a finite nonzero γ. The receiver may also introduce a certain phase perturbation δ in the detected phase of the signal because of imperfections present in various parts of the receiver. The methods of evaluating its effect on both CPSK and DCPSK systems have also been given in this paper.

Error Rates and Error Detection on Telegraph Circuits

This paper reviews error rates obtained in tests on the full range of telegraph facilities provided by an administration at signalling speeds not exceeding 200 bauds. The telegraph facilities tested include: an inland private circuit, automatic switched circuits, international circuits, intercontinental circuits in a submarine telephone cable, a speech plus telegraphy facility on inland telephone circuits. For transmission of data with a character error rate of better than <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/10^{6}</tex> on cable circuits, an improvement in performance will be necessary, and as this is unlikely to be obtained economically by attention to the cable or telegraph networks; the improvement will have to be achieved by error detection and retransmission. Where a short renter-to-renter circuit is involved, it may be satisfactory to use automatic error detection and manual retransmission, but over other telegraph facilities automatic retransmission may be necessary. Tests have also been made on both unprotected radio telegraph circuits and circuits employing the Van Duuren automatic error correction (ARQ) system. An appraisal is made of the over-all performance of this system, with suggestions for improvements necessary for the transmission of data. Where a duplex telegraph circuit is available, a simple method of error detection is to feed the received signal back to the sender where it can be checked against the original.

The NBS Meteor Burst Communication System

This project was undertaken in 1955 to investigate the properties of the intermittent reception of VHF signals over long distances by meteoric propagation and their communication usefulness. To accomplish this, a complete duplex teletype system operating at about 50 mc was constructed, and results are reported from tests made over a 1277-km east-west path. Comparisons are made of burst transmissions of 10, 20, 40, and 80 times normal teletype speed and of varaitions in a number of control system parameters. For the system under test, the optimum speedup was 40X, which produced a daily average channel capacity of about 40 wpm with a character error rate of about 0.0035 (with the best control system settings). Higher speedup ratios are advocated for future systems. The most serious causes of outages in this type of system are atmospheric noise and sustained multipath distortion from competing modes such as E and auroral propagation.