Japanese Speech Recognition Research Articles

Sound signal analysis in Japanese speech recognition based on deep learning algorithm

Sound signal analysis in Japanese speech recognition based on deep learning algorithm

Analysis of Japanese Expressions and Semantics Based on Link Sequence Classification

Based on locness corpus, this paper uses Wordsmith 6.0, SPSS 24, and other software to explore the use of temporal connectives in Japanese writing by Chinese Japanese learners. This paper proposes a method of tense classification based on the Japanese dependency structure. This method analyzes the results of the syntactic analysis of Japanese dependence and combines the tense characteristics of the target language to extract tense‐related information and construct a maximum entropy tense classification model. The model can effectively identify the tense, and its classification accuracy shows the effectiveness of the classification method. This paper proposes a temporal feature extraction algorithm oriented to the hierarchical phrase expression model. The end‐to‐end speech recognition system has become the development trend of large‐scale continuous speech recognition because of its simplicity and efficiency. In this paper, the end‐to‐end technology based on link timing classification is applied to Japanese speech recognition. Taking into account the characteristics of Japanese hiragana, katakana, and Japanese kanji writing forms, through experiments on the Japanese data set, different suggestions are explored. The final effect is better than mainstream speech recognition systems based on hidden Markov models and two‐way long and short‐term memory networks. This algorithm can extract the temporal characteristics of rules that meet certain conditions while extracting expression rules. These tense characteristics can guide the selection of rules in the expression process, make the expression results more in line with linguistic knowledge, and ensure the choice of relevant vocabulary and the structural ordering of the language. Through the analysis of time series and static information, we combine the time and space dimensions of the network structure. Using connectionist temporal classification (CTC) technology, an end‐to‐end speech recognition method for pronunciation error detection and diagnosis tasks is established. This method does not require phonemic information nor does it require forced alignment. The extended initials and finals are the error primitives, and 64 types of errors are designed. The experimental results show that the method can effectively detect the wrong pronunciation, the detection accuracy rate is 87.07%, the false rejection rate is 7.83%, and the error rate is 87.07%. The acceptance rate is 25.97%. This method uses network information more comprehensively than traditional methods, and the model is more effective. After detailed experiments, this article evaluates the prediction effect of this method and previous methods on the data set. This method improves the prediction accuracy by about 15% and achieves the expected goal of the work in this paper.

Consonant discrimination scores and recognition errors in adults with cochlear implants―Findings of Japanese speech recognition tests (CI2004)―

Consonant discrimination scores and recognition errors in adults with cochlear implants―Findings of Japanese speech recognition tests (CI2004)―

Non-native speakers perform poorer than native speakers in Japanese speech recognition in reverberation

Reverberation affects perceptual performance, especially non-native listeners, elderly people, and people with hearing impairments. Native Japanese speakers confused some Japanese consonants and they failed to distinguish Japanese length contrast in reverberation [Arai et al., Proc. Autumn Meet. Acoust. Soc. Jpn., 2016]. The current study examined degradation degree of non-native speakers (native English speakers) in Japanese speech recognition in reverberation, using the same experiment settings with the previous study. There were two sets of stimulus: 1) Japanese consonant-vowel (CV) syllables, 2) non-words varying in duration of a certain vowel/consonant along durational continuum. In the results, although non-native speakers confused consonants even in non-reverberant condition, the number of their confusion in reverberation was larger than that in non-reverberant condition. Confusion of non-native speakers in reverberation was much more than that of native speakers. In addition, the results indicated that non-native speakers could hardly distinguish the length contrast of Japanese in reverberation. The proportion of “long” responses did not change sharply in the perception of vowel continuum, while native speakers changed their responses at some points on the continuum. The results suggest that learners need training for listening to speech sounds in reverberation.

End-to-end neural network modeling for Japanese speech recognition

This study proposes end-to-end neural network modeling to adapt direct speech-to-text decoding to Japanese. End-to-end speech recognition systems using deep neural networks (DNNs) are currently being investigated. These systems do not need intermediate phonetic representation. Instead, many of them utilize Recurrent Neural Networks (RNNs) trained by using much more data than ever before. The end-to-end approach makes acoustic models simpler to train. Typically, previous works have dealt with phonogram labels such alphabetic characters. Ideograms such as Kanji, however, make end-to-end speech recognition more complex. A single Kanji can have multiple readings, such as On-yomi (Chinese reading) and Kun-yomi (Japanese reading). In addition, whereas alphabets have at most 100 labels, Japanese has over 2000 labels to predict, such as Kanji, Hiragana, Katakana, the Roman alphabet, digits, and punctuation marks. To resolve this problem, we attempt to make end-to-end neural network modeling allows speech recognition of Japanese without phonetic representation. This method trains RNN and adopts the connectionist temporal classification (CTC) objective function. The proposed method was able to deal with a large amount of character labels. We also analyzed the decoding results and examined ideas for improving the accuracy of word error rate (WER).

Japanese Vowel Sound Classification Using Fuzzy Inference System

An automatic speech recognition system is one of the popular research problems. There are many research groups working in this field for different language including Japanese. Japanese vowel recognition is one of important parts in the Japanese speech recognition system. The vowel classification system with the Mamdani fuzzy inference system was developed in this research. We tested our system on the blind test data set collected from one male native Japanese speaker and four male non-native Japanese speakers. All subjects in the blind test data set were not the same subjects in the training data set. We found out that the classification rate from the training data set is 95.0 %. In the speaker-independent experiments, the classification rate from the native speaker is around 70.0 %, whereas that from the non-native speakers is around 80.5 %.

Generative approach for robust acoustic model training for blindly separated speech recognition

We propose a generative acoustic model training method for robust speech recognition with blind sound source separation as a front-end. Multiple microphone systems are often used for the separation. In such situation, separated speech is severely distorted and thus the recognition rate significantly drops. If we can measure transmission characteristics from the sound sources with various directions to the microphones, we can simulate to receive various mixed speech made by multiple speakers speaking with overlaps to each other. Then we separate the simulated overlapped speech using a blind source separation method such as frequency domain independent component analysis (FDICA) and use the separated speech to train HMM acoustic models to recognize such separated speech. Our method can generate such distorted speech enormously without recording the real speech spoken to the microphone system. We evaluate the models in the continuous Japanese speech recognition and show the effectiveness.

Speech Perception in Quiet and Noise Using the Hearing in Noise Test and the Japanese Hearing in Noise Test by Japanese Listeners

The aim of this study was to evaluate the first-language (L1) and second-language (L2) perception abilities in quiet and noise of native Japanese listeners who acquired English late in life and lived in the United States. The study addressed two primary questions: (1) whether native Japanese listeners who developed some fluency in English showed poorer English speech perception ability in quiet and noise than native English listeners, and (2) whether native Japanese listeners living in an English-speaking environment demonstrated poorer speech perception ability in Japanese than native Japanese listeners who reside in Japan and rarely use a second language. Ten native Japanese adults who had excellent English word recognition ability in quiet were evaluated using the English and Japanese versions of the Hearing in Noise Test (HINT). In addition, 10 native English adults were evaluated using the English version of the HINT. All Japanese participants started learning English between the ages of 12 and 13 yrs while at school in Japan and had lived in the United States more than 4 yrs. An adaptive procedure was used to measure English and Japanese speech recognition thresholds (SRTs) in quiet and signal-to-noise ratios (SNRs) at which SRTs were obtained in noise conditions. In addition, a fixed-level procedure was used to obtain percent correct recognition scores at five different SNRs. Both HINT sentence stimuli and noise were presented at a simulated 0° azimuth under headphones. The Japanese group had significantly poorer English speech perception ability than the native English group in all the test conditions under both the adaptive and fixed-level procedures. The psychometric function for the Japanese listeners was shifted by approximately 3 to 4 dB SNR from that of the native English listeners. Japanese speech recognition ability by the Japanese group was not significantly different from that of native Japanese speakers who live in Japan. The Japanese participants who had excellent English word recognition ability in quiet listening conditions failed to reach native-like English speech perception when presented with sentences in quiet conditions. In addition, all 10 Japanese listeners examined in this study had HINT SNRs (i.e., SNRs at which SRTs were obtained in noise conditions) outside the normal range. However, the possible deterioration of L1 speech perception after living in the L2 environment for an extended period of time was not observed among the Japanese, late L2 learners in this study. The implication of this study for clinical settings is that it is important to use L1 sentence materials in both quiet and noise to examine the speech recognition performance of L2 (English) users who had acquired L2 after puberty.

Evaluation of speech perception for cochlear implant users using the Japanese speech recognition test battery, “CI-2004”.

Evaluation of speech perception for cochlear implant users using the Japanese speech recognition test battery, “CI-2004”.

A pronunciation and vocabulary teaching aid for Spanish as a Foreign Language programs at Japanese universities utilizing a Spanish speech recognition system and corpora of academic Spanish

<p>This paper reports on a project which researchers at two universities in Japan developed for Spanish as a Foreign Language CALL materials, which has two features, a Spanish speech recognition system based on the AmiVoice Japanese speech recognition system; and a Spanish verb collocation-based vocabulary teaching aid based on a corpus of academic Spanish. The first step was the creation of a Spanish speech recognition system for Japanese university students based on the AmiVoice Japanese pronunciation recognition system developed by Advanced Media Incorporated. The second step was the selection of vocabulary for a specific genre of academic Spanish, which was done in the following way: (1) Articles in a Spanish-language academic journal on Spanish Literature, Culture, History, and Language (themes typically chosen by students for graduation theses) compiled and published annually in Japan were used as the source of data for the compilation of a corpus. (2) All the verbs were extracted from the data. (3) The extracted verbs were compared with a corpus of verbs from graduation theses written in Japanese by Japanese university students and (4) by taking into account their needs for learning verbs, a list of verb-centered collocations was compiled. Finally, we developed a user-friendly Spanish pronunciation and verb collocation-based vocabulary teaching aid. The aid displays the verb collocations on a computer screen during pronunciation training. The training had two phases. The first phase required Japanese university students to carefully and repeatedly listen to model pronunciation examples of the vocabulary items, and to pronounce the vocabulary. This pronunciation training is intended to contribute to effective memorizing of the verb collocations. The next phase required the students to repeat the vocabulary items at the rate of the model utterance as well as to imitate the stress and other intonation features. This training is expected to help Japanese students speak Spanish more fluently.</p>

CENSREC-3: An Evaluation Framework for Japanese Speech Recognition in Real Car-Driving Environments

This paper introduces a common database, an evaluation framework, and its baseline recognition results for in-car speech recognition, CENSREC-3, as an outcome of the IPSJ-SIG SLP Noisy Speech Recognition Evaluation Working Group. CENSREC-3, which is a sequel to AURORA-2J, has been designed as the evaluation framework of isolated word recognition in real car-driving environments. Speech data were collected using two microphones, a close-talking microphone and a hands-free microphone, under 16 carefully controlled driving conditions, i.e., combinations of three car speeds and six car conditions. CENSREC-3 provides six evaluation environments designed using speech data collected in these conditions.

Erratum: Language modeling by stochastic dependency grammer for Japanese speech recognition

Erratum: Language modeling by stochastic dependency grammer for Japanese speech recognition

A Transliteration Algorithm for Adapting a Japanese Voice Controlled Browser to English

We propose a novel algorithm to transliterate English to Japanese and its application to a voice controlled browser, which enable ordinary Japanese people to browse English Web site by voice. Speech recognition software designed for native English speakers do not work for most Japanese because Japanese can't pronounce English as native English speakers do. Therefore, we combined Japanese speech recognition software with English-to-Japanese transliteration software. The accuracy of our transliteration algorithm is 80% recall for the top candidate, and 92% recall for the top three candidates. The browser using this transliteration algorithm makes it possible for Japanese to navigate English Web pages almost as accurate as Japanese pages by voice commands.

Language modeling by stochastic dependency grammar for Japanese speech recognition

AbstractThis article has been removed by the publisher.

An instance-based model of Japanese speech recognition by native and non-native listeners

Japanese has a short and long segment contrast (oji-san versus ojii-san, kita versus kitta, and kana versus kanna). Previous research has shown that this durational property is one of the characteristics of moras in Japanese. It has also been reported that Japanese learners generally have difficulty in acquiring this contrast, but after extensive language experience with Japanese, they do learn the contrast. This suggests that the contrasts and similarities are based on remembered instances of linguistic objects in various environments during a lifetime. This paper investigates whether an instance-based approach of speech recognition can replicate performance by native listeners and second language learners of Japanese. An instance-based model was trained with naturally produced utterances of koma spin and komma comma and then tested on a 24-step nasal duration continuum from koma to komma [T. Uchida, doctoral dissertation, Nagoya University (1996)]. The identification function produced by the model was very similar to identification responses produced by native Japanese listeners. Additional simulations trained with utterances produced by English native speakers (e.g., coma, comma) will be reported. These simulations will address the role of linguistic experience in second language acquisition. [Supported by NIDCD R29-016435-06.]

Associative information for speech recognition using semantic attributes

Speech recognition has been studied extensively in recent years with regard to human interface and the goal has been realization of speech recognition systems. However, the problem is that these systems have numerous limitations, despite being accompanied by implementation of speech-based interactive systems. There have been a few attempts to use associative information in Japanese speech recognition and understanding. As a result, this paper introduces a new concept of association between semantic attributes and associations between words under the assumption of so-called “indirect associations” in human communication. Hence, nearly successful processing of associations close to common sense relations is achieved. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(2): 63–77, 1997

Clustered inter-phrase or word context-dependent models for continuously read Japanese.

This paper investigates methods to model inter-phrase or word context for continuous Japanese speech recognition. It is well known that in continuous speech, coarticulation between words or phrases induces allophonic variation of the beginning and ending phones in words or phrases. It was found that by compiling a network of contextdependent phonetic models which models these inter-word or inter-phrase context, recognition error reduction by 32 % can be achieved compared to models which do not account for inter-word context with task-dependent training, i.e. models that were trained with the same vocabulary as the test set. A more dramatic error reduction of up to 43% was possible with task-independent training. However, this will significantly increase the number of phonetic models required to model the vocabulary. With digit models, the increase in the number of models is 4 to 5 fold. To overcome this increase, we clustered the inter-word/phrase context into a few phonetic classes. Using one class for consonant inter-word context and two classes for vowel context, the recognition accuracy on digit string recognition was found to be virtually equal to the accuracy with unclustered models, while the number of phonetic models required was reduced by more than 50%.

Feature extraction using a matrix coefficient filter for speech recognition

This paper provides a new interpretation of the so-called “delta-cepstrum” and extends the formulation of the conventional delta-cepstrum towards an optimal design of the filter, which extracts important spectral dynamics from a cepstrum sequence. The algorithm to obtain new feature parameters is unified to a formulation using a matrix coefficient filter and is tested through Japanese speech recognition experiments. The average recognition error rate in a Japanese 24 phoneme recognition experiment for four speakers was reduced from 12.2% to 10.3%.

Recognition of speech produced in a simulated noisy environment

Pronunciation manner varies depending on the noise of the speech environment. This is known as the Lombard effect. It affects the acoustic features of speech such as intensity, pitch, duration, and spectral shape. Because vowels play an important role in Japanese speech recognition, special attention was paid to the variability of the acoustic features of vowels uttered in noise. First, the variability of the five Japanese vowels was examined using speech uttered while hearing noise through headphones. It was observed that the higher formants are unstable and that the energy in the middle frequency range of the vowel spectrum increases depending on the noise level. Based on these observations, a normalization method combining band limitation and spectral tilt compensation was proposed, and its effectiveness in the recognition of vowels was shown by experiment. Next, a new word recognition method utilizing this normalization technique was proposed. This method performs vowel normalization at the portion where both test and reference patterns are vowel‐like sounds. The experimental results show the significant effectiveness of the proposed method. The recognition rate obtained by the proposed method with normalization is 96% for 50 words uttered by two male speakers, while the recognition rate without normalization is 80%.

文節スポッティングに基づく日本語文音声の認識

文節スポッティングに基づく日本語文音声の認識