Hindi Speech Research Articles

IndicCONAN: A Multilingual Dataset for Combating Hate Speech in Indian Context

Hate speech (HS) is a growing concern in many parts of the world, including India, where it has led to numerous instances of violence and discrimination. The development of effective counter-narratives (CNs) is a critical step in combating hate speech, but there is a lack of research in this area, especially in non-English languages. In this paper, we introduce a new dataset, IndicCONAN, of counter-narratives against hate speech in Hindi and Indian English. We propose a scalable human-in-the-loop approach for generating counter-narratives by an auto-regressive language model through machine generation - human correction cycle, where the model uses augmented data from previous cycles to generate new training samples. These newly generated samples are then reviewed and edited by annotators, leading to further model refnement. The dataset consists of over 2,500 exam- ˜ ples of counter-narratives each in both English and Hindi corresponding to various hate speeches in the Indian context. We also present a framework for generating CNs conditioned on specifc CN type with a mean perplexity of 3.85 for English and 3.70 for Hindi, a mean toxicity score of 0.04 for English and 0.06 for Hindi, and a mean diversity of 0.08 for English and 0.14 for Hindi. Our dataset and framework provide valuable resources for researchers and practitioners working to combat hate speech in the Indian context.

Sentiment Analysis of Custom Speech Corpus: A proof of concept for NLP

This paper presents an innovative, custom Hindi Speech Corpus dedicated to assessing the positive intensity in Hindi language. By meticulously gathering voice samples from individuals across diverse demographics, this corpus is well crafted for acoustic research in Sentiment Analysis. Several sentences spanning a spectrum of sentiments (extremely positive, positive, neutral, negative) were acquired from participants. The resulting corpus comprises of 225 audio files. To guarantee the consistency and to accommodate domain knowledge based nuances, the sentiments and its intensity (high, medium, low) were classified by Expert annotation. We used PowerBI for attaining insights into emotional landscape of the corpus. Employing exploratory analysis and Sentiment Analysis, the corpus is analyzed to unveil linguistic features indicative of positive intensity in Hindi speech. This study advances the understanding of positivity expression in Hindi and furnishes valuable resources for linguistic investigations in the scope of Hindi language. It is observed that positive sentiment has the highest count at 58 and the lowest count is for very negative at 27. The main contribution of current work lies in creating the custom dataset in Hindi which to our knowledge is first of its kind. Also, we did not come across any dataset focused on intensity of positivity. These contributions shall definitely be beneficial to future researchers in exploring positive affect related research.

To think of interdisciplinarity as intercurrence: Or, working as an interdisciplinary team to develop an ML tool to tackle online gender-based violence and hate speech

The paper reflects on the working of an interdisciplinary team consisting of researchers and activists from the field of computer science and social sciences involved in developing a user-facing, browser plug-in to detect and moderate instances of online gender-based violence, hate speech and harassment in Hindi, Indian English, and Tamil. There have been multiple calls within the field of Human-Computer Interaction (HCI) to include qualitative methods in one’s research design. These calls, while attuned to the importance of qualitative methods for HCI, ignore the intercurrent nature of different research methods, disciplines and practices. The paper borrows the concept of intercurrence from Orren & Skowronek (1996) and reorients it to explicate the practice of interdisciplinary research. It argues that intercurrence i.e. (in between, an occurrence within an occurrence) is a useful image to perceive interdisciplinarity wherein we argue that at any given point, an interdisciplinary team navigates multiple, yet simultaneously occurring temporal dimensions of differently disciplined bodies. An awareness of these multiple temporalities adds another dimension to thinking about conflicts and possibilities emerging from interdisciplinary practices and reorients interdisciplinary research towards unexpected outcomes.

Identifying acoustic cues for dialect profiling: Policing in multilingual communities of India

A multilingual country such as India with numerous languages and dialects provides fertile grounds for evasive language crimes. From threat letters to ransom demands, the scope of crime is huge. The cases of illegal immigrants have only added to the fragility of international boundaries especially, during political upheavals. This leads to further vulnerability of society and also creates challenges for the police and law enforcement agencies towards timely intervention. The purpose of the study is to exhibit dialectal variation in Indian English by comparing two varieties. The current paper is based on the acoustic analysis of Indian English spoken by two distinct groups with different mother tongues. Ten native speakers of Hindi and Bangla were recorded in an anechoic chamber. A phonetically balanced passage was selected to be read. The analysis is based on Native Language Influence Detection (Perkins Grant, 2018) to derive acoustic phonetic correlates that can be used as significant identifying markers to distinguish Indian English speakers of Bangla and Hindi speech communities. The paper highlights that dialect profiling in the Indian context can be efficiently correlated with formant frequencies and Voice Onset Time for speech data. Acoustic analysis was done on PRAAT. PRAAT was used in this study because it has often been used by other similar studies to measure desired acoustic parameters simultaneously. Formant frequencies were measured at the midpoint of the vowels in the PRAAT using the LPC formant measurement algorithm. The normalization procedure was applied to the measured formant frequencies of vowels. The research affirms that acoustic analysis can provide verifiable cues for NLID. The framework can be used in the detection of native language influence in speech-centric criminal cases. The acoustic analysis shows that Indian English has subvarieties that could help in dialect profiling. The variation in Indian English vowel patterns could be due to the influence of the native language of the speakers.

SmHeSol (IoT-BC): Smart Healthcare Solution for Future Development Using Speech Feature Extraction Integration Approach with IoT and Blockchain

Voice of any human plays an important role in communication and sharing information among each other. Through voice, internal behavior can be identified as to whether the person is happy or angry which is reflected. A person’s behavior is not exactly reflected by their face; variation in his/her voice reflects somehow their behavior as there will be variation in voice and variation in frequency and pitch. Feelings and natural behaviors are important features, and there are many biological aspects through which they can be identified. Therefore, in this paper, we consider a Hindi speech specimen of different groups to identify the person’s behavior and natural feelings under different acoustic conditions. Many research papers show emotion recognition based on neural networks with different models using speech signals to identify the present status of any patient, and it helps to build a way for a smart healthcare system. Enabling service in terms of Blockchain means the sufferer does not require communicating with complex and failed tasks for collecting information from various sources to send to their expert. Blockchain provides experts access to systems and enables entry to the dataset section. Patients have total control over their data, and they no longer require monitoring to keep their data managed and up to date. Also, manually coordinating with data is required for multiple visitors, which can be a very tedious one. In this paper, we focused on feature removal of speech using different extraction approaches which were used to know the quality or state of voice specimens and also understand which feature extraction plays a vital role in gaining a close state of speech. Internet of Things-based learning platforms are used to gather the voice sample, and also, a deep gaining method was followed to reach and achieve the best accuracy and identify the error rate which will help to gather close behavior and state of mind. Finally, a proposed model based on the Gaussian mixture model as a classifier was used for its spotting and attestation.

Text to Speech and Language Conversion in Hindi and English Using CNN

Abstract: Deaf and dumb persons use sign language to communicate with other people in their society. Because sign language is the only means of communication for persons who are deaf or hard of hearing, it is mostly utilized by them. Ordinary folks are unfamiliar with this language. A real-time sign language recognition system has been developed in this article to allow those who do not know sign language to communicate with hearing-impaired people more readily. In this case, we employed American Sign Language. We have used American Sign Language in this paper. We introducing the development and implementation of an American Sign Language (ASL) derived from convolutional neural network. Deep Learning Method is used to train a classifier to recognize Sign Language and Convolutional Neural Network (CNN) is used to extract features from the images. We have also used Text-To-Speech Synthesis to convert the detected output into speech format. With use of MATLAB function the obtained text is converted into voice. In our system we are converting text to speech in Hindi language. Therefore hand gesture made by deaf and dumb people has been anatomized and restated into text and voice for better communication. Keywords: Sign Language Recognition, Image Processing, Text-To-Speech, American Sign Language, Classification, Convolutional Neural Network, Deep Learning real-time

Hindi/Bengali Sentiment Analysis using Transfer Learning and Joint Dual Input Learning with Self Attention

Sentiment Analysis typically refers to using natural language processing, text analysis, and computational linguistics to extract effect and emotion-based information from text data. Our work explores how we can effectively use deep neural networks in transfer learning and joint dual input learning settings to effectively classify sentiments and detect hate speech in Hindi and Bengali data. We start by training Word2Vec word embeddings for Hindi HASOC dataset and Bengali hate speech [24] and then train LSTM and subsequently, employ parameter sharing based transfer learning to Bengali sentiment classifiers by reusing and fine-tuning the trained weights of Hindi classifiers with both classifiers being used as the baseline in our study. Finally, we use BiLSTM with self-attention in a joint dual input learning setting where we train a single neural network on the Hindi and Bengali datasets simultaneously using their respective embeddings.

An Investigation of Multilingual TDNN-BLSTM Acoustic Modeling for Hindi Speech Recognition

Background: In India, thousands of languages or dialects are in use. Most Indian dialects are low asset dialects. A well-performing Automatic Speech Recognition (ASR) system for Indian languages is unavailable due to a lack of resources. Hindi is one of them as large vocabulary Hindi speech datasets are not freely available. We have only a few hours of transcribed Hindi speech dataset. There is a lot of time and money involved in creating a well-transcribed speech dataset. Thus, developing a real-time ASR system with a few hours of the training dataset is the most challenging task. The different techniques like data augmentation, semi-supervised training, multilingual architecture, and transfer learning, have been reported in the past to tackle the fewer speech data issues. In this paper, we examine the effect of multilingual acoustic modeling in ASR systems for the Hindi language. Objective: This article’s objective is to develop a high accuracy Hindi ASR system with a reasonable computational load and high accuracy using a few hours of training data. Method: To achieve this goal we used Multilingual training with Time Delay Neural Network- Bidirectional Long Short Term Memory (TDNN-BLSTM) acoustic modeling. Multilingual acoustic modeling has significantly improved the ASR system's performance for low and limited resource languages. The common practice is to train the acoustic model by merging data from similar languages. In this work, we use three Indian languages, namely Hindi, Marathi, and Bengali. Hindi with 2.5 hours of training data and Marathi with 5.5 hours of training data and Bengali with 28.5 hours of transcribed data, was used in this work to train the proposed model. Results: The Kaldi toolkit was used to perform all the experiments. The paper is investigated over three main points. First, we present the monolingual ASR system using various Neural Network (NN) based acoustic models. Second, we show that Recurrent Neural Network (RNN) language modeling helps to improve the ASR performance further. Finally, we show that a multilingual ASR system significantly reduces the Word Error Rate (WER) (absolute 2% WER reduction for Hindi and 3% for the Marathi language). In all the three languages, the proposed TDNN-BLSTM-A multilingual acoustic models help to get the lowest WER. Conclusion: The multilingual hybrid TDNN-BLSTM-A architecture shows a 13.67% relative improvement over the monolingual Hindi ASR system. The best WER of 8.65% was recorded for Hindi ASR. For Marathi and Bengali, the proposed TDNN-BLSTM-A acoustic model reports the best WER of 30.40% and 10.85%.

Ceasing hate with MoH: Hate Speech Detection in Hindi–English code-switched language

Warning: This manuscript may contain upsetting language.Social media has become a bedrock for people to voice their opinions worldwide. Due to the greater sense of freedom with the anonymity feature, it is possible to disregard social etiquette online and attack others without facing severe consequences, inevitably propagating hate speech. The current measures to sift the online content and offset the hatred spread do not go far enough. One factor contributing to this is the prevalence of regional languages in social media and the paucity of language flexible hate speech detectors. The proposed work focuses on analyzing hate speech in Hindi–English code-switched language. Our method explores transformation techniques to capture precise text representation. To contain the structure of data and yet use it with existing algorithms, we developed ‘MoH’ or (Map Only Hindi), which means ‘Love’ in Hindi. ‘MoH’ pipeline which consists of language identification, Roman to Devanagari Hindi transliteration using a knowledge base of Roman Hindi words, and finally employs the fine-tuned Multilingual Bert, and MuRIL language models. We conducted several quantitative experiment studies on three datasets, and evaluated performance using Precision, Recall and F1 metrics. The first experiment studies ‘MoH’ mapped text’s performance with classical machine learning models and shows an average increase of 13% in F1 scores. The second compares the proposed work’s scores with those of the baseline models and shows a rise in performance by 6%. Finally, the third compares the proposed ‘MoH’ technique with various data simulations using the existing transliteration library. Here, ‘MoH’ outperforms the rest by 15%. Our results demonstrate a significant improvement in the state-of-the-art scores on all three datasets.

Effects of the Dynamic and Energy Based Feature Extraction on Hindi Speech Recognition

Background: Speech Recognition is the most effective and suitable way of communication. Extracted features play an important role in speech recognition. Previous research works for Hindi speech recognition lack detailed comparative analysis of the feature extraction methods using dynamic and energy parameters. Objective: The research work presents experimental work done to explore the effects of integrating dynamic coefficients and energy parameters with different feature extraction techniques on Connected word Hindi Speech recognition. As extracted features play a significant role in speech recognition, a comparative analysis is presented to show the effects of integration of dynamic and energy parameters to basic extracted features. Method: Speaker dependent system was proposed with monophones based five states Hidden Markov Model (HMM) using HTK Tool kit. Speech data set of connected words in Hindi was created. The feature extraction techniques such as Linear Predictive Coding Cepstral coefficients (LPCCs), Mel Frequency Cepstral Coefficients (MFCCs), and Perceptual Linear Prediction (PLPs) coefficients were applied integrating delta, delta2, and energy parameters to evaluate the performance of the proposed methodology for speaker dependent recognition. Results: Experimental results show that the system achieved the highest recognition word accuracy of 89.97% using PLP coefficients. The PLP coefficients achieved 4% increment in word accuracy than original MFCCs and a 16% increment in word accuracy than LPCCs. Adding energy parameters to original MFCCs increased word accuracy by 1.5% only while adding dynamic coefficients delta and double delta had no significant effect on speech recognition accuracy. Conclusion: Research findings reveal that PLP coefficients outperformed. Explorations reveal that the integration of energy parameters are better than original MFCCs. Investgations also reveal that adding energy parametres improved recognition score while adding delta and delta2 coefficients to basic features did not improve the recognition scores. Research findings could be used to enhance the performance of a speech recognition system by using a suitable feature extraction technique and combining the different feature extraction techniques. Further, investigations can be used to develop language resources for refining speech recognition. The work can be extended to develop a continuous Hindi speech recognition system.

Teaching Indefinite Articles in English Using Hindi Speech Sounds: A Classroom Action Research

This paper focuses on teaching the use of indefinite articles ‘a’ and ‘an’ appropriately to the students who have had their schooling from Hindi medium schools and are presently pursuing bachelor and diploma courses. It was seen that these students fail to use correct indefinite articles before English words because of the lack of knowledge of consonant and vowel sounds in English language, further to solve this issue a technique was introduced to the students where they could use their prior knowledge of sounds associated to Hindi letters of alphabet. Classroom action research was performed for the fulfillment of the purpose of this research.

QLSTM-based Joint-Training for Noise Robust Hindi Speech Recognition

QLSTM-based Joint-Training for Noise Robust Hindi Speech Recognition

Monophone-based connected word Hindi speech recognition improvement

Monophone-based connected word Hindi speech recognition improvement

An exploration of semi-supervised and language-adversarial transfer learning using hybrid acoustic model for hindi speech recognition

Semi-supervised training and language adversarial transfer learning are two different techniques to improve the Automatic Speech Recognition (ASR) performance in limited resource conditions. In this paper, we combined these two techniques and presented a common framework for the Hindi ASR system. For acoustic modeling, we proposed a hybrid architecture of SincNet-Convolutional Neural Network (CNN)-Light Gated Recurrent Unit (LiGRU), which shows increased interpretability, high accuracy, and fewer parameter size. We investigate the impact of the proposed hybrid model on monolingual Hindi ASR with semi-supervised training, and multilingual Hindi ASR with language adversarial transfer learning. In this work, we have chosen three Indian languages (Hindi, Marathi, Bengali) of the same Indo-Aryan family for multilingual training. All experiments were conducted using Kaldi and Py-Torch Kaldi toolkits. The proposed model with combined learning strategies helps to get the lowest 5.5% Word Error Rate (WER) for Hindi ASR.

Feature Extraction Techniques with Analysis of Confusing Words for Speech Recognition in the Hindi Language

The research work presents experimental work to build a speaker-independent connected word Hindi speech recognition system using different feature extraction techniques with comparative analysis of confusing words. Comparative analysis of confusing words is essential to understand the reason for the speech recognition errors. Based on the error analysis, different feature extraction techniques, classification techniques, acoustic models, and pronunciation dictionaries can be selected to improve the speech recognition system's performance. Earlier studies for Hindi speech recognition lack detailed comparative analysis of confusing words for different feature extractions methods. As speaker-independent systems are developed for all, comparative analysis of confusing words is also presented for all feature extraction techniques. Speaker independent system was proposed with five states monophone based hidden Markov model (HMM) using HMM-based tool kit HTK. A Self-created data set of Hindi speech corpus has been used in the experiment. Feature extraction techniques such as linear predictive coding cepstral coefficients (LPCCs), mel frequency cepstral coefficients (MFCCs), and perceptual linear prediction coefficients (PLPs) were applied using delta, double delta, and energy parameters to evaluate the performance of the proposed methodology. The system was assessed by using different feature extraction techniques for speaker-independent mode. Research findings reveal that PLP coefficients show the highest recognition score, while LPCCs got the lowest recognition scores.Investigations also reveal that both PLP and MFCC coefficients are better than LPCC in speech recognition. Comparative analysis of confusing words shows that PLPs and MFCCs show fewer confusions than LPCCs and exhibit mostly the same pattern in the confusion analysis. Research outcomes also reveal that substitution errors are a significant cause of low recognition. It was also found that some words were recognized with individual feature extraction techniques only. Confusion analysis of the words indicates that words which have nasals, liquid, and fricative sound in first place exhibit more confusions. The investigation could improve speech recognition by choosing an appropriate feature extraction method and mixing the various feature extraction methods. The research outcomes can also be utilized to build linguistic resources for improving speech recognition. The results show that the developed recognition framework achieved the highest recognition word accuracy of 76.68% with PLPs for the speaker-independent model. The proposed system was also compared with existing similar work available.

Hindi/Bengali Sentiment Analysis using Transfer Learning and Joint Dual Input Learning with Self Attention

Sentiment Analysis typically refers to using natural language processing, text analysis, and computational linguistics to extract effect and emotion-based information from text data. Our work explores how we can effectively use deep neural networks in transfer learning and joint dual input learning settings to effectively classify sentiments and detect hate speech in Hindi and Bengali data. We start by training Word2Vec word embeddings for Hindi HASOC dataset and Bengali hate speech [1] and then train LSTM and subsequently, employ parameter sharing based transfer learning to Bengali sentiment classifiers by reusing and fine-tuning the trained weights of Hindi classifiers with both classifiers being used as the baseline in our study. Finally, we use BiLSTM with self-attention in a joint dual input learning setting where we train a single neural network on the Hindi and Bengali datasets simultaneously using their respective embeddings.

Hindi speech recognition in noisy environment using hybrid technique

Automatic speech recognition is generally analyzed for two types of word utterances; isolated and continuous-words speech. Continuous-words speech is almost natural way of speaking but is difficult to be recognized through machines (speech recognizers). It is also highly sensitive to environmental variations. There are various parameters which are directly affecting the performance of automatic speech recognition like size of datasets/corpus, type of data sets (isolated, spontaneous or continuous) and environment variations (noisy/clean). The performance of speech recognizers is generally good in clean environments for isolated words, but it becomes typical in noisy environments especially for continuous words/sentences and is still a challenge. In this paper, a hybrid feature extraction technique is proposed by joining core blocks of PLP (perceptual linear predictive) and Mel frequency cepstral coefficients (MFCC) that can be utilized to improve the performance of speech recognizers under such circumstances. Voice activity and detection (VAD)-based frame dropping formula has been used solely within the training part of ASR (automatic speech recognition) procedure obviating its need in actual implementations. The motivation to use this formula is for removal of pauses and distorted elements of speech improving the phonemes modeling further. The proposed method shows average improvement in performance by 12.88% for standard datasets.

Qualitative Spectral Parameter Coding for Hindi and English Speech Signals

Qualitative Spectral Parameter Coding for Hindi and English Speech Signals

Performance of deer hunting optimization based deep learning algorithm for speech emotion recognition

This paper proposes a speech emotion recognition technique based on Optimized Deep Neural Network. The speech signals are denoised by presenting a novel adaptive wavelet transform with a modified galactic swarm optimization algorithm (AWT_MGSO). From the noise removed speech signals, the spectral features like LPC (Linear Prediction Coefficients), MFCC (Mel frequency cepstral coefficients), PSD (power spectral density) and prosodic features like energy, entropy, formant frequencies and pitch are extracted and certain features are selected by ASFO (Adaptive Sunflower Optimization Algorithm). The optimized DNN-DHO (Deep Neural Network with Deer Hunting Optimization Algorithm) is proposed for emotion classification. An enhanced squirrel search algorithm is proposed to update the weight in the optimized DNN_DHO classifier. In this study, all the eight emotions of the speech from RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song) and TESS (Toronto Emotional Speech Set) databases for English and IITKGP-SEHSC (Indian Institute of Technology Kharagpur Simulated Emotion Hindi Speech Corpus) database for Hindi are classified. The experimental results are obtained and compared with the classifiers such as DNN_DHO, DNN (Deep Neural Network) and DAE (Deep Auto Encoder). The experimental results show that the proposed algorithm obtains maximum accuracy as 97.85% by the TESS dataset, 97.14% by the RAVDESS dataset and 93.75% by the IITKGP-SEHSC dataset by the DNN-HHO classifier.

A hybrid CNN-LiGRU acoustic modeling using raw waveform sincnet for Hindi ASR

Deep Neural Network (DNN) is currently playing the most vital role in Automatic Speech Recognition (ASR). Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) are the advanced versions of DNN. CNN and RNN are right to deal with spatial and temporal properties of the speech signal, respectively, and both properties have a higher impact on accuracy. In today’s scenario, many acoustic modeling techniques often switches due to the battle of CNNs and RNNs. In the last few years, CNN, with raw speech signal, shows their superiority over precomputed acoustic features. Recently, a novel first convolution layer named SincNet was proposed to produce the interpretable filters with better accuracy. In this work, we proposed a hybrid SincNet-CNN-RNN architecture with low computation cost and high accuracy. Different configurations of the hybrid model were extensively examined to achieve this goal. All experiments were performed on the Hindi speech dataset.