Hindi Text Research Articles

Detecting Gurmukhi and Hindi text object present in images

This paper resolves the trouble detecting Gurmukhi and Hindi text objects present in images. Therefore, we propose a new region-based bottom-up method to extract text from images, where we first identify elementary substructures using connected components and edges and then merge them successively. As each character counter has high contrast as compared to its neighbours, all character pixels and several non-character pixels which exhibit excessive neighbourhood. Although a lot of noise is removed by this step a second pass filter is applied based on stroke width. The stroke width of Hindi and Gurmukhi characters are approximately uniform. The candidates with similar properties are aggregated into chains, according to the observation that the figures of a book. For performance evaluation, we created our own dataset containing 60 images of varied size, resolution and type. The performance is evaluated on this dataset containing caption, document, and scene text images using different performance metrics.

An Application of Zipf's Law for Prose and Verse Corpora Neutrality for Hindi and Marathi Languages

Availability of the text in different languages has become possible, as almost all websites have offered multilingual option. Hindi is considered as official language in one of the states of India. Hindi text analysis is dominated by the corpus of stories and poems. Before performing any text analysis token extraction is an important step and supports many applications like text summarization , categorizing text and so on. Token extraction is a part of Natural language processing (NLP). NLP includes many steps such as preprocessing the corpus, lemmatization and so on. In this paper the tokens are extracted by two methods and on two corpora. BaSa, a context-based term extraction technique having different NLP activities, e.g. Term Frequency Inverse Document Frequency (TF-IDF) and Zipf ‘s law are used to count and compare extracted tokens. Further token comparison between both of the methods is achieved. The corpus contains proses and verses of Hindi as well as the Marathi language. Common tokens from corpora of verses and proses of Marathi as well as Hindi are identified to prove that both of them behave same as per as NLP activities are concerened. The betterment of BaSa over Zipf’s law is proved. Hindi Corpus includes 820 stories and 710 poems and Marathi corpus includes 610 stories and 505 poems.

Detecting Gurmukhi and Hindi text object present in images

This paper resolves the trouble detecting Gurmukhi and Hindi text objects present in images. Therefore, we propose a new region-based bottom-up method to extract text from images, where we first identify elementary substructures using connected components and edges and then merge them successively. As each character counter has high contrast as compared to its neighbours, all character pixels and several non-character pixels which exhibit excessive neighbourhood. Although a lot of noise is removed by this step a second pass filter is applied based on stroke width. The stroke width of Hindi and Gurmukhi characters are approximately uniform. The candidates with similar properties are aggregated into chains, according to the observation that the figures of a book. For performance evaluation, we created our own dataset containing 60 images of varied size, resolution and type. The performance is evaluated on this dataset containing caption, document, and scene text images using different performance metrics.

Machine Translation Evaluation: Unveiling the Role of Dense Sentence Vector Embedding for Morphologically Rich Language

Machine Translation (MT) evaluation metrics like BiLingual Evaluation Understudy (BLEU) and Metric for Evaluation of Translation with Explicit Ordering (METEOR) are known to have poor performance for word-order and morphologically rich languages. Application of linguistic knowledge to evaluate MTs for morphologically rich language like Hindi as a target language, is shown to be more effective and accurate [S. Tripathi and V. Kansal, Using linguistic knowledge for machine translation evaluation with Hindi as a target language, Comput. Sist.21(4) (2017) 717–724]. Leveraging the recent progress made in the domain of word vector and sentence vector embedding [T. Mikolov and J. Dean, Distributed representations of words and phrases and their compositionality, Adv. Neural Inf. Process. Syst. 2 (2013) 3111–3119], authors have trained a large corpus of pre-processed Hindi text ([Formula: see text] million tokens) for obtaining the word vectors and sentence vector embedding for Hindi. The training has been performed on high end system configuration utilizing Google Cloud platform resources. This sentence vector embedding is further used to corroborate the findings through linguistic knowledge in evaluation metric. For morphologically rich language as target, evaluation metric of MT systems is considered as an optimal solution. In this paper, authors have demonstrated that MT evaluation using sentence embedding-based approach closely mirrors linguistic evaluation technique. The relevant codes used to generate the vector embedding for Hindi have been uploaded on code sharing platform Github. a

Error tolerant global search incorporated with deep learning algorithm to automatic Hindi text summarisation

Error tolerant global search incorporated with deep learning algorithm to automatic Hindi text summarisation

“UTTAM”

In this article, we propose a system called “UTTAM,” for correcting spelling errors in Hindi language text using supervised learning. Unlike other languages, Hindi contains a large set of characters, words with inflections and complex characters, phonetically similar sets of characters, and so on. The complexity increases the possibility of confusion and occasionally leads to entering a wrong character in a word. The existence of spelling errors in text significantly decreases the accuracy of the available resources, like search engine, text editor, and so on. The proposed work is the first approach to correct non-word (Out of Vocabulary) errors as well as real-word errors simultaneously in a sentence of Hindi language. The proposed method investigates the human behavior, i.e., the type and frequency of spelling errors done by humans in Hindi text. Based on the type and frequency of spelling errors, the heterogeneous data is collected in matrices. This data in matrices is used to generate the suitable candidate words for an input word. After generating candidate words, the Viterbi algorithm is applied to perform the word correction. The Viterbi algorithm finds the best sequence of candidate words to correct the input sentence. For Hindi, this work is the first attempt for real-word error correction. For non-word errors, the experiments show that “UTTAM” performs better than the existing systems SpellGuru and Saksham.

The Hindi to Dogri machine translation system: grammatical perspective

This paper is a part of the research work done to develop a machine translation system (MTS) to convert Hindi text into Dogri. The process of machine translation includes various key activities such as preprocessing, POS tagging, transliteration, word order alignment, translation, morphological analysis, etc.; but it is not mandatory to include all of them in every MTS. The inclusion of these activities varies depending on the languages undertaken and also the approach used for its development. The first and the foremost step for the development of any MTS, is to study and analyze the languages undertaken. This paper presents a comparative study of Hindi and Dogri keeping in view machine translation. This study has helped in the selection of the approach of translation, the preprocessing activities to be applied on the source text, for developing rules for inflectional analysis and other activities to be done for processing some special cases pertaining to Dogri. The grammatical and inflectional analysis of Hindi and Dogri is presented in this paper.

Automatic Construction of Generic Stop Words List for Hindi Text

In this technological world, Hindi text is speedily increasing on web and attracting many users and researchers to retrieve useful information from this data. In Information Retrieval (IR) process, user comes across many words of least or no semantic importance are called as stop words. Removal of such common words can result in to effective indexing of corpus and enhancement of IR systems performance. To achieve performance of IR systems construction of stop word list becomes the basic requirement of any IR process. Many stop words lists are created for English, Chinese and other European languages, but no such standard list is available for Hindi language over Internet. In order to save user time and overhead of manual picking of stop words; we have tried to implement aggregation model based on social choice theory of election process. In aggregation model, both statistical and information based methods are implemented to construct stop words list for Hindi. Results are shown with the help of tables and graphs.

Word sense-based approach for Hindi to Tamil machine translation using English as pivot language

Machine translation is defined as the translation of source text to a desired target text. As there is resource availability in different languages in the internet world, there is need to share the knowledge to a different set of audience who knows only their native language. The proposed system is aimed to build a word sense-based statistical machine translation system (Hindi to Tamil). Since there is a lack of resources in these languages, there is need of some other intermediate pivot language which has high resource availability and English language is chosen as one. Initially, the Hindi text is subjected to pre-processing phase where the text is morphologically and syntactically analysed. After analysis, the senses of the words are identified using latent semantic analysis (LSA) in order to provide a meaningful translation. Once these analyses are done, the sentence is subjected to statistical translation from source to target language through the intermediate pivot language. This system has an improved efficiency when compared with the system that does not have sense identification and pivot language.

Word Sense Based Approach for Hindi to Tamil Machine Translation Using English as Pivot Language

Machine translation is defined as the translation of source text to a desired target text. As there is resource availability in different languages in the internet world, there is need to share the knowledge to a different set of audience who knows only their native language. The proposed system is aimed to build a word sense-based statistical machine translation system (Hindi to Tamil). Since there is a lack of resources in these languages, there is need of some other intermediate pivot language which has high resource availability and English language is chosen as one. Initially, the Hindi text is subjected to pre-processing phase where the text is morphologically and syntactically analysed. After analysis, the senses of the words are identified using latent semantic analysis (LSA) in order to provide a meaningful translation. Once these analyses are done, the sentence is subjected to statistical translation from source to target language through the intermediate pivot language. This system has an improved efficiency when compared with the system that does not have sense identification and pivot language.

Indian Influences in the Supernatural Elements Found in the Malay Classic Films of Dewi Murni, Gul Bakawali and Selendang Delima

The paper examines that the supernatural elements found in the Malay movies DewiMurni 1950 directed by B.S. Rajhans, SelendangDelima 1958 directed by K.M. Basker and Gul Bakawali 1963 directed by B. Narayanan Rao which are heavily embedded with Indian elements. Analysis into the films’ repertoires have shown striking parallels and significant differences between the Malay films and its Indian versions. Dewi Murni is derived from Kalidasa’s Shakuntala. Excerpts from the Brahmavaivarta Purana and Ramayana are used in the narration of Selendang Delima which is the story of a nymph’s offspring who is cursed and subjected to cruelty. GulBakawali is obtained from a Persian story and deciphered to Hindustani in 1702 by Niphal Chand (Zainal Abidin Ahmad, 1942: 144). Sheikh Mohd. Ali bin Ghulam Hussein Alhandi (an Indian Muslim) is reponsible for translating Gul Bakawali from an Urdu / Hindi text. Elements that have come to be accepted as the culture of the Malay world is the outcome of Indianisation in the region (Noor 2005: 32). One has to agree with Farish Noor when he states ‘that the Malay world has a number of lasting legacies’ (Noor 2005: 32). The pluralist culture that is now inherent amongst the Malays was deepen through a long process of socio-cultural growth and exposure to other traditions. As such, it is not surprising that encounters between native cultures namely Hinduism and Islam in their daily routines were a part of the lives of the local Malay people in the past. The study on the films Dewi Murni, Gul Bakawali and Selendang Delima illustrate that a fusion of culture produces films that is able to withstand the test of time and that cultural convolution is possible.

Text Translation of Scanned Hindi Document to Braille via Image Processing

Objective: Braille - a model to reduce the illiteracy rate among the visually challenged people. So, this paper focuses on text translation/ conversion of scanned Hindi document to Braille, a single mode of communication for visually challenged people via Image processing techniques. Method/Analysis: The translation of scanned Hindi text into Braille code involves the following four processes: (i) Hindi database generation of consonants and matras using image segmentation. (ii) Segmentation of test images into lines followed by words and finally into letters. This step involves three types of segmentation: (a) line segmentation (b) word segmentation (c) letter segmentation. (iii) Letter matching of segmented letters with the generated Hindi database using Principal Component Analysis (PCA). (iv) Conversion of matched letter into corresponding Braille code. The algorithm takes test documents as input and reads the letter wise and maps the letters in the corresponding Braille code. Findings: Three Hindi documents containing single, multiple lines are taken as test documents. These test documents are successfully converted into their corresponding Braille code using the proposed algorithm. A lot of research work has been conducted which emphasizes the conversion of scanned Braille code into the text, but the conversion of scanned text to Braille has not been attempted so far. These results are a step further in the field of Braille communication, adding a new way through which Hindi literature can easily be made available to visually impaired people for improving their knowledge and easy access to documents and books as needed. Improvement/Application: The results obtained will provide easy access of Hindi literature to visually challenged people as there is limited availability of Braille textbooks in the market. The memory consumption is very less as the size of each image in Hindi database is 187x128 pixels. This work can be modified for real time applications, like instant conversion of text to Braille using handheld devices.

N-GRAMS SOLUTION FOR ERROR DETECTION AND CORRECTION IN HINDI LANGUAGE

Hindi is the National language of India, which is still in its early stage of research and development regarding natural language processing applications in comparison to other languages like English, Chinese. Natural language processing is a field of Artificial Intelligence, which includes major tasks such as information retrieval, word segmentation, speech recognition, parsing, part of speech tagging, text classification, automatic text summarization etc. Spelling detection and correction in Hindi language is an important task of NLP which has not gotten sufficient attention till date. Spelling detection and correction for Indian languages such as Hindi is considered as a difficult task. Hindi Language is very different from English language in its phonetic properties and grammatical rules. Thus the existing techniques and methods that are being used to check the errors in English language can’t be used for Hindi Language. There are mainly two types of error: Non word error and real word error. Error detection for non-word error in Hindi language has been done but for real word error no work has been done till date. This paper focused on Real word spelling error detection and correction in Hindi text by using N Grams Model and Levensthein edit distance algorithm.

ASPECT BASED SENTIMENT ANALYSIS OF HINDI TEXT REVIEW

Sentiment analysis (SA) is one of the fastest growing research areas in Natural Language Processing, making it challenging to keep track of all the activities in the area. Increase in user-generated content (UGC) has provided an important aspect for the researchers, industries and government(s) to mine this information. SA mine information from UGC on the basis of polarity as positive, negative or neutral. The problem domain, to which this research is concerned, is to find the sentiment and its respective aspect in the sentence and finally to calculate the overall sentiment score of entered Hindi text to classify each sentence as positive, negative and neutral. In this thesis, we work on the sentiment analysis by devolving an algorithm that identifying the sentiment according to proposed rules based on positions of conjunction, negation and aspects (nouns).

English

English

A Review on Character Segmentation of Touching and Half Character in Handwritten Hindi Text

Segmentation is one of the challenging phases of OCR in case of handwritten text. It has been observed that different languages have different nature of writing the text that causes a number of issues in segmentation aspect. Hindi is one of the official languages of India having the complex structural features. The major problem in handwritten character segmentation is to deal with the touching characters such as conjunct, half-characters, overlapping characters, highly skewed, uneven header lines found in writing pattern of well known Hindi language[]. Segmentation is done on the basis of observed structural properties examined from the writing style of different individuals. This paper reviews different techniques available to dissect the simple, touching, overlapping Hindi text. Also, it discusses the existing techniques with their merits and demerits. It is observed on the basis of related work, few techniques are available for touching and half characters. Hybrid approach including projection profile combined along with the structural property such as width, height, pixel intensity etc is giving quite promising results but still, there exist some challenges due to the constraints that are taken in stated algorithms which need concerned efforts and immense research to improve the segmentation level. Keywords: character segmentation, touching character, devnagari script, header line modifier, half character

Analysis of MWES in Hindi Text Using NLTK

Analysis of MWES in Hindi Text Using NLTK

Understanding Document Semantics from Summaries

Summary of a document contains words that actually contribute to the semantics of the document. Latent Semantic Analysis (LSA) is a mathematical model that is used to understand document semantics by deriving a semantic structure based on patterns of word correlations in the document. When using LSA to capture semantics from summaries, it is observed that LSA performs quite well despite being completely independent of any external sources of semantics. However, LSA can be remodeled to enhance its capability to analyze correlations within texts. By taking advantage of the model being language independent, this article presents two stages of LSA remodeling to understand document semantics in the Indian context, specifically from Hindi text summaries. One stage of remodeling is done by providing supplementary information, such as document category and domain information. The second stage of remodeling is done by using a supervised term weighting measure in the process. The remodeled LSA’s performance is empirically evaluated in a document classification application by comparing the accuracies of classification to plain LSA. An improvement in the performance of LSA in the range of 4.7% to 6.2% is achieved from the remodel when compared to the plain model. The results suggest that summaries of documents efficiently capture the semantic structure of documents and is an alternative to full-length documents for understanding document semantics.

AUTOMATIC TEXT SUMMARIZATION USING SUPERVISED MACHINE LEARNING TECHNIQUE FOR HINDI LANGAUGE

Automatic text summarization is a technique which compresses large text into a shorter text which includes the important information. Hindi is the top-most language used in India and also in a few neighboring countries there is a lack of proper summarization system for Hindi text. Hence,in this paper, we present an approach to the design an automatic text summarizer for Hindi text that generates a summary by extracting sentences. It deals with a single document summarization based on machine learning approach. Each sentence in the document is represented by a set of various features namelysentence paragraph position, sentence overall position, numeric data, presence of inverted commas, sentence length and keywords in sentences. The sentences are classified into one of four classes namelymost important, important, less important and not important. The classes are in turn having ranks from 4 to 1 respectively with “4”indicating most important sentence and “1” being least relevant sentence . Next a supervised machine learning tool SVM rank is used to train the summarizer to extract important sentences, based on the feature vector. The sentences are ordered according to the ranking of classes. Then based on the required compression ratio, sentences are included in the final summary. The experiment was performed on news articles of different category such as bollywood, politics and sports. The performance of the technique is compared with the human generated summaries. The average result of experiments indicates 72% accuracy at 50% compression ratio and 60% accuracy at 25% compression ratio.

Multi Script Text Transliteration to Punjabi

Machine Transliteration is becoming an important research area in Machine Translation, which ensures phonology of target language word. This paper describes the transliteration system that has been developed for multi script text. The proposed system is capable of transliterating English and Hindi text into equivalent Punjabi text using a rule based transliteration approach. Transliteration is more phonetic process because transliteration is more closely related with utterance. Thus there is need to perform phonetic analysis in addition to orthography for forming rules. The accuracy of English-Punjabi and Hindi-Punjabi transliteration is found to be 94% and 98% respectively.