Hindi Text Research Articles

SMDDH: Singleton Mention Detection using Deep Learning in Hindi Text

Mention detection is an important component of the Coreference Resolution (CR) system, where mentions such as name, nominal, and pronominals are identified. These mentions can be purely coreferential mentions or singleton mentions (non-coreferential mentions). Coreferential mentions are those mentions in a text that refer to the same entities in the real world. Whereas, singleton mentions are mentioned only once in the text and do not participate in the coreference as they are not mentioned again in the following text. Filtering of these singleton mentions can substantially improve the performance of a CR process. This paper proposes a singleton mention detection module based on a Fully Connected Network (FCN) and a Long Short-Term Memory for Hindi text and model identifying singleton mentions so that these mentions can be filtered out to reduce the search space for CR. A CR system can look for the previous reference of that mention in the text and if these mentions are removed from the list of mentions, then it reduces the searching time and also space time. This model utilizes a few hand-crafted features, context information, and embedding for words from word2vec and a multilingual Bidirectional Encoder Representations from Transformers (mBERT) language model. The coreference annotated Hindi dataset comprising 3.6K sentences, and 78K tokens are used for the task. The singleton mention detection model is analyzed extensively by experimenting with various lengths of context windows for each mention. The performance of the model is significant with two window sizes of context as compared to other various window sizes of contexts such as 2,3,4,5, etc., and all previous and all next words of each mention. The Precision, Recall, and F-measure of the LSTM-FCN model with mBERT (Word + Context + Syntactic) with two window sizes of context for identifying the singleton mentions are 63%, 71%, and 67% respectively.

Am I Hurt?: Evaluating Psychological Pain Detection in Hindi Text Using Transformer-based Models

The automated evaluation of pain is critical for developing effective pain management approaches that seek to alleviate pain while preserving patients’ functioning. Transformer-based models can aid in detecting pain from Hindi text data gathered from social media by leveraging their ability to capture complex language patterns and contextual information. By understanding the nuances and context of Hindi text, transformer models can effectively identify linguistic cues and sentiments and expressions associated with pain, enabling the detection and analysis of pain-related content present in social media posts. The purpose of this research is to analyze the feasibility of utilizing NLP techniques to automatically identify pain within Hindi textual data, providing a valuable tool for pain assessment in Hindi-speaking populations. The research showcases the HindiPainNet model, a deep neural network that employs the IndicBERT model, classifying the dataset into two class labels {pain, no_pain} for detecting pain in Hindi textual data. The model is trained and tested using a novel dataset, दर्द-ए-शायरी (pronounced as Dard-e-Shayari ), curated using posts from social media platforms. The results demonstrate the model's effectiveness, achieving an accuracy of 70.5%. This pioneer research highlights the potential of utilizing textual data from diverse sources to identify and understand pain experiences based on psychosocial factors. This research could pave the path for the development of automated pain assessment tools that help medical professionals comprehend and treat pain in Hindi-speaking populations. Additionally, it opens avenues to conduct further NLP-based multilingual pain detection research, addressing the needs of diverse language communities.

Sentiment classification in Hindi text using hybrid deep learning method

Sentiment classification in Hindi text using hybrid deep learning method

PARIBHAASHA

As in world increasingly reliant on digital communication, effective writing skills are paramount. However, for “Hindi” language users, finding tools that take their unique linguistic needs presents a challenge. This abstract will come with a novel solution: a Grammarly-like platform precisely designed to empower Hindi writers by offering advanced language correction and enhancement features tailored specifically for the nuances of the Hindi language. At the core of our platform lies a sophisticated text analysis engine, powered by state-of-the-art natural language processing algorithms meticulously trained on vast corpora of Hindi text. Our platform offers a seamless user experience, with a user-friendly interface designed to facilitate effortless interaction in single time of corrections. Users can simply input their “Hindi” text into the platform, and within moments, receive comprehensive feedback and suggestions aimed at refining their writing to perfection. In addition to its high dynamic language correction capabilities, our platform also features advanced plagiarism detection functionality. Leveraging cutting-edge algorithms that is particular focusing on platform thoroughly scans user-submitted text against an extensive database of online sources, academic papers, and published works to ensure originality and integrity, vulnerable and fully secured. Furthermore, our platform goes beyond mere error correction to offer personalized writing insights and guidance. Users can set custom writing goals based on their intended audience, style preferences, and communication objectives. Our platform then provides tailored suggestions and recommendations aimed at helping users achieve their desired writing outcomes, whether it be crafting formal business documents, creative literature, or casual social media posts. This engine enables our platform to accurately identify and rectify a myriad of grammatical errors, spelling mistakes, and stylistic inconsistencies commonly encountered in Hindi writing.

Enhanced word vector space with ensemble deep learning model for COVID-19 Hindi text sentiment analysis

Enhanced word vector space with ensemble deep learning model for COVID-19 Hindi text sentiment analysis

Automatic Extractive Text Summarization using Multiple Linguistic Features

Automatic text summarization (ATS) provides a summary of distinct categories of information using natural language processing (NLP). Low-resource languages like Hindi have restricted applications of these techniques. This study proposes a method for automatically generating summaries of Hindi documents using extractive technique. The approach retrieves pertinent sentences from the source documents by employing multiple linguistic features and machine learning (ML) using maximum likelihood estimation (MLE) and maximum entropy (ME). We conducted pre-processing on the input documents, such as eliminating Hindi stop words and stemming. We have obtained 15 linguistic feature scores from each document to identify the phrases with high scores for summary generation. We have performed experiments over BBC News articles, CNN News, DUC 2004, Hindi Text Short Summarization Corpus, Indian Language News Text Summarization Corpus, and Wikipedia Articles for the proposed text summarizer. The Hindi Text Short Summarization Corpus and Indian Language News Text Summarization Corpus datasets are in Hindi, whereas BBC News articles, CNN News, and the DUC 2004 datasets have been translated into Hindi using Google, Microsoft Bing, and Systran translators for experiments. The summarization results have been calculated and shown for Hindi as well as for English to compare the performance of a low and rich-resource language. Multiple ROUGE metrics, along with precision, recall, and F-measure, have been used for the evaluation, which shows the better performance of the proposed method with multiple ROUGE scores. We compare the proposed method with the supervised and unsupervised machine learning methodologies, including support vector machine (SVM), Naive Bayes (NB), decision tree (DT), latent semantic analysis (LSA), latent Dirichlet allocation (LDA), and K-means clustering, and it was found that the proposed method outperforms these methods.

Hindi Abstractive Text Summarization using Transliteration with Pre-trained Model

Automatic text summarization is a subarea of natural language processing that generates a summary of the text by keeping its key points. The research work done on summarizing low-resourced language text is very limited. In India, the Hindi language is being spoken by central and north Indian people and only a few research works have been done on abstractive summarization of Hindi language. Having matras in Hindi makes it difficult to tokenize so it is difficult to summarize Hindi text using abstractive text summarization. In the proposed method, abstractive Hindi text summarization is done using transliteration and fine-tuning. In this work, the model is trained to generate both summaries and headlines. ROUGE-score and BERT-score have been utilized to check summary quality. A new semantic similarity score-based performance measure is also proposed to measure semantic similarity between reference summaries and predicted summaries. Using the proposed method, we have achieved the highest 55.16 ROUGE score, 0.80 BERT score, and 0.98 similarity score. Along with these performance measures, human evaluation of predicted summaries is also done and it is found that summaries and headlines were generated at a human-acceptable level.

AI unveiled personalities: Profiling optimistic and pessimistic attitudes in Hindi dataset using transformer‐based models

AbstractBoth optimism and pessimism are intricately intertwined with an individual's inherent personality traits and people of all personality types can exhibit a wide range of attitudes and behaviours, including levels of optimism and pessimism. This paper undertakes a comprehensive analysis of optimistic and pessimistic tendencies present within Hindi textual data, employing transformer‐based models. The research represents a pioneering effort to define and establish an interaction between the personality and attitude chakras within the realm of human psychology. Introducing an innovative “Chakra” system to illustrate complex interrelationships within human psychology, this work aligns the Myers‐Briggs Type Indicator (MBTI) personality traits with optimistic and pessimistic attitudes, enriching our understanding of emotional projection in text. The study employs meticulously fine‐tuned transformer models—specifically mBERT, XLM‐RoBERTa, IndicBERT, mDeBERTa and a novel stacked mDeBERTa—trained on the novel Hindi dataset ‘मनोभाव’ (pronounced as Manobhav). Remarkably, the proposed Stacked mDeBERTa model outperforms others, recording an accuracy of 0.7785 along with elevated precision, recall, and F1 score values. Notably, its ROC AUC score of 0.7226 underlines its robustness in distinguishing between positive and negative emotional attitudes. The comparative analysis highlights the superiority of the Stacked mDeBERTa model in effectively capturing emotional attitudes in Hindi text.

Classifying Hindi News Using Various Machine Learning and Deep Learning Techniques

Text classification involves organizing textual information into predefined classes, a task which is particularly useful in domains like sentiment analysis, spam detection, and content labeling. In India, where a massive amount of information is generated daily through newspapers and social media, Hindi is one of the most widely used and spoken languages. However, there is limited research on Hindi text classification and, particularly, regarding Hindi news classification. This paper presents a research study to classify Hindi news articles published in Hindi-language newspapers in India by using and comparing various Machine Learning (ML) and Deep Learning (DL) algorithms. To prepare the textual news data for classification, pre-processing and feature engineering techniques, such as count vectorizer, Tf-Idf vectorizer and Doc2Vec, were used and applied to convert texts into vectors. This pre-processing step on the textual data was very challenging due to the presence of multimodal words, conjunctions, punctuation, and special characters in Hindi texts. The study considered Hindi news headlines from predetermined categories (Science, Sports, Entertainment and Business) and, among the different ML and DL models tested and evaluated, Linear Regression with Doc2Vec vectorizer and SGD classifier with Tf-Idf vectorizer produced best accuracies of 97.04% and 96.59%, respectively. The best performing DL model was found to be the Bi-LSTM with an accuracy of approximately 97% on the testing data.

Particle swarm optimization model for Hindi text summarization

Particle swarm optimization model for Hindi text summarization

Optimized Hybrid Model for COVID-19 Vaccine Sentiment Analysis for Hindi Text

Optimized Hybrid Model for COVID-19 Vaccine Sentiment Analysis for Hindi Text

Multi-Class Classification of Hindi Text using Machine Learning

This study focuses on the multi-class classification of Hindi texts using ML techniques. The goal is to develop an efficient model that can accurately classify Hindi text documents into various predefined categories. We explore the preprocessing steps necessary for cleaning and preparing the text data, tokenization, stop-word removal, and stemming. Feature extraction methods such as TF-IDF and word embeddings are employed to represent the text data numerically. It includes ML algorithm like Supervised Learning algorithm (Naïve Bayes, SVM, Decision tree and Random Forest) are experimented with to determine the best-performing model. The dataset used for training and testing consists of a diverse range of Hindi texts from different domains. The experimental results provide insights into the effectiveness of different approaches and algorithms, shedding light on the challenges and opportunities of Hindi texts. Keywords: Multi-Class Classification, Hindi Texts, Machine Learning Algorithms, Natural Language Processing, Text Classification, Feature Extraction, Evaluation Metrics.

HINDI TEXT CATEGORIZATION & TRANSLATION

The Hindi Text Translation and Categorization aims to develop a comprehensive system for translating and categorizing Hindi text content. This project addresses the growing need for effective language processing tools in the context of Hindi, one of the most widely spoken languages globally. The project consists of two primary components: translation and categorization. The translation module employs state-of-the-art machine translation techniques to provide accurate and contextually relevant translations between Hindi and other languages. This functionality is essential for breaking down language barriers and facilitating communication across diverse linguistic communities. The categorization module focuses on organizing and classifying Hindi text content into relevant categories or topics. Leveraging advanced natural language processing (NLP) algorithms, the system can analyze the semantic meaning of text and assign it to appropriate categories. This categorization enhances content management, information retrieval, and overall user experience, particularly in applications such as content recommendation systems and information filtering.

A Novel Ensemble Model for Complex Entities Identification in Low Resource Language

The fundamental method for pre-processing speech or text data that enables computers to comprehend human language is known as natural language processing. Numerous models have been developed to date to pre-process data in the English language; however, the Hindi language does not support these models. India's national tongue is Hindi. In order to help the locals, the authors of this study used supervised learning methods like Linear Regression, SVM, and Naive Bayes algorithm to investigate a dataset of complicated terms in the Hindi language. Additionally, a sophisticated Hindi word classification model is suggested employing several methods based on the forecasts as well as collective learning strategies like Random Forest, Adaboost, and Decision Tree. Depending on how well the user's language is understood, the suggested model will assist in simplifying Hindi text. Authors attempt to classify the uncharted dataset using deep learning algorithms like Bi-LSTM and GRU approaches in further processing.

LiHiSTO: a comprehensive list of Hindi stopwords

A preliminary preprocessing step in text analytics is the removal of words with no semantic meaning, otherwise known as stopwords. English stopwords are very easily accessible and created due to the broad usability of the English language. However, a standard list of Hindi stopwords is still missing. This paper proposes an exhaustive list of generic Hindi stopwords and a Python package for easy distribution and usage. The methodology uses a dual mechanism for creating a list of Hindi stopwords. First, the famous English stopwords are collected and translated into meaningful Hindi words (group 1). Second, unique Hindi stopwords from multiple sources are fetched (group 2). Finally, the respective Hindi stopwords from groups 1 and 2 are combined, which resulted in a significantly large set of 820 Hindi stopwords. Additionally, the list of Hindi stopwords is made openly available for use at the Python Package Index (PyPI) repository as a Python package, which is named LiHiSTO. With the help of illustrative implementations, it is shown that LiHiSTO provides abstract and easy access to the list of stopwords for users to perform Hindi text analytics.

Hindi Text Summarization Using Sequence to Sequence Neural Network

Text summarizing reduces a large block of text data to a precise, short, and intelligible text that conveys the whole meaning of the actual text in a few words while maintaining the original context. Due to a lack of relevant summaries, it is hard to understand the main idea of the document. Text summarization using the abstractive technique is well-studied in English, although it is still in its infancy in Indian regional languages. In this study, we investigate the effectiveness of using a sequence-to-sequence (Seq2Seq) neural network based on attention and its optimization for text summarization for the Hindi language (HiATS), explicitly comparing the Adam and RMSprop optimizers. Our method allows the model to take the Hindi language dataset and, as output, provides a concise summary that accurately reflects the gist of the original text. The performance of the models will be evaluated using Rouge-1 and Rouge-2 metrics.

HINDI FAKE NEWS DETECTOR

Fake news has proliferated on the internet in recent decades. More people than ever before are creating and sharing knowledge because of social networks, many of which have no connection to reality. This has led to the rapid dissemination of false information used for various political and business objectives. Finding reliable news sources has become more difficult due to online newspapers. In this work, we gathered news articles of Hindi text from various news sources. Techniques for pre-processing, feature extraction, classification, and prediction are all extensively covered. A “Fake news detection system” has been developed in this project. Various Hindi news articles have been collected from multiple sources to help diversify the dataset and train the model better. The project first pre-processes the dataset and uses the pre-trained Bert model for feature extraction. Then, the data is classified from the dataset and prediction processes are employed on the dataset. Various machine learning algorithms and deep learning models like Naïve Bayes, Long Short-Term Memory, Logistic Regression have been employed in previous works for the purpose of detecting fake news. Pre-processing steps include data cleaning, stop words removal, tokenizing, stemming. The testing and training of the dataset include using the BERT for sequence classification model. The model is trained and tested against the validation dataset

Identifying and Analyzing Reduplication Multiword Expressions in Hindi Text Using Machine Learning

The task of identifying and analyzing Reduplication Multiword Expressions (RMWEs) in Natural Language Processing (NLP) involves extracting repeated words from various text forms and classifying them into Onomatopoeic, non-Onomatopoeic, partial, or semantic types. With the increasing use of low-resource languages in news, opinions, comments, hashtags, reviews, posts, and journals, this study proposes a machine learning-based RMWE identification method for Hindi text. The method employs linguistic patterns and statistical data, along with a proposed threshold boundary detection in statistical filtering. The Jaccard distance of dissimilarity and Sorensen Dice Coefficient of Similarity are used for semantic relation analysis. The proposed approach was evaluated using the publicly available Hindi corpus from IITB, measuring performance between two consecutive thresholds with the lowest error and highest recall. This study proposes an effective method for Indian computational linguistics, with experimental results highlighting its viability and utility, and providing a blueprint for current procedures.

Psquad: Plagiarism detection and document similarity of Hindi text

Psquad: Plagiarism detection and document similarity of Hindi text

Hybrid Optimization Based Hindi Document Summarization Using Deep Learning Technique

The proliferation of textual information today is a result of the internet's recent development, which is widely accessible to anybody, at any time. Generally speaking, several Natural Language Processing (NLP) techniques can be used to analyze the textual information that is offered on the basis of text documents. In recent years, various text summarization techniques have been implemented in English text documents but a little amount of work is carried out in Hindi text documents summarization. In this research investigation, the Coot Remora Optimization (CRO) technique based on Deep Recurrent Neural Network (DRNN) is used to summarize Hindi documents. Here, the CRO algorithm is used to train the DRNN, which is used to compute the sentence scores.The highest scored sentences are going to included in the summary. When compared to recent optimization algorithmic techniques, such as MCRMR-SSO, Graph-based_PSO, Genetic Algorithms (GA), and Political Elephant Herding Optimization (PEHO) based Deep Long Short Term Memory (DLSTM) algorithm, the developed method is shown to be superior. Additionally, three evaluation metrics such as precision, recall, f-measure are used to analyze the performance of the CRO based DRNN technique and obtained high performance.