Area Of Language Processing Research Articles

An Autonomous Approach for Type Categorization of Product Data Based on Numerous Text and Attribute Factorizations

Product categorization or Product classification is an area of Natural Language Processing. This special area happens to have a lot of challenges, especially for e-commerce organizations. Scientists and researchers have actively worked on this area over the years using various machine learning & artificial intelligence techniques. Product categorization is the placement and organization of products into their respective categories.

Review Sentimental Analysis

Sentiment analysis or opinion mining is the computational study of people’s opinions, sentiments, attitudes, and emotions expressed in written language. It is one of the most active research areas in natural language processing and text mining in recent years. Its popularity is mainly due to two reasons. First, it has a wide range of applications because opinions are central to almost all human activities and are key influencers of our behaviors. Whenever we need to make a decision, we want to hear others’ opinions.

Entity understanding with hierarchical graph learning for enhanced text classification

Text classification plays an important role in the areas of natural language processing and data mining. In general, a text is usually described around a collection of entities, i.e., the entities are the core part of the text. As a result, a deep understanding of the entities in a text benefits the classification of texts. To understand entities, traditional work tends to introduce concepts or web data for entities. However, we argue that the potential relations between entities are also important for the understanding of entity semantics, thus further supporting the classification of texts. In this paper, we focus on enhancing the performance of the existing text classification models by extracting features from entities with hierarchical graph learning. To this end, we mine the concepts of entities and the relations between them for a given text simultaneously, and further construct the semantic graph of the text. Then a novel hierarchical graph learning model is proposed to learn the graph embedding that well captures the node, relation, and graph structure information. Our experiments show that the proposed method has the ability to effectively improve the performance of the existing text classifiers.

A White-Box Sociolinguistic Model for Gender Detection

Within the area of Natural Language Processing, we approached the Author Profiling task as a text classification problem. Based on the author’s writing style, sociodemographic information, such as the author’s gender, age, or native language can be predicted. The exponential growth of user-generated data and the development of Machine-Learning techniques have led to significant advances in automatic gender detection. Unfortunately, gender detection models often become black-boxes in terms of interpretability. In this paper, we propose a tree-based computational model for gender detection made up of 198 features. Unlike the previous works on gender detection, we organized the features from a linguistic perspective into six categories: orthographic, morphological, lexical, syntactic, digital, and pragmatics-discursive. We implemented a Decision-Tree classifier to evaluate the performance of all feature combinations, and the experiments revealed that, on average, the classification accuracy increased up to 3.25% with the addition of feature sets. The maximum classification accuracy was reached by a three-level model that combined lexical, syntactic, and digital features. We present the most relevant features for gender detection according to the trees generated by the classifier and contextualize the significance of the computational results with the linguistic patterns defined by previous research in relation to gender.

Dynamic Embedding Projection-Gated Convolutional Neural Networks for Text Classification

Text classification is a fundamental and important area of natural language processing for assigning a text into at least one predefined tag or category according to its content. Most of the advanced systems are either too simple to get high accuracy or centered on using complex structures to capture the genuinely required category information, which requires long time to converge during their training stage. In order to address such challenging issues, we propose a dynamic embedding projection-gated convolutional neural network (DEP-CNN) for multi-class and multi-label text classification. Its dynamic embedding projection gate (DEPG) transforms and carries word information by using gating units and shortcut connections to control how much context information is incorporated into each specific position of a word-embedding matrix in a text. To our knowledge, we are the first to apply DEPG over a word-embedding matrix. The experimental results on four known benchmark datasets display that DEP-CNN outperforms its recent peers.

Wiki sense bag creation using multilingual word sense disambiguation

Performance of word sense disambiguation (WSD) is one of the challenging tasks in the area of natural language processing (NLP). Generation of sense annotated corpus for multilingual word sense disambiguation is out of reach for most languages even if resources are available. In this paper we propose an unsupervised method using word and sense embedding or improving the performance of these systems using untagged. Corpora and create two bags namely ontological bag and wiki sense bag to generate the senses with highest similarity. Wiki sense bag provides external knowledge to the system required to boost the disambiguation accuracy. We explore Word2Vec model to generate the sense bag and observe significant performance gain for our dataset.

A novel hybrid methodology for computing semantic similarity between sentences through various word senses

In the area of natural language processing, measuring sentence similarity is an essential problem. Searching for semantic meaning in natural language is a related issue. The task of measuring sentence similarity is to find semantic symmetry in two sentences, not matter how they are arranged. It is important to measure the similarity of sentences accurately. To compute the similarity between sentences, existing methods have been constructed from approaches for large texts. Since these methods work in very high-dimensional spaces, they are inefficient, require human input, and are not flexible enough for some applications. In this study, we propose a hybrid method (HydMethod) which considers not only semantic information including lexical databases, word embeddings, and corpus statistics, but also implied word order information. With lexical databases, our method models human common sense knowledge, and that knowledge can then be adapted to be used in different domains with the incorporation of corpus statistics. Therefore, the methodology is applicable across several domains. As part of our experiments, we used two standard datasets - Pilot Short Text Semantic Similarity Benchmark and MS paraphrase - in order to demonstrate the efficacy of our proposed method. As a result, the proposed method outperforms the existing approaches when tested on these two datasets, giving the highest correlation value for both word and sentence similarity. Moreover, it achieves a maximum of 32% higher increase than only using word vector or WorldNet based methodology. With Rubenstein and Goodenough word & sentence pairs, our algorithm's similarity measure shows a high Pearson correlation coefficient of 0.8953.

Functional Network Changes After High-Frequency rTMS Over the Most Activated Speech-Related Area Combined With Speech Therapy in Chronic Stroke With Non-fluent Aphasia.

ObjectiveHigh-frequency repetitive transcranial magnetic stimulation (HF-rTMS) to the lesional hemisphere requires prudence in selecting the appropriate stimulation spot. Functional near-IR spectroscopy (fNIRS) can be used in both selecting the stimulation spot and assessing the changes of the brain network. This study aimed to evaluate the effect of HF-rTMS on the most activated spot identified with fNIRS and assess the changes of brain functional network in the patients with poststroke aphasia.MethodsA total of five patients received HF-rTMS to the most activated area on the lesional hemisphere, followed by 30 min of speech therapy for 10 days. The Korean version of the Western aphasia battery (K-WAB) and fNIRS evaluation were done 1 day before the treatment, 1 day and 1 month after the last treatment session. Changes of K-WAB and paired cortical interaction and brain network analysis using graph theory were assessed.ResultsAphasia quotient in K-WAB significantly increased after the treatment (P = 0.043). The correlation analysis of cortical interactions showed increased connectivity between language production and processing areas. Clustering coefficients of the left hemisphere were increased over a sparsity range between 0.45 and 0.58 (0.015 < p < 0.031), whereas the clustering coefficients of the right hemisphere, decreased over a sparsity range 0.15–0.87 (0.063 < p < 0.095). The global efficiency became lower over a network sparsity range between 0.47 and 0.75 (0.015 < p < 0.063).ConclusionImprovement of language function and changes of corticocortical interaction between language-related cortical areas were observed after HF-rTMS on the most activated area identified by fNIRS with combined speech therapy in the patients with poststroke aphasia.

Investigating the Efficient Use of Word Embedding with Neural-Topic Models for Interpretable Topics from Short Texts.

With the rapid proliferation of social networking sites (SNS), automatic topic extraction from various text messages posted on SNS are becoming an important source of information for understanding current social trends or needs. Latent Dirichlet Allocation (LDA), a probabilistic generative model, is one of the popular topic models in the area of Natural Language Processing (NLP) and has been widely used in information retrieval, topic extraction, and document analysis. Unlike long texts from formal documents, messages on SNS are generally short. Traditional topic models such as LDA or pLSA (probabilistic latent semantic analysis) suffer performance degradation for short-text analysis due to a lack of word co-occurrence information in each short text. To cope with this problem, various techniques are evolving for interpretable topic modeling for short texts, pretrained word embedding with an external corpus combined with topic models is one of them. Due to recent developments of deep neural networks (DNN) and deep generative models, neural-topic models (NTM) are emerging to achieve flexibility and high performance in topic modeling. However, there are very few research works on neural-topic models with pretrained word embedding for generating high-quality topics from short texts. In this work, in addition to pretrained word embedding, a fine-tuning stage with an original corpus is proposed for training neural-topic models in order to generate semantically coherent, corpus-specific topics. An extensive study with eight neural-topic models has been completed to check the effectiveness of additional fine-tuning and pretrained word embedding in generating interpretable topics by simulation experiments with several benchmark datasets. The extracted topics are evaluated by different metrics of topic coherence and topic diversity. We have also studied the performance of the models in classification and clustering tasks. Our study concludes that though auxiliary word embedding with a large external corpus improves the topic coherency of short texts, an additional fine-tuning stage is needed for generating more corpus-specific topics from short-text data.

Sign and Spoken Language Processing Differences in the Brain: A Brief Review of Recent Research.

Background:It is currently accepted that sign languages and spoken languages have significant processing commonalities. The evidence supporting this often merely investigates frontotemporal pathways, perisylvian language areas, hemispheric lateralization, and event-related potentials in typical settings. However, recent evidence has explored beyond this and uncovered numerous modality-dependent processing differences between sign languages and spoken languages by accounting for confounds that previously invalidated processing comparisons and by delving into the specific conditions in which they arise. However, these processing differences are often shallowly dismissed as unspecific to language.Summary:This review examined recent neuroscientific evidence for processing differences between sign and spoken language modalities and the arguments against these differences’ importance. Key distinctions exist in the topography of the left anterior negativity (LAN) and with modulations of event-related potential (ERP) components like the N400. There is also differential activation of typical spoken language processing areas, such as the conditional role of the temporal areas in sign language (SL) processing. Importantly, sign language processing uniquely recruits parietal areas for processing phonology and syntax and requires the mapping of spatial information to internal representations. Additionally, modality-specific feedback mechanisms distinctively involve proprioceptive post-output monitoring in sign languages, contrary to spoken languages’ auditory and visual feedback mechanisms. The only study to find ERP differences post-production revealed earlier lexical access in sign than spoken languages. Themes of temporality, the validity of an analogous anatomical mechanisms viewpoint, and the comprehensiveness of current language models were also discussed to suggest improvements for future research.Key message:Current neuroscience evidence suggests various ways in which processing differs between sign and spoken language modalities that extend beyond simple differences between languages. Consideration and further exploration of these differences will be integral in developing a more comprehensive view of language in the brain.

Deep Learning Approach for Spoken Digit Recognition in Gujarati Language

Speech Recognition is an emerging field in the area of Natural Language Processing which provides ease for human machine interaction with speech. Speech recognition for digits is useful for numbers oriented communication such as mobile number, scores, account number, registration code, social security code etc. This research paper seeks to achieve recognition of ten Gujarati digits from zero to nine (૦ to ૯) by using a deep learning approach. Dataset is generated with total 8 native speakers 4 male 4 female with the age group of 20 to 40. The dataset includes 2400 labeled audio clips of both genders. To implement a deep learning approach, Convolutional Neural Network (CNN) with MFCC is used to analyze audio clips to generate spectrograms. For the proposed approach three different experiments were performed with different dataset sizes as 1200, 1800 and 2400. With this approach maximum 98.7% accuracy is achieved for spoken digits in Gujarati language with 98% Precision and 98% Recall. It is analyzed from various experiments that increase in dataset size improves the accuracy rate for spoken digit recognition. No of epochs in CNN also improves accuracy to some extent.

Analytics of machine learning-based algorithms for text classification

Text classification is the most vital area in natural language processing in which text data is automatically sorted into a predefined set of classes. The application of text classification is wide in commercial works like spam filtering, decision making, extracting information from raw data, and many other applications. Text classification is more significant for many enterprises since it eliminates the need for manual data classification, a more expensive and time-consuming mechanism. In this paper, a comparative analysis of text classification is done in which the efficiency of different machine learning algorithms on different datasets is analyzed and compared. Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Logistic Regression (LR), Multinomial Naïve Bayes (MNB), and Random Forest (RF) are Machine Learning based algorithms used in this work. Two different datasets are used to make a comparative analysis of these algorithms. This paper further analyzes the machine learning techniques employed for text classification on the basis of performance metrics viz accuracy, precision, recall and f1- score. The resullltsss reveals that Logistic Regression and Support Vector Machine outperforms the other models in the IMDB dataset, and kNN outperforms the other models for the SPAM dataset as per the results obtained from the proposed system.

A Long-Text Classification Method of Chinese News Based on BERT and CNN

Text Classification is an important research area in natural language processing (NLP) that has received a considerable amount of scholarly attention in recent years. However, real Chinese online news is characterized by long text, a large amount of information and complex structure, which also reduces the accuracy of Chinese long text classification as a result. To improve the accuracy of long text classification of Chinese news, we propose a BERT-based local feature convolutional network (LFCN) model including four novel modules. First, to address the limitation of Bidirectional Encoder Representations from Transformers (BERT) on the length of the max input sequence, we propose a named Dynamic LEAD-n (DLn) method to extract short texts within the long text based on the traditional LEAD digest algorithm. In Text-Text Encoder (TTE) module, we use BERT pretrained language model to complete the sentence-level feature vector representation of a news text and to capture global features by using the attention mechanism to identify correlated words in text. After that, we propose a CNN-based local feature convolution (LFC) module to capture local features in text, such as key phrases. Finally, the feature vectors generated by the different operations over several different periods are fused and used to predict the category of a news text. Experimental results show that the new method further improves the accuracy of long text classification of Chinese news.

Word Sense Disambiguation Models Emerging Trends: A Comparative Analysis

Word Sense Disambiguation (WSD) arises due to the presence of ambiguity in the text during the semantic analysis of natural languages. It is a major unsolved problem in the area of Natural Language Processing (NLP) and its applications. This paper explores and reviews WSD algorithms that have contributed to, or created state-of-art solutions in recent years. Moreover, this paper also aims to analyze the recent technological trends in the domain of WSD which can give us leverage to identify the possible future trajectory of the search for better WSD solutions.

Natural language inference by deep learning method

Natural Language inference refers to the problem of determining the relationships between a premise and a hypothesis, it is an emerging area of natural language processing. The paper uses deep learning methods to complete natural language inference task. The dataset includes 3GPP dataset and SNLI dataset. Gensim library is used to get the word embeddings, there are 2 methods which are word2vec and doc2vec to map the sentence to array. 2 deep learning models DNNClassifier and Attention are implemented separately to classify the relationship between the proposals from the telecommunication area dataset. The highest accuracy of the experiment is 88% and we found that the quality of the dataset decided the upper bound of the accuracy.

Aspect-Level Sentiment Analysis Using CNN Over BERT-GCN

The increase in the volume of user-generated content on Twitter has resulted in tweet sentiment analysis becoming an essential tool for the extraction of information about Twitter users’ emotional state. Consequently, there has been a rapid growth of tweet sentiment analysis in the area of natural language processing. Tweet sentiment analysis is increasingly applied in many areas, such as decision support systems and recommendation systems. Therefore, improving the accuracy of tweet sentiment analysis has become practical and an area of interest for many researchers. Many approaches have tried to improve the performance of tweet sentiment analysis methods by using the feature ensemble method. However, most of the previous methods attempted to model the syntactic information of words without considering the sentiment context of these words. Besides, the positioning of words and the impact of phrases containing fuzzy sentiment have not been mentioned in many studies. This study proposed a new approach based on a feature ensemble model related to tweets containing fuzzy sentiment by taking into account elements such as lexical, word-type, semantic, position, and sentiment polarity of words. The proposed method has been experimented on with real data, and the result proves effective in improving the performance of tweet sentiment analysis in terms of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> score.

SPAMID-PAIR: A Novel Indonesian Post–Comment Pairs Dataset Containing Emoji

The detection of spam content is an important task especially in social media. It has become a topic to be continuely studied in Natural Language Processing (NLP) area in the last few years. However, limited data sets are available for this research topic because most researchers collect the data by themselves and make it private. Moreover, most available data sets only provide the post content without considering the comment content. This data becomes a limitation because the post-comment pair is needed when determining the context of a comment from a particular post. The context may contribute to the decision of whether a comment is a spam or not. The scarcity of non-English data sets, including Indonesian, is also another issue. To solve these problems, the authors introduce SPAMID-PAIR, a novel post-comment pair data set collected from Instagram (IG) in Indonesian. It is collected from selected 13 Indonesian actress/actor accounts, each of which has more than 15 million followers. It contains 72874 pairs of data. This data set has been annotated with spam/non-spam labels in Unicode (UTF-8) text format. The data also includes a lot of emojis/emoticons from IG. To test the baseline performance, the data is tested with some machine learning methods using several scenarios and achieves good performance. This dataset aims to be used for the replicable experiment in spam content detection on social media and other tasks in the NLP area.

Pathological resting-state executive and language system perfusion in first-episode psychosis

Background and hypothesisCortical (e.g., Broca’s area and Wernicke’s area) and subcortical (e.g., putamen) language-related areas and executive control areas (e.g., inferior frontal gyrus (IFG), dorsolateral prefrontal cortex (DLPFC)) show functional and structural dysconnectivity in long-term psychosis. We examined whether resting-state basal perfusion levels revealed selective pathophysiology (likely hypo- and hyper-activation) of language-related and executive areas in first-episode psychosis (FEP). Study designBasal resting-state perfusion was measured using pseudo-continuous Arterial Spin Labeling (pcASL). Relative cerebral blood flow (rCBF) was compared between 32 FEP and 34 matched healthy comparison (HC) individuals. Structural and functional MRI scans were acquired using a 3T Prisma scanner during the same session. Study resultsWhole-brain comparison of resting rCBF identified 8 clusters with significant between-group differences. Reduced rCBF was found in executive control areas in left and right IFG, right DLPFC, and right parietal cortex. Increased rCBF was found in left and right temporal cortex (including Wernicke’s area), and left and right putamen. A positive correlation was observed between auditory hallucination severity and rCBF in the left putamen. ConclusionsTo the degree that perfusion implies activation, language and auditory processing areas in bilateral temporal lobe and putamen showed pathological hyper-activity, and cognitive control areas (IFG, DLPFC, right parietal) showed pathological hypo-activity in FEP at rest. Pathological basal activity was present across the range of symptom severity, suggesting it may be a common underlying pathology for psychosis that may be targeted with non-invasive brain stimulation to normalize resting activity levels.

Brain responses during auditory word recognition vary with reading ability in Chinese school-age children.

While the close relationship between the brain system for speech processing and reading development is well-documented in alphabetic languages, whether and how such a link exists in children in a language without systematic grapheme-phoneme correspondence has not been directly investigated. In the present study, we measured Chinese children's brain activation during an auditory lexical decision task with functional magnetic resonance imaging. The results showed that brain areas distributed across the temporal and frontal lobes activated during spoken word recognition. In addition, the left occipitotemporal cortex (OTC) was recruited, especially under the real word condition, thus confirming the involvement of this orthographic-related area in spoken language processing in Chinese children. Importantly, activation of the left temporoparietal cortex (TPC) in response to words and pseudowords was positively correlated with children's reading ability, thus supporting the salient role phonological processing plays in Chinese reading in the developing brain. Furthermore, children with higher reading scores also increasingly recruited the left anterior OTC to make decisions on the lexical status of pseudowords, indicating that higher-skill children tend to search abstract lexical representations more deeply than lower-skill children in deciding whether spoken syllables are real. In contrast, the precuneus was more related to trial-by-trial reaction time in lower-skill children, suggesting that effort-related neural systems differ among pupils with varying reading abilities. Taken together, these findings suggest a strong link between the neural correlates of speech processing and reading ability in Chinese children, thus supporting a universal basis underlying reading development across languages.

Load Classification: A Case Study for Applying Neural Networks in Hyper-Constrained Embedded Devices

The application of Artificial Intelligence to the industrial world and its appliances has recently grown in popularity. Indeed, AI techniques are now becoming the de-facto technology for the resolution of complex tasks concerning computer vision, natural language processing and many other areas. In the last years, most of the the research community efforts have focused on increasing the performance of most common AI techniques—e.g., Neural Networks, etc.—at the expenses of their complexity. Indeed, many works in the AI field identify and propose hyper-efficient techniques, targeting high-end devices. However, the application of such AI techniques to devices and appliances which are characterised by limited computational capabilities, remains an open research issue. In the industrial world, this problem heavily targets low-end appliances, which are developed focusing on saving costs and relying on—computationally—constrained components. While some efforts have been made in this area through the proposal of AI-simplification and AI-compression techniques, it is still relevant to study which available AI techniques can be used in modern constrained devices. Therefore, in this paper we propose a load classification task as a case study to analyse which state-of-the-art NN solutions can be embedded successfully into constrained industrial devices. The presented case study is tested on a simple microcontroller, characterised by very poor computational performances—i.e., FLOPS –, to mirror faithfully the design process of low-end appliances. A handful of NN models are tested, showing positive outcomes and possible limitations, and highlighting the complexity of AI embedding.