Word Representations Research Articles

Sentiment analysis of Algerian Arabic dialect on social media Using Bi-LSTM recurrent neural networks

This paper presents a sentiment analysis approach using Bidirectional Long Short-Term Memory (Bi-LSTM) Recurrent Neural Networks to train predictive models for sentiment analysis on social media, particularly focusing on Algerian Arabic Dialect. The method leverages word-to-vector embedding for word representation and incorporates natural language understanding of emojis to improve semantic interpretation. The model achieves a high accuracy of 94%, demonstrating its effectiveness in analyzing sentiments in online discussions. The originality lies in applying Bi-LSTM to handle multilingual challenges on social platforms. The findings have practical implications for business, policymaking, and public sentiment evaluation, while also contributing positively to fostering informed online discourse.

Temporal coherence shapes cortical responses to speech mixtures in a ferret cocktail party

Perceptual segregation of complex sounds such as speech and music simultaneously emanating from multiple sources is a remarkable ability that is common in humans and other animals alike. Unlike animal physiological experiments with simplified sounds or human investigations with spatially broad imaging techniques, this study combines insights from animal single-unit recordings with segregation of speech-like sound mixtures. Ferrets are trained to attend to a female voice and detect a target word, both in presence and absence of a concurrent equally salient male voice. Recordings are made in primary and secondary auditory cortical fields, and in frontal cortex. During task performance, representation of the female words becomes enhanced relative to the male in all, but especially in higher cortical regions. Analysis of the temporal and spectral response characteristics during task performance reveals how speech segregation gradually emerges in the auditory cortex. A computational model evaluated on the same voice mixtures replicates and extends these results to different attentional targets (attention to female or male voices). These findings underscore the role of the principle of temporal coherence whereby attention to a target voice binds together all neural responses coherently modulated with the target, thus ultimately forming and extracting a common auditory stream.

Neural correlates of masked priming: only morphologically derived words facilitate lexical decisions

ABSTRACT The visual processing of morphologically complex words has been studied for decades now. One influential account proposes initial sublexical parsing, based on surface structure, before semantic information comes in (form-first models; Rastle & Davis, 2008, Morphological decomposition based on the analysis of orthography. Language and Cognitive Processes, 23(7-8), 942–971. https://doi.org/10.1080/01690960802069730). We tested this account in German, a morphologically rich language in a masked-priming lexical decision study with (pseudo-)derived German words. Behavioural data showed masked priming for truly morphologically complex (e.g. farmer – FARM), but not for pseudo-complex (e.g. corner – CORN) or merely form-related primes (e.g. cartel – CAR). MEG data revealed an early sensitivity to form (140–330 ms), followed by a differentiation between derived and other primes (260–480 ms), challenging the notion of early blind decomposition. The results align with the AUSTRAL model (Taft, 2023, Localist lexical representation of polymorphemic words. In D. Crepaldi (Ed.), Linguistic morphology in the mind and brain (1st ed., pp. 152–166). Routledge. https://doi.org/10.4324/9781003159759-11) and suggest that German speakers have heightened sensitivity to morpho-semantic information, unlike English speakers, which likely due to differences in morphological complexity between languages.

Distributional hypothesis as isomorphism between word-word co-occurrence and analogical parallelograms.

Most of the modern natural language processing (NLP) techniques are based on the vector space models of language, in which each word is represented by a vector in a high dimensional space. One of the earliest successes was demonstrated by the four-term analogical reasoning task: what is to C as B is to A? The trained word vectors form "parallelograms" representing the quadruple of words in analogy. This discovery in NLP offers us insight into our understanding of human semantic representation of words via analogical reasoning. Despite successful applications of the large-scale language models, it has not been fully understood why such parallelograms emerge by learning through natural language data. As the vector space model is not optimized to form parallelograms, the key structure related to geometric shapes of word vectors is expected to be in the data, rather than the models. In the present article, we test our hypothesis that such parallelogram arrangement of word vectors readily exists in the co-occurrence statistics of language. Our approach focuses more on the data itself, and it is different from the existing theoretical approach trying to find the mechanism of parallelogram formation in the algorithms and/or vector arithmetic operations on word vectors. First, our analysis suggested that analogical reasoning is possible by decomposition of the bigram co-occurrence matrix. Second, we demonstrated the formation of a parallelepiped, a more structured geometric object than a parallelogram, by creating a small artificial corpus and its word vectors. With these results, we propose a refined form of distributional hypothesis pointing out an isomorphism between a sort of symmetry or exchangeability and word co-occurrence statistics.

Piecewise convolutional neural network relation extraction with self-attention mechanism

The task of relation extraction in natural language processing is to identify the relation between two specified entities in a sentence. However, the existing model methods do not fully utilize the word feature information and pay little attention to the influence degree of the relative relation extraction results of each word. In order to address the aforementioned issues, we propose a relation extraction method based on self-attention mechanism (SPCNN-VAE) to solve the above problems. First, we use a multi-head self-attention mechanism to process word vectors and generate sentence feature vector representations, which can be used to extract semantic dependencies between words in sentences. Then, we introduce the word position to combine the sentence feature representation with the position feature representation of words to form the input representation of piecewise convolutional neural network (PCNN). Furthermore, to identify the word feature information that is most useful for relation extraction, an attention-based pooling operation is employed to capture key convolutional features and classify the feature vectors. Finally, regularization is performed by a variational autoencoder (VAE) to enhance the encoding ability of model word information features. The performance analysis is performed on SemEval 2010 task 8, and the experimental results show that the proposed relation extraction model is effective and outperforms some competitive baselines.

Traduction et paratraduction en tant que stratégies de résistance à la réification à l’ère de l’Intelligence Artificielle (IA)

Philosopher Byung-Chul Han argues that we are living in a transition from the “era of things to the era of non-things”. According to his perspective, things do not determine the world we live in: information does. Consequently, the profession of translation has become increasingly mechanized to meet the high demand for “information”, turning readers into compulsive consumers of words. Thus, there is a significant risk of confusing the symbolic representation of words with essential reality, neglecting the importance of context and paratext in the construction of meaning. This phenomenon can lead to a loss of meaning in the act of translation, diminishing its role as a bridge between cultures and languages. Borrowing from the axiom of Portuguese intellectual Rui Pereira in a society of “non-thought”, it is crucial to reflect on how “translating in times of non-translation” operates. In this regard, the objective of this essay is to analyze how paratranslation, from the Vigo School, can be understood as a space of resistance against a process of reification of the other’s existence in the act of translating. In the “AI era”, where translation has become an increasingly mechanized act, the examination of the paratexts surrounding this process serves as a way to return to the fundamental condition of translation: the human condition.

An embedded computational framework of memory: Accounting for the influence of semantic information in verbal short-term memory

We introduce the Embedded Computational Framework of Memory (eCFM), a model that integrates structured semantic word representations with an instance-based memory model to account for the influence of semantic information in verbal short-term memory. The eCFM combines principles from the episodic MINERVA 2 model and the semantic Latent Semantic Analysis model. After reviewing how semantic information impacts verbal short-term memory performance, we demonstrate eCFM’s ability to reconcile various phenomena within a common computational framework. Our model captures key findings, such as the influence of semantic information in serial recall, its reduction in serial reconstruction, and the impact of task difficulty on semantic information. In five experiments, we tested predictions derived from the eCFM. Experiments 1 and 2 manipulated list organization, with Experiment 1 using alternating lists of related or unrelated words and Experiment 2 using blocked lists. Experiment 3 varied presentation rates, Experiment 4 revisited the detrimental effect of semantic information on order information, and Experiment 5 explored false recall. We found that semantic information interacts with list composition, presentation rate affects the magnitude of its influence, and semantic information impacts order information contrary to the dominant view. Additionally, increasing the number of related study words to a non-studied semantic lure boosts false recall. The eCFM captured these findings as well as memory at the item level. Our demonstration provides insight into the cognitive mechanisms underlying verbal short-term memory and the interplay of semantic and episodic memory processes in recall.

FRGEM: Feature integration pre-training based Gaussian embedding model for Chinese word representation

Most Chinese word representation approaches mine the lexeme of Chinese words by predicting the target word or the contextual words. However, Chinese words may have multiple meanings, above ways may only capture one aspect implication. In this paper, for better modeling multiple connotations of Chinese words, a Feature integRation pre-training based Gaussian Embedding Model (FRGEM) is proposed with three stages, feature integration pre-training, relevancy Gaussian representation and similarity-based training. In the first stage, the internal sequences with three inner-character stroke, structure and pinyin features of Chinese words are generated to learn their semantic relevancy, then we combine BERT with internal sequences to extract sentence relevance and integrate the internal and sentence features to learn pre-training representations. In the second stage, relevancy Gaussian representation with pre-training embeddings is proposed to fuse inner-character features with Gaussian through estimating its mean to analyze multiple implications of Chinese words. In the third stage, a similarity-based objective is proposed to distinguish true contextual words and learn Chinese word representations. Extensive experiments on word similarity, word analogy, named entity recognition and text classification show that FRGEM is superior to majority state-of-the-art algorithms.

Comparing Machine Learning Models for Sentiment Analysis and Rating Prediction of Vegan and Vegetarian Restaurant Reviews

The paper investigates the relationship between written reviews and numerical ratings of vegan and vegetarian restaurants, aiming to develop a predictive model that accurately determines numerical ratings based on review content. The dataset was obtained by scraping reviews from November 2022 until January 2023 from the TripAdvisor website. The study applies multidimensional scaling and clustering using the KNN algorithm to visually represent the textual data. Sentiment analysis and rating predictions are conducted using neural networks, support vector machines (SVM), random forest, Naïve Bayes, and BERT models. Text vectorization is accomplished through term frequency-inverse document frequency (TF-IDF) and global vectors (GloVe). The analysis identified three main topics related to vegan and vegetarian restaurant experiences: (1) restaurant ambiance, (2) personal feelings towards the experience, and (3) the food itself. The study processed a total of 33,439 reviews, identifying key aspects of the dining experience and testing various machine learning methods for sentiment and rating predictions. Among the models tested, BERT outperformed the others, and TF-IDF proved slightly more effective than GloVe for word representation.

Distribution of words across the first years of life: A longitudinal analysis of everyday language input to three English-learning infants.

Many in-lab studies have demonstrated that the distribution of word learning moments affects the strength and quality of word representations. How are words distributed in speech to children in their daily lives, and how is distribution related to other input characteristics? The present study analyzes transcripts of language input to English-learning infants from three longitudinal, naturalistic corpora captured between 6 and 39months of age. To describe how word frequency varies across time, we calculated dispersion scores for all word types for each child. Dispersion quantifies the deviation of observed frequencies in each recording session from expected (uniform across sessions) word frequency, providing a measure of how evenly word utterances were spread across sessions. Dispersion is strongly correlated with frequency and moderately correlated with concreteness across all corpora, such that high frequency and low concreteness words are more evenly dispersed. Correlations with measures of age of acquisition (AoA) varied across corpora, and dispersion did not reliably predict AoA above and beyond frequency and concreteness. The contradiction between the current results and results from in-lab experiments is discussed. This study provides a foundation to explore how word learning unfolds across time and contexts in the real world.

HubNet: An E2E Model for Wheel Hub Text Detection and Recognition Using Global and Local Features.

Automatic detection and recognition of wheel hub text, which can boost the efficiency and accuracy of product information recording, are undermined by the obscurity and orientation variability of text on wheel hubs. To address these issues, this paper constructs a wheel hub text dataset and proposes a wheel hub text detection and recognition model called HubNet. The dataset captured images on real industrial production line scenes, including 446 images, 934 word instances, and 2947 character instances. HubNet is an end-to-end text detection and recognition model, not only comprising conventional detection and recognition heads but also incorporating a feature cross-fusion module, which improves the accuracy of recognizing wheel hub texts by utilizing both global and local features. Experimental results show that on the wheel hub text dataset, the HubNet achieves an accuracy of 86.5%, a recall of 79.4%, and an F1-score of 0.828, and the feature cross-fusion module increases the accuracy by 2% to 4%. The wheel hub dataset and the HubNet offer a significant reference for automatic detection and recognition of wheel hub text.

Advancements in Word Embeddings: A Comprehensive Survey and Analysis

In recent years, the field of Natural Language Processing (NLP) has seen significant growth in the study of word representation, with word embeddings proving valuable for various NLP tasks by providing representations that encapsulate prior knowledge. We reviewed word embedding models, their applications, cross-lingual embeddings, model analyses, and techniques for model compression. We offered insights into the evolving landscape of word representations in NLP, focusing on the models and algorithms used to estimate word embeddings and their analysis strategies. To address this, we conducted a detailed examination and categorization of these evaluations and models, highlighting their significant strengths and weaknesses. We discussed a prevalent method of representing text data to capture semantics, emphasizing how different techniques can be effectively applied to interpret text data. Unlike traditional word representations, such as Word to Vector (word2vec), newer contextual embeddings, like Bidirectional Encoder Representations from Transformers (BERT) and Embeddings from Language Models (ELMo), have pushed the boundaries by capturing the use of words through diverse contexts and encoding information transfer across different languages. These embeddings leverage context to represent words, leading to innovative applications in various NLP tasks.

Four Cases of Children With Phonological Impairment and Precocious Vocabulary: Making Sense of a Clinical Conundrum.

The purpose of this study was to describe the speech production, speech perception, vocabulary, and word learning abilities of lexically precocious 4-year-old children with phonological impairment, in an effort to better understand the underlying nature of phonological impairment in children. Using a case series approach, we identified four children with phonological impairment and precocious vocabulary abilities. Each child completed routine speech production and vocabulary assessments, as well as experimental speech perception and word learning tasks. The results from these tasks were then used to create profiles of each child's individual strengths and needs across the abilities assessed. Although all four children presented with phonological impairment and lexically precocious receptive and expressive vocabulary, they differed in their specific speech errors. One child presented with phonological speech errors only, while the other three children presented with an interdental lisp alongside their phonological errors. Three children presented with average speech perception abilities, and one child presented with poorer speech perception. The same three children also showed some learning of novel nonwords 1 week post-initial exposure, while the other child showed no evidence of word learning 1 week post-initial exposure. The clinical profiles of lexically precocious children with phonological impairment offered different insights into the nature of the problem. Although one child appeared to present with underspecified underlying representations of words, the other three children appeared to present with well-specified underlying representations. Of the three children with well-specified underlying representations, two appeared to have difficulty abstracting particular rules of the ambient phonological system. Further research is needed to improve our understanding of the underlying nature of phonological impairment. https://doi.org/10.23641/asha.26307640.

Homogenization of word relationships in schizophrenia: Topological analysis of cortical semantic representations.

Patients with schizophrenia typically exhibit symptoms of disorganized thought and display concreteness and over-inclusion in verbal reports, depending on the level of abstraction. While concreteness and over-inclusion may appear contradictory, the underlying psychopathology that explains these symptoms remains unclear. In the current study, we used functional magnetic resonance imaging with an encoding modeling approach to examine how concepts of various words, represented as brain activity, are anomalously connected at different levels of abstraction in patients with schizophrenia. Fourteen individuals diagnosed with schizophrenia and 17 healthy controls underwent functional magnetic resonance imaging to measure brain activity representing concepts of various words. We used a persistent homology (PH) method to analyze the topological structures of word representations in schizophrenia patients, healthy controls, and random data, across different levels of abstraction by varying dissimilarity scales in the representation space. The results revealed that patients with schizophrenia exhibited more homogeneous word relationships across different levels of abstraction compared with healthy controls. Additionally, topological structures exhibited a shift toward a random network structure in patients with schizophrenia compared with controls. The PH method successfully distinguished semantic representations of patients with schizophrenia from those of controls. The current results provide an explanation for the mechanisms underlying the deficits in abstraction ability observed in schizophrenia. The isotopic connection of individual concepts reflects both the reduction of contextual connections at a semantically fine-grained scale and the absence of clear boundaries between related concepts at a coarse scale, which lead to concreteness and over-inclusion, respectively.

GLOBAL READING - (NON)CONVENTIONAL MODEL OF READING

Reading as a complex process is presented as an important factor for the proper organization of the general activity of the individual, and especially for his intellectual development. As a rather complicated activity, reading includes different psychological, social-ecological, as well as physical viewpoints. This "hitting" of the alphabetic code to determine the words and the construction of meaning from them, in an organized and systematized form, begins to develop in the first developmental cycle. From the phonetic awareness that helps to decode the word, students create a "visual dictionary". This visual image facilitates "seeing" and matching words to create meaning. Similar to the teaching of the spoken language that is presented as a complete form, and not with analysis and synthesis, the graphic representation of the read word is visualized in consciousness as an image, during which the meaning of the word also appears reflexively. Given that 65% of the population are visual learners, today the use of the whole word method is an unpretentious necessity. The whole word method stems from the premise that students living in this century are surrounded by technology and are in constant contact with written material. The basic unit of reading is the word, the graphic image of which is transmitted as a picture, so television announcements, computers, food labels, street advertising, and books are great causes and triggers of early literacy. Internet providers use the universality of symbols to make reading easy for everyone, including children who are in constant contact with this medium. Technology gives them the ability to read without knowing the letters, so they show awareness of the displayed symbols through computer tools. Although the Albanian language has a morphological structure and phonetic spelling, the use of this alternative model of reading is encountered more and more in school practice every day due to the need and many benefits which will be elaborated below in the paper. 50 children aged 6 years with different demographic characteristics were checked. The same approach is the checking of the skill to read words with the whole word and phonological method. The obtained results were processed with the t-test statistical method, results which allowed the findings presented which show a very pronounced difference.

Karakteristik Ungkapan Makian dalam Podcast Akun Youtube Deddy Corbuzier

This research examines the use of profanity in Deddy Corbuzier's YouTube podcasts in Indonesia. The study aims to analyze the structure, types, and purposes of cuss words used. Employing a qualitative approach with descriptive methodology, data was collected through observation and recording of words, phrases, and clauses containing profanity. The analysis revealed 95 instances of cuss words, with word form being the most common structure (77 instances). Five categories were identified: conditional, animal-related, object-related, body part-related, and profession-related, with conditional being the most frequent (46 instances). Four functions of cuss word usage were observed: conformity to group norms, insults, expressions of shock or threats, and articulation of intense emotions, with the latter being most prominent (33 instances). Cuss words in these podcasts primarily take the form of single words and serve to describe negative qualities or express emotions, particularly surprise. Notably, they are used for both negative and positive purposes, including humor, to enhance entertainment value in Corbuzier's infotainment-focused podcasts.

Mental representation of words and concepts in late multilingualism

Abstract Speakers of multiple languages must store the respective lexical items efficiently to enable correct access. Importantly, all items must be linked to semantic information and world knowledge. One prominent model of the mental lexicon of late bilinguals is the Revised Hierarchical Model (Kroll & Stewart, 1994), which postulates bidirectional but asymmetrical connections between separate stores for L1 (native language) and L2 (second/foreign language), and a shared conceptual store. Using German native speakers with advanced English proficiency, Experiment 1 largely confirmed model predictions regarding different preferred mental routes and processing times depending on translation direction. Moreover, the original design was extended by including abstract stimuli and picture naming in L2. A series of additional measures, such as proficiency and age of acquisition, served to specify the language experience of the participants and made it possible to compare the results with a group of non-native speakers of German (Experiment 2). Interestingly, the results suggest that the model also applies to two or more non-native languages, potentially influenced by the experimental and environmental language context.

Cancer Prevention and Treatment on Chinese Social Media: Machine Learning-Based Content Analysis Study.

Nowadays, social media plays a crucial role in disseminating information about cancer prevention and treatment. A growing body of research has focused on assessing access and communication effects of cancer information on social media. However, there remains a limited understanding of the comprehensive presentation of cancer prevention and treatment methods across social media platforms. Furthermore, research comparing the differences between medical social media (MSM) and common social media (CSM) is also lacking. Using big data analytics, this study aims to comprehensively map the characteristics of cancer treatment and prevention information on MSM and CSM. This approach promises to enhance cancer coverage and assist patients in making informed treatment decisions. We collected all posts (N=60,843) from 4 medical WeChat official accounts (accounts with professional medical backgrounds, classified as MSM in this paper) and 5 health and lifestyle WeChat official accounts (accounts with nonprofessional medical backgrounds, classified as CSM in this paper). We applied latent Dirichlet allocation topic modeling to extract cancer-related posts (N=8427) and identified 6 cancer themes separately in CSM and MSM. After manually labeling posts according to our codebook, we used a neural-based method for automated labeling. Specifically, we framed our task as a multilabel task and utilized different pretrained models, such as Bidirectional Encoder Representations from Transformers (BERT) and Global Vectors for Word Representation (GloVe), to learn document-level semantic representations for labeling. We analyzed a total of 4479 articles from MSM and 3948 articles from CSM related to cancer. Among these, 35.52% (2993/8427) contained prevention information and 44.43% (3744/8427) contained treatment information. Themes in CSM were predominantly related to lifestyle, whereas MSM focused more on medical aspects. The most frequently mentioned prevention measures were early screening and testing, healthy diet, and physical exercise. MSM mentioned vaccinations for cancer prevention more frequently compared with CSM. Both types of media provided limited coverage of radiation prevention (including sun protection) and breastfeeding. The most mentioned treatment measures were surgery, chemotherapy, and radiotherapy. Compared with MSM (1137/8427, 13.49%), CSM (2993/8427, 35.52%) focused more on prevention. The information about cancer prevention and treatment on social media revealed a lack of balance. The focus was primarily limited to a few aspects, indicating a need for broader coverage of prevention measures and treatments in social media. Additionally, the study's findings underscored the potential of applying machine learning to content analysis as a promising research approach for mapping key dimensions of cancer information on social media. These findings hold methodological and practical significance for future studies and health promotion.

Fiscal policy in the Bundestag: Textual analysis and macroeconomic effects

Fiscal policy is made in parliaments. We go to the roots of changes of fiscal policy in Germany and use a novel data set on all parliamentary speeches in the Bundestag from 1960 to 2021. We propose an embedding-based approach, which allows the representation of words and documents in a shared vector space, in order to measure fiscal policy-related sentiment in parliamentary debates at a scale from contractionary to expansionary. We also distinguish between sentiment related to exogenous and endogenous fiscal policy. We put fiscal sentiment into a series of recursively-identified vector autoregressive models to show that a change in fiscal sentiment causes a shift in government spending and has significant effects on the macroeconomy. The results support the notion that the debate in parliament contains information for the identification of government spending shocks.

Tracking the brain signature of (mis)spelled logotypes via letter transpositions and replacements

All leading models of visual word recognition assume a hierarchical process that progressively converts the visual input into abstract letter and word representations. However, the results from recent behavioral studies suggest that the mental representations of words with a highly consistent visual format, such as logotypes, may comprise not only purely abstract information but also perceptual information. This hypothesis would explain why participants often misperceive transposed-letter misspellings with the original base words to a larger degree in logotypes (e.g., SASMUNG, but not SARVUNG, is perceived as SAMSUNG) than in common words. The present experiment examined the electrophysiological signature behind the identification of correctly spelled and misspelled logotypes (via letter transposition or replacement) in an ERP go/no-go semantic categorization experiment. Results showed that N400 amplitudes for transposed-letter misspelled logotypes (SASMUNG) and intact logotypes (SAMSUNG) did not differ significantly across various time windows (until 600 ms), whereas replacement-letter misspelled logotypes (SARVUNG) yielded consistently larger N400 amplitudes. These findings reveal that the mental representations of logotypes are particularly resistant to minor orthographic changes, which has important theoretical and applied (e.g., marketing) implications.