Distribution Of Words Research Articles

Text Mining technologies in sociological analysis (using the example of studying students`ideas about the mission of a modern university)

There are discussed in the article the possibilities of using Text Mining methods in the practice of analyzing the information received on the base of open questionnaire questions. The paper presents an example of unigrams and bigrams analysis, as well as the search for latent topic using thematic modeling. Empirical materials present the data of survey conducted in 2022, in which 929 students of one Moscow economics university took part. In the open question of the questionnaire, it was proposed to define the mission of the university. Information made it possible to get the subjective interpretation of the main significancy of higher education in modern conditions. The frequency analysis of unigrams, supplemented by a qualitative analysis of respondents’ statements, allowed reflecting the vocabulary of student discourse about the mission of the university. The articulation of bigrams was carried out on the basis of several statistical metrics, which made it possible to rank phrases and highlight a key set of concepts. The procedure revealed that in the perception of students, the priorities of the university are aimed at the transferring of professional knowledge and skills, in a broad sense – the training of qualified specialists. The social functions of the university, focused on meeting the needs of society and the state, are less pronounced in the conceptual interpretations of the interviewed students. At the next stage of the study the task of articulation and research of latent topics was put forward. The specific feature of thematic modeling is that the words combined into one topic reflect the distribution of words identified by the program, but not a topic that is literally understandable to a person. Taking into account the specifics of the method used, the authors demonstrated the results of search analysis in the practice of processing an open question. As it turned out, the keywords concentrated in the core of the main topics are mainly related to meeting the needs of the students themselves, leaving on the periphery of the verbalized definitions any understanding of the importance of the university as a platform for innovation, scientific research, entrepreneurial and other initiatives for the benefit of society and the country. The results of the presented research can be useful in rethinking the research tools of sociologists in the context of the active development of digital technologies, which requires testing new methods, understanding their real capabilities and limitations in solving the tasks of sociological research.

Short text classification using semantically enriched topic model

Modelling short text is challenging due to the small number of word co-occurrence and insufficient semantic information that affects downstream Natural Language Processing (NLP) tasks, for example, text classification. Gathering information from external sources is expensive and may increase noise. For efficient short text classification without depending on external knowledge sources, we propose Expressive Short text Classification (EStC). EStC consists of a novel document context-aware semantically enriched topic model called the Short text Topic Model (StTM) that captures words, topics and documents semantics in a joint learning framework. In StTM, the probability of predicting a context word involves the topic distribution of word embeddings and the document vector as the global context, which obtains by weighted averaging of word embeddings on the fly simultaneously with the topic distribution of words without requiring an additional inference method for the document embedding. EStC represents documents in an expressive (number of topics × number of word embedding features) embedding space and uses a linear support vector machine (SVM) classifier for their classification. Experimental results demonstrate that EStC outperforms many state-of-the-art language models in short text classification using several publicly available short text data sets.

Understanding latent affective bias in large pre-trained neural language models

Groundbreaking inventions and highly significant performance improvements in deep learning based Natural Language Processing are witnessed through the development of transformer based large Pre-trained Language Models (PLMs). The wide availability of unlabeled data within human generated data deluge along with self-supervised learning strategy helps to accelerate the success of large PLMs in language generation, language understanding, etc. But at the same time, latent historical bias/unfairness in human minds towards a particular gender, race, etc., encoded unintentionally/intentionally into the corpora harms and questions the utility and efficacy of large PLMs in many real-world applications, particularly for the protected groups. In this paper, we present an extensive investigation towards understanding the existence of “Affective Bias” in large PLMs to unveil any biased association of emotions such as anger, fear, joy, etc., towards a particular gender, race or religion with respect to the downstream task of textual emotion detection. We conduct our exploration of affective bias from the very initial stage of corpus level affective bias analysis by searching for imbalanced distribution of affective words within a domain, in large scale corpora that are used to pre-train and fine-tune PLMs. Later, to quantify affective bias in model predictions, we perform an extensive set of class-based and intensity-based evaluations using various bias evaluation corpora. Our results show the existence of statistically significant affective bias in the PLM based emotion detection systems, indicating biased association of certain emotions towards a particular gender, race, and religion.

An Analysis of Students’ Obstacles in Learning English for Grade VIII Students at SMP Negeri 1 Selesai

This research explores the obstacles faced by students at SMP Negeri 1 SELESAI in learning English, particularly in reading. The study was conducted through preliminary analysis and a questionnaire, revealing a strong preference among students for topics related to the internet and entertainment, which might distract them from focusing on reading English materials. The research also revealed that students prefer to learn in groups rather than individually, suggesting a strong influence of peer interactions. Despite the high interest in English, students still face numerous obstacles in their learning process, including a lack of understanding about word distribution and pronoun usage in English, and difficulty in understanding and interpreting complex terms and phrases. The study suggests potential solutions to overcome these challenges, such as the use of more varied and interesting teaching methods, increasing students' understanding of English sentence structure, and enhancing students' interest and motivation in learning English, especially in reading. Keywords: Obstacle;, English learning; motivation

A Semantics-enhanced Topic Modelling Technique: Semantic-LDA

Topic modelling is a beneficial technique used to discover latent topics in text collections. But to correctly understand the text content and generate a meaningful topic list, semantics are important. By ignoring semantics, that is, not attempting to grasp the meaning of the words, most of the existing topic modelling approaches can generate some meaningless topic words. Even existing semantic-based approaches usually interpret the meanings of words without considering the context and related words. In this article, we introduce a semantic-based topic model called semantic-LDA that captures the semantics of words in a text collection using concepts from an external ontology. A new method is introduced to identify and quantify the concept–word relationships based on matching words from the input text collection with concepts from an ontology without using pre-calculated values from the ontology that quantify the relationships between the words and concepts. These pre-calculated values may not reflect the actual relationships between words and concepts for the input collection, because they are derived from datasets used to build the ontology rather than from the input collection itself. Instead, quantifying the relationship based on the word distribution in the input collection is more realistic and beneficial in the semantic capture process. Furthermore, an ambiguity handling mechanism is introduced to interpret the unmatched words, that is, words for which there are no matching concepts in the ontology. Thus, this article makes a significant contribution by introducing a semantic-based topic model that calculates the word–concept relationships directly from the input text collection. The proposed semantic-based topic model and an enhanced version with the disambiguation mechanism were evaluated against a set of state-of-the-art systems, and our approaches outperformed the baseline systems in both topic quality and information filtering evaluations.

How Fast Do Distribution and Semantics of Polysemic Words Change?

Creation of large diachronic text corpora triggered studies of evolution of natural languages using quantitative methods in the last decade. For the first time, it became possible to raise questions about general patterns of lexical semantic change. In the work by W. Hamilton et al., 2016, a hypothesis was formulated about a more rapid change in the meanings of polysemic words. In this paper, we consider how frequency of words influence variations of semantic metrics. We use explicit word vector representations and Jensen-Shannon divergence as a measure of change in the distribution and semantics of a word. The obtained results show that changes in the frequency ratio of a word in different meanings can themselves lead to a change in a word distribution, even in the absence of any changes in its meaning. Therefore, the observations made in the work by W. Hamilton et al about correlation between the rate of change of words and their local cluster coefficient in the semantic connection network (considered as a quantitative measure of polysemy) can be fully explained by the effect described in the article.

Association of lexical and collocation knowledge: A comparative analysis of native speakers and nonnative learners of English

To provide a comprehensive estimate of Korean college students’ lexical competence, this study compared L2 learners’ (EFL) use of single words and collocations with that of native speakers (NSs). The study compiled two text corpora, one from an existing native corpus and the other with texts from Korean EFL learners at three proficiency levels. There were significant differences between the two corpora in lexical diversity, although the difference between advanced EFL learners and NSs was non-significant, indicating that advanced learners were on par with NSs in terms of using a diverse range of words. In a comparison of lexical distribution of single words, however, these advanced learners were found to use words between the 1K and 4K level, whereas NSs used words beyond this level. This suggests the need to cover a wider range of words in classroom teaching and assessment. Regarding collocational knowledge, the difference between the NSs and EFL group was apparent and statistically significant, regardless of learners’ proficiency levels. Namely, EFL learners used far fewer collocations in a smaller range than their counterparts. The learners’ limited collocational competence indicates that collocations should be considered an integral component of the curriculum, instruction, and assessment.

N-gram Language Model for Chinese Function-word-centered Patterns

N-gram language modelling, a proven and effective method in NLP, is widely used to calculate the probability of a sentence in natural language. Language pattern is a linguistic level between word/character and sentence, which exists in pattern grammar. In this research, the approach of language model and language pattern are combined for the first time, and language patterns are studied by use of the N-gram model. Chinese function-word-centered patterns are extracted from the LCMC corpus, and aligned into pattern chains. The language model is trained from these chains to investigate the properties and distribution of Chinese function words, the interaction of content words and function words, and the interaction between patterns. The results indicate that there are approximately 10,000 function-word-centered patterns in the texts, which are distributed exponentially. This research summarizes the most common function-word-centered patterns and content-word-centered patterns, and discusses the interactions of patterns based on corpus data. The bigram language model of these patterns reflects the restrictions of function words. In addition, the research adopts an innovative method to visualize the interactions between patterns. This research fills the research gap between word/character and sentence and reveals basic Chinese pattern categories and the interactions between patterns, which makes a significant contribution to Chinese linguistic research, and improves the efficiency of NLP.

MIT: Mutual Information Topic Model for Diverse Topic Extraction.

To automatically mine structured semantic topics from text, neural topic modeling has arisen and made some progress. However, most existing work focuses on designing a mechanism to enhance topic coherence but sacrificing the diversity of the extracted topics. To address this limitation, we propose the first neural-based topic modeling approach purely based on mutual information maximization, called the mutual information topic (MIT) model, in this article. The proposed MIT significantly improves topic diversity by maximizing the mutual information between word distribution and topic distribution. Meanwhile, MIT also utilizes Dirichlet prior in latent topic space to ensure the quality of mined topics. The experimental results on three publicly benchmark text corpora show that MIT could extract topics with higher coherence values (considering four topic coherence metrics) than competitive approaches and has a significant improvement on topic diversity metric. Besides, our experiments prove that the proposed MIT converges faster and more stable than adversarial-neural topic models.

Lexical Characterization of Ancient and Modern Chinese Combined with Semantic Association Network Modeling

Abstract This paper explores the linguistic features of ancient and modern Chinese, focuses on capturing the knowledge of the relevant domains of ancient and contemporary Chinese using semantic models, and carries out the corresponding semantic resource extraction and semantic representation through the resource expression mechanism in the framework of the Semantic Association Network Model (SANM), and then analyzes the historical evolution of the lexicalization patterns of the Chinese language and the linguistic features of the Chinese language in different periods. In the historical evolution of Chinese lexicalization patterns, it is found that the distribution of lexicalization patterns of ancient Chinese table-cutting action lexical items is [action+object] > [action+method] > [action result] > [action+tool], and the distribution of lexicalization patterns of modern Chinese table-cutting action lexical items is [action+method] > [action+tool] > [action+result] > [action+object]. The linguistic features of different periods show that all networks have small-world properties and scale-free properties, and the statistical parameters of the networks in each period are relatively close to each other, with the modern network having the most significant C (8.59) and the smallest L (0.92) and γ (0.40). Ancient and modern Chinese words and phrases with relational word collocations in the interval [0,23] have the most extensive and densest distribution of relational words. These data present the similarities and differences between linguistic features between ancient and modern Chinese, which is significant for studying ancient and modern Chinese.

Using Natural Language Processing Techniques to Study the Development Dynamics of the Subei Mongolian Language and Its Shaping of the Chinese National Community Consciousness

Abstract Language and society are a pair of covariance; language reflects the changes in society, and the changes in society have an important impact on the language. The urbanization process in the Subei area is reflected in the current Mongolian language use situation. This paper explores the intrinsic connection between language and the Chinese national community, in which the emotional identity of Mongolian script is one of the factors affecting the development of Mongolian language in Subei. Mongolian emotions are represented semantically using the SOFT self-attention mechanism. The computation of emotion word weights and distributions is carried out sequentially to enhance the emotion word tokens that semantic rules represent. Through the mining of emotion prior knowledge, new emotion words are found from the Mongolian emotion corpus with the help of emotion seed words, and a set of candidate emotion words is constructed. The emotion words that were mined in Mongolia are merged into emotion sentences, and then they are emotionally masked. An empirical survey is designed to visit and examine the local Mongolian residents in Subei using random sampling. The psychological distance of Mongolians towards intermarriage is the lowest among the subject ethnic groups, with a mean value of 1.7585. The development of the Mongolian language in Subei is a map of the progress of the Chinese nation’s consciousness and contributes to the formation of a sense of community among its people.

Analysis of Hot Topics and Evolution of Research in World-class Agricultural Universities Based on BERTopic

Abstract In this paper, the cited papers and cited patents of world-class agriculture-related universities are taken as the research objects, and the preprocessing process of the research hotspot text of agriculture-related universities is accomplished by combining the participle processing method and the TF-IDF method. To make up for the shortcomings of the semantic level of the LDA model, BERTopic is utilized to obtain the document-topic distribution probability and the topic-topic word distribution probability, and the topic hotness of different topics is calculated. Then the Glove method is used to extract the word vector features of the topic words, normalize them, calculate the word vectors by taking the distribution probability values as the weighted weights, solve the topic vectors, and then calculate the similarity of the hot topics in different time windows by the cosine similarity to explore the hot topics and the evolution of the world’s first-class agriculture-related colleges and universities research.

Exploring the Inheritance and New Changes in Pre-Qin Literature in Cao Zhi’s “Luoshen Fu” Using Network Textual Analysis

Abstract Through network textual analysis, this paper explores the inheritance and development of Cao Zhi’s Luoshen Fu on pre-Qin literature, revealing its innovations in linguistic style, emotional expression and literary concepts. In-depth discussion of Cao Zhi’s Luoshen Fu on the inheritance and innovation of pre-Qin literature. The study firstly analyzes the main themes and emotional tendencies of Luoshen Fu by modeling textual themes through natural language processing techniques. In the word distribution analysis, we find that the frequency of verb and noun usage in Luoshen Fu is 221‰ and 178‰, respectively, which is significantly higher than the average level of pre-Qin literature (232‰ and 140‰). In addition, Luoshen Fu demonstrates more complex and rich emotional changes in the emotional analysis than Li Sao. By calculating the information entropy value of Luoshen Fu (10.5), compared with the average level of pre-Qin literature (9.8), it reflects its richness and innovation in vocabulary use. Luoshen Fu demonstrates remarkable innovation in language style, emotional expression, and literary concepts while demonstrating a profound ancestry in pre-Qin literature.

Communication about rebranding: the case of Polish listed companies

RESEARCH OBJECTIVE: Rebranding is a crucial driver of modern marketing strategies that can only be successful if appropriately communicated to society. This research aims to explain which rebranding-related information companies disclose in their communication on this process using various information dissemination channels.
 THE RESEARCH PROBLEM AND METHODS: To examine the content of 200 disclosures about rebranding, we used topic modelling, which identifies latent patterns of word co-occurrence using computer algorithms and the distribution of words in the analyzed corpus (set of documents). We applied an unsupervised Bayesian machine-learning approach for topic modelling called latent Dirichlet allocation (LDA), a cutting-edge method that is still not widely used by scholars.
 THE PROCESS OF ARGUMENTATION: Rebranding pays out over the lifetime of a company, but society has to be appropriately informed of this decision. The information should be comprehensive, disclosed through multiple channels, and addressed to all stakeholders. However, the question is whether this theoretical knowledge finds its application in companies’ practice. We researched each rebranding case of all Polish companies listed on the WSE since 1991 and analyzed how they communicated their rebranding through diverse types information disclosure.
 RESEARCH RESULTS: Polish companies are very selective, both in the content of information disclosed and the communication channels they use. They tend to avoid explaining the reasons for rebranding that are vital for society. Information is mainly disclosed through company websites or press releases and primarily has an informal character. They ignore official stock exchange communication channels, even though rebranding information could greatly influence financial performance and value creation.
 CONCLUSIONS, INNOVATIONS AND RECOMMENDATIONS: Our findings implicate a need to increase listed companies’ awareness concerning the importance of communicating marketing strategy and its potential benefits in the creation of investor relations.

APPROACHES TO AUTOMATIC WORD SENSE DISAMBIGUATION BASED ON UNEVEN DISTRIBUTION OF WORD SENSES IN CORPUS

Word Sense Disambiguation (WSD) is a key task of automatic semantic analysis that affects other upstream tasks. Nevertheless, the selection of appropriate sense of ambiguous word in context is a complicated task even for human native speakers. It is even more relevant for automatic disambiguation models. That is why we need any observations and heuristics able to make the WSD task simpler or performance higher. Researchers have noticed that the distribution of ambiguous word senses follow certain laws. In the paper we discuss three hypotheses about word senses distribution in corpus: 1) Most Frequent Sense, MFS; 2) One Sense per Discourse and 3) One Sense per Collocation. The following results were obtained on the material of a corpus of Russian texts. Most Frequent Sense based algorithm demonstrates relatively high precision on both training and test set (85.7% and 71.1% respectively). The One Sense per Discourse hypothesis has been confirmed in 93% of texts. The One Sense per Collocation hypothesis has been confirmed in 84.46% word pairs from texts. The exceptions are related to difficulties and errors by manual word sense labeling. Heuristics based on uneven distribution of ambiguous words in corpus allow to make WSD task simpler and can be used by collecting training data sets for WSD models.

Multi-Granularity Interaction and Integration Network for Video Question Answering

Video question answering, aiming to answer a natural language question related to the given video, has gained popularity in the last few years. Although significant improvements have been achieved, it is still confronted with two challenges: the sufficient comprehension of video content and the long-tailed answers. To this end, we propose a multi-granularity interaction and integration network for video question answering. It jointly explores multi-level intra-granularity and inter-granularity relations to enhance the comprehension of videos. To be specific, we first build a word-enhanced visual representation module to achieve cross-modal alignment. And then we advance a multi-granularity interaction module to explore the intra-granularity and inter-granularity relationships. Finally, a question-guided interaction module is developed to select question-related visual representations for answer prediction. In addition, we employ the seesaw loss for open-ended tasks to alleviate the long-tailed word distribution effect. Both the quantitative and qualitative results on TGIF-QA, MSRVTT-QA, and MSVD-QA datasets demonstrate the superiority of our model over several state-of-the-art approaches.

The Analysis of the Binary Opposition of “Good” and “Evil” in Modern English and French in the Context of Foreign Language Teaching

The research examines the binary opposition of “good” and “evil” in the English and French languages within the context of foreign language teaching. The aim is to analyze these concepts’ lexical-semantic and cultural aspects and explore their implications for foreign language learners. The study adopts a comparative approach, considering the similarities and differences in representing “good” and “evil” in the two distantly related languages. The analysis focuses on the semantic structure and distribution of lexical units associated with these concepts and the cultural connotations embedded in their usage. The research focuses on how the binary opposition of “good” and “evil” impacts language learners’ understanding of the target culture and their ability to communicate effectively in the foreign language. The latter gain a deeper appreciation of the target culture’s moral values and social dynamics by examining the cultural meanings and connotations associated with these oppositions. The findings highlight the asymmetry of lexical units within the lexical-semantic groups of “good” and “evil”, revealing discrepancies in the number and distribution of words and their meanings. Antonymy and enantiosemy are identified as regularities in forming vocabulary, denoting good and evil in the compared languages. These linguistic tools enable learners to express contrasting concepts and enhance their linguistic and intercultural awareness. Incorporating the binary opposition of “good” and “evil” in foreign language teaching can promote a deeper understanding of the target culture’s values, norms, and worldview. Learners become more sensitive to cultural aspects and better equipped to navigate intercultural interactions by unravelling the underlying cultural codes embedded in language.

Do Self-translating Poets Have Equally Distributed Equivalent Words in the Target and Original Texts? A Corpus Examination of Yu Guangzhong’s Poems

This article examines the utilization of high-frequency words in Yu Guangzhong’s self-translated poetry through a corpus-driven analysis. The objective is to explore the presence of equivalence and inequivalence in the translations executed by Yu Guangzhong himself. The utilization of modal verbs, lexical bundles, and keywords has been analyzed in various contexts, such as novels and speeches. However, studies that compare disparities in the use of high-frequency words between the source and target languages within poetry translation are scarce. The original and self-translated poetry corpora of Yu Guangzhong have been constructed at the word level to enhance the original texts’ co-occurrence and corresponding translations. In the comprehensive self-translation of his eighty-five poems, Yu Guangzhong encounters challenges in achieving an equitable distribution of equivalent words between the target and original texts. It is observed that most high-frequency words in Yu Guangzhong’s original and target texts lack equivalence in use and meaning. Although two of the three poems analyzed individually mainly achieve equivalence, this discrepancy might be attributed to the translator’s utilization of literal and word-for-word translations.

How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure

Abstract Language models are typically evaluated on their success at predicting the distribution of specific words in specific contexts. Yet linguistic knowledge also encodes relationships between contexts, allowing inferences between word distributions. We investigate the degree to which pre-trained transformer-based large language models (LLMs) represent such relationships, focusing on the domain of argument structure. We find that LLMs perform well in generalizing the distribution of a novel noun argument between related contexts that were seen during pre-training (e.g., the active object and passive subject of the verb spray), succeeding by making use of the semantically organized structure of the embedding space for word embeddings. However, LLMs fail at generalizations between related contexts that have not been observed during pre-training, but which instantiate more abstract, but well-attested structural generalizations (e.g., between the active object and passive subject of an arbitrary verb). Instead, in this case, LLMs show a bias to generalize based on linear order. This finding points to a limitation with current models and points to a reason for which their training is data-intensive.1

Are Textbooks Authentic From a Developmental Perspective? A Corpus Analysis of Word Use in Chinese Textbooks in US Universities

Abstract There is a prevailing belief that second language (L2) textbooks should strive for authenticity, aiming to accurately and reliably represent natural language use. However, assessing the authenticity of language textbooks is not a straightforward task, as it requires both a comparison between instructional texts and real-world language use and a consideration of the comparison within the broader context of language development. This study employs a corpus-based approach to examine word use in 31 widely used commercial textbooks across different proficiency levels, which serve as the primary resource for teaching, learning, and assessment in Chinese language programs at US universities. Using hierarchical cluster analysis, Zipf’s law analysis, and descriptive statistics, we found supportive evidence for the authenticity of word use in Chinese textbooks from a developmental perspective. Specifically, our findings reveal a nuanced gradation in the distribution of words across proficiency levels within these Chinese textbooks. Moreover, as proficiency levels increase, there is a discernible register shift in word use from spoken to written discourse. These observations carry significant implications for the utility of textbooks in Chinese language programs, particularly in the realms of instruction and assessment.