Text Model Research Articles

A Disturbance-Driven Textual Model for Train Running Time Prediction on High-Speed Railways

A Disturbance-Driven Textual Model for Train Running Time Prediction on High-Speed Railways

Advancing Financial Resilience: A Systematic Review of Default Prediction Models and Future Directions in Credit Risk Management

This research presents a systematic review of a substantial body of high-quality research articles on Default Prediction Models published from 2015 to 2024. It is a comprehensive analysis of a DPM wide spectrum approaches including Textual Models, Systematic Review Studies, Hybrid Models, Intelligent Models and Statistical Models. The reason behind this study is rooted in the critical need to mitigate and understand the credit default risk that poses a significant threat to financial stability worldwide. By employing an evidence-based approach and methodological rigorously, this research critically evaluates the gaps, effectiveness and evolution in existing DPM methodologies. It is not only synthesized the current landscape of DPM study but also identified the direction for the future research, by offering novel insights and bridging theoretical gaps for enhancing the strategies of credit risk management. This study stands out by focusing on high citation research from top tier publishers, ensuring the quality and relevance of its analysis. The findings of this study have profound implications for stakeholders across the financial sector, including bankers, investors, regulatory bodies, and researchers. It aims to advance financial stability by providing a comprehensive overview of DPM advancements and pointing towards areas that require further exploration. By doing so, it contributes significantly to the development of more effective and sophisticated DPM strategies, thereby enhancing the robustness of financial institutions against potential defaults.

Der gut strukturierte Tod

Abstract Based on the example of King Conrad I’s death, this paper wants to contribute to the question of how medieval historiography chose and arranged its contents. It postulates that chroniclers narrated their histories just like medieval authors of fictional literature by using plot schemes. Before King Conrad I died, Ottonian sources tell us, he had called for his brother and designated the Saxonian Duke Henry I king. Former research saw vast similarities between the sources as either evidence for their reliability, as textual references, or regarded them as written result of an oral tradition at the Ottonian court. A comparison with several ( East-/West- ) Franconian and English narratives about other rulers’ deaths ( e. g. Pepin the Younger, William the Conqueror or Saladin ) shows us that not only Ottonian, but chroniclers in general used the same textual elements, which they arranged in a similar order. One reason for this might be references to the Bible and hagiography. However, since there are no obvious linguistic borrowings tangible between the narratives, the death of Conrad I in the Ottonian sources can be interpreted as a variation of narratives, based on a more widespread scheme used probably all over Latin Europe. This scheme was distributed in different ways: by the use of textual models, but also by the adaption of oral narratives.

Using boundary objects and methodological island (BOMI) modeling in large-scale agile systems development

AbstractLarge-scale systems development commonly faces the challenge of managing relevant knowledge between different organizational groups, particularly in increasingly agile contexts. Here, there is a conflict between coordination and group autonomy, and it is challenging to determine what necessary coordination information must be shared by what teams or groups, and what can be left to local team management. We introduce a way to manage this complexity using a modeling framework based on two core concepts: methodological islands (i.e., groups using different development methods than the surrounding organization) and boundary objects (i.e., artifacts that create a common understanding across team borders). However, we found that companies often lack a systematic way of assessing coordination issues and the use of boundary objects between methodological islands. As part of an iterative design science study, we have addressed this gap by producing a modeling framework (BOMI: Boundary Objects and Methodological Islands) to better capture and analyze coordination and knowledge management in practice. This framework includes a metamodel, as well as a list of bad smells over this metamodel that can be leveraged to detect inter-team coordination issues. The framework also includes a methodology to suggest concrete modeling steps and broader guidelines to help apply the approach successfully in practice. We have developed Eclipse-based tool support for the BOMI method, allowing for both graphical and textual model creation, and including an implementation of views over BOMI instance models in order to manage model complexity. We have evaluated these artifacts iteratively together with five large-scale companies developing complex systems. In this work, we describe the BOMI framework and its iterative evaluation in several real cases, reporting on lessons learned and identifying future work. We have produced a matured and stable modeling framework which facilitates understanding and reflection over complex organizational configurations, communication, governance, and coordination of knowledge artifacts in large-scale agile system development.

FETILDA: Evaluation Framework for Effective Representations of Long Financial Documents

In the financial sphere, there is a wealth of accumulated unstructured financial data, such as the textual disclosure documents that companies submit on a regular basis to regulatory agencies, such as the Securities and Exchange Commission. These documents are typically very long and tend to contain valuable soft information about a company’s performance that is not present in quantitative predictors. It is therefore of great interest to learn predictive models from these long textual documents, especially for forecasting numerical key performance indicators. In recent years, there has been great progress in natural language processing via pre-trained language models (LMs) learned from large corpora of textual data. This prompts the important question of whether they can be used effectively to produce representations for long documents, as well as how we can evaluate the effectiveness of representations produced by various LMs. Our work focuses on answering this critical question, namely, the evaluation of the efficacy of various LMs in extracting useful soft information from long textual documents for prediction tasks. In this article, we propose and implement a deep learning evaluation framework that utilizes a sequential chunking approach combined with an attention mechanism. We perform an extensive set of experiments on a collection of 10-K reports submitted annually by U.S. banks, and another dataset of reports submitted by U.S. companies, to investigate thoroughly the performance of different types of language models. Overall, our framework using LMs outperforms strong baseline methods for textual modeling as well as for numerical regression. Our work provides better insights into how utilizing pre-trained domain-specific and fine-tuned long-input LMs for representing long documents can improve the quality of representation of textual data and, therefore, help in improving predictive analyses.

Automatic Algerian Sarcasm Detection from Texts and Images

In recent years, the number of Algerian Internet users has significantly increased, providing a valuable opportunity for collecting and utilizing opinions and sentiments expressed online. They now post not just texts but also images. However, to benefit from this wealth of information, it is crucial to address the challenge of sarcasm detection, which poses a limitation in sentiment analysis. Sarcasm often involves the use of non-literal and ambiguous language, making its detection complex. To enhance the quality and relevance of sentiment analysis, it is essential to develop effective methods for sarcasm detection. By overcoming this limitation, we can fully harness the expressed online opinions and benefit from their valuable insights for a better understanding of trends and sentiments among the Algerian public. In this work, our aim is to develop a comprehensive system that addresses sarcasm detection in Algerian dialect, encompassing both text and image analysis. We propose a hybrid approach that combines linguistic characteristics and machine learning techniques for text analysis. Additionally, for image analysis, we utilized the deep learning model VGG-19 for image classification, and employed the EasyOCR technique for Arabic text extraction. By integrating these approaches, we strive to create a robust system capable of detecting sarcasm in both textual and visual content in the Algerian dialect. Our system achieved an accuracy of 92.79% for the textual models and 89.28% for the visual model.

MISTRA: Misogyny Detection through Text–Image Fusion and Representation Analysis

Detecting misogynous memes poses a significant challenge due to the presence of multiple modalities (image + text). The inherent complexity arises from the lack of direct correspondence between the textual and visual elements, where an image and overlaid text often convey disparate meanings. Additionally, memes conveying messages of hatred or taunting, particularly targeted towards women, present additional comprehension difficulties. This article introduces the MISTRA framework, which leverages variational autoencoders for dimensionality reduction of the large-sized image features before fusing multimodal features. The framework also harnesses the capabilities of large language models through transfer learning to develop fusion embeddings by extracting and concatenating features from different modalities (image, text, and image-generated caption text) for the misogynous classification task. The components of the framework include state-of-the-art models such as the Vision Transformer model (ViT), textual model (DistilBERT), CLIP (Contrastive Language–Image Pre-training), and BLIP (Bootstrapping Language–Image Pre-training for Unified Vision-Language Understanding and Generation) models. Our experiments are conducted on the SemEval-2022 Task 5 MAMI dataset. To establish a baseline model, we perform separate experiments using the Naive Bayes machine learning classifier on meme texts and ViT on meme images. We evaluate the performance on six different bootstrap samples and report evaluation metrics such as precision, recall, and Macro-F1 score for each bootstrap sample. Additionally, we compute the confidence interval on our evaluation scores and conduct paired t-tests to understand whether our best-performing model has significant differences from the other experiments or not. The experimental results demonstrate that the dimensionality reduction approach on multimodal features with a multilayer perceptron classifier achieved the highest performance with a Macro–F1 score of 71.5 percent, outperforming the baseline approaches in individual modalities.

“They created segregation with the economy”: Using AI for a student-driven inquiry into redlining in the social studies classroom

ABSTRACT This article investigates students’ engagement with a historical inquiry into redlining—a practice of discriminatory lending that originated in the 1930s as part of the New Deal. The authors developed and implemented a week-long curricular intervention for high school sophomores using StoryQ—an Artificial Intelligence (AI) textual modeling platform designed for high school students without technical expertise—to examine hundreds of neighborhood descriptions produced for the Home Owners Loan Corporation’s “residential security maps” in the late 1930s. In this article, we ask: What kinds of historical and present-day racial awareness do high school students demonstrate through instruction focused on AI-assisted analysis of patterns in redlining? Analyzing field notes, interviews, and student-generated digital work showed that many students were drawn to structural explanations of racism and worked to unpack the way primary sources presented Whiteness through “coded language.” We argue that it is not only possible for teachers to construct historical inquiries that aim to identify patterns in a large set of primary sources with the aid of AI, but this approach to inquiry offers students an important avenue to engage with alternatives to individual conceptions of racial oppression.

Pelatihan Pembelajaran Berbasis Teks Secara Digital Dalam Kerangka Penerapan Kurikulum Merdeka

Text-based teaching (TBT) has characterized and been emphasized in our national curriculum at elementary and secondary levels. As a consequence, English teaching is based on the TBT principles. Along with the vast development of technology, the implementation of TBT is also integrated with the use of digital technology that is to increase the quality of teaching and learning and to increase students’ technology literacy. This paper reports a workshop on digital TBT in the implementation of “Kurikulum Merdeka.” The objective is to find out teachers’ implementation of digital TBT for teaching English using “Kurikulum Merdeka.” The participants were 26 English teachers under Musyawarah Guru Mata Pelajaran (MGMP) Bahasa Inggris (an English Teachers Forum), Salatiga, Central Java. The methods used in this program were reflective practice and Focus Group Discussion (FGD). The findings revealed that most teachers used all the steps of the TBT cycle. All teachers applied Modelling of Text (MOT), but only a few applied Joint Construction of Text (JCOT). The findings on the use of technology for the TBT cycle demonstrated that the teachers used digital technology more in Building Knowledge of the Fields (BKOF) and Independent Construction of Text (ICOT).

Radioport: a radiomics-reporting network for interpretable deep learning in BI-RADS classification of mammographic calcification

Objective. Generally, due to a lack of explainability, radiomics based on deep learning has been perceived as a black-box solution for radiologists. Automatic generation of diagnostic reports is a semantic approach to enhance the explanation of deep learning radiomics (DLR). Approach. In this paper, we propose a novel model called radiomics-reporting network (Radioport), which incorporates text attention. This model aims to improve the interpretability of DLR in mammographic calcification diagnosis. Firstly, it employs convolutional neural networks to extract visual features as radiomics for multi-category classification based on breast imaging reporting and data system. Then, it builds a mapping between these visual features and textual features to generate diagnostic reports, incorporating an attention module for improved clarity. Main results. To demonstrate the effectiveness of our proposed model, we conducted experiments on a breast calcification dataset comprising mammograms and diagnostic reports. The results demonstrate that our model can: (i) semantically enhance the interpretability of DLR; and, (ii) improve the readability of generated medical reports. Significance. Our interpretable textual model can explicitly simulate the mammographic calcification diagnosis process.

WITHDRAWN: Evaluating the generation capabilities of large Chinese language models

This paper unveils CG-Eval, the first-ever comprehensive and automated evaluation framework designed for assessing the generative capabilities of large Chinese language models across a spectrum of academic disciplines. CG-Eval stands out for its automated process, which critically assesses models based on their proficiency in generating precise and contextually relevant responses to a diverse array of questions within six key domains: Science and Engineering, Humanities and Social Sciences, Mathematical Calculations, Medical Practitioner Qualification Examination, Judicial Examination, and Certified Public Accountant Examination. Alongside this, we introduce Gscore, an innovative composite index developed from a weighted sum of multiple metrics. Gscore uniquely automates the quality measurement of a model's text generation against reference standards, providing a detailed and nuanced assessment of model performance. This automation not only enhances the efficiency and scalability of the evaluation process but also ensures objective and consistent assessment across various models. The detailed test data and results, highlighting the robust capabilities and comparative performance of the evaluated models, are accessible at http://cgeval.besteasy.com/.

Automated football match reports as models of textuality

Abstract This paper deals with automated football match reports as a common genre of automated journalism. Based on a corpus of automated and human-written reports (n = 1,302) on the same set of matches and with reference to linguistic concepts of text and textuality, the textual properties of these texts are analyzed both quantitatively and qualitatively. The analysis is based on the idea that the task of text generation can be described as the task of automatically selecting cues of textuality such as connectives or signals of thematic relatedness. The results show that automated and human-written texts differ significantly in the use of these cues, particularly in the use of linguistic means for creating evaluation and contrast, and thus allow to trace in detail, how these cues contribute to cohesion, coherence and narrative qualities. Different from computational linguistic approaches focused on optimizing text generation algorithms, this paper proposes to use automated texts, which are to some extent imperfect, as models of textuality that through their imperfection can say something about the nature of texts in general. The paper thus contributes to the field of (mostly communication studies) research on automated journalism in which the texts themselves are rarely investigated.

Overcoming linguistic barriers in code assistants: creating a Qlora adapter to improve support for russian-language code writing instructions

This article describes an approach to learning and evaluating an adapter model for the popular zephyr-7b-beta language model. The adapter was designed to improve the performance of the basic model in tasks related to programming and understanding the Russian language. Taking into account the high quality of the original model's work in problems in English, the purpose of the study was to expand its linguistic and technical spectrum. The proposed adapter was trained using a large and diverse dataset, including question-answer pairs related to programming, as well as a text with a discussion of the code in Russian. The applied training methodology ensures an improvement in the quality of the model's responses in understanding and generating Python code based on Russian-language instructions. The authors evaluated the performance of the basic model with the adapter installed using various metrics, comparing it with the basic model, as well as with other advanced models in this area. The results obtained by the authors showed significant improvement both in tasks related to writing Python code and in processing the Russian language, confirming the effectiveness of the proposed adapter. The analysis of the experiment results showed that the adapter not only adds new functionality to the model but also significantly changes the priority of the model's output in favor of the instructions that were included during the retraining stage. To eliminate the fact of changing the model's output priority mentioned above, the following methods are proposed: 1) Using a different format for writing the query (prompt) when formatting the dataset for training the model, different from the original format in Zephyr. 2) Including instructions from the first stage of fine-tuning (retraining) in the instruction set. 3) Creating a "mixture of experts" from several different models, where each model will specialize in the most accurate solution to one or several tasks from the overall range of solvable tasks. The methods of retraining the model without losing the original characteristics are promising directions for future research in the field of resource-efficient modification of large language models. In the conducted experiment, a large language model "HuggingFaceH4/zephyr-7b-beta" was successfully fine-tuned (using the QLoRA adapter creation method) to improve its capabilities in writing Python code based on Russian instructions and providing explanations in Russian. The testing results showed that when the created adapter is installed, the model's text generation priority shifts towards generating code with explanations. Synthetic tests also demonstrated significant improvements in the model's ability to solve programming and mathematical problems. Therefore, it is advisable to consider the possibility of changing adapters depending on the type of task being solved. Changing the adapter installed on the model (loaded into memory) does not take a significant amount of time, which allows for the creation of a set of adapters tailored to specific tasks and their interchange based on the type of tasks being solved. Changing adapters instead of models eliminates the need to load multiple full versions of the model into the memory of the computational accelerator, which, depending on the implementation method, either significantly reduces the use of memory or the model loading time. Thus, the use of adapters speeds up training and optimizes the utilization of computational resources during model operation. This presented research contributes to the current efforts in the field of machine learning and natural language processing towards creating more versatile AI models. The observed potential of adapter models in improving the quality of responses in specific domains without the need for extensive retraining helps meet the growing demand for models for multilingual natural language processing and code generation. In the conducted experiment, an assessment was made of the possibility of retraining a model that had already been trained on a set of instructions. The methods of additional training of a language model with new instructions without losing the ability to perform original instructions are not well-studied, which is why a decision was made to test the feasibility and effectiveness of such retraining. Based on the description of the LoRA method, this approach is possible because during such training, the original weights of the model remain unchanged and are only supplemented with new coefficients from the adapter model (the total number of weights equals that of the base model), which was trained on new instructions. The experiment determined that multilingual adaptation of language models is viable, and that it also could be implemented using parametrically efficient methods on comparatively low-end hardware.

Weakening the Dominant Role of Text: CMOSI Dataset and Multimodal Semantic Enhancement Network.

Multimodal sentiment analysis (MSA) is important for quickly and accurately understanding people's attitudes and opinions about an event. However, existing sentiment analysis methods suffer from the dominant contribution of text modality in the dataset; this is called text dominance. In this context, we emphasize that weakening the dominant role of text modality is important for MSA tasks. To solve the above two problems, from the perspective of datasets, we first propose the Chinese multimodal opinion-level sentiment intensity (CMOSI) dataset. Three different versions of the dataset were constructed: manually proofreading subtitles, generating subtitles using machine speech transcription, and generating subtitles using human cross-language translation. The latter two versions radically weaken the dominant role of the textual model. We randomly collected 144 real videos from the Bilibili video site and manually edited 2557 clips containing emotions from them. From the perspective of network modeling, we propose a multimodal semantic enhancement network (MSEN) based on a multiheaded attention mechanism by taking advantage of the multiple versions of the CMOSI dataset. Experiments with our proposed CMOSI show that the network performs best with the text-unweakened version of the dataset. The loss of performance is minimal on both versions of the text-weakened dataset, indicating that our network can fully exploit the latent semantics in nontext patterns. In addition, we conducted model generalization experiments with MSEN on MOSI, MOSEI, and CH-SIMS datasets, and the results show that our approach is also very competitive and has good cross-language robustness.

Protest Slogans of the Snow Revolution in Russia (2011–2012) as an Expression of Peaceful Resistance against the Authorities

The expression of resistance against the authorities during the street protests of the so-called snow revolution in the years 2011-2012 reached a level of mass creativity and visuality previously and since unrecorded in Russian social life and, despite its lack of success, it is worth attention due to its openness and intensity. During the snow revolution a certain corpus of texts (slogans) was shaped, with repeatability typical for folklore – it was a recurrence of themes, motifs, characters, linguistic tricks, images, textual models. The aim of analyzing selected protest slogans is to showcase contexts that constitute the source of oppositional expression on the textual (slogan on a poster), visual (image on a poster) and object (artifacts) level. The examples of the protest creativity presented in the article prove that verbal, visual and material ways of resistance realized functions typical for political folklore, such as integrating the rebels, mocking the stronger enemy as a defense strategy, unmasking the crimes of the government, and expressing the expectations of the protesters.

Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity

As Large Language Models (LLMs) become increasingly sophisticated and ubiquitous in natural language processing (NLP) applications, ensuring their robustness, trustworthiness, and alignment with human values has become a critical challenge. This paper presents a novel framework for contextual grounding in textual models, with a particular emphasis on the Context Representation stage. Our approach aims to enhance the reliability and ethical alignment of these models through a comprehensive, context-aware methodology. By explicitly capturing and representing relevant situational, cultural, and ethical contexts in a machine-readable format, we lay the foundation for anchoring a model's behavior within these contexts. Our approach leverages techniques from knowledge representation and reasoning, such as ontologies, semantic web technologies, and logic-based formalisms. We evaluate our framework on real-world textual datasets, demonstrating its effectiveness in improving model performance, fairness, and alignment with human expectations, while maintaining high accuracy. Furthermore, we discuss the other key components of the framework, including context-aware encoding, context-aware learning, interpretability and explainability, and continuous monitoring and adaptation. This research contributes to the growing body of work on responsible AI, offering a practical approach to developing more reliable, trustworthy, and ethically-aligned language models. Our findings have significant implications for the deployment of LLMs in sensitive domains such as healthcare, legal systems, and social services, where contextual understanding is paramount.

Requirements development for IoT systems with UCM4IoT

The engineering of IoT (Internet of Things) systems brings about various challenges due to the inherent complexities associated with such adaptive systems. Addressing the adaptive nature of IoT systems in the early stages of the development life cycle is essential for developing a complete and precise system specification. In this paper, we propose a use case-based modelling language, UCM4IoT, to support requirements elicitation and specification of IoT systems. UCM4IoT takes into account the heterogeneity of IoT systems and provides domain-specific language constructs to model the different facets of IoT systems. The language also incorporates the notion of exceptional situations and adaptive system behaviour. Our language is supported with a textual modelling environment to assist modellers in writing use cases. The environment supports syntax-directed editing, validation of use case models, and requirements analysis. The proposed language and tool is demonstrated and evaluated with two case studies: smart store system and smart fire alarm system.

Formulate Arabic Learning Objectives in Early Childhood Education Unit

As a characteristic of Islam, the Arabic language curriculum has been taught in Islamic-based early childhood education units. This research is important because Arabic has become an international language used as a tool of international communication. Hence, skills and abilities in speaking Arabic properly are competencies that students at all levels of education must possess. This research aims to find, identify, and explain the objectives of learning Arabic in Islamic-based early childhood education units. The research was conducted using a qualitative-descriptive-and textual model, namely research using a database of textual documents that were studied and presented descriptively from a scientific point of view of learning objectives, early childhood, and early childhood education units. The research results in areFirst, The instrumental objective of Arabic is taught so that early childhood can use Arabic to understand, get used to, and practice the teachings, worship, and prayers found in the Al-Qur'an and Al-Hadith. Second, for Integrative-communicative purposes, Arabic is taught so that children can have communication skills in Arabic. Third, pedagogical goals include Arabic in learning activities to master Arabic knowledge, skills, and values ​​in early childhood.

Medieval roots of antisemitism in Sweden

The lack of a local Jewish community did not prevent medieval Swedish clerics and lay people from being interested in Jews and Jewish questions. They bought, translated, read and preached from most of the available textual sources and thus spread the widely available views of the hermeneutical Jew: a cruel, stubborn and ugly person and at the same time a cipher for the entire Jewish people both in biblical times and today. This article gives an overview of the Latin and vernacular manuscripts with anti-Jewish motifs and texts and shows that the main and most common textual models and motifs were available in Swedish libraries and collections, from legends via apocryphal texts to fake disputations – adding up to a relatively complete ‘hermeneutical Jew’. A focus was, as in the rest of Europe, on Passion-related piety, which was the most common form of piety in the late Middle Ages – and usually connected with distinct anti-Jewish features. The fact that we can establish direct and indirect textual and narrative lines of tradition between the medieval codices and modern printed booklets of the nineteenth century proves the long-lasting intelligibility of anti-Jewish stereotypes in Sweden – developed and spread completely independently from the Jewish minority. The medieval perspective thus adds a much-needed nuance to the debate about antisemitism in the North: it did not need any actual Jews; it simply made up its own, based on the general Christian tradition.

The OpenESEA Modeling Language and Tool for Ethical, Social, and Environmental Accounting

Assessing business operations’ ethical, social, and environmental impacts is a key practice for establishing sustainable development. There is a multitude of methods that describes how to perform such assessments. Often these methods are supported by an ICT tool. In most cases, the tools are developed to support a single method only and do not allow any tailoring. Therefore, they are rigid and inflexible. In this article, we present a novel model-driven approach for alleviating managerial issues that arise as a consequence of the complex landscape of ethical, social, and environmental accounting methods and tools. We have developed an open-source, model-driven tool, called openESEA. OpenESEA parses and interprets textual models, that are specified according to a domain-specific language (DSL). We have performed another iteration of the DSL engineering process, which is in line with the design science paradigm. We have validated the new DSL version by means of a user study. As a result, we present a new version of the openESEA modeling language and interpreter. The results of the user study with regards to performance, perceived usefulness, and perceived ease of use of modeling language are encouraging and provide us with a basis to continue developing new versions with more functionalities. The contributions of this work include a new version of the modeling language, a new version of the interpreter, knowledge surrounding the development of these artifacts, and a protocol for evaluating the quality of textual DSLs. The modeling language and interpreter are relevant for sustainability practitioners and consultants since our tool support has the potential to reduce redundancy in ethical, social, and environmental accounting. Our work is valuable to researchers that aim to assess and reduce the complexity of their modeling languages.