Rich Text Research Articles

From Shi`a to White: Race and Colonialism in Kamel Murouwwa’s Nahnu fi Ifriqiya

In 1938, Lebanese Syrian publisher Kamel Murouwwa extensively toured West Africa and produced a 329-page travelogue entitled Nahnu Fi Ifriqiya: al-Hijra al-Lubnaniyya al-Suriyya ila Ifriqiya al-Gharbiyya, Madiha, Hadirha, Mustaqbaliha (We are in Africa: Lebanese Syrian Migration to West Africa, its Past, Present, and Future). In this article, I use Murouwwa’s travelogue as a lens to understand the process of racial identity formation in West Africa from a Lebanese visitor’s perspective in the 1930s under French colonialism. Murouwwa's source has been underutilized as a rich text of thick racial description. This article argues that through the process of racial identity formation in West Africa, Lebanese Shi’is were able to transcend emerging sectarian identities prevalent in newly carved Lebanon which marked Shi’is as “metwali” (backward). Lebanese migrants’ “whiteness” became the most pronounced marker of their presence amongst local Africans, overshadowing religious identity and erasing any “backwardness” associated with Shi’is in Lebanon, elevating the community to the class of colonizer vs. colonized. Sectarianism carries significant weight within the confines of the nation-state but a broadened transnational approach to sectarianism provides a widened view to see how racial identity supplanted sectarian identity outside the nation-state.

Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning.

Emerging multimodal large language models (MLLMs) exhibit great potential for chart question answering (CQA). Recent efforts primarily focus on scaling up training datasets (i.e., charts, data tables, and question-answer (QA) pairs) through data collection and synthesis. However, our empirical study on existing MLLMs and CQA datasets reveals notable gaps. First, current data collection and synthesis focus on data volume and lack consideration of fine-grained visual encodings and QA tasks, resulting in unbalanced data distribution divergent from practical CQA scenarios. Second, existing work follows the training recipe of the base MLLMs initially designed for natural images, under-exploring the adaptation to unique chart characteristics, such as rich text elements. To fill the gap, we propose a visualization-referenced instruction tuning approach to guide the training dataset enhancement and model development. Specifically, we propose a novel data engine to effectively filter diverse and high-quality data from existing datasets and subsequently refine and augment the data using LLM-based generation techniques to better align with practical QA tasks and visual encodings. Then, to facilitate the adaptation to chart characteristics, we utilize the enriched data to train an MLLM by unfreezing the vision encoder and incorporating a mixture-of-resolution adaptation strategy for enhanced fine-grained recognition. Experimental results validate the effectiveness of our approach. Even with fewer training examples, our model consistently outperforms state-of-the-art CQA models on established benchmarks. We also contribute a dataset split as a benchmark for future research. Source codes and datasets of this paper are available at https://github.com/zengxingchen/ChartQA-MLLM.

Language Model Crossover: Variation through Few-Shot Prompting

This article pursues the insight that language models naturally enable an intelligent variation operator similar in spirit to evolutionary crossover. In particular, language models of sufficient scale demonstrate in-context learning, i.e., they can learn from associations between a small number of input patterns to generate outputs incorporating such associations (also called few-shot prompting). This ability can be leveraged to form a simple but powerful variation operator, i.e., to prompt a language model with a few text-based genotypes (such as code, plain-text sentences, or equations), and to parse its corresponding output as those genotypes’ offspring. The promise of such language model crossover (which is simple to implement and can leverage many different open source language models) is that it enables a simple mechanism to evolve semantically rich text representations (with few domain-specific tweaks), and naturally benefits from current progress in language models. Experiments in this article highlight the versatility of language-model crossover, through evolving binary bit-strings, sentences, equations, text-to-image prompts, and Python code. The conclusion is that language model crossover is a flexible and effective method for evolving genomes representable as text.

Enhancing Text-to-Image Generation: Integrating CLIP and Diffusion Models for Improved Visual Accuracy and Semantic Consistency

Abstract. Text-to-Image (T2I) generation focuses on producing images that precisely match given textual descriptions by combining techniques from computer vision and natural language processing (NLP). Existing studies have shown an innovative approach to enhance T2I generation by integrating Contrastive Language-Image Pretraining (CLIP) embeddings with a Diffusion Model (DM). The method involves initially extracting rich and meaningful text embeddings using CLIP, which are then transformed into corresponding images. These images are progressively refined through an iterative denoising process enabled by diffusion models. Comprehensive experiments conducted on the MS-COCO dataset validate the proposed method, demonstrating significant improvements in image fidelity and the alignment between text and images. When compared to traditional models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), which often struggle with maintaining both visual quality and semantic accuracy, this hybrid model shows superior performance. Future research could explore optimizing hybrid models further and applying T2I technology to specialized fields, such as medical imaging and scientific visualization, expanding its potential use cases.

Glitching the simulated carnival: “Girls like us” in MTV's Ex on the Beach

ABSTRACT During the fourth season of MTV’s Ex on the Beach, Adore Delano disrupts a rich and complicated tradition of queer visibility and representation. Unlike her queer predecessors, who often used unscripted talk shows and reality television to seek normalization, Adore demonstrates a conspicuous lack of appeals to normality while actively working toward coalition among her cis, straight counterparts. Recognizing the mediated nature of reality television as a simulation of Bakhtin’s carnival, this paper argues that Adore’s queer disruptions coalesce to glitch the homogenizing function of a simulated carnival. Throughout her time on the show, Adore actively and consciously produces discourses of gender that examine illegibility; enact and deconstruct hegemonic masculinities; and use the rhetorics of gender to hail cis and trans women into coalition. Together, these interventions provide a rich text through which we might understand alternative notions of visibility and coalition-building in the face of a political climate that intends to annihilate queer communities.

People's Voice: Exploring Discussion Themes on Generative AI and Libraries on X

ABSTRACTThe launch of ChatGPT and other Generative Artificial Intelligence (GenAI) models has significantly influenced people's perceptions of AI, making its impact a widely discussed topic in various areas. In Library and Information Science (LIS), researchers experience the opportunities and challenges presented by GenAI, such as using the rich text from books and digital collections as training data for GenAI and enabling quick summarization and reference consultation for patrons. Meanwhile, concerns have arisen about the generation of fake information and nonexistent text by GenAI and their potential effects on users. This project explores the themes in public discussions regarding the impact of GenAI on libraries. To explore public opinions about the impact of GenAI in libraries, we collected over 50,000 posts from X (formerly Twitter) between November 1, 2022, and May 6, 2024. We manually sampled 1,000 posts focusing on GenAI in the context of libraries. We conducted content analysis to identify the key themes and relationships among these themes. This poster presents our initial findings on different themes and highlights the trends in public opinion. Our analysis provides a curated dataset for training APIs for future investigation.

Black Informational Past: History, Data, and Archives of Chicago's ‘New Negro’ Wonder books

ABSTRACTIn 1927, an organized group of Black college and graduate students from the Washington Intercollegiate Club of Chicago compiled a rich informational reference text reflecting the mosaic of Black life in Chicago, IL, USA. This effort resulted in the publication of two editions of the Intercollegian Wonder Book in 1927 and 1929. These books are reflective of the New Negro movement and cultural renaissance of the Black urban population in the early 20th Century. Our project examines the influence of the Wonder Books through methods that express the layered informational context of a Black community. The poster reflects the iterative development of a phased descriptive, archival, data and visual analysis of an historic Black Community. such as citation tracking, genealogical research of individual members, and analysis of data usage by the club's active students.

Real Time Collaborative Code Editor

Abstract: The main goal of our project is to provide an online platform especially for group coding interviews. This platform provides a comprehensive environment for seamless communication between interviewers and candidates, with the goal of addressing the challenges faced during remote technical assessments. The creation ofa feature-rich web platform is the goal of our project. Thisincludes an adaptable rich text editor that enables coding in real time, a mechanism for instantaneous code evaluation via execution and compilation, integrated video conferencing for efficient communication, and the abilityto set up interview rooms to hold several interviews at once. We're designing a user intuitive interface using technology and strong programming frameworks. The main objectives of the platform are to ensure real time code execution and a seamless interview process. This is achieved by incorporating code compilation engines implementing editing tools and utilizing WebRTC for video conferencing. By creating a user-friendly and productive environment that benefits both interviewers and candidates, our platform seeks to expedite the remote technical interview process. This means reducing logistical obstacles, making thorough coding evaluations possible, and creating a positive interview environment.

IQAGPT: computed tomography image quality assessment with vision-language and ChatGPT models

Large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities in various tasks and attracted increasing interest as a natural language interface across many domains. Recently, large vision-language models (VLMs) that learn rich vision–language correlation from image–text pairs, like BLIP-2 and GPT-4, have been intensively investigated. However, despite these developments, the application of LLMs and VLMs in image quality assessment (IQA), particularly in medical imaging, remains unexplored. This is valuable for objective performance evaluation and potential supplement or even replacement of radiologists’ opinions. To this end, this study introduces IQAGPT, an innovative computed tomography (CT) IQA system that integrates image-quality captioning VLM with ChatGPT to generate quality scores and textual reports. First, a CT-IQA dataset comprising 1,000 CT slices with diverse quality levels is professionally annotated and compiled for training and evaluation. To better leverage the capabilities of LLMs, the annotated quality scores are converted into semantically rich text descriptions using a prompt template. Second, the image-quality captioning VLM is fine-tuned on the CT-IQA dataset to generate quality descriptions. The captioning model fuses image and text features through cross-modal attention. Third, based on the quality descriptions, users verbally request ChatGPT to rate image-quality scores or produce radiological quality reports. Results demonstrate the feasibility of assessing image quality using LLMs. The proposed IQAGPT outperformed GPT-4 and CLIP-IQA, as well as multitask classification and regression models that solely rely on images.

Research on Resident Behavioral Activities Based on Social Media Data: A Case Study of Four Typical Communities in Beijing

With the support of big data mining techniques, utilizing social media data containing location information and rich semantic text information can construct large-scale daily activity OD flows for urban populations, providing new data resources and research perspectives for studying urban spatiotemporal structures. This paper employs the ST-DBSCAN algorithm to identify the residential locations of Weibo users in four communities and then uses the BERT model for activity-type classification of Weibo texts. Combined with the TF-IDF method, the results are analyzed from three aspects: temporal features, spatial features, and semantic features. The research findings indicate: ① Spatially, residents’ daily activities are mainly centered around their residential locations, but there are significant differences in the radius and direction of activity among residents of different communities; ② In the temporal dimension, the activity intensities of residents from different communities exhibit uniformity during different time periods on weekdays and weekends; ③ Based on semantic analysis, the differences in activities and venue choices among residents of different communities are deeply influenced by the comprehensive characteristics of the communities. This study explores methods for OD information mining based on social media data, which is of great significance for expanding the mining methods of residents’ spatiotemporal behavior characteristics and enriching research on the configuration of public service facilities based on community residents’ activity spaces and facility demands.

Differentially processed optimized collaborative rich text editor

Differentially processed optimized collaborative rich text editor

Key Information Extraction and Recognition from Rich Text Images

Key information extraction and recognition from rich text images are crucial for various applications. There are two main tasks involved in this process: Line Item Recognition (LIR) and Key Information Localization and Extraction (KILE). LIR aims at identifying and interpreting data line items in a document. The essential information in each line item is then classified or extracted, a task known as KILE. A widely used approach for this problem is sequence based, which relies on the generalization of a language model and requires a significant amount of training time. We present an effective and reliable solution to the problem by using RoBERTa, a transformer model trained on a large corpus, along with the LION optimizer to improve the training process. A comprehensive evaluation was conducted on two different benchmarks, emphasizing two different languages, English and Vietnamese. Experimental results on DocILE indicate that the proposed framework significantly improves the KILE task with a 7.24% increase in accuracy compared to the baseline and also enhances the correct recognition rate at the LIR stage. On MCOCR, the method achieved a Character Error Rate (CER) of 28.6%, which is competitive with the state-of-the-art on this dataset.

DSDRec: Next POI recommendation using deep semantic extraction and diffusion model

Semantics play a crucial role in many AI tasks, yet the lack of high-quality textual data in LBSNs hampers deep semantic feature learning. Sparse user check-in records, characterized by significant spatial or temporal intervals (cut points), create challenges in understanding users' real mobile preferences. However, prior studies often rely on shallow semantic signals from independent textual properties and overlook these cut points when modeling spatio-temporal context. This work proposes a recommender model named DSDRec for the next POI recommendation. To alleviate the shortage of high-quality textual data in LBSNs, we take prompt engineering to freely obtain rich semantic texts (a.k.a., prompt sentences) from the discrete trajectory sequences and then use a pre-trained language model to acquire high-quality deep semantic features based on the functional prompt sentences. To mitigate the risk caused by the cut points, we propose an effective method, Spatio-Temporal Area Encoding, to depict the global spatial and temporal relationship of check-ins in trajectory. Besides, to study the potential of diffusion models in the next POI recommendation, we stacked an improved diffusion model to assist the recommender in learning more abstract interaction information. Comprehensive experiments conducted on two real-world datasets validate the competitive performance of DSDRec over previous SOTA approaches.

Multiple Personalities of Pulp Fiction 's Musical Soundtrack: Liner Notes to a 30th Anniversary

Abstract: Pulp Fiction: Music from the Motion Picture was released in 1994 in coordination with the film Pulp Fiction (1994). Across the film's soundtrack album, variously considered an auteur mixtape, a compilation score, and its own rich text, music formulates a cast of characters alongside those more recognizable from the film's story. Critical analysis reveals multiple and multiply associative and disassociative personalities that afford innovative approaches to the role of the soundtrack and sound-plus-visual imagery in Pulp Fiction .

ChineseEEG: A Chinese Linguistic Corpora EEG Dataset for Semantic Alignment and Neural Decoding

An Electroencephalography (EEG) dataset utilizing rich text stimuli can advance the understanding of how the brain encodes semantic information and contribute to semantic decoding in brain-computer interface (BCI). Addressing the scarcity of EEG datasets featuring Chinese linguistic stimuli, we present the ChineseEEG dataset, a high-density EEG dataset complemented by simultaneous eye-tracking recordings. This dataset was compiled while 10 participants silently read approximately 13 hours of Chinese text from two well-known novels. This dataset provides long-duration EEG recordings, along with pre-processed EEG sensor-level data and semantic embeddings of reading materials extracted by a pre-trained natural language processing (NLP) model. As a pilot EEG dataset derived from natural Chinese linguistic stimuli, ChineseEEG can significantly support research across neuroscience, NLP, and linguistics. It establishes a benchmark dataset for Chinese semantic decoding, aids in the development of BCIs, and facilitates the exploration of alignment between large language models and human cognitive processes. It can also aid research into the brain’s mechanisms of language processing within the context of the Chinese natural language.

Female Image in Netochka Nezvanova: A Study Based on Feminist Perspective

This study delves into Fyodor Dostoevsky's unfinished novel, Netochka Nezvanova through a feminist lens, exploring the nuanced portrayals of female characters and their significance in the narrative. By examining the interactions, relationships, and development of key female figures such as Netochka, Sophia Alexandrovna, and Alexandra Mikhailovna, the paper highlights how these characters challenge and are limited in the patriarchal structures of 19th-century Russian society. The study argues that Dostoevsky, perhaps unconventionally for his time, presents a complex and empathetic exploration of the female experience, showcasing the struggles and resilience of his female protagonists. Through the character of Netochka, the narrative foregrounds the quest for identity and autonomy amidst societal constraints, emphasizing the importance of education, artistic expression, and female solidarity as means of resistance and self-realization. Netochka Nezvanova emerges as a rich text for feminist analysis, offering insights into the complexities of female agency, the critique of gendered societal expectations, and the enduring relevance of Dostoevsky's portrayal of women's pursuit of freedom and dignity.

A CNN-BiLSTM algorithm for Weibo emotion classification with attention mechanism

Weibo short text information contains a large amount of network language, emoticons, etc., and due to the long-time span of the content, the emotions of the posts posted by people often change due to time or the occurrence of certain special events. Therefore, traditional sentiment analysis methods are not suitable for this task. This article proposes a CNNs-Bi LSTM sentiment analysis method that integrates attention mechanism. It combines convolutional neural networks and bidirectional short-term memory networks to obtain keyword information in text through attention mechanism, efficiently and accurately realizing data temporal and semantic information mining. Through experimental verification using Weibo public opinion data, the results show that this method achieves higher accuracy compared to other benchmark models and can fully utilize multidimensional matrices to capture rich text features, with certain advantages.

Learning to compose diversified prompts for image emotion classification

Image emotion classification (IEC) aims to extract the abstract emotions evoked in images. Recently, language-supervised methods such as contrastive language-image pretraining (CLIP) have demonstrated superior performance in image understanding. However, the underexplored task of IEC presents three major challenges: a tremendous training objective gap between pretraining and IEC, shared suboptimal prompts, and invariant prompts for all instances. In this study, we propose a general framework that effectively exploits the language-supervised CLIP method for the IEC task. First, a prompt-tuning method that mimics the pretraining objective of CLIP is introduced, to exploit the rich image and text semantics associated with CLIP. Subsequently, instance-specific prompts are automatically composed, conditioning them on the categories and image content of instances, diversifying the prompts, and thus avoiding suboptimal problems. Evaluations on six widely used affective datasets show that the proposed method significantly outperforms state-of-the-art methods (up to 9.29% accuracy gain on the EmotionROI dataset) on IEC tasks with only a few trained parameters. The code is publicly available at https://github.com/dsn0w/PT-DPC/for research purposes.

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model’s performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

Irregular text block recognition via decoupling visual, linguistic, and positional information

Scene text recognition has made great progress in regular formats, and the recent research has focused on irregular text recognition. In this work, we investigate a new challenge problem of recognizing a text block instance with irregular arrangement of characters, which is referred to as irregular text block recognition (ITBR). This problem is prevalent in daily scenarios, especially with the increasing use of rich text designs in signboards, logos, posters, and other mediums. The primary challenge arises from the weakened position clues and the highly complex reading order, which can often only be deciphered by a heavy reliance on understanding the intrinsic linguistic information. Hence, conventional recognition methods that employ inflexible character grouping rules, coupled with positional information, or constrained by vocabulary reliance, may struggle with the ITBR problem. To this end, we propose a progressive layout reasoning network (PLRN) to recognize the irregular text block by decoupling visual, linguistic, and positional information. PLRN comprises a character spotting module that recognizes the character set based solely on visual features with a new TopK-rank decoding mechanism, and a linkage reasoning module to interpret the character relationships within this set with a progressive refinement strategy. The linkages are initially reasoned by linguistic information and then progressively refined through the incorporation of proximity and tendency information, allowing for explicit decoupling and improved reasoning accuracy. To assess the effectiveness of the proposed method, we construct a new dataset called TextBlock600. This dataset consists of 600 images of irregular text blocks, each with complete sequence annotations. Experimental results demonstrate that PLRN shows promising performance in ITBR, opening up possibilities for further research in this field. Code and datasets will be available at https://github.com/eezyli/PLRN.