Textual Descriptions Research Articles

A Multimodal Sentiment Analysis Model for Graphic Texts Based on Deep Feature Interaction Networks

Due to the widespread adoption of social networks, image-text comments have become a prevalent mode of emotional expression compared to traditional text descriptions. However, there are currently two major challenges. The first is the question of how to extract rich representations effectively from both text and images, and the second is the question of how to extract cross-modal shared emotion features. This study proposes a multimodal sentiment analysis method based on a deep feature interaction network (DFINet). It leverages word-to-word graphs and deep attention interaction networks (DAIN) to learn text representations effectively from multiple subspaces. Additionally, it introduces a cross-modal attention interaction network to extract cross-modal shared emotion features efficiently. This approach helps alleviate the difficulties associated with acquiring image-text features and representing cross-modal shared emotion features. Experimental results on the Yelp dataset demonstrate the effectiveness of the DFINet method.

DEVELOPMENT OF ENGLISH POP-UP BOOK MEDIA TO TEACH WRITING SKILL AT JUNIOR HIGH SCHOOL

This study aims to ensure that the use of Pop-Up Book Media affects the writing abilities of junior high school students. This media focuses with Descriptive Text material to teach writing skill to class VII students. This research use ADDIE model. The sample for this research was class VIIA MTs Sunan Ampel Kediri, totaling 14 students. Data collection was carried out by observation, unstructured interviews with teacher and questionnaires. Data were analyzed using a manual percentage formula. These findings show that the validation percentage of media experts reached 98%, that means "very valid", then the percentage of material validation reached 96%, with means "very valid". The total percentage of student satisfaction questionnaires reached 83.9% which is classified as "Very Satisfied". In conclusion, the use of Pop-Up Book media in learning can increase students' motivation in learning English and increase the creativity of students' in making paragraphs this is immediately effective for developing students' writing skills and the Pop-Up Book learning media is ready to be used by teachers as a learning media.

Animating Text Descriptions into Characters: A Comparative Review of Generative Models

Animating Text Descriptions into Characters: A Comparative Review of Generative Models

Zero-shot prompt-based video encoder for surgical gesture recognition.

In order to produce a surgical gesture recognition system that can support a wide variety of procedures, either a very large annotated dataset must be acquired, or fitted models must generalize to new labels (so-called zero-shot capability). In this paper we investigate the feasibility of latter option. Leveraging the bridge-prompt framework, we prompt-tune a pre-trained vision-text model (CLIP) for gesture recognition in surgical videos. This can utilize extensive outside video data such as text, but also make use of label meta-data and weakly supervised contrastive losses. Our experiments show that prompt-based video encoder outperforms standard encoders in surgical gesture recognition tasks. Notably, it displays strong performance in zero-shot scenarios, where gestures/tasks that were not provided during the encoder training phase are included in the prediction phase. Additionally, we measure the benefit of inclusion text descriptions in the feature extractor training schema. Bridge-prompt and similar pre-trained+prompt-tuned video encoder models present significant visual representation for surgical robotics, especially in gesture recognition tasks. Given the diverse range of surgical tasks (gestures), the ability of these models to zero-shot transfer without the need for any task (gesture) specific retraining makes them invaluable.

Language Model-Assisted Machine Learning, Photoelectrochemical, and First-Principles Investigation of Compatible Solvents for a CH3NH3PbI3 Film in Water.

Machine learning and data-driven methods have attracted a significant amount of attention for the acceleration of the design of molecules and materials. In this study, a material design protocol based on multimode modeling that combines literature modeling, numerical data collection, textual descriptor design, genetic modeling, experimental validation, first-principles calculation, and theoretical efficiency calculation is proposed, with a case study on designing compatible complex solvent molecules for a halide perovskite film, which is notorious for optoelectronic deactivation under hostile conditions, especially in water. In the multimode modeling design process, the textual descriptors play the central role and store rich literature scientific knowledge, which starts from the construction of a high-dimension literature model based on scientific articles and is realized by a genetic algorithm for materials predictions. The prediction is substantiated by follow-up experiments and first-principles calculations, leading to the successful identification of effective molecular combinations delivering an unprecedented large aqueous photocurrent (increasing by 3 orders of magnitude compared with that of CH3NH3PbI3) and remarkable aqueous stability (improving from 36% to 89% after immersion in water) under the hostile condition. This study provides a practical route via multimode modeling for accelerating the design of molecule-modified and solution-processed materials in a real scenario.

FICE: Text-conditioned fashion-image editing with guided GAN inversion

Fashion-image editing is a challenging computer-vision task where the goal is to incorporate selected apparel into a given input image. Most existing techniques, known as Virtual Try-On methods, deal with this task by first selecting an example image of the desired apparel and then transferring the clothing onto the target person. Conversely, in this paper, we consider editing fashion images with text descriptions. Such an approach has several advantages over example-based virtual try-on techniques: (i) it does not require an image of the target fashion item, and (ii) it allows the expression of a wide variety of visual concepts through the use of natural language. Existing image-editing methods that work with language inputs are heavily constrained by their requirement for training sets with rich attribute annotations or they are only able to handle simple text descriptions. We address these constraints by proposing a novel text-conditioned editing model called FICE (Fashion Image CLIP Editing) that is capable of handling a wide variety of diverse text descriptions to guide the editing procedure. Specifically, with FICE, we extend the common GAN-inversion process by including semantic, pose-related, and image-level constraints when generating images. We leverage the capabilities of the CLIP model to enforce the text-provided semantics, due to its impressive image–text association capabilities. We furthermore propose a latent-code regularization technique that provides the means to better control the fidelity of the synthesized images. We validate the FICE through rigorous experiments on a combination of VITON images and Fashion-Gen text descriptions and in comparison with several state-of-the-art, text-conditioned, image-editing approaches. Experimental results demonstrate that the FICE generates very realistic fashion images and leads to better editing than existing, competing approaches. The source code is publicly available from: https://github.com/MartinPernus/FICE.

The Enhancement of Statistical Literacy: A Cross-Institutional Study Using Data Analysis and Text Mining to Identify Statistical Issues in the Transition to University Education

Statistics modules are included in most university degrees, independent of the degree area, and this means that many students face these modules underprepared and struggle because of a lack of statistics knowledge. The Maths Support Centre (MSC) in the University College Dublin (UCD) provides support for various mathematics-related subjects, with statistics students being the second-largest cohort of visitors. The overall goal of this paper is to identify the common statistical issues students face during the transition from secondary education to tertiary education. The main data set for this study is the data from UCD students who have accessed the UCD MSC since 2015/16 for statistics support; the categorization of statistical concepts has been made with the statistics module description for each statistics subject at the Universidad Rey Juan Carlos (URJC). First, we conducted a categorization of statistical concepts taught in university (based on URJC’s catergorization); after that, UCD MSC tutor comments were categorized and validated, and subsequently descriptive analyses and text mining were used on the UCD MSC comments to achieve a deeper understanding of the statistical issues. The statistical issues presented were categorized as descriptive statistics (22.8%), probability (44%), statistical inference (29.2%), and statistical software (4%). Students struggled with material that was introduced at university level rather than material seen at secondary level. Our findings on students’ main statistical issues contribute to the development of a suite of evidence-based educational applications and games to support undergraduate students internationally in first- and second-year statistical modules.

MPNTEXT: An Interactive Platform for Automatically Extracting Metal-Polyphenol Networks and Their Applications from Scientific Literature.

In recent years, metal-polyphenol networks (MPNs) have gained significant attention due to their unique properties and broad applications across various fields. However, the burgeoning volume of MPN literature necessitates the automation of chemical information extraction from the extensive corpus of unstructured data, including scientific publications. To address this challenge, we proposed a platform named MPNTEXT, which utilized natural language processing techniques and machine learning algorithms to efficiently identify and extract pertinent information, thereby assisting users in comprehending complex MPNs and their textual descriptions of applications. Users can enter keywords, such as "Fe", "drug delivery", or "tannic acid", to retrieve relevant information, which is then presented in a structured format. This study aims to provide a user-friendly tool for collecting and retrieving MPN data and promotes data-driven material design. The platform offers researchers a more convenient and efficient way to design versatile MPNs and explore their applications.

Exploring Text Classification Systems for Automatically Coding Historical Occupations and Causes of Death

ObjectivesText classification models can be used to automatically categorize occupations and causes of death within historical documents. It is important to classify/code these categories as different words or textual descriptions could refer to the same occupation or cause of death. Given the many historical documents that are becoming available for research, accurate classification systems can be valuable resources. ApproachWe explore different text classification techniques, from traditional machine learning to deep learning, and investigate methodologies that transform occupations and causes of death into a vectorial space and use these representations as features to train text classification systems. Our data come from IPUMS USA/International, and SCADR. ResultsHistorians have coded occupations and causes of death for some census collections (e.g., US, Canada), but not yet for others (e.g., Scotland). We train and evaluate our classification systems using data from the US and Canada and then deploy it on data from Scotland. We quantitatively measure the performance of the classification systems for historical documents that have codes available. Additionally, once we deploy the model to data that does not yet have codes, we qualitatively evaluate our results by engaging with historians working on those data. We report and discuss these results to understand where the models are performing well and where the models are underperforming. ConclusionsResults suggest that there is value in building and deploying these classification models. We recommend the use of such models in conjunction with engaging with domain experts.

Multi-modal transformer with language modality distillation for early pedestrian action anticipation

Language-vision integration has become an increasingly popular research direction within the computer vision field. In recent years, there has been a growing recognition of the importance of incorporating linguistic information into visual tasks, particularly in domains such as action anticipation. This integration allows anticipation models to leverage textual descriptions to gain deeper contextual understanding, leading to more accurate predictions. In this work, we focus on pedestrian action anticipation, where the objective is the early prediction of pedestrians’ future actions in urban environments. Our method relies on a multi-modal transformer model that encodes past observations and produces predictions at different anticipation times, employing a learned mask technique to filter out redundancy in the observed frames. Instead of relying solely on visual cues extracted from images or videos, we explore the impact of integrating textual information in enriching the input modalities of our pedestrian action anticipation model. We investigate various techniques for generating descriptive captions corresponding to input images, aiming to enhance the anticipation performance. Evaluation results on available public benchmarks demonstrate the effectiveness of our method in improving the prediction performance at different anticipation times compared to previous works. Additionally, incorporating the language modality in our anticipation model proved significant improvement, reaching a 29.5% increase in the F1 score at 1-second anticipation and a 16.66% increase at 4-second anticipation. These results underscore the potential of language-vision integration in advancing pedestrian action anticipation in complex urban environments.

Progress and developments towards making historic administrative data research ready

ObjectivesTo make research ready data from transcribed historic civil registration records: birth, marriage and death (from mid-19th century onwards). To use historic records effectively for large-scale research they must not only be made machine-readable, but also coded in a suitable format – along with classification of transcribed information. Included on the digitised historic records are textual descriptions of occupations and causes of death. Thus, to code transcribed occupations to HISCO and causes of death to ICD. ApproachIt is impractical to hand-code the records manually (34 million occupations and 8 million causes of death), especially for deaths where more than one cause can be given. As such, coding is viewed as a text classification task and the process is automated. To facilitate auto-coding, a proportion of records were hand-coded as training data (90,000 occupations and 102,000 deaths). Ahead of the auto-coding, initial pre-processing, cleaning and standardising is done on both the occupations and deaths. ResultsPreliminary experiments undertaken obtained reasonable results from a combination of exact matching and statistical classification. Experiments using a larger pilot uncovered that since some occupations are very common, the training data set covers a very large proportion of the records (ie exact match). This proportion is not as high for deaths given the different ways causes are written. ConclusionThis is work in progress to create the research ready data, and the poster will include the results from experiments using the full training data (90,000 occupations and 102,000 deaths).

Making Country Proud: The Role of Athletics in the Development of Eldoret City, Kenya

This study examined the impact of athletics on the development of Eldoret City from 1920-2020. The study was informed by the need to discuss in detail and document the development of athletics among the Kalenjin community in Kenya. To realize the above, the study utilized Social Constructivism and Self-efficacy theories. The study was guided by one objective: the development and impact of athletics among the Kalenjin community in Kenya from 1920 through 2020. The literature reviewed, global, regional, and local, helped identify the knowledge gap, hence the study’s justification. The philosophical paradigm that anchored this study is constructivist philosophy. The study is qualitative and has an applied historical research design. The sampling procedure entailed using purposive and snowball sampling to arrive at knowledgeable informants. The sample size of 20 informants was determined by saturation. Data collected was from primary and secondary sources. Primary sources entailed archival materials such as artefacts and photographs, while unstructured interview schedules were used to collect oral data. The secondary sources were the books, theses, journals, magazines, and newspapers. Using the qualitative method, the study analyzed data thematically and presented the findings through narration, direct quotes and descriptive texts. Photographs were also used to present the findings. According to the research findings, the Kalenjin community had traditional sports activities, primarily athletics. The thrust of the athletics was reminiscent of their cultural aspirations. Further, the study established that the development of athletics grew gradually in phases marked by the arrival of the European colonialists, the internationalization of sports in the 1950s and the emergence of Kenya as a world athletics player. The study additionally found that scholarship, employment opportunities, and acquisition of wealth through diverse investments are evident in Eldoret City. The town has recently grown into the 5th Kenyan City following a significant economic contribution from the athletics arena. The rise of the Kalenjin diaspora and the attraction of multinationals to local athletics constitute substantial socioeconomic and political impacts on athletics among the Kalenjin. The study thus recommends that the government improve funding for sports and fully implement the existing laws and policies

Impact of consumer familiarity on acceptability and purchase intent of a novel amaranth‐based coffee creamer

SummaryThe objective of the study was to identify the impact of consumer familiarity on liking, acceptability, purchase intention and textual description of a novel cream substitute formulated with popped amaranth flour. Four formulations were evaluated in different proportions of amaranth vs. a control. A total of 400 consumers from two regions participated (Veracruz: familiar with coffee and Tlaxcala: familiar with amaranth). Consumers from both regions had significant differences in liking the attributes of the different formulations. Liking across regions was mainly a function of product aroma. Regarding the acceptance of the product, it was found that the attributes flavour, colour and overall liking were significant (P < 0.05) for consumers from Veracruz, whereas appearance and mouthfeel were significant (P < 0.05) for consumers from Tlaxcala. In purchase intent, overall liking was significant for both regions (P < 0.05). In relation to the impact of amaranth on purchase intention, Tlaxcala showed higher purchase willingness. The words used for sensory characterisation and that are important for both regions were the amaranth aroma, and the creamy flavour in the formulation with the highest amaranth content. The findings of this study allowed for understanding the potential of amaranth to formulate new creamers in a cross‐cultural context, considering consumers from two different levels of familiarity with specific crops.

An Intelligent Image Caption Generator Model using Deep Learning

Replicating the innate human capability to understand image content and create descriptive text is a formidable challenge for machines. The application of image captioning is highly versatile and holds great importance. It involves the creation of concise and descriptive captions for images using a variety of advanced techniques such as Natural Language Processing (NLP), Computer Vision (CV), and Deep Learning (DL). These techniques enable the automated generation of accurate and meaningful captions, which have numerous practical applications across industries such as healthcare, entertainment, marketing to mention but few. This study employed the machine learning algorithms of Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) to train computer vision using the Hibs dataset. The model leveraged input images with associated captions to identify and analyze predicted captions within the computer system. Model evaluation is a step in assessing the model's performance. This involves metrics such as BLEU score and METEOR score to measure the quality of the model's output in comparison to reference translation. These metrics help in quantifying the accuracy and fluency of the generated text, and the results are then presented to provide insights into the model's performance.

Deep scene understanding with extended text description for human object interaction detection

Human–object interaction (HOI) detection plays a pivotal role in scene understanding, enabling the identification, localization, and behavioral intention prediction of humans and objects within a visual scene. Conventional approaches, such as graph-based networks, have demonstrated effectiveness in capturing spatio-temporal interaction cues. However, relying solely on visual information within these networks limits their ability to comprehensively grasp the intricate aspects of human interactions. To address this shortcoming, we propose a novel deep scene understanding graph network that harnesses the power of extended text descriptions to represent and interpret interactions between humans and objects effectively. Text serves as a rich source of information, directly conveying the nature of interactions within a visual scene. Our model seamlessly integrates text descriptions with visual features extracted from the entire video sequence, enabling it to capture the context and nuances of human–object interactions. This approach significantly enhances the model’s ability to accurately predict ambiguous human actions and anticipate future interactions. Extensive experiments on two benchmark datasets, CAD-120, and Something-Else, demonstrate that our proposed method achieves remarkable improvements in the F1-Score for both HOI detection and anticipation tasks. This work paves the way for more accurate and comprehensive HOI understanding in various real-world scenarios, highlighting the potential of text-augmented graph networks for effective interaction modeling.

A Vision-Language Model-Based Traffic Sign Detection Method for High-Resolution Drone Images: A Case Study in Guyuan, China.

As a fundamental element of the transportation system, traffic signs are widely used to guide traffic behaviors. In recent years, drones have emerged as an important tool for monitoring the conditions of traffic signs. However, the existing image processing technique is heavily reliant on image annotations. It is time consuming to build a high-quality dataset with diverse training images and human annotations. In this paper, we introduce the utilization of Vision-language Models (VLMs) in the traffic sign detection task. Without the need for discrete image labels, the rapid deployment is fulfilled by the multi-modal learning and large-scale pretrained networks. First, we compile a keyword dictionary to explain traffic signs. The Chinese national standard is used to suggest the shape and color information. Our program conducts Bootstrapping Language-image Pretraining v2 (BLIPv2) to translate representative images into text descriptions. Second, a Contrastive Language-image Pretraining (CLIP) framework is applied to characterize not only drone images but also text descriptions. Our method utilizes the pretrained encoder network to create visual features and word embeddings. Third, the category of each traffic sign is predicted according to the similarity between drone images and keywords. Cosine distance and softmax function are performed to calculate the class probability distribution. To evaluate the performance, we apply the proposed method in a practical application. The drone images captured from Guyuan, China, are employed to record the conditions of traffic signs. Further experiments include two widely used public datasets. The calculation results indicate that our vision-language model-based method has an acceptable prediction accuracy and low training cost.

Penerapan Gamifikasi dalam Pembelajaran Konjungsi Teks Deskripsi Kelas 9I SMPN 13 Surabaya

This research examines the use of gamification in learning descriptive text conjunctions for class 9I of SMP Negeri 13 Surabaya. How the use of gamification in learning can improve the results and motivation of learning Indonesian for class 9I students. The aim of this research is (1) to increase the learning activities of class 9I students at SMP Negeri 13 Surabaya through the application of gamification in learning (2) to improve the learning outcomes of descriptive text conjunctions for class 9I students at SMP Negeri 13 Surabaya. This research is Classroom Action Research (PTK) which includes planning, implementation, observation and reflection. Data collection methods use questionnaires and observation. The instrument used is quantitative descriptive data analysis. The results of this research show an increase in students' motivation and learning participation, as well as a better understanding of conjunctions in Indonesian descriptive texts.

Efektivitas Model Pembelajaran Think Pair Share dalam Meningkatkan Pemahaman Siswa Terhadap Materi Ide Pokok dan Ide Pendukung Teks Deskripsi di Kelas IX-I SMP Negeri 13 Surabaya

Through classroom action research conducted in two cycles, this study aims to improve the understanding of grade IX-I students of SMP Negeri 13 Surabaya on the material of main ideas and supporting ideas of descriptive texts by implementing the TPS cooperative learning model. The results of the study showed a significant increase in student understanding after the implementation of the Think Pair Share learning model. In the pre-cycle, the average student test score was 65. After cycle I, the average score increased to 78, indicating an increase of 20%. In cycle II, the average score reached 85, or an increase of 9% from cycle I. This increase is indicated by the increasing number of students who can correctly identify the main idea and supporting ideas of descriptive texts. In addition, student activeness in discussing and giving opinions also increased significantly. This shows that the Think Pair Share cooperative learning model is effective in improving students' understanding of the main idea and supporting ideas of descriptive texts.

Exploring the potential of NLP technology to generate textual descriptions of space for indoor navigation applications

Exploring the potential of NLP technology to generate textual descriptions of space for indoor navigation applications

Multi-modal fusion deep learning model for excavated soil heterogeneous data with efficient classification

Multimodal fusion, a cutting-edge method aiming to integrate heterogeneous data for prediction tasks, has received limited attention in geotechnical engineering. This study focuses on classifying excavated soils with large quantities rapidly and finely. By using an excavated soil information collecting system at the largest soil transferring platform in China, 3,243 groups of multi-source heterogenous data were created, including soil images, spatial series data of cone index curves, time series data of TDR waveforms, discrete net weight data, and textual descriptions of soil morphology. After data augmentation, a big dataset containing 23,122 sets with labels based on the Unified Soil Classification System, moisture content, and mineral composition was created. Multimodal deep learning models were trained using different fusion strategies: 7 early fusion, 3 intermediate fusion, and 2 late fusion cases. Performance metrics were evaluated, including loss, accuracy, F1 Score, precision, recall, specificity, NPV, FPR, and AUROC. Results showed that intermediate fusions performed best, while late fusions performed the poorest. The two-stage intermediate fusion with five modalities achieved the best results, achieving an accuracy above 0.99 on the test set. This multimodal fusion approach effectively explores the correlation between multi-source heterogeneous data of soils, leading to scientific and engineering value in geotechnics.