Textual Descriptions Research Articles

An Intelligent Image Caption Generator Model using Deep Learning

Replicating the innate human capability to understand image content and create descriptive text is a formidable challenge for machines. The application of image captioning is highly versatile and holds great importance. It involves the creation of concise and descriptive captions for images using a variety of advanced techniques such as Natural Language Processing (NLP), Computer Vision (CV), and Deep Learning (DL). These techniques enable the automated generation of accurate and meaningful captions, which have numerous practical applications across industries such as healthcare, entertainment, marketing to mention but few. This study employed the machine learning algorithms of Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) to train computer vision using the Hibs dataset. The model leveraged input images with associated captions to identify and analyze predicted captions within the computer system. Model evaluation is a step in assessing the model's performance. This involves metrics such as BLEU score and METEOR score to measure the quality of the model's output in comparison to reference translation. These metrics help in quantifying the accuracy and fluency of the generated text, and the results are then presented to provide insights into the model's performance.

Deep scene understanding with extended text description for human object interaction detection

Human–object interaction (HOI) detection plays a pivotal role in scene understanding, enabling the identification, localization, and behavioral intention prediction of humans and objects within a visual scene. Conventional approaches, such as graph-based networks, have demonstrated effectiveness in capturing spatio-temporal interaction cues. However, relying solely on visual information within these networks limits their ability to comprehensively grasp the intricate aspects of human interactions. To address this shortcoming, we propose a novel deep scene understanding graph network that harnesses the power of extended text descriptions to represent and interpret interactions between humans and objects effectively. Text serves as a rich source of information, directly conveying the nature of interactions within a visual scene. Our model seamlessly integrates text descriptions with visual features extracted from the entire video sequence, enabling it to capture the context and nuances of human–object interactions. This approach significantly enhances the model’s ability to accurately predict ambiguous human actions and anticipate future interactions. Extensive experiments on two benchmark datasets, CAD-120, and Something-Else, demonstrate that our proposed method achieves remarkable improvements in the F1-Score for both HOI detection and anticipation tasks. This work paves the way for more accurate and comprehensive HOI understanding in various real-world scenarios, highlighting the potential of text-augmented graph networks for effective interaction modeling.

A Vision-Language Model-Based Traffic Sign Detection Method for High-Resolution Drone Images: A Case Study in Guyuan, China.

As a fundamental element of the transportation system, traffic signs are widely used to guide traffic behaviors. In recent years, drones have emerged as an important tool for monitoring the conditions of traffic signs. However, the existing image processing technique is heavily reliant on image annotations. It is time consuming to build a high-quality dataset with diverse training images and human annotations. In this paper, we introduce the utilization of Vision-language Models (VLMs) in the traffic sign detection task. Without the need for discrete image labels, the rapid deployment is fulfilled by the multi-modal learning and large-scale pretrained networks. First, we compile a keyword dictionary to explain traffic signs. The Chinese national standard is used to suggest the shape and color information. Our program conducts Bootstrapping Language-image Pretraining v2 (BLIPv2) to translate representative images into text descriptions. Second, a Contrastive Language-image Pretraining (CLIP) framework is applied to characterize not only drone images but also text descriptions. Our method utilizes the pretrained encoder network to create visual features and word embeddings. Third, the category of each traffic sign is predicted according to the similarity between drone images and keywords. Cosine distance and softmax function are performed to calculate the class probability distribution. To evaluate the performance, we apply the proposed method in a practical application. The drone images captured from Guyuan, China, are employed to record the conditions of traffic signs. Further experiments include two widely used public datasets. The calculation results indicate that our vision-language model-based method has an acceptable prediction accuracy and low training cost.

Penerapan Gamifikasi dalam Pembelajaran Konjungsi Teks Deskripsi Kelas 9I SMPN 13 Surabaya

This research examines the use of gamification in learning descriptive text conjunctions for class 9I of SMP Negeri 13 Surabaya. How the use of gamification in learning can improve the results and motivation of learning Indonesian for class 9I students. The aim of this research is (1) to increase the learning activities of class 9I students at SMP Negeri 13 Surabaya through the application of gamification in learning (2) to improve the learning outcomes of descriptive text conjunctions for class 9I students at SMP Negeri 13 Surabaya. This research is Classroom Action Research (PTK) which includes planning, implementation, observation and reflection. Data collection methods use questionnaires and observation. The instrument used is quantitative descriptive data analysis. The results of this research show an increase in students' motivation and learning participation, as well as a better understanding of conjunctions in Indonesian descriptive texts.

Efektivitas Model Pembelajaran Think Pair Share dalam Meningkatkan Pemahaman Siswa Terhadap Materi Ide Pokok dan Ide Pendukung Teks Deskripsi di Kelas IX-I SMP Negeri 13 Surabaya

Through classroom action research conducted in two cycles, this study aims to improve the understanding of grade IX-I students of SMP Negeri 13 Surabaya on the material of main ideas and supporting ideas of descriptive texts by implementing the TPS cooperative learning model. The results of the study showed a significant increase in student understanding after the implementation of the Think Pair Share learning model. In the pre-cycle, the average student test score was 65. After cycle I, the average score increased to 78, indicating an increase of 20%. In cycle II, the average score reached 85, or an increase of 9% from cycle I. This increase is indicated by the increasing number of students who can correctly identify the main idea and supporting ideas of descriptive texts. In addition, student activeness in discussing and giving opinions also increased significantly. This shows that the Think Pair Share cooperative learning model is effective in improving students' understanding of the main idea and supporting ideas of descriptive texts.

Exploring the potential of NLP technology to generate textual descriptions of space for indoor navigation applications

Exploring the potential of NLP technology to generate textual descriptions of space for indoor navigation applications

Multi-modal fusion deep learning model for excavated soil heterogeneous data with efficient classification

Multimodal fusion, a cutting-edge method aiming to integrate heterogeneous data for prediction tasks, has received limited attention in geotechnical engineering. This study focuses on classifying excavated soils with large quantities rapidly and finely. By using an excavated soil information collecting system at the largest soil transferring platform in China, 3,243 groups of multi-source heterogenous data were created, including soil images, spatial series data of cone index curves, time series data of TDR waveforms, discrete net weight data, and textual descriptions of soil morphology. After data augmentation, a big dataset containing 23,122 sets with labels based on the Unified Soil Classification System, moisture content, and mineral composition was created. Multimodal deep learning models were trained using different fusion strategies: 7 early fusion, 3 intermediate fusion, and 2 late fusion cases. Performance metrics were evaluated, including loss, accuracy, F1 Score, precision, recall, specificity, NPV, FPR, and AUROC. Results showed that intermediate fusions performed best, while late fusions performed the poorest. The two-stage intermediate fusion with five modalities achieved the best results, achieving an accuracy above 0.99 on the test set. This multimodal fusion approach effectively explores the correlation between multi-source heterogeneous data of soils, leading to scientific and engineering value in geotechnics.

Directional Texture Editing for 3D Models

AbstractTexture editing is a crucial task in 3D modelling that allows users to automatically manipulate the surface materials of 3D models. However, the inherent complexity of 3D models and the ambiguous text description lead to the challenge of this task. To tackle this challenge, we propose ITEM3D, a Texture Editing Model designed for automatic 3D object editing according to the text Instructions. Leveraging the diffusion models and the differentiable rendering, ITEM3D takes the rendered images as the bridge between text and 3D representation and further optimizes the disentangled texture and environment map. Previous methods adopted the absolute editing direction, namely score distillation sampling (SDS) as the optimization objective, which unfortunately results in noisy appearances and text inconsistencies. To solve the problem caused by the ambiguous text, we introduce a relative editing direction, an optimization objective defined by the noise difference between the source and target texts, to release the semantic ambiguity between the texts and images. Additionally, we gradually adjust the direction during optimization to further address the unexpected deviation in the texture domain. Qualitative and quantitative experiments show that our ITEM3D outperforms the state‐of‐the‐art methods on various 3D objects. We also perform text‐guided relighting to show explicit control over lighting. Our project page: https://shengqiliu1.github.io/ITEM3D/.

Graph-based Consistent Reconstruction and Alignment for imbalanced text-image person re-identification

Text-image person re-identification (TIReID) has emerged as a versatile approach for retrieving target pedestrians using textual descriptions. However, current TIReID research has been overly idealistic and has overlooked the issues of data incompleteness and modal imbalance in real-world application scenarios. Therefore, in this paper, we propose imbalanced text-image person re-identification (ITIReID) to address these problems. In comparison to TIReID, ITIReID contains a larger proportion of unimodal data, which leads to modal imbalance. The setting of ITIReID is more aligned with real-world scenarios, and studying ITIReID can expand the application scalability of TIReID. We propose a Graph-based Consistent Reconstruction and Alignment framework (GCRA), for ITIReID, which achieves modal balance by completing missing modality features for training implementation. By treating the accessible modality features as graph nodes, GCRA firstly builds an adjacency graph where a new semantic distance that establishes semantic relevance between nodes by comprehensively measuring both intra-modality and inter-modality correlation, serves as the measurement of graph’s edges. GCRA further reconstructs the missing nodes - thus re-establishing missing modality features - using existing nodes connected with high semantic relevance. To ensure the reliability and effectiveness of reconstructed features, we propose a proxy-based identity constraint and a reconstruction constraint. In addition, to enable effective semantic alignment using both the reconstructed features and original features, we introduce a cross-modal semantic constraint. Extensive experiments demonstrate that GCRA can effectively handle issues of data incompleteness and modal imbalance, exhibiting its effectiveness and superiority.

Harmonizing Fine-tuned Llama 2 for Content Generation with Stable Diffusion for Image Synthesis in Article Creation

The research explores the integration of generative AI in multimedia content production using a fine-tuned Llama 2 model for text generation and the Stable Diffusion algorithm for image synthesis. The research analyses the fine-tuned Llama 2-7b-chat model's adaptability to specific content generation contexts, enhanced by a unique dataset and QLoRa, a Quantized Low-Rank Adaptation for parameter-efficient fine-tuning, achieving significant reductions in training loss and nuanced quality in the generated content. Notably, the model's evaluation yielded an impressive perplexity score of 1.49, indicating advanced predictive performance. Additionally, stable diffusion's ability to transform textual descriptions into intricate images, highlighting its potential in AI-mediated content creation is demonstrated. The experiments and qualitative analyses reveal improvements in efficiency and creativity, emphasizing the collaborative potential of these models to revolutionize multidisciplinary content generation. The research underscores the transformative impact of fine-tuned generative models on content creation and offers insights into the broader implications for future AI research, while acknowledging the critical need for ethical considerations in the deployment of such technologies.

Design and Implementation of a Cognitive Cloud Assistant: A Text Classification Model Approach

Abstract The main objective of this research was to develop a classification model for the design of a cloud cognitive assistant. The classification model should be able to classify the textual description of the cloud architecture into desired multiple classes. For the purpose of implementation of such assistant, the analysis of current state of cognitive assistants and cloud computing was researched. As for the implementation of classifier, the overview of the possible ways how to create such classifier was also added to the research. Based on the analysis, a solution was proposed for the implementation of a model for text classification, and also a proposal for the implementation of an assistant that would use the proposed model. The Keras library was used to create a sequential text classifier. The IBM Watson cloud services were used to deploy the created model into live environment, and the services from the same group were used to develop a proposed cognitive assistant that was in the end connected to the classifier model.

Insights into decision‐maker's perceptions of good versus bad decisions in emergency services—A modified Delphi study

AbstractDecision‐making in emergency situations, such as those faced by fire, police, and health service personnel, presents unique challenges due to the high‐stakes and time‐pressured environment. Here we aim to better understand what emergency responders regard as constituting ‘good’ and ‘bad’ decisions in emergency situations. We administered a modified Delphi study, eliciting opinions from decision‐makers across all these sectors towards consensus around the key elements of good and bad decision‐making. Participants were first asked to define what makes a ‘good’ and a ‘bad’ decision, and subsequently to identify the top five most important elements of each. While consensus was not found, important insights were identified that can assist improve the standard of decision making at all levels of emergency response and management. We observed (i) a lack of a common understanding between participants of what a decision is, and how a decision differs from pre‐decision and postdecision components; (ii) responses varied according to whether a free text description or the identification of separate elements was requested; (iii) respondents valued ‘goodness’ across different and at times unrelated components a scaled measure of decision quality as opposed to a binary evaluation of ‘rightness’; and (iv) pre‐ and postdecision elements are considered more important than the decision itself when determining the quality of a decision. To address the issues highlighted by the study we recommend improvements in training, and improvements to organization doctrine related to decision making, risk tolerance, assessment of decision quality and development of intent‐based or principle‐based operational guidance. Ultimately, incident controllers from all services must remain cognizant that they will face considerable scrutiny if they cannot explain how they arrived at the decision they made.

A audiodescrição como mediação para estudar o papel da luz no teatro visual de Robert Wilson

This study proposes an audio description of four photographs taken from the work Shakespeare’s Sonnets, directed by Robert Wilson and performed with the Berliner Ensemble in 2009, considering the presence of stage lighting in these images. These descriptive texts were created for the enjoyment of people with visual impairments and aim to promote the understanding of the poetics of stage lighting in this play. Such audio-described images are part of the teaching material of a stage lighting class aimed at blind people or people with low vision. To do this, we articulate parameters for audio description in the classroom by Livia Motta, with the ideas of Adolphe Appia on the role of light in modern theater. The conclusions point to aspects of the visual composition of this staging, shown by the verbal language of the audio-descriptive text, to expand such mediation to audiences with and without visual impairment.

Implementation of Grasp (Guide Reading And Summary Procedure) Strategy in Descriptive Text on Students Reading Comprehension

Implementation of Grasp (Guide Reading And Summary Procedure) Strategy in Descriptive Text on Students Reading Comprehension

Applying Engagement Marketing And Human-Centered Design to Cocreate a Digital Decision Support Tool for Research Participation with LGBTQIA+ Community Members.

We used engagement marketing and human-centered design principles to cocreate a digital decision support tool for research participation with LGBTQIA+ community members to help them make an informed decision about joining the All of Us Research Program. Building on results from the research phase, we conducted eight problem validation and solutioning workshops with 48 LGBTQIA+ community members. Community members validated barriers to engagement with All of Us and brainstormed 47 potential digital solutions. We developed potential solutions into 27 concepts (descriptive text and visual storyboards) and assessed acceptability, appropriateness, feasibility, and engagement in a set of 10 concept testing workshops with 57 community members. We developed one of the highest rated concepts, the "Decide Later Tool," into a prototype and tested it with 45 LGBTQIA+ community members and 14 community advisory group members to assess acceptability, appropriateness, feasibility, usability, and engagement. Prototype testing participants indicated that the tool provides information to help with decision making, provides a clear value or benefit to them, was designed for someone like them, provides the right amount of information, and is easy to use; they also offered constructive feedback to improve it. Across the design and development phases, community members indicated that the process of engaging them demonstrated integrity, competence, dependability, trust, and collaboration; fostered a sense of connection to All of Us; and will enhance future engagement with All of Us. Our next steps are to develop the prototype into a fully functioning web tool and pilot test it in community and health care settings.

Integrating Generative AI and IoT for Sustainable Smart Tourism Destinations

This paper aims to develop a groundbreaking approach to fostering inclusive smart tourism destinations by integrating generative artificial intelligence (Gen AI) with natural language processing (NLP) and the Internet of Things (IoT) into an intelligent platform that supports tourism decision making and travel planning in smart tourism destinations. The acquisition of this new technology was conducted using Agile methodology through requirements analysis, system architecture analysis and design, implementation, and user evaluation. The results revealed that the synergistic combination of these technologies was organized into three tiers. The system provides information, including place names, images, descriptive text, and an audio option for users to listen to the information, supporting tourists with disabilities. Employing advanced AI algorithms alongside NLP, developed systems capable of generating predictive analytics, personalized recommendations, and conducting real-time, multilingual communication with tourists. This system was implemented and evaluated in Suphan Buri and Ayutthaya, UNESCO World Heritage sites in Thailand, with 416 users participating. The results showed that system satisfaction was influenced by (1) the tourism experience, (2) tourism planning and during-trip factors (attention, interest, and usage), and (3) emotion. The relative Chi-square (χ2/df) of 1.154 indicated that the model was suitable. The Comparative Fit Index (CFI) was 0.990, the Goodness-of-Fit Index (GFI) was 0.965, and the model based on the research hypothesis was consistent with the empirical data. This paper contributions significant advancements in the field of smart tourism by demonstrating the integration of Gen AI, NLP, and the IoT and offering practical solutions and theoretical insights that enhance accessibility, personalization, and environmental sustainability in tourism.

A feature-enhanced knowledge graph neural network for machine learning method recommendation.

Large amounts of machine learning methods with condensed names bring great challenges for researchers to select a suitable approach for a target dataset in the area of academic research. Although the graph neural networks based on the knowledge graph have been proven helpful in recommending a machine learning method for a given dataset, the issues of inadequate entity representation and over-smoothing of embeddings still need to be addressed. This article proposes a recommendation framework that integrates the feature-enhanced graph neural network and an anti-smoothing aggregation network. In the proposed framework, in addition to utilizing the textual description information of the target entities, each node is enhanced through its neighborhood information before participating in the higher-order propagation process. In addition, an anti-smoothing aggregation network is designed to reduce the influence of central nodes in each information aggregation by an exponential decay function. Extensive experiments on the public dataset demonstrate that the proposed approach exhibits substantial advantages over the strong baselines in recommendation tasks.

USING POWER STRATEGY ON STUDENTS’ DESCRIPTIVE TEXT WRITING ABILITY: IT’S EFFECTIVENESS

This study examines how well ninth-grade students at SMP Negeri 1 Rantau Utara use the POWER approach to improve their writing skills in descriptive texts. Because they don't want to learn how to write in English and because the approaches aren't engaging enough, the students' writing skills are poor. This issue needs to be taken into account because students' motivation to write in English will be impacted if they are not interested in learning the language. The study utilized a quasi-experimental design featuring a pre-test/post-test control group structure, as well as quantitative technique. The experimental group received teaching utilizing the POWER technique (Plan, Organize, Write, Edit, Revise), whereas the control group received standard writing instruction. Students from SMP Negeri 1 Rantau Utara's ninth grade made up the research's population. The study included sixty students, divided equally into thirty in the experimental group and thirty in the control group. Data analysis was conducted using the t-test in SPSS version 23, revealing a difference in writing proficiency between the students in the experimental and control groups. Evidence for this is the significance, or Sig. (2-tailed), which was reached 0.000<0,05, indicating the acceptance of Ha and the rejection of H0. According to the results of the aforementioned study, ninth-grade students at SMP Negeri 1 Rantau Utara can enhance their writing skills by utilizing the POWER Strategy.

THE ROLE OF LINGUISTIC BACKGROUND IN SHAPING STUDENTS’ READING PROFICIENCY IN VIRTUAL SETTINGS: A CORRELATIONAL STUDY

This research draws from the linguistic relativity hypothesis and sociocultural literacy perspectives to investigate how students’ linguistic backgrounds influence their digital reading practices in English as a Foreign Language (EFL) settings. Specifically, it examines the impact of Algerian Arabic and Berber languages on digital literacy habits among university students. Utilizing a sociocultural approach that views reading as a social activity, a mixed-methods approach combining qualitative and quantitative analyses was employed to ensure the credibility of the findings. The study surveyed students from two different Algerian universities situated in cities where the inhabitants primarily speak different native languages, namely Kabyle and Arabic, respectively. The data were collected through a questionnaire and analysed using both statistical and content analysis methods. Results reveal contrasting reading practices between Arabic and Berber speakers, with Arabic speakers emphasizing cognitive benefits of reading, while Berber speakers view it as a cultural tradition, particularly evident in their digital literacy habits. Digital literacy, for Berber speakers, represents both freedom and a continuation of their oral heritage, shaping their preference for narrative texts. In contrast, Arabic speakers lean towards descriptive texts. In conclusion, this study underscores the influence of linguistic backgrounds on Algerian EFL students’ digital reading behaviours.

Lexical Density of Reading Materials in An English Textbook: A Content Analysis

In reading, students are required to learn to comprehend the content of the text. Lexical density is one of the factors that influences students’ reading comprehension. Lexical density is a condition of words’ proportion in a text, which shows the ratio of content words (CW) and grammatical function words (GFW). This research used the “Passport to the World” English textbook for the 8th grade of junior high school to be analyzed. This study aims to find out the genres of reading materials offered by the curriculum 2013 found in the textbook and to examine the lexical density of the reading materials in the textbook. A qualitative approach was applied, while content analysis was used as the research design. Data were collected with document analysis. The findings showed that there are 3 genres of reading materials, among 5 genres offered by the curriculum 2013, included in the textbook such as descriptive, recount, and functional texts in the form of announcements. From those, it was found that two texts were categorized as low lexical density, 11 texts were categorized as medium lexical density, and four texts were labeled as high lexical density. In addition, the composition of this text did not reach the ideal ratio of 1:2:1 for index difficulty. However, this textbook can still be suggested to be used in the class with some usage modifications in the classroom.