Description Language Research Articles

Towards Retrieval-Augmented Architectures for Image Captioning

The objective of image captioning models is to bridge the gap between the visual and linguistic modalities by generating natural language descriptions that accurately reflect the content of input images. In recent years, researchers have leveraged deep learning-based models and made advances in the extraction of visual features and the design of multimodal connections to tackle this task. This work presents a novel approach toward developing image captioning models that utilize an external k NN memory to improve the generation process. Specifically, we propose two model variants that incorporate a knowledge retriever component that is based on visual similarities, a differentiable encoder to represent input images, and a k NN-augmented language model to predict tokens based on contextual cues and text retrieved from the external memory. We experimentally validate our approach on COCO and nocaps datasets and demonstrate that incorporating an explicit external memory can significantly enhance the quality of captions, especially with a larger retrieval corpus. This work provides valuable insights into retrieval-augmented captioning models and opens up new avenues for improving image captioning at a larger scale.

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation

Commit messages are natural language descriptions of code changes, which are important for software evolution such as code understanding and maintenance. However, previous methods are trained on the entire dataset without considering the fact that a portion of commit messages adhere to good practice (i.e., good-practice commits), while the rest do not. On the basis of our empirical study, we discover that training on good-practice commits significantly contributes to the commit message generation. Motivated by this finding, we propose a novel knowledge-aware denoising learning method called KADEL. Considering that good-practice commits constitute only a small proportion of the dataset, we align the remaining training samples with these good-practice commits. To achieve this, we propose a model that learns the commit knowledge by training on good-practice commits. This knowledge model enables supplementing more information for training samples that do not conform to good practice. However, since the supplementary information may contain noise or prediction errors, we propose a dynamic denoising training method. This method composes a distribution-aware confidence function and a dynamic distribution list, which enhances the effectiveness of the training process. Experimental results on the whole MCMD dataset demonstrate that our method overall achieves state-of-the-art performance compared with previous methods.

The role of linguistic perspectives in the language description

Abstract The paper discusses the language of linguistic description (metalanguage), in particular the question of how the language of description can influence the language that is the object of the description. I approach the topic from the aspect of the use of linguistic terms and their personification. I summarize the results of my three previous case studies, in which I tracked the formulations associated with the selected terms. I focused on the actions that, according to the literal understanding of the formulations, are performed by the terms. Based on these case studies, I concluded that the more frequently a term is used, the more likely it is that it will “perform” an activity and, if this happens, the further away they go, the more their “competencies” expand, i.e. they perform progressively more complex acts. I have characterized this phenomenon as the perspective through which we view linguistic reality through terms (the perspective of the term), and in contrast to it, I have presented the speaker’s perspective, through which linguistic reality is perceived from the perspective of everyday speakers. I have pointed out that the perspective of the term can affect the description of language in a way that shifts the linguist’s attention from the real behaviour of speakers to the “behaviour” of abstract linguistic entities.

Generative AI Generating Buzz: Volume, Engagement, and Content of Initial Reactions to ChatGPT in Discussions Across Education-Related Subreddits

The emergence of generative artificial intelligence (GenAI) has ignited intense debates regarding its potential benefits and detriments for education. Despite such widespread discussions, insights into GenAI's impact have often been limited to anecdotal evidence and a narrow scope of educational stakeholders. To understand the broader response to GenAI tools in education, we collected and analyzed public online discussions of ChatGPT, a leading GenAI tool, in the first four months following the tool’s release. We collected 345 posts and 6,463 comments about ChatGPT from 25 education-focused subreddits. We analyzed the volume, engagement, and content of ChatGPT discussions through descriptive statistics and natural language processing techniques. Findings show relatively low volume of ChatGPT discussions, unevenly spread across education-related subreddits—with the majority of the discussions occurring in two subreddits while six subreddits did not have any discussions. Despite this, the level of engagement within ChatGPT posts was substantial; for instance, a ChatGPT post hosted a median of 15 comments, and these comments were lengthy, indicating rich engagement rather than superficial. The content of ChatGPT discussions across the six largest education-related subreddits differed in the degree of analytical thinking and emotional tone even while sharing a predominant focus on students and AI. Diverse reactions to and perspectives on GenAI—observed from varied levels of volume, engagement, and content of ChatGPT across educational-related subreddits—highlights how diverse educational stakeholders reacted to GenAI differently, offering insights into how to explore, analyze, comprehend the spread and adoption of technological innovation in education.

Detecting Vulnerability in Hardware Description Languages: Opcode Language Processing

Detecting Vulnerability in Hardware Description Languages: Opcode Language Processing

Fast synchronization with enhanced switching control for grid-tied single-phase square wave inverter using FPGA

Research on grid synchronization has been conducted worldwide by researchers in conjunction with the development of innovative technologies, such as dedicated short-range communication (DSRC) and cellular vehicle-to-everything (C-V2X). However, grid-connected inverters face several challenges, mainly the mismatch in voltage amplitude, frequency, and phase angle, as well as grid voltage disturbance and grid faults. Thus, the control algorithm of this research mainly focused on a half-cycle algorithm to design an enhanced digital switching control for fast synchronization using an FPGA. The control algorithm was developed based on zero-crossing detection (ZCD) and digital phase-locked loop (PLL) modeling techniques using the hardware description language (HDL) and a combination of digital logic blocks in Quartus II software, where the proposed switching was applied using the square-wave switching technique through a 300-watt full-bridge experimental prototype. The performance of the proposed technique was studied, where the total harmonic distortion (THD) for voltage and current resulted in a percentage reduction of 89.29% and 78.05% for voltage and current, respectively, after filter implementation. Also, the resulting signal synchronized in every half cycle and matched the voltage amplitude, frequency, and phase angle of the grid signal in 10 ms.

Deep Learning–Based Clustering of OCT Images for Biomarker Discovery in Age-Related Macular Degeneration (PINNACLE Study Report 4)

PurposeWe introduce a deep-learning-based biomarker proposal system for the purpose of accelerating biomarker discovery in age-related macular degeneration (AMD). DesignRetrospective analysis of a large dataset of retinal optical coherence tomography (OCT) images. ParticipantsA total of 3,456 adults aged between 51 and 102 years of whom OCT images were collected under the PINNACLE project. MethodsOur system proposes candidates for novel AMD imaging biomarkers in OCT. It works by first training a neural network using self-supervised contrastive learning to discover, without any clinical annotations, features relating to both known and unknown AMD biomarkers present in 46,496 retinal OCT images. To interpret the learned biomarkers, we partition the images into 30 subsets, termed clusters, that contain similar features. We conduct two parallel 1.5-hour semi-structured interviews with two independent teams of retinal specialists to assign descriptions in clinical language to each cluster. Descriptions of clusters achieving consensus can potentially inform new biomarker candidates. Main Outcome MeasuresWe checked if each cluster showed clear features comprehensible to retinal specialists, if they related to AMD and how many described established biomarkers used in grading systems as opposed to recently proposed or potentially new biomarkers. We also compared their prognostic value for late stage wet and dry AMD against an established clinical grading system and a demographic baseline model. ResultsOverall, both teams independently identified clearly distinct characteristics in 27 of 30 clusters, of which 23 were related to AMD. Seven were recognised as known biomarkers used in established grading systems and 16 depicted biomarker combinations or subtypes that are either not yet used in grading systems, were only recently proposed, or were unknown. Clusters separated incomplete from complete retinal atrophy, intraretinal from subretinal fluid and thick from thin choroids, and in simulation outperformed clinically-used grading systems in prognostic value. ConclusionsUsing self-supervised deep learning we were able to automatically propose AMD biomarkers going beyond the set used in clinically established grading systems. Without any clinical annotations, contrastive learning discovered subtle differences between fine-grained biomarkers. Ultimately, we envision that equipping clinicians with discovery-oriented deep-learning tools can accelerate discovery of novel prognostic biomarkers.

The Artistic Expression of Character Shaping and Dramatic Costume Design

Character shaping and theatrical costume design complement each other in theatrical performance. Character shaping requires consideration of the character's background, personality traits, and goals, and showcases the character's characteristics through techniques such as dialogue, body language, and psychological description. Dramatic costume design should reflect the personality of the characters, conform to the theatrical style and historical background, and reflect the development of the characters in the plot. By selecting appropriate materials, colors, and designing clothing styles, as well as cleverly matching accessories, theatrical costumes can enhance the effect of character shaping. The interaction between characters and clothing is reflected in clothing strengthening character characteristics, assisting in expressing character emotions, and creating a suitable plot atmosphere. By integrating character shaping with theatrical costume design, the connotation of characters can be better interpreted, and the overall expressive power of drama can be enhanced.

TraumaFlow-development of a workflow-based clinical decision support system for the management of severe trauma cases.

The treatment of severely injured patients in the resuscitation room of an emergency department requires numerous critical decisions, often under immense time pressure, which places very high demands on the facility and the interdisciplinary team. Computer-based cognitive aids are a valuable tool, especially in education and training of medical professionals. For the management of polytrauma cases, TraumaFlow, a workflow management-based clinical decision support system, was developed. The system supports the registration and coordination of activities in the resuscitation room and actively recommends diagnosis and treatment actions. Based on medical guidelines, a resuscitation room algorithm was developed according to the cABCDE scheme. The algorithm was then modeled using the process description language BPMN 2.0 and implemented in a workflow management system. In addition, a web-based user interface that provides assistance functions was developed. An evaluation study was conducted with 11 final-year medical students and three residents to assess the applicability of TraumaFlow in a case-based training scenario. TraumaFlow significantly improved guideline-based decision-making, provided more complete therapy, and reduced treatment errors. The system was shown to be beneficial not only for the education of low- and medium-experienced users but also for the training of highly experienced physicians. 92% of the participants felt more confident with computer-aided decision support and considered TraumaFlow useful for the training of polytrauma treatment. In addition, 62% acknowledged a higher training effect. TraumaFlow enables real-time decision support for the treatment of polytrauma patients. It improves guideline-based decision-making in complex and critical situations and reduces treatment errors. Supporting functions, such as the automatic treatment documentation and the calculation of medical scores, enable the trauma team to focus on the primary task. TraumaFlow was developed to support the training of medical students and experienced professionals. Each training session is documented and can be objectively and qualitatively evaluated.

Using drawings and deep neural networks to characterize the building blocks of human visual similarity.

Early in life and without special training, human beings discern resemblance between abstract visual stimuli, such as drawings, and the real-world objects they represent. We used this capacity for visual abstraction as a tool for evaluating deep neural networks (DNNs) as models of human visual perception. Contrasting five contemporary DNNs, we evaluated how well each explains human similarity judgments among line drawings of recognizable and novel objects. For object sketches, human judgments were dominated by semantic category information; DNN representations contributed little additional information. In contrast, such features explained significant unique variance perceived similarity of abstract drawings. In both cases, a vision transformer trained to blend representations of images and their natural language descriptions showed the greatest ability to explain human perceptual similarity-an observation consistent with contemporary views of semantic representation and processing in the human mind and brain. Together, the results suggest that the building blocks of visual similarity may arise within systems that learn to use visual information, not for specific classification, but in service of generating semantic representations of objects.

Object Detection for the Visually Impaired

This paper presents the design and development of a mobile application, built using Flutter, that leverages object detection to enhance the lives of visually impaired individuals. The application addresses a crucial challenge faced by this community – the lack of real-time information about their surroundings. We propose a solution that utilizes pre-trained machine learning models, potentially through TensorFlow Lite for on-device processing, to identify objects within the user's field of view captured by the smartphone camera. The application goes beyond simple object recognition. Detected objects are translated into natural language descriptions through text-to-speech functionality, providing crucial auditory cues about the environment. This real-time information stream empowers users to navigate their surroundings with greater confidence and independence. Accessibility is a core principle of this project. The user interface will be designed with compatibility for screen readers, ensuring seamless interaction for users who rely on assistive technologies. Haptic feedback mechanisms will be incorporated to provide non-visual cues and enhance the user experience. The ultimate goal of this project is to create a user-friendly and informative application that empowers visually impaired people to gain greater independence in their daily lives. The application has the potential to improve spatial awareness, foster a sense of security, and promote overall inclusion within society

Thinking of food: The mental representation of healthy foods as unprepared

We find that people implicitly and explicitly represent healthy foods they categorize as healthy in their purest, least prepared forms but represent foods they categorize as unhealthy in their most prepared forms (e.g., a veggie patty is represented as frozen while a beef burger is represented in a bun with melted cheese and ready to eat). We find this effect across several studies using both image and word sorting measures in explicit tasks and implicit association tasks. The effect results from the perception of health and taste as two conflicting goals. Preparation (e.g., cooking, adding toppings) makes food more delicious, which creates categorization ambiguity. Hence, healthy food is thought of as unprepared. Indeed, individual differences in perceived health-taste goal conflict moderate the effect. Critically, the representation of healthy foods matters for food decisions. In an experiment that manipulated the descriptive language on a restaurant menu, emphasizing the preparation of foods increased participants’ preference for healthy foods (with no improvement for unhealthy foods).

An aesthetic approach to emotional intelligence: The development of the Aesthetic Descriptors of Emotion Measure (ADEM)

The four-factor model of emotional intelligence (EI) abilities has faced challenges in the inconsistency of the facilitation branch leading Mayer et al. (2024) to propose a provisional factor- Connecting Emotions- that consists of relating emotions to sensations. The Aesthetic Descriptors of Emotion Measure (ADEM) has emerging potential as a measure of this new EI ability factor. The ADEM is a performance measure based on the Aesthetic Approach to Emotions (AAE) that matches descriptive aesthetic-sensory language to emotional states providing a non-evaluative representation of emotions. The ADEM presents six emotions and found distinct aesthetic profiles for each. Cronbach Alpha and factor analyses respectively found consistency and validity with the internal structure of the ADEM. Bivariate and partial bivariate analyses (controlling for cognitive reasoning) indicated that the ADEM corresponds with self-perceived measures of emotional clarity and a similar performance-based EI measure (MEIS-Synesthesia subtest). The ADEM has potential as a method for developing EI in educational and clinical settings.

Optimization research of handwritten digit recognition algorithm based on hardware neural networks

In the modern era of digitization, the widespread application of handwritten numerical data has led to an exponential increase in data volume, necessitating efficient real-time recognition and processing systems. This project is grounded in the foundational structure of neural networks, with a specific focus on integrating these networks into hardware through Very-Large-Scale Integration (VLSI). The primary objective is to achieve faster processing speeds, lower power consumption, and real-time operations. The central emphasis of this study is the design and implementation of a hardware neural network system tailored specifically for handwritten digit recognition. Drawing from neural network theory, we explore the construction of a hardware-based handwritten digit classifier. The fundamental model employed is the single-layer perceptron neuron model. Upon receiving 28x28 pixel grayscale images, the image classifier utilizes pre-trained weights, activation functions, and a maximum selector to compare and output recognition results for numerical digits. The concrete implementation is realized using the Verilog hardware description language, coupled with algorithm optimization strategies to enhance performance and efficiency. This research endeavor aims to provide an effective hardware solution for real-time handwritten digit recognition in the digital age.

Accelerating handwritten number classifier with pipelined Multiply-Accumulate units in neural networks

This research project explores the development and assessment of a Handwritten Number Classifier based on Neural Networks, employing the widely used MNIST dataset. The primary objective of this study is to enhance computational efficiency, and to achieve this, it concentrates on integrating and contrasting non-pipelined and pipelined structures within the Multiply-Accumulate (MAC) unit. The initial phase involved conducting MATLAB simulations, which yielded promising results regarding the accuracy of weight calculations. Subsequently, Hardware Description Language (HDL) testing was carried out to further validate the classifiers performance. In the HDL testing phase, the classifier incorporating the pipelined MAC unit demonstrated a substantial 42.9% enhancement in processing speed when compared to its non-pipelined counterpart. These results highlight the potential advantages of employing pipeline processing in neural network architectures, emphasizing its effectiveness in achieving faster and more efficient image classification, particularly when dealing with extensive datasets. In conclusion, this research project not only presents valuable insights into improving the efficiency of neural network-based image classifiers but also lays the groundwork for potential future endeavors. These future directions may include adapting the classifier to handle more complex datasets and addressing emerging challenges.

Text-to-3D Non-Deformable Animation: A Proof-of-Concept System using Natural Language Descriptions

The ability to generate 3D animations from natural language descriptions has significant potential in various fields, including entertainment, education, and computer-aided design. This research paper presents a novel proof-of-concept system that transforms text prompts into complete and simple 3D animations. The system architecture comprises two main components: extraction and scene creation. In the extraction phase, a transformer model identifies the necessary objects from the text prompt, generates their geometric representations as vertices and edges, and creates a script describing the animations. The scene creation phase involves constructing the 3D objects, adding them to the scene, and rendering the animation based on the generated script. The implementation details, including the technologies and tools used, code snippets, and challenges faced, are thoroughly discussed. Qualitative and quantitative evaluations are conducted to assess the system's performance, highlighting its strengths and limitations compared to existing text-to-animation approaches. Potential applications of the proposed system are explored, ranging from rapid prototyping of 3D animations to interactive storytelling and educational tools. Furthermore, future research directions, such as improving scalability, generalization, and real-world deployment, are outlined. The research presented in this paper contributes to the emerging field of text-to-animation generation and paves the way for further advancements in transforming natural language descriptions into engaging and immersive 3D experiences.

Making Information About Cladribine Tablets Accessible to People with Multiple Sclerosis: A Patient-Survey-Led Narrative Review for Healthcare Professionals.

Cladribine tablets have been granted marketing authorization in Europe and approved by the Food and Drug Administration (FDA) in the USA to treat relapsing forms of multiple sclerosis (MS). However, people with MS (PwMS) may be more familiar, and therefore more confident, with treatments requiring long-term and frequent dosing. Differences in such treatment strategies can lead to questions relating to how short-course non-continuous treatments, such as cladribine tablets, can work and how well they are tolerated. In response to this, we aimed to create an evidence-based report on patient-focused aspects of treatment with cladribine. To inform development, MS experts, including healthcare professionals (HCPs) and PwMS, proposed topics that PwMS and their families and caregivers would most like to discuss with HCPs during consultations to help them better understand cladribine treatment. The statements regarding each topic were then ranked by PwMS and used to inform the topics covered in this report. We explain here the use of cladribine tablets, which includes explanations of how cladribine tablets work, how to take cladribine tablets, and considerations required prior to and while taking cladribine tablets. We also describe how cladribine tablets affect relapse rate and quality of life and detail side effects, when they are likely to happen, and for how long. We also discuss how cladribine tablets affect family planning, fertility, and the use of vaccines. Alongside each section is a brief, plain language description of what is covered and an accompanying visual to aid conversations between HCPs and PwMS. Improved understanding by PwMS of treatments, such as cladribine, can empower them to play a bigger role in shared decision-making regarding their treatment. Additionally, the open dialogue we aim to promote with this type of report could lead to treatment choices being better tailored for individuals with chronic diseases on the basis of personal experiences, preferences, and circumstances.

A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization

Code summarization is the task of creating short, natural language descriptions of source code. It is an important part of code comprehension, and a powerful method of documentation. Previous work has made progress in identifying where programmers focus in code as they write their own summaries (i.e., Writing). However, there is currently a gap studying programmers’ attention as they read code with pre-written summaries (i.e., Reading). As a result, it is currently unknown how these two forms of code comprehension compare: Reading and Writing. Also, there is a limited understanding of programmer attention with respect to program semantics. We address these shortcomings with a human eye-tracking study ( n =27) comparing Reading and Writing. We examined programmers’ attention with respect to fine-grained program semantics, including their attention sequences (i.e., scan paths). We find distinctions in programmer attention across the comprehension tasks, similarities in reading patterns between them, and differences mediated by demographic factors. This can help guide code comprehension in both CS education and automated code summarization. Furthermore, we mapped programmers’ gaze data onto the Abstract Syntax Tree to explore another representation of human attention. We find that visual behavior on this structure is not always consistent with that on source code.

Design and Simulation of Clock Divider using VHDL

This paper presents the diesign and simulation of clock divider circuit using VHDL(VHSIC Hardware Description Language) on an FPGA(Field Programmable Gate Array). The clock divider circuit is a fundamental component in digital system for generating lower frequency clocks from a higher frequency reference clock. The paper starts up with simple divider where the clock is divided by even numbers, odd numbers and then later expands it into non- integer dividers. Keywords:- clock divider, D flipflop, FPGA

AN EFFICIENT VLSI IMPLEMENTATION OF EDGE DETECTION OF IMAGES

Edge detection is a fundamental process in image processing with significant implications across multiple domains. Within the realm of object detection, the accurate delineation of image perimeters forms the bedrock for subsequent analysis. In this study, we present an optimized Laplacian Edge Detection algorithm tailored for efficient implementation in hardware.Leveraging approximation methodologies and strategic pipelining, we streamline complex operations, thereby reducing latency and resource consumption. Our implementation, realized through the Verilog Hardware Description Language (VHDL), stands as a testament to the efficacy of our approach. Through rigorous optimization and meticulous comparison with conventional Sobel edge detection, we demonstrate superior performance metrics, including enhanced Area utilization, reduced Time Delay, and improved Power Efficiency. This work underscores the importance of methodological refinement and technological innovation in advancing edge detection techniques for practical applications. Keywords: Laplacian, Sobel, Kernel, Pixel range, Area, Power