Text Labels Research Articles

Agent-DA: Enhancing low-resource event extraction with collaborative multi-agent data augmentation

Low-resource event extraction presents a significant challenge in real-world applications, particularly in domains like pharmaceuticals, military and law, where data is frequently insufficient. Data augmentation, as a direct method for expanding samples, is considered an effective solution. However, existing data augmentation methods often suffers from text fluency issues and label hallucination. To address these challenges, we propose a framework called Agent-DA, which leverages multi-agent collaboration for event extraction data augmentation. Specifically, Agent-DA follows a three-step process: data generation by the large language model, collaborative filtering by both the large language model and small language model to discriminate easy samples, and the use of an adjudicator to identify hard samples. Through iterative and selective augmentation, our method significantly enhances both the quantity and quality of event samples, improving text fluency and label consistency. Extensive experiments on the ACE2005-EN and ACE2005-EN+ datasets demonstrate the effectiveness of Agent-DA, with F1-score improvements ranging from 0.15% to 16.18% in trigger classification and from 2.2% to 15.67% in argument classification. The code and models can be found at https://github.com/GYK-CASIC/Agent-DA.

Preserving text space integrity for robust compositional zero-shot learning via mixture of pretrained experts

In the current landscape of Compositional Zero-Shot Learning (CZSL) methods that leverage CLIP, the predominant approach is based on prompt learning paradigms. These methods encounter significant computational complexity when dealing with a large number of categories. Additionally, when confronted with new classification tasks, there is a necessity to learn the prompts again, which can be both time-consuming and resource-intensive. To address these challenges, We present a new methodology, named the Mixture of Pretrained Expert (MoPE), for enhancing Compositional Zero-shot Learning through Logit-Level Fusion with Mutile Expert Fusion Module. The MoPE skillfully blends the benefits of extensive pre-trained models like CLIP, Bert, GPT-3 and Word2Vec for effectively tackling Compositional Zero-shot Learning. Firstly, we extract the text label space for each language model individually, then map the visual feature vectors to their respective text spaces. This maintains the integrity and structure of the original text space. During this process, the pre-trained expert parameters are kept frozen. The mapping of visual features to the corresponding text spaces is subject to learning and could be considered as multiple learnable visual experts. In the model fusion phase, we propose a new fusion strategy that features a gating mechanism that adjusts the contributions of various models dynamically. This enables our approach to adapt more effectively to a range of tasks and data sets. The method’s robustness is demonstrated by the fact that the language model is not tailored to specific downstream task datasets or losses. This preserves the larger model’s topology and expands the potential for application. Preliminary experiments conducted on the UT-Zappos, AO-Clever, and C-GQA datasets indicate that MoPE performs competitively when compared to existing techniques.

GPT-4 as an X data annotator: Unraveling its performance on a stance classification task.

Data annotation in NLP is a costly and time-consuming task, traditionally handled by human experts who require extensive training to enhance the task-related background knowledge. Besides, labeling social media texts is particularly challenging due to their brevity, informality, creativity, and varying human perceptions regarding the sociocultural context of the world. With the emergence of GPT models and their proficiency in various NLP tasks, this study aims to establish a performance baseline for GPT-4 as a social media text annotator. To achieve this, we employ our own dataset of tweets, expertly labeled for stance detection with full inter-rater agreement among three annotators. We experiment with three techniques: Zero-shot, Few-shot, and Zero-shot with Chain-of-Thoughts to create prompts for the labeling task. We utilize four training sets constructed with different label sets, including human labels, to fine-tune transformer-based large language models and various combinations of traditional machine learning models with embeddings for stance classification. Finally, all fine-tuned models undergo evaluation using a common testing set with human-generated labels. We use the results from models trained on human labels as the benchmark to assess GPT-4's potential as an annotator across the three prompting techniques. Based on the experimental findings, GPT-4 achieves comparable results through the Few-shot and Zero-shot Chain-of-Thoughts prompting methods. However, none of these labeling techniques surpass the top three models fine-tuned on human labels. Moreover, we introduce the Zero-shot Chain-of-Thoughts as an effective strategy for aspect-based social media text labeling, which performs better than the standard Zero-shot and yields results similar to the high-performing yet expensive Few-shot approach.

A multi-task framework based on decomposition for multimodal named entity recognition

Given a text-image pair, Multimodal Named Entity Recognition (MNER) is the task of identifying and categorizing entities in the text. Most existing work performs named entity labeling directly using final token representations derived by fusing image and text representations. Although they achieve promising results, these work may fail to effectively exploit text and image modalities. This is because they neglect the difference in the role of the two modalities: text modality can detect the boundary of an entity, while image modality is introduced to disambiguate the category of the entity. Based on these findings, in this paper, we construct two auxiliary tasks based on the decomposition strategy and propose a multi-task framework for MNER. Specifically, we first decompose MNER into two auxiliary tasks: entity boundary detection task and entity category classification task. Here, the former treats only the text modality as input and outputs the boundary labels, since it can achieve satisfactory boundary results by itself. The latter uses two modalities to yield category labels where image modality is dedicated to disambiguating categories. These two auxiliary tasks allow the effective exploitation of text and image modalities and put them back into their respective roles. Then, we vectorize their results to improve entity recognition using label clues from auxiliary tasks. Finally, we fuse features from text and image modalities and label embeddings from auxiliary tasks to fulfill MNER. Experimental results on two widely used MNER datasets show that our framework can yield new SOTA performance.

Dynamic training for handling textual label noise

Dynamic training for handling textual label noise

Towards Visualization Thumbnail Designs That Entice Reading Data-Driven Articles.

As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, cropping, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. To this end, we first survey visualization thumbnails collected online and discuss visualization thumbnail practices with data journalists and news graphics designers. Based on the survey and discussion results, we then define a design space for visualization thumbnails and conduct a user study with four types of visualization thumbnails derived from the design space. The study results indicate that different chart components play different roles in attracting reader attention and enhancing reader understandability of the visualization thumbnails. We also find various thumbnail design strategies for effectively combining the charts' components, such as a data summary with highlights and data labels, and a visual legend with text labels and Human Recognizable Objects (HROs), into thumbnails. Ultimately, we distill our findings into design implications that allow effective visualization thumbnail designs for data-rich news articles. Our work can thus be seen as a first step toward providing structured guidance on how to design compelling thumbnails for data stories.

Does CLIP Know My Face?

With the rise of deep learning in various applications, privacy concerns around the protection of training data have become a critical area of research. Whereas prior studies have focused on privacy risks in single-modal models, we introduce a novel method to assess privacy for multi-modal models, specifically vision-language models like CLIP. The proposed Identity Inference Attack (IDIA) reveals whether an individual was included in the training data by querying the model with images of the same person. Letting the model choose from a wide variety of possible text labels, the model reveals whether it recognizes the person and, therefore, was used for training. Our large-scale experiments on CLIP demonstrate that individuals used for training can be identified with very high accuracy. We confirm that the model has learned to associate names with depicted individuals, implying the existence of sensitive information that can be extracted by adversaries. Our results highlight the need for stronger privacy protection in large-scale models and suggest that IDIAs can be used to prove the unauthorized use of data for training and to enforce privacy laws. This article appears in the AI & Society track.

Developing icons for older adults: The roles of icon design categories, text labels, and learning strategies

When using electronic devices, older adults usually face challenges associated with difficulties in understanding icons. This study investigated age-related differences in icon identification using various design principles (Experiment 1), as well as the impacts of adding text labels (Experiment 2) and learning strategies (Experiment 3) on improving the performance of older adults. Participants, including older adults (aged 60 or older) and younger individuals, were asked to identify icons matching previous descriptions. The results revealed that the identification performance between older and younger adults varied across icon categories. The addition of text labels resolved differences in icon comprehension related to age of participant and design principles. Learning strategies had diverse effects on the identification performance of older and younger participants, depending on the icon category. These findings provide guidance for designers to apply effective principles and identify optimal strategies to support older adults in the identification and understanding of icons.

Are Tobacco Companies in Nigeria Complying With Health Warning Label Regulations on Cigarettes and Other Tobacco Products?

BACKGROUND: Nigeria has approved 50% coverage of text and images of pictorial health warning labels (PHWLs) on packages of cigarettes and other tobacco products such as cigars, smokeless tobacco, and loose tobacco. While there is knowledge on the extent to which LMICs like Nigeria adopt the WHO Framework Convention of Tobacco Control (FCTC), there is need to examine the compliance with adopted packaging and labeling policies to generate knowledge for strengthening existing policies and efforts at enforcement. We assessed agreement with FCTC regulations as well as compliance with Nigerian requirements which are not adequate and do not effectively capture all FCTC requirements to identify the gaps being explored by these companies in an environment of poor regulation in Lagos, Nigeria. METHODS: This descriptive study was conducted in Lagos State, Nigeria, from October 2022 to January 2023. A multistage random sampling method was utilized. Four local government areas (LGAs) with a combined population of 16 million, accounting for 70% of the population in Lagos, Nigeria were selected while three points‑of‑sale were randomly picked from two selected wards in each LGA. Empty discarded packs of cigarette and other tobacco products were collected from these outlets. The pack dimensions were measured and assessed for PHWLs, health warning manipulations and cessation assistance. Findings were assessed for compliance with the FCTC and national legislation. For each compliance feature, we calculated the frequencies and percentages of the packs that were compliant. RESULTS: 704 packs: 434 packs of 22 cigarette brands and 270 packs of 17 different tobacco products including cigars, snuff and loose tobacco were included in the study. Of 434 cigarette packs assessed, 50.3% were compliant with 9 indicators measured. These ranged from 48.8% to 55.5% for individual brands. For cessation assistance, image variability and image manipulation, compliance was 0% across all cigarette packs. Out of 270 tobacco product packs assessed, 10.6% were compliant with the 9 indicators assessed. Of the 12 multinational parent companies and product distributors assessed, the overall level of compliance among them was 24.9%. Level of compliance was noted to very low for companies involved in marketing other tobacco products, ranging from 0% to 20% but moderate for those marketing cigarettes, ranging from 37% to 55.5%. Companies that had no presence or subsidiaries in Nigeria but only export their products were highly non-compliant. CONCLUSIONS: Tobacco companies had low compliance with PHWL on cigarettes and negligible compliance on other tobacco products like cigars, loose and smokeless tobacco. The government should expand and strictly enforce all regulations applying to health warnings, especially on non-cigarette tobacco products and closely monitor companies that merely export their products to the country.

Fast and Lightweight Vision-Language Model for Adversarial Traffic Sign Detection

Several attacks have been proposed against autonomous vehicles and their subsystems that are powered by machine learning (ML). Road sign recognition models are especially heavily tested under various adversarial ML attack settings, and they have proven to be vulnerable. Despite the increasing research on adversarial ML attacks against road sign recognition models, there is little to no focus on defending against these attacks. In this paper, we propose the first defense method specifically designed for autonomous vehicles to detect adversarial ML attacks targeting road sign recognition models, which is called ViLAS (Vision-Language Model for Adversarial Traffic Sign Detection). The proposed defense method is based on a custom, fast, lightweight, and salable vision-language model (VLM) and is compatible with any existing traffic sign recognition system. Thanks to the orthogonal information coming from the class label text data through the language model, ViLAS leverages image context in addition to visual data for highly effective attack detection performance. In our extensive experiments, we show that our method consistently detects various attacks against different target models with high true positive rates while satisfying very low false positive rates. When tested against four state-of-the-art attacks targeting four popular action recognition models, our proposed detector achieves an average AUC of 0.94. This result achieves a 25.3% improvement over a state-of-the-art defense method proposed for generic image attack detection, which attains an average AUC of 0.75. We also show that our custom VLM is more suitable for an autonomous vehicle compared to the popular off-the-shelf VLM and CLIP in terms of speed (4.4 vs. 9.3 milliseconds), space complexity (0.36 vs. 1.6 GB), and performance (0.94 vs. 0.43 average AUC).

Novel approach to implementing natural language processing for clinical staging of non-small-cell lung cancer.

e13624 Background: Cancer staging is instrumental in driving clinical management and trial enrollment, but staging data are generally unreliable and unstructured in the electronic health record (EHR). Advances in natural language processing (NLP) may facilitate clinical staging and documentation [1], but challenges to real-world implementation include (1) automatically identifying appropriate patients and reports from the EHR and (2) developing an unbiased dataset for training and validation [2]. We describe our institution’s novel approach to overcome these barriers while building an in-house NLP pipeline for clinical tumor staging of non-small-cell lung cancer (NSCLC). Methods: We identified patients by searching our EHR (Epic) for a molecular analysis test ordered specifically for pathological diagnoses of NSCLC at our institution. We used the test order date as the diagnosis proxy date (DPD). For each patient, we extracted imaging reports up to 16 weeks before and 6 weeks after the DPD. To derive primary tumor size, we analyzed the CT Chest or PET/CT report closest to the DPD using an oncology-trained NLP text extraction and labeling tool (John Snow Labs). We cleaned all extracted tumor size entities and identified the largest measurement linked to the lungs. We compared primary tumor measurements from the NLP pipeline to those in a preexisting, manually compiled cancer registry (CNEXT). We manually analyzed discrepancies through chart review. Results: 542 patients with a DPD between 11/2016 - 9/2023 were processed through the NLP pipeline. Of 443 patients with valid values in both the pipeline and CNEXT, 53% (234) were exact matches, and 20% (90) had a close match (within 0-5mm), yielding a 73% accuracy rate for values within 5mm. When mismatched values were manually reviewed, several cases in CNEXT were found to have a DPD differing by more than 3 months and tumor sizes derived from external reports. When these cases were excluded, 320 of the remaining 349 patients had valid values in both the pipeline and the updated manual review. In this refined population, 66% (213) were exact matches, and 15% (48) had a close match, yielding an 82% accuracy rate for values within 5mm. Conclusions: To our knowledge, this is the first report of a pathology-based method to automatically and reliably identify patients with NSCLC and their relevant imaging reports directly from the EHR. We used a prebuilt NLP tool to derive primary tumor sizes with relatively high accuracy and found that adding flags for timeline discrepancies and external reports can further improve validity. As we near completion of analogous pipelines for node and metastasis staging, we will develop methodology to identify subgroups of patients that can be clinically staged with near-perfect accuracy, ultimately aiming to substantially limit manual staging of uncomplicated cases. 1. Puts 2023. 2. Wang 2022.

The impacts of packaging on preferences for cannabis edibles: A discrete choice experiment

BackgroundsCannabis edibles recently gained considerable market share in the United States. The tobacco and food literatures consistently suggest that product packaging regulations are crucial to substance control, but little is known about how product packaging may impact cannabis edible use. This study aims to estimate the impacts of packaging on individual preferences for cannabis edibles and explore heterogeneities in preferences by cannabis use status and use purposes. Methods1578 adults were recruited, who lived in 18 states and Washington D.C. in the United States that legalized recreational cannabis by the time of data collection in August and September of 2022. An online discrete choice experiment was conducted to elicit individual choices between cannabis edibles with variations in five packaging attributes: package style, health claim, potency indicator, warning label position, and warning label text. Mixed logit regressions were used to assess associations between package attributes and package choices. Subsample analysis was conducted by cannabis use status (users vs. nonusers) and use purposes (medical-only, recreational-only, and dual-purpose) to detect heterogeneities. ResultsAlmost all subsamples prefer branded packages to plain packages, any health claim to no health claim, and any potency indicator to no potency indicator. Cannabis users, particularly recreational-only users and dual-purpose users, also prefer youth-appealing packages to branded packages. Warning label position and text have limited impacts on choices. Overall, package style is perceived to be the most important attribute among the five (relative importance 33.2–50.8%), followed by health claim (relative importance 22.6–30.5%). ConclusionIn the United States, adults’ preferences for cannabis edibles are associated with packaging features. Policies requiring plain package and prohibiting youth-appealing package and unsubstantiated health claims may be effective methods of cannabis control.

Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases.

Artificial Intelligence (AI), particularly AI-Generated Imagery, has the potential to impact medical and patient education. This research explores the use of AI-generated imagery, from text-to-images, in medical education, focusing on congenital heart diseases (CHD). Utilizing ChatGPT's DALL·E 3, the research aims to assess the accuracy and educational value of AI-created images for 20 common CHDs. In this study, we utilized DALL·E 3 to generate a comprehensive set of 110 images, comprising ten images depicting the normal human heart and five images for each of the 20 common CHDs. The generated images were evaluated by a diverse group of 33 healthcare professionals. This cohort included cardiology experts, pediatricians, non-pediatric faculty members, trainees (medical students, interns, pediatric residents), and pediatric nurses. Utilizing a structured framework, these professionals assessed each image for anatomical accuracy, the usefulness of in-picture text, its appeal to medical professionals, and the image's potential applicability in medical presentations. Each item was assessed on a Likert scale of three. The assessments produced a total of 3630 images' assessments. Most AI-generated cardiac images were rated poorly as follows: 80.8% of images were rated as anatomically incorrect or fabricated, 85.2% rated to have incorrect text labels, 78.1% rated as not usable for medical education. The nurses and medical interns were found to have a more positive perception about the AI-generated cardiac images compared to the faculty members, pediatricians, and cardiology experts. Complex congenital anomalies were found to be significantly more predicted to anatomical fabrication compared to simple cardiac anomalies. There were significant challenges identified in image generation. Based on our findings, we recommend a vigilant approach towards the use of AI-generated imagery in medical education at present, underscoring the imperative for thorough validation and the importance of collaboration across disciplines. While we advise against its immediate integration until further validations are conducted, the study advocates for future AI-models to be fine-tuned with accurate medical data, enhancing their reliability and educational utility.

Chinese text recognition and knowledge graph of Shen Nong Ben Cao Jing based on BERT pre-trained language models

Background: The research and utilization of ancient traditional Chinese medicine (TCM) books are relatively limited at present. With the rapid development of artificial intelligence (AI), knowledge graph related technology has brought light on this field. Objective: To construct the knowledge graph of Shen Nong Ben Cao Jing, analyze basic knowledge of materia medica, explore implicit knowledge, and conduct visualization display, as well to provide methodological references for the study of ancient TCM books. Methods: The types of knowledge entities and relationships between entities in Shen Nong Ben Cao Jing were analyzed. A training corpus dataset was produced by using the BIO sequence labeling method; a self-developed CNLP text labeling system was used for text labeling; the BERT model was used to recognize named entities; the relationships between entities were set based on rules and semantic associations; the data into the Neo4j-community 4.4.9 graph database was imported by using Cypher language for storage and visualization display after knowledge fusion; finally, a knowledge graph was constructed. Results: The knowledge graph of Shen Nong Ben Cao Jing included 5, 273 nodes and 11, 064 relationships. The schema layer contained 14 entity types and 15 relationship types. Through the query, knowledge can be visualized from the aspects of classification, property, and seven mutual relationships of herbal combination. Conclusion: The knowledge graph constructed in this study directly reflects the knowledge recorded in Shen Nong Ben Cao Jing and the relationship between them, which is suitable for knowledge mining and intuitive multi-dimensional display of ancient TCM books.

PEDESTRIAN ALERT SYSTEM USING DEEP LEARNING

Pedestrian detection is an essential and significant task in any intelligent video surveillance system, as it provides the fundamental information for semantic understanding of the video footages. It has an obvious extension to automotive applications due to the potential for improving safety systems. Many car manufacturers (e.g., Volvo, Ford, GM, Nissan) offer this as an ADAS option in 2017. It is used in many cars and vehicles for automatic driver assistance systems. But when the ADAS system detects pedestrians but if fails to apply brakes on time accidents might happened. To overcome this situation our project provides an audio alert to the human passengers or driver in the vehicle so that they can apply manual brakes. In this project we will require Python libraries such as Pytorch and OpenCV. Pytorch is a widely used Machine Learning library. It is popular for the YOLO (You Only Look Once) algorithm which is built for Object Detection, we are using the YOLO algorithm and customize it to detect objects on a pedestrian dataset. Python has a library pyttsx3, that is capable to convert text-to-speech offline. Itextracts the label from the detected pedestrians in the video and converts the text label into speech. The algorithm detects the pedestrians when they are in a close approach to the vehicles and sends the identifies labels to the pyttsx3 speaker engine. An audio alert is generated by the engine and alerts the passengers or drivers. Keywords - YOLO, Pytorch, Pyttsx3, Object Detection, Python.

Bridging the Communication Gap - Sign Language Communication System

Now, imagine a world where words are not spoken, and emotions are not expressed. For the Deaf community, this is not a mere thought experiment but a daily reality. Indeed, they are forced to communicate primarily through sign language e, therefore, regularly feel alienated in a society where words are the primary tool for translating thoughts and feelings. Under such circumstances, most Deaf and hard-of-hearing people are struggling to communicate the most effectively. They usually opt for interpreters or their own signaling, but these alternatives are not the most efficient or effective choices. While signaling is the most intuitive and meaningful alternative, its grammar and semantic variations make it impossible to comprehend for those people who are not within the culture. Therefore, we have also developed a software prototype that could interpret sign language automatically. This paper presents a new method to recognize Indian sign language alphabets A-Z and digits (0-9) in a real- time video feed by employing the Bag of Visual Words model (BOVW). Furthermore, it not only predicts the labels of the signs but also gives the output as text and speech. The segmentation process here comprehends skin color detection and background subtraction. Speeded Up Robust FeaturesSurf features have been extracted again from the images, and histograms are built to map the signs to text labels. Furthermore, we have utilized algorithms such as Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) for binary classification. Finally, we have established an interactive Graphical User Interface (GUI to make this process more user-friendly

Are Tobacco Companies in Nigeria Complying With Health Warning Label Regulations on Cigarettes and Other Tobacco Products?

BACKGROUND: Nigeria has approved 50% coverage of text and images of pictorial health warning labels (PHWLs) on packages of cigarettes and other tobacco products such as cigars, smokeless tobacco, and loose tobacco. While there is knowledge on the extent to which LMICs like Nigeria adopt the WHO Framework Convention of Tobacco Control (FCTC), there is need to examine the compliance with adopted packaging and labeling policies to generate knowledge for strengthening existing policies and efforts at enforcement. We assessed agreement with FCTC regulations as well as compliance with Nigerian requirements which are not adequate and do not effectively capture all FCTC requirements to identify the gaps being explored by these companies in an environment of poor regulation in Lagos, Nigeria. METHODS: This descriptive study was conducted in Lagos State, Nigeria, from October 2022 to January 2023. A multistage random sampling method was utilized. Four local government areas (LGAs) with a combined population of 16 million, accounting for 70% of the population in Lagos, Nigeria were selected while three points‑of‑sale were randomly picked from two selected wards in each LGA. Empty discarded packs of cigarette and other tobacco products were collected from these outlets. The pack dimensions were measured and assessed for PHWLs, health warning manipulations and cessation assistance. Findings were assessed for compliance with the FCTC and national legislation. For each compliance feature, we calculated the frequencies and percentages of the packs that were compliant. RESULTS: 704 packs: 434 packs of 22 cigarette brands and 270 packs of 17 different tobacco products including cigars, snuff and loose tobacco were included in the study. Of 434 cigarette packs assessed, 50.3% were compliant with 9 indicators measured. These ranged from 48.8% to 55.5% for individual brands. For cessation assistance, image variability and image manipulation, compliance was 0% across all cigarette packs. Out of 270 tobacco product packs assessed, 10.6% were compliant with the 9 indicators assessed. Of the 12 multinational parent companies and product distributors assessed, the overall level of compliance among them was 24.9%. Level of compliance was noted to very low for companies involved in marketing other tobacco products, ranging from 0% to 20% but moderate for those marketing cigarettes, ranging from 37% to 55.5%. Companies that had no presence or subsidiaries in Nigeria but only export their products were highly non-compliant. CONCLUSIONS: Tobacco companies had low compliance with PHWL on cigarettes and negligible compliance on other tobacco products like cigars, loose and smokeless tobacco. The government should expand and strictly enforce all regulations applying to health warnings, especially on non-cigarette tobacco products and closely monitor companies that merely export their products to the country.

Large language models and their possible uses in law

AbstractThe paper explores the potential applications of Large Language Models (LLMs) like ChatGPT in the legal field, focusing on how they can enhance access to law. We begin by elucidating the fundamental workings of LLMs and their current and future general applications. The core of our study predicts the utilization of LLMs in various legal domains, especially where tasks like text retrieval, generation, labeling, and classification are prevalent. We argue that tools like ChatGPT could play a pivotal role in these areas. Additionally, we discuss the limitations and customization requirements of LLMs, particularly for legal uses. An experiment conducted by one of the authors, involving a tailored version of GPT for small law firms, serves as a practical example, but building on this, the paper also proposes ways in which LLM-based applications could democratize access to justice, making legal assistance more accessible and efficient for the broader public. This study contributes to the understanding of the intersection between AI technology and legal services, highlighting both the opportunities and challenges in this field.

CLIM: Contrastive Language-Image Mosaic for Region Representation

Detecting objects accurately from a large or open vocabulary necessitates the vision-language alignment on region representations. However, learning such a region-text alignment by obtaining high-quality box annotations with text labels or descriptions is expensive and infeasible. In contrast, collecting image-text pairs is simpler but lacks precise object location information to associate regions with texts. In this paper, we propose a novel approach called Contrastive Language-Image Mosaic (CLIM), which leverages large-scale image-text pairs effectively for aligning region and text representations. CLIM combines multiple images into a mosaicked image and treats each image as a ‘pseudo region’. The feature of each pseudo region is extracted and trained to be similar to the corresponding text embedding while dissimilar from others by a contrastive loss, enabling the model to learn the region-text alignment without costly box annotations. As a generally applicable approach, CLIM consistently improves different open-vocabulary object detection methods that use caption supervision. Furthermore, CLIM can effectively enhance the region representation of vision-language models, thus providing stronger backbones for open-vocabulary object detectors. Our experimental results demonstrate that CLIM improves different baseline open-vocabulary object detectors by a large margin on both OV-COCO and OV-LVIS benchmarks. The code is available at https://github.com/wusize/CLIM.

Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks

Textual label names (descriptions) are typically semantically rich in many natural language understanding (NLU) tasks. In this paper, we incorporate the prompting methodology, which is widely used to enrich model input, into the label side for the first time. Specifically, we propose a Mask Matching method, which equips an input with a prompt and its label with another, and then makes predictions by matching their mask representations. We evaluate our method extensively on 8 NLU tasks with 14 datasets. The experimental results show that Mask Matching significantly outperforms its counterparts of fine-tuning and conventional prompt-tuning, setting up state-of-the-art performances in several datasets. Mask Matching is particularly good at handling NLU tasks with large label counts and informative label names. As pioneering efforts that investigate the label-side prompt, we also discuss open issues for future study.