Bidirectional Encoder Representations From Transformers Research Articles

Empowering geoportals HCI with task-oriented chatbots through NLP and deep transfer learning

ABSTRACT In the past ten years, chatbot development has matured to become one of the most well-distinguished outcomes of artificial intelligence. Despite some criticism, Bing AI, ChatGPT and other natural language processing (NLP) products of similar nature are becoming more popular. The creation of chatbots can close several gaps in geographic information retrieval as well. This research introduces and successfully implements, for the first time, a model for integrating task-oriented chatbots into geoportals, with the goal of easing user requests, improving access to geospatial services, and fostering human-computer interactions (HCI). Additionally, it presents a novel recommendation solution for matching the most appropriate volunteer to the user’s geospatial needs based on expertise similarity, semantic similarity, and community feedback. The three categories of finding map services, discovering geoprocessing services, and volunteer expert recommendations were shown to be the most significant geoportal bot intents. Depending on the requirement, each intent additionally includes various entities such as time, place, description, skill, etc. The notion of deep transfer learning (DTL) was then put into practice by customizing a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model for our particular aim and creating a task-oriented conversational agent. According to the results, effective intent classification and entity recognition in the geospatial domain could arise from this approach. We performed the training process with 200 sample data, 20% of which were utilized in a stratified manner for testing, and we obtained f1-scores of at least 0.75. Finally, a pilot Geoportal Chabot that combines crowdsourcing and conversational agents’ approaches was put into use and tested with success. In keeping with SDI technical purposes, the system might direct users to common geospatial web services, namely WMS and WPS, in addition to including natural language understanding (NLU) and natural language generation (NLG) capabilities. Result of user-centered evaluation indicated that the integration of a chatbot significantly reduces the average time required to access geospatial data and processing services by more than 50%. Notably, this effect is even more pronounced when locating an expert, with a fivefold decrease in the time required. Finally, overall user satisfaction rose from 86% to 94%.

Machine learning and natural language processing to assess the emotional impact of influencers’ mental health content on Instagram

Background This study aims to examine, through artificial intelligence, specifically machine learning, the emotional impact generated by disclosures about mental health on social media. In contrast to previous research, which primarily focused on identifying psychopathologies, our study investigates the emotional response to mental health-related content on Instagram, particularly content created by influencers/celebrities. This platform, especially favored by the youth, is the stage where these influencers exert significant social impact, and where their analysis holds strong relevance. Analyzing mental health with machine learning techniques on Instagram is unprecedented, as all existing research has primarily focused on Twitter. Methods This research involves creating a new corpus labelled with responses to mental health posts made by influencers/celebrities on Instagram, categorized by emotions such as love/admiration, anger/contempt/mockery, gratitude, identification/empathy, and sadness. The study is complemented by modelling a set of machine learning algorithms to efficiently detect the emotions arising when faced with these mental health disclosures on Instagram, using the previous corpus. Results Results have shown that machine learning algorithms can effectively detect such emotional responses. Traditional techniques, such as Random Forest, showed decent performance with low computational loads (around 50%), while deep learning and Bidirectional Encoder Representation from Transformers (BERT) algorithms achieved very good results. In particular, the BERT models reached accuracy levels between 86–90%, and the deep learning model achieved 72% accuracy. These results are satisfactory, considering that predicting emotions, especially in social networks, is challenging due to factors such as the subjectivity of emotion interpretation, the variability of emotions between individuals, and the interpretation of emotions in different cultures and communities. Discussion This cross-cutting research between mental health and artificial intelligence allows us to understand the emotional impact generated by mental health content on social networks, especially content generated by influential celebrities among young people. The application of machine learning allows us to understand the emotional reactions of society to messages related to mental health, which is highly innovative and socially relevant given the importance of the phenomenon in societies. In fact, the proposed algorithms’ high accuracy (86–90%) in social contexts like mental health, where detecting negative emotions is crucial, presents a promising research avenue. Achieving such levels of accuracy is highly valuable due to the significant implications of false positives or false negatives in this social context.

Cascaded Adaptive Dilated Temporal Convolution Network-Based Efficient Sentiment Analysis Model from Social Media Posts

Sentimental analysis is one of the most complicated tasks in text mining. It is the process of converting unstructured text to structural text to identify the meaningful pattern of data. Different kinds of sentiments in social media posts are classified into positive and negative. Natural Language Processing (NLP) plays an imperative task in examining sentiments on social media. Sentimental analysis techniques frequently use text data to help to monitor customer feedback about products and understand the needs of customers in the field of business. The text data are gathered from social media comments that lead to several drawbacks such as spelling errors, noise, and unstructured data, and some of the data are in abbreviation form in which abbreviations are hard to understand from the text data. Therefore, the innovative sentiment analysis method is implemented to identify the kinds of sentiments from the collected social media posts. At first, the text data are assembled from the social media posts. The collected text data are subjected to text preprocessing techniques to improve the data quality. NLP techniques such as a bag of n-grams, glove embedding, and Bidirectional Encoder Representations from Transformers (BERT) are applied to separate the features from the text data. The removed features from these techniques are given to the fusion of the optimal feature stage, where the weight optimization takes place via the Mutated Random Parameter-based Serval Optimization Algorithm (MRP-SOA). The attained optimal weighted fused features contain the most relevant information, and it is fed to the classifier for analyzing the sentiments. For analyzing the sentiments, the Cascaded Adaptive Dilated Temporal Convolutional Network (CADTCN) is utilized, and the parameters from ADTCN are optimized with the same to get higher classification outcomes. The analysis outcomes are compared with the traditional sentiment analysis models in terms of various metrics to show the success of the advanced design.

Exploring Players’ Perspectives: A Comprehensive Topic Modeling Case Study on Elden Ring

Game reviews heavily influence public perception. User feedback is crucial for developers, offering valuable insights to enhance game quality. In this research, Metacritic game reviews for Elden Ring were analyzed for topic modeling using Latent Dirichlet Allocation (LDA), Bidirectional Encoder Representations from Transformers (BERT), and a hybrid model combining both to identify effective methods for extracting underlying themes in player feedback. We analyzed and interpreted these models’ outputs to learn the game reviews. We aimed to identify the differences, similarities, and variations between the three to determine which provided more valuable and instructive information. Our findings indicate that each method successfully identified keywords with some similarities in identified words. The LDA model had the highest silhouette score, indicating the most distinct clustering. The LDA-BERT model had a 1% higher coherence score than LDA, indicating more meaningful topics.

An advanced learning approach for detecting sarcasm in social media posts: Theory and solutions

AbstractObjectiveUsers of social media platforms such as Facebook, Instagram, and Twitter can view and share their daily life events through text, photographs, or videos. These platforms receive many sarcastic posts daily because there were fewer limits on what could be posted. The presence of multiple languages and media types in a single post makes it harder to identify sarcastic messages on the current platform than on posts written solely in English.MethodsThis study provides both the theory and solutions about sarcastic post detection on social platforms. Hindi–English code‐mixed data were used to train and test the automated models for sarcasm detection. The models in this study were constructed using traditional machine learning, deep neural networks, LSTM (long short‐term memory), CNN (convolutional neural network), and the combinations of BERT (Bidirectional Encoder Representations from Transformers) with LSTM.ResultsThe experimental results confirm that in the Hindi–English code‐mixed data set, the CNN, LSTM, and BERT‐LSTM ensemble perform best for sarcasm detection. The proposed model achieved an accuracy of 96.29 percent and outperformed by 2.29 percent compared to the existing models.ConclusionThe performance of the proposed system strengthens the code‐mixed sarcastic post detection on social platforms. The model will help filter not only English but also Hindi‐English code‐mixed sarcastic posts on social platforms.

On the development and validation of large language model-based classifiers for identifying social determinants of health

The assessment of social determinants of health (SDoH) within healthcare systems is crucial for comprehensive patient care and addressing health disparities. Current challenges arise from the limited inclusion of structured SDoH information within electronic health record (EHR) systems, often due to the lack of standardized diagnosis codes. This study delves into the transformative potential of large language models (LLM) to overcome these challenges. LLM-based classifiers-using Bidirectional Encoder Representations from Transformers (BERT) and A Robustly Optimized BERT Pretraining Approach (RoBERTa)-were developed for SDoH concepts, including homelessness, food insecurity, and domestic violence, using synthetic training datasets generated by generative pre-trained transformers combined with authentic clinical notes. Models were then validated on separate datasets: Medical Information Mart for Intensive Care-III and our institutional EHR data. When training the model with a combination of synthetic and authentic notes, validation on our institutional dataset yielded an area under the receiver operating characteristics curve of 0.78 for detecting homelessness, 0.72 for detecting food insecurity, and 0.83 for detecting domestic violence. This study underscores the potential of LLMs in extracting SDoH information from clinical text. Automated detection of SDoH may be instrumental for healthcare providers in identifying at-risk patients, guiding targeted interventions, and contributing to population health initiatives aimed at mitigating disparities.

A Domain-Specific Lexicon for Improving Emergency Management in Gas Pipeline Networks through Knowledge Fusing

Emergencies in gas pipeline networks can lead to significant loss of life and property, necessitating extensive professional knowledge for effective response and management. Effective emergency response depends on specialized knowledge, which can be captured efficiently through domain-specific lexicons. The goal of this research is to develop a specialized lexicon that integrates domain-specific knowledge to improve emergency management in gas pipeline networks. The process starts with an enhanced version of Term Frequency–Inverse Document Frequency (TF-IDF), a statistical method used in information retrieval, combined with filtering logic to extract candidate words from investigation reports. Simultaneously, we fine tune the Chinese Bidirectional Encoder Representations from Transformers (BERT) model, a state-of-the-art language model, with domain-specific data to enhance semantic capture and integrate domain knowledge. Next, words with similar meanings are identified through word similarity analysis based on standard terminology and risk inventories, facilitating lexicon expansion. Finally, the domain-specific lexicon is formed by amalgamating these words. Validation shows that this method, which integrates domain knowledge, outperforms models that lack such integration. The resulting lexicon not only assigns domain-specific weights to terms but also deeply embeds domain knowledge, offering robust support for cause analysis and emergency management in gas pipeline networks.

Unveiling the hidden patterns: A novel semantic deep learning approach to fake news detection on social media

The rise of social media as a source of news consumption has led to the spread of fake news, posing serious consequences for both individuals and society. The detection and prevention of fake news are essential, and previous research has shown that incorporating news content along with its associated headlines and user comments can improve detection performance. However, the semantic relationships between these elements have not been fully explored. This paper proposes a novel approach that models the relationships between news bodies and associated headlines/user comments using deep learning techniques, such as fine-tuned Bidirectional Encoder Representations from Transformers (BERT) and cross-level cross-modality attention sub-networks. In our proposed model, we utilize two different configurations of BERT: pool-based representation, which provides a representation of the entire document, and sequence representation, which represents each token within the document (i.e., at the word and text levels). The approach also encodes user-posting behavioural features and fuses the output of these components to detect fake news using a classification layer. Our experiments on benchmark datasets demonstrate the superiority of the proposed method over existing state-of-the-art (SOTA) approaches, highlighting the importance of utilizing semantic relationships for improved fake news detection (FND). These findings have significant implications for combating the spread of fake news and protecting society from its negative effects.

CACER: Clinical concept Annotations for Cancer Events and Relations.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48000 medical problems and drug events and 10000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

Unveiling the drivers of satisfaction in mobile trading: Contextual mining of retail investor experience through BERTopic and generative AI

The proliferation of mobile stock trading has introduced various apps with distinct features, emphasizing the need to understand users’ evaluations after adopting the service. This study explores the determinants of retail investors’ satisfaction with mobile stock trading services by employing an advanced textual analysis of customer reviews for four leading trading applications. We utilized Bidirectional Encoder Representations from Transformers (BERT) based Topic modeling (BERTopic modeling) to identify key topics within customer reviews and used the results as input for generative AI to discern the theme and sentiment of each topic. Based on Service Quality (SERVQUAL) theory, topics are categorized into key quality dimensions: functionality, usability, information quality, customer service, and system quality. Regression models were employed to assess the impact of the quality dimensions on investor satisfaction, revealing positive feedback on usability, information quality, and service quality as primary enhancers of satisfaction. In contrast, negative feedback on service quality, system quality, and functionality was identified as the primary inhibitor of satisfaction. This study explores how the influence of each quality dimension varies among different types of brokers (full-service vs. online-only brokerages). Finally, we propose a visualization tool called Topic Rating Impact and Frequency Analysis (TRIFA), which is designed to categorize topics based on their frequency of occurrence and impact on satisfaction. This tool aids in identifying the strengths and areas for improvement in services by effectively visualizing the results of text review analysis. This research not only deepens our understanding of the quality dimensions of mobile financial services but also offers valuable insights for service providers by suggesting predictive models that could help increase customer retention.

Fine-grained vulnerability detection for medical sensor systems

The Internet of Things (IoT) has revolutionized the healthcare system by connecting medical sensors to the internet, while also posing challenges to the security of medical sensor networks (MSN). Given the extreme sensitivity of medical data, any vulnerability may result in data breaches and misuse, impacting patient safety and privacy. Therefore, safeguarding MSN security is critical. As medical sensor devices rely on smart healthcare software systems for data management and communication, precisely detecting system code vulnerabilities is essential to ensuring network security. Effective software vulnerability detection targets two key objectives: (i) achieving high accuracy and (ii) directly identifying vulnerable code lines for developers to fix. To address these challenges, we introduce Vulcoder, a novel vulnerability-oriented, encoder-driven model based on the Bidirectional Encoder Representations from Transformers (BERT) architecture. We propose a one-to-one mapping function to capture code semantics through abstract syntax trees (AST). Combined with multi-head attention, Vulcoder achieves precise function- and line-level detection of software vulnerabilities in MSN. This accelerates the vulnerability remediation process, thereby strengthening network security. Experimental results on various datasets demonstrate that Vulcoder outperforms previous models in identifying vulnerabilities within MSN. Specifically, it achieves a 1%–419% improvement in function-level prediction F1 scores and a 12.5%–380% increase in line-level localization precision. Therefore, Vulcoder helps enhance security defenses and safeguard patient privacy in MSN, facilitating the development of smart healthcare.

Exploring RoBERTa model for cross-domain suggestion detection in online reviews

Detecting suggestions in online review requires contextual understanding of review text, which is an important real-world application of natural language processing. Given the disparate text domains found in product reviews, a common strategy involves fine-tuning bidirectional encoder representations from transformers (BERT) models using reviews from various domains. However, there hasn't been an empirical examination of how BERT models behave across different domains in tasks related to detecting suggestion sentences from online reviews. In this study, we explore BERT models for suggestion classification that have been fine-tuned using single-domain and cross-domain Amazon review datasets. Our results indicate that while single-domain models achieved slightly better performance within their respective domains compared to cross-domain models, the latter outperformed single-domain models when evaluated on cross-domain data. This was also observed for single-domain data not used for fine-tuning the single-domain model and on average across all tests. Although fine-tuning single-domain models can lead to minor accuracy improvements, employing multi-domain models that perform well across domains can help in cold start problems and reduce annotation costs.

Do firms listen to the ESG voices of minority investors? evidence from China

This study examines whether and how the expression of ESG(Environmental, Social, and Governance) concerns by minority investors in their communication with firms affects firms' ESG performance. We employ the bidirectional encoder representation from transformer (BERT) model to identify ESG statements in questions asked by minority investors and construct sentence-level ESG concern indicators. The findings indicate that the more ESG-related questions investors ask in their interactions with firms, the better the firms' future ESG performance. This effect is more pronounced when investors’ ESG-related questions are more negative, when managers show greater willingness to learn from investor interactions, when listed firms emphasize investor relations management, and when listed firms are held by ESG funds. Additional analyses reveal that pressure from the market and regulation may explain why listed companies listen to the ESG voice of minority investors. In addition, we find that minority investor voices do not lead to opportunistic "greenwashing" when driving improvements in firms' ESG performance. Overall, our findings suggest that minority investors promote corporate sustainability through active voice.

Virality classification from Twitter data using pre-trained language model and multi-layer perceptron

Twitter is one of the well-known text-based social media that is often used to disseminate content. According to Katadata, Indonesia ranked fifth in the world in 2023. So many people or organizations want to make tweets go viral. Therefore, this research aims to develop a model that uses tweet data from the Indonesian language Twitter social media to categorize the level of virality. There are several tasks in classifying the level of virality, such as upsampling data, predicting sentiment and emotion, and text embedding. Upsampling data was carried out because the dataset used was an imbalanced dataset. Data upsampling, emotions, and text embedding is carried out using the bidirectional encoder representation from transformers (BERT) model. Meanwhile, sentiment prediction uses the Ro-bustly optimized BERT pretraining approach (RoBERTa). The results of text embedding, sentiment, emotion, will be combined with Twitter metadata then all features will be fed into the multi-layer perceptron (MLP) model to classifying the level of virality which is divided into 3 classes based on the number of retweets, namely low, medium and high. The proposed method produces an F1-score of 49% and an accuracy of 95% and performs better than the baseline model.

Hidden emotional trends on social media regarding the Thailand–China high-speed railway project: a deep learning approach with ChatGPT integration

Leveraging sentiment analysis on social media reveals hidden emotional trends by providing deep insights into the public’s collective opinions and feelings because social media can reflect most opinions based on true feelings. However, there is no time-series analysis of Thai opinions on social media regarding government projects in Thailand. Therefore, the focus of this research is to survey and dynamically analyze sentiment on social media platforms, particularly YouTube, regarding the Thailand–China High-Speed Rail Project (Thai–China HSR Project). This is a major project involving an investment of over 500 billion baht, a collaborative effort between the Thai and Chinese governments aimed at developing transportation infrastructure. It leads the public concerns of Thai people about the potential economic and social impacts. The deep learning (DL) technique was used for this analysis by the Bidirectional Encoder Representations from Transformers (BERT) model, together with WangchanBERTa, a Thai language model specifically designed to analyze opinions from Thai comments received from YouTube from July 2015 to January 2024. The analysis reveals that citizens’ sentiments are primarily neutral, with more negative than positive sentiments with the passing years. Results also indicate that the data volume has increased since 2019 and peaked in 2022. Negative comments peaked in January 2022, and based on our analysis, such negative comments followed news covering the opening of the China–Laos Railway on December 1, 2021. Video contents were related to the Thai–China HSR Project, which provides comparative reviews and analysis of government operations. Additionally, we explore the possibility of integrating AI technology based on the ChatGPT 4 model to support data interpretation. Although the AI model can interpret data quickly and efficiently serve as a convenient method for users, the results lack the in-depth analysis that should be done by experts in the field. The proposed framework could be useful to government agencies and can be used to quickly and efficiently survey and track public opinion. The results can be used for policy planning, strategic decision-making, and improving communications to better inform citizens about projects and operations, leading to the country’s sustainable economic and social development.

Multi-Feature Fusion in Graph Convolutional Networks for Data Network Propagation Path Tracing

With the rapid development of information technology, the complexity of data networks is increasing, especially in electric power systems, where data security and privacy protection are of great importance. Throughout the entire distribution process of the supply chain, it is crucial to closely monitor the propagation paths and dynamics of electrical data to ensure security and quickly initiate comprehensive traceability investigations if any data tampering is detected. This research addresses the challenges of data network complexity and its impact on the security of power systems by proposing an innovative data network propagation path tracing model, which is constructed based on graph convolutional networks (GCNs) and the BERT model. Firstly, propagation trees are constructed based on the propagation structure, and the key attributes of data nodes are extracted and screened. Then, GCNs are utilized to learn the representation of node features with different attribute feature combinations in the propagation path graph, while the Bidirectional Encoder Representations from Transformers (BERT) model is employed to capture the deep semantic features of the original text content. The core of this research is to effectively integrate these two feature representations, namely the structural features obtained by GCNs and the semantic features obtained by the BERT model, in order to enhance the ability of the model to recognize the data propagation path. The experimental results demonstrate that this model performs well in power data propagation and tracing tasks, and the data recognition accuracy reaches 92.5%, which is significantly better than the existing schemes. This achievement not only improves the power system’s ability to cope with data security threats but also provides strong support for protecting data transmission security and privacy.

Movie Review Based Sentiment Analysis

Sentiment analysis, an automated method for detecting emotions and opinions in text, has become a versatile tool applied across various domains, from assessing online customer feedback to monitoring sentiment trends on social media. While Bag of Words (BoW) has been the traditional method, more advanced techniques like Bidirectional Encoder Representations from Transformers (BERT) and Long Short-Term Memory (LSTM) have revolutionized sentiment analysis by elevating accuracy and context comprehension. This study conducts an extensive comparison between BERT, a hybrid model that combines CNN and LSTM in the realm of sentiment analysis within movie reviews. Through meticulous experimentation, the study strongly proves that both the hybrid model of CNN and LSTM models significantly outshine the techniques like BERT and BoW with noticeably higher accuracy in precisely categorizing sentiments expressed in movie reviews. These results highlight how deep learning methodologies remarkably refine the precision and effectiveness of sentiment analysis, allowing for more nuanced and context-aware sentiment classification. The study's significance goes beyond performance contrasts; it demonstrates the exceptional capacity of BERT along with the hybrid model of CNN and LSTM to comprehend the complexities of language and contextual nuances within movie reviews. This heightened contextual understanding is particularly crucial in sentiment analysis, enabling the models to discern subtle shifts in sentiment that simpler methods like BoW might overlook. By showcasing the supremacy of hybrid model that combines CNN and LSTM in movie review sentiment analysis, this research opens new avenues for the field's advancement. It emphasizes the potential of deep learning techniques in revolutionizing sentiment analysis across diverse domains, offering more accurate and contextually attuned sentiment classification for deeper insights and well-informed decision-making. This study stands as a valuable asset for researchers and practitioners aiming to leverage deep learning for sentiment analysis applications

Unmasking large language models by means of OpenAI GPT-4 and Google AI: A deep instruction-based analysis

Large Language Models (LLMs) have become a hot topic in AI due to their ability to mimic human conversation. This study compares the open artificial intelligence generative pretrained transformer-4 (GPT-4) model, based on the (GPT), and Google's artificial intelligence (AI), which is based on the Bidirectional Encoder Representations from Transformers (BERT) framework in terms of the defined capabilities and the built-in architecture. Both LLMs are prominent in AI applications. First, eight different capabilities were identified to evaluate these models, i.e. translation accuracy, text generation, factuality, creativity, intellect, deception avoidance, sentiment classification, and sarcasm detection. Next, each capability was assessed using instructions. Additionally, a categorized LLM evaluation system has been developed by means of using ten research questions per category based on this paper's main contributions from a prompt engineering perspective. It should be highlighted that GPT-4 and Google AI successfully answered 85 % and 68,7 % of the study prompts, respectively. It has been noted that GPT-4 better understands prompts than Google AI, even with verbal flaws, and tolerates grammatical errors. Moreover, the GPT-4 based approach was more precise, accurate, and succinct than Google AI, which was sometimes verbose and less realistic. While GPT-4 beats Google AI in terms of translation accuracy, text generation, factuality, intellectuality, creativity, and deception avoidance, Google AI outperforms the former when considering sarcasm detection. Both sentiment classification models did work properly. More importantly, a human panel of judges was used to assess and evaluate the model comparisons. Statistical analysis of the judges' ratings revealed more robust results based on examining the specific uses, limitations, and expectations of both GPT-4 and Google AI-based approaches. Finally, the two approaches' transformers, parameter sizes, and attention mechanisms have been examined.

Language conditioned multi-scale visual attention networks for visual grounding

Visual grounding (VG) is a task that requires to locate a specific region in an image according to a natural language expression. Existing efforts on the VG task are divided into two-stage, one-stage and Transformer-based methods, which have achieved high performance. However, most of the previous methods extract visual information at a single spatial scale and ignore visual information at other spatial scales, which makes these models unable to fully utilize the visual information. Moreover, the insufficient utilization of linguistic information, especially failure to capture global linguistic information, may lead to failure to fully understand language expressions, thus limiting the performance of these models. To better address the task, we propose a language conditioned multi-scale visual attention network (LMSVA) for visual grounding, which can sufficiently utilize visual and linguistic information to perform multimodal reasoning, thus improving performance of model. Specifically, we design a visual feature extractor containing a multi-scale layer to get the required multi-scale visual features by expanding the original backbone. Moreover, we exploit pooling the output of the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model to extract sentence-level linguistic features, which can enable the model to capture global linguistic information. Inspired by the Transformer architecture, we present the Visual Attention Layer guided by Language and Multi-Scale Visual Features (VALMS), which is able to better learn the visual context guided by multi-scale visual and linguistic features, and facilitates further multimodal reasoning. Extensive experiments on four large benchmark datasets, including ReferItGame, RefCOCO, RefCOCO+ and RefCOCOg, demonstrate that our proposed model achieves the state-of-the-art performance.

Development of an Internet-based Product-related Child Injury Textual Data Platform (IPCITDP) in China.

Internet-based media stories provide valuable information for emerging risks of product-related child injury prevention and control, but critical methodological challenges and high costs of data acquisition and processing restrict practical use by stakeholders. We constructed a data platform through literature reviews and multi-round research group discussions. Developed components included standard search strategies, filtering criteria, textual document classification, information extraction standards and a keyword dictionary. We used ten thousand manually labelled media stories to validate the textual document classification model, which was established using the Bidirectional Encoder Representation from Transformers (BERT). Multiple information extraction methods based on natural language processing algorithms were adopted to extract data for 29 structured variables from media stories. They were evaluated through manual validation of 1000 media stories about product-related child injury. We mapped the geographic distribution of media sources and media-reported product-related child injury events. We developed an internet-based product-related child injury textual data platform, IPCITDP, consisting of four layers - automatic data search and acquisition, data processing, data storage, and data application - concerning product-related child injury online media stories in China. Each layer occurred daily. External validation demonstrated high performance for the BERT classification model we established (accuracy = 0.9703) and the combined information extraction strategies (accuracy >0.70 for 25 variables). As of 31 December 2023, IPCITDP collected 35 275 eligible product-related child injury reports from 13 261 news media websites or social media platform accounts which were geographically located across all 31 mainland Chinese provinces and covered over 97% of prefecture-level cities. The injury cases in IPCITDP were typically reported several months or years earlier than official announcements about the product-related child injury risks. Our data platform added data concerning 15 supplementary variables that the national product-related injury surveillance system lacks. Two examples demonstrate the value of IPCITDP in supplementing existing data and providing early epidemiological detection of emerging signals concerning product-related child injury: magnetic beads and electric self-balancing scooters. Our data platform provides injury data that can support early detection of new product-related child injury characteristics and supplement existing data sources to reduce the burden of product-related injury among Chinese children.