Bidirectional Encoder Representations Research Articles

Attentive Review Semantics-Aware Recommendation Model for Rating Prediction

Online reviews are crucial when building a recommendation model because they contain the specific and rich preferences of users related to different aspects of a particular item. Incorporating these online reviews into the recommendation model mitigates the data sparsity issue to some extent and contributes to better recommendation performance. Despite this success, review-based recommender systems have the limitation that they do not fully consider the relevance of the review text to the target item. Specifically, the review text should reflect the user’s detailed opinion about the target item to extract detailed preference information. Meanwhile, the review content must be directly related to the target item to extract the customer’s specific preferences for the item. However, previous studies have overlooked both of these aspects. Therefore, it is necessary to build a recommendation model that considers the relevance of the review content to the target item. To address this issue, this study proposes a novel recommendation model that accurately estimates users’ preferences by carefully considering the relevance of the review content to the items. The proposed model effectively extracts feature representations from the text using bidirectional encoder representations from a transformer and obtains fused features by considering the importance of features through the attention mechanism. To evaluate the performance of the model, we used a publicly accessible dataset of reviews from Amazon.com and compared it to various baseline models. The experimental results demonstrated better recommendation performance of the proposed model compared to other baseline models.

Effectiveness of ELMo embeddings, and semantic models in predicting review helpfulness

Online product reviews (OPR) are a commonly used medium for consumers to communicate their experiences with products during online shopping. Previous studies have investigated the helpfulness of OPRs using frequency-based, linguistic, meta-data, readability, and reviewer attributes. In this study, we explored the impact of robust contextual word embeddings, topic, and language models in predicting the helpfulness of OPRs. In addition, the wrapper-based feature selection technique is employed to select effective subsets from each type of features. Five feature generation techniques including word2vec, FastText, Global Vectors for Word Representation (GloVe), Latent Dirichlet Allocation (LDA), and Embeddings from Language Models (ELMo), were employed. The proposed framework is evaluated on two Amazon datasets (Video games and Health & personal care). The results showed that the ELMo model outperformed the six standard baselines, including the fine-tuned Bidirectional Encoder Representations from Transformers (BERT) model. In addition, ELMo achieved Mean Square Error (MSE) of 0.0887 and 0.0786 respectively on two datasets and MSE of 0.0791 and 0.0708 with the wrapper method. This results in the reduction of 1.43% and 1.63% in MSE as compared to the fine-tuned BERT model on respective datasets. However, the LDA model has a comparable performance with the fine-tuned BERT model but outperforms the other five baselines. The proposed framework demonstrated good generalization abilities by uncovering important factors of product reviews and can be evaluated on other voting platforms.

TB-BGAT With TinyBERT and BiGRU in Personalized Course Recommendations

Aiming at the problems of inadequate user feature extraction, cold start and sparse data, a personalized course recommendation algorithm that utilizes TB-BGAT is suggested. First, the Tiny Bidirectional Encoder Representation from Transformers (TinyBERT) model is utilized to output character-level word vectors; then, Bidirectional Recurrent Neural Network (BiGRU) model is utilized to obtain the embedded contextual semantic features. Finally, the attention mechanism is utilized to allocate weights to various course features by assigning their importance and to obtain the output results. The results of experiment on the publicly available dataset MOOCs-Course prove that the proposed method improves at least 3.62%, 3.04%, and 3.33% in precision, recall, and F1-score, correspondingly, in contrast to several other state-of-the-art course resource recommendation algorithms. The proposed method can enhance the effectiveness of the course recommendation model, enhance the quality of learners' online learning, and provide good technical support for online education learning platforms.

TransLSTM: A hybrid LSTM-Transformer model for fine-grained suggestion mining

Digital platforms on the internet are invaluable for collecting user feedback, suggestions, and opinions about various topics, such as company products and services. This data is instrumental in shaping business strategies, enhancing product development, and refining service delivery. Suggestion mining is a key task in natural language processing, which focuses on extracting and analysing suggestions from these digital sources. Initially, suggestion mining utilized manually crafted features, but recent advancements have highlighted the efficacy of deep learning models, which automatically learn features. Models like Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Bidirectional Encoder Representations from Transformers (BERT) have been employed in this field. However, considering the relatively small datasets and the faster training time of LSTM compared to BERT, we introduce TransLSTM, a novel LSTM-Transformer hybrid model for suggestion mining. This model aims to automatically pinpoint and extract suggestions by harnessing both local and global text dependencies. It combines the sequential dependency handling of LSTM with the contextual interaction capabilities of the Transformer, thus effectively identifying and extracting suggestions. We evaluated our method against state-of-the-art approaches using the SemEval Task-9 dataset, a benchmark for suggestion mining. Our model shows promising performance, surpassing existing deep learning methods by 6.76% with an F1 score of 0.834 for SubTask A and 0.881 for SubTask B. Additionally, our paper presents an exhaustive literature review on suggestion mining from digital platforms, covering both traditional and state-of-the-art text classification techniques.

Identifying e-cigarette content on TikTok: Using a BERTopic Modeling approach.

The use of hashtags is a common way to promote e-cigarette content on social media. Analysis of hashtags may provide insight into e-cigarette promotion on social media. However, the examination of text data is complicated by the voluminous amount of social media data. This study used machine learning approaches (i.e., Bidirectional Encoder Representations from Transformers [BERT] topic modeling) to identify e-cigarette content on TikTok. We used 13 unique hashtags related to e-cigarettes (e.g., #vape) for data collection. The final analytic sample included 12,573 TikTok posts. To identify the best fitting number of topic clusters, we used both quantitative (i.e., coherence test) and qualitative approaches (i.e., researchers checked the relevance of text from each topic). We, then, grouped and characterized clustered text to each theme. We evaluated that N=18 was the ideal number of topic clusters. The 9 overarching themes were identified: Social media and TikTok-related features (N=4; "duet", "viral"), Vape shops and brands (N=3; "store"), Vape tricks (N=3; "ripsaw"), Modified use of e-cigarettes (N=1; "coil", "wire"), Vaping and girls (N=1; "girl"), Vape flavors (N=1; "flavors"), Vape and cigarettes (N=1; "smoke"), Vape identities and communities (N=1; "community"), and Non-English language (N=3; Romanian and Spanish). This study used a machine learning method, BERTopic modeling, to successfully identify relevant themes on TikTok. This method can inform future social media research examining other tobacco products, and tobacco regulatory policies such as monitoring of e-cigarette marketing on social media. This study can inform future social media research examining other tobacco products, and tobacco regulatory policies such as monitoring of e-cigarette marketing on social media.

A new risk level identification model for aviation safety

The aviation safety reporting system (ASRS) data, which are accident investigation reports issued by the national transportation safety board (NTSB), are often used for aviation risk level identification modeling. However, ASRS data suffer from sample imbalance and inaccurate encoding of textual information. To address these issues, this paper proposes a novel risk identification model. Initially, the pre-trained sentence bidirectional encoder representations from transformers (SBERT) model is fine-tuned using the textual information in the ASRS data, improving the accuracy of the text encoding. Subsequently, a generative adversarial network (GAN) is employed to generate samples for underrepresented categories in the ASRS data, thereby addressing the sample imbalance problem. Finally, a bayesian optimization algorithm is integrated to automatically search for the hyperparameters of the risk identification model, further enhancing the model's performance in risk level identification. Experimental results demonstrate that not only does the SBERT fine-tuning substantially improve the accuracy of text information encoding, but both data augmentation and automatic hyperparameter search also significantly contribute to the performance improvement of the risk level recognition model.

Automated classification of brain MRI reports using fine-tuned large language models.

This study aimed to investigate the efficacy of fine-tuned large language models (LLM) in classifying brain MRI reports into pretreatment, posttreatment, and nontumor cases. This retrospective study included 759, 284, and 164 brain MRI reports for training, validation, and test dataset. Radiologists stratified the reports into three groups: nontumor (group 1), posttreatment tumor (group 2), and pretreatment tumor (group 3) cases. A pretrained Bidirectional Encoder Representations from Transformers Japanese model was fine-tuned using the training dataset and evaluated on the validation dataset. The model which demonstrated the highest accuracy on the validation dataset was selected as the final model. Two additional radiologists were involved in classifying reports in the test datasets for the three groups. The model's performance on test dataset was compared to that of two radiologists. The fine-tuned LLM attained an overall accuracy of 0.970 (95% CI: 0.930-0.990). The model's sensitivity for group 1/2/3 was 1.000/0.864/0.978. The model's specificity for group1/2/3 was 0.991/0.993/0.958. No statistically significant differences were found in terms of accuracy, sensitivity, and specificity between the LLM and human readers (p ≥ 0.371). The LLM completed the classification task approximately 20-26-fold faster than the radiologists. The area under the receiver operating characteristic curve for discriminating groups 2 and 3 from group 1 was 0.994 (95% CI: 0.982-1.000) and for discriminating group 3 from groups 1 and 2 was 0.992 (95% CI: 0.982-1.000). Fine-tuned LLM demonstrated a comparable performance with radiologists in classifying brain MRI reports, while requiring substantially less time.

Early identification of high attention content for online mental health community users based on multi-level fusion model

PurposeThe overload of user-generated content in online mental health community makes the focus and resonance tendencies of the participating groups less clear. Thus, the purpose of this paper is to build an early identification mechanism for users' high attention content to promote early intervention and effective dissemination of professional medical guidance.Design/methodology/approachWe decouple the identification mechanism from two processes: early feature combing and algorithmic model construction. Firstly, based on the differentiated needs and concerns of the participant groups, the multiple features of “information content + source users” are refined. Secondly, a multi-level fusion model is constructed for features processing. Specifically, Bidirectional Encoder Representation from Transformers (BERT)-Bi-directional Long-Short Term Memory (BiLSTM)-Linear are used to refine the semantic features, while Graph Attention Networks (GAT) is used to capture the entity attributes and relation features. Finally, the Convolutional Neural Network (CNN) is used to optimize the multi-level fusion features.FindingsThe results show that the ACC of the multi-level fusion model is 84.42%, F1 is 79.43% and R is 76.71%. Compared with other baseline models and single feature elements, the ACC and F1 values are improved to different degrees.Originality/valueThe originality of this paper lies in analyzing multiple features based on early stages and constructing a new multi-level fusion model for processing. Further, the study is valuable for the orientation of psychological patients' needs and early guidance of professional medical care.

A method of identifying domain-specific academic user information needs based on academic Q&A communities

PurposeTo address the shortcomings of existing academic user information needs identification methods, such as low efficiency and high subjectivity, this study aims to propose an automated method of identifying online academic user information needs.Design/methodology/approachThis study’s method consists of two main parts: the first is the automatic classification of academic user information needs based on the bidirectional encoder representations from transformers (BERT) model. The second is the key content extraction of academic user information needs based on the improved MDERank key phrase extraction (KPE) algorithm. Finally, the applicability and effectiveness of the method are verified by an example of identifying the information needs of academic users in the field of materials science.FindingsExperimental results show that the BERT-based information needs classification model achieved the highest weighted average F1 score of 91.61%. The improved MDERank KPE algorithm achieves the highest F1 score of 61%. The empirical analysis results reveal that the information needs of the categories “methods,” “experimental phenomena” and “experimental materials” are relatively high in the materials science field.Originality/valueThis study provides a solution for automated identification of academic user information needs. It helps online academic resource platforms to better understand their users’ information needs, which in turn facilitates the platform’s academic resource organization and services.

Smart contract vulnerabilities detection with bidirectional encoder representations from transformers and control flow graph

Smart contract vulnerabilities detection with bidirectional encoder representations from transformers and control flow graph

Advancing predictive maintenance: a deep learning approach to sensor and event-log data fusion

PurposeThis study aims to introduce an innovative approach to predictive maintenance by integrating time-series sensor data with event logs, leveraging the synergistic potential of deep learning models. The primary goal is to enhance the accuracy of equipment failure predictions, thereby minimizing operational downtime.Design/methodology/approachThe methodology uses a dual-model architecture, combining the patch time series transformer (PatchTST) model for analyzing time-series sensor data and bidirectional encoder representations from transformers for processing textual event log data. Two distinct fusion strategies, namely, early and late fusion, are explored to integrate these data sources effectively. The early fusion approach merges data at the initial stages of processing, while late fusion combines model outputs toward the end. This research conducts thorough experiments using real-world data from wind turbines to validate the approach.FindingsThe results demonstrate a significant improvement in fault prediction accuracy, with early fusion strategies outperforming traditional methods by 2.6% to 16.9%. Late fusion strategies, while more stable, underscore the benefit of integrating diverse data types for predictive maintenance. The study provides empirical evidence of the superiority of the fusion-based methodology over singular data source approaches.Originality/valueThis research is distinguished by its novel fusion-based approach to predictive maintenance, marking a departure from conventional single-source data analysis methods. By incorporating both time-series sensor data and textual event logs, the study unveils a comprehensive and effective strategy for fault prediction, paving the way for future advancements in the field.

The Fill-Mask Association Test (FMAT): Measuring propositions in natural language.

Recent advances in large language models are enabling the computational intelligent analysis of psychology in natural language. Here, the Fill-Mask Association Test (FMAT) is introduced as a novel and integrative method leveraging Masked Language Models to study and measure psychology from a propositional perspective at the societal level. The FMAT uses Bidirectional Encoder Representations from Transformers (BERT) models to compute semantic probabilities of option words filling in the masked blank of a designed query (i.e., a clozelike contextualized sentence). The current research presents 15 studies that establish the reliability and validity of the FMAT in predicting factual associations (Studies 1A-1C), measuring attitudes/biases (Studies 2A-2D), capturing social stereotypes (Studies 3A-3D), and retrospectively delineating lay perceptions of sociocultural changes over time (Studies 4A-4D). Empirically, the FMAT replicated seminal findings previously obtained with human participants (e.g., the Implicit Association Test) and other big-data text-analytic methods (e.g., word frequency analysis, the Word Embedding Association Test), demonstrating robustness across 12 BERT model variants and diverse training text corpora. Theoretically, the current findings substantiate the propositional (vs. associative) perspective on how semantic associations are represented in natural language. Methodologically, the FMAT allows for more fine-grained language-based psychological measurement, with an R package developed to streamline its workflow for use on broader research questions. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

3MTox: A motif-level graph-based multi-view chemical language model for toxicity identification with deep interpretation

Toxicity identification plays a key role in maintaining human health, as it can alert humans to the potential hazards caused by long-term exposure to a wide variety of chemical compounds. Experimental methods for determining toxicity are time-consuming, and costly, while computational methods offer an alternative for the early identification of toxicity. For example, some classical ML and DL methods, which demonstrate excellent performance in toxicity prediction. However, these methods also have some defects, such as over-reliance on artificial features and easy overfitting, etc. Proposing novel models with superior prediction performance is still an urgent task. In this study, we propose a motifs-level graph-based multi-view pretraining language model, called 3MTox, for toxicity identification. The 3MTox model uses Bidirectional Encoder Representations from Transformers (BERT) as the backbone framework, and a motif graph as input. The results of extensive experiments showed that our 3MTox model achieved state-of-the-art performance on toxicity benchmark datasets and outperformed the baseline models considered. In addition, the interpretability of the model ensures that the it can quickly and accurately identify toxicity sites in a given molecule, thereby contributing to the determination of the status of toxicity and associated analyses. We think that the 3MTox model is among the most promising tools that are currently available for toxicity identification.

Impact of Effective Word Vectors on Deep Learning Based Subjective Classification of Online Reviews

Sentiment Analysis tasks are made considerably simpler by extracting subjective statements from online reviews, thereby reducing the overhead of the classifiers. The review dataset encompasses both subjective and objective sentences, where subjective writing expresses the author's opinions, and objective text presents factual information. Assessing the subjectivity of review statements involves categorizing them as objective or subjective. The effectiveness of word vectors plays a crucial role in this process, as they capture the semantics and contextual cues of a subjective language. This study investigates the significance of employing sophisticated word vector representations to enhance the detection of subjective reviews. Several methodologies for generating word vectors have been investigated, encompassing both conventional approaches, such as Word2Vec and Global Vectors for word representation, and recent innovations, such as like Bidirectional Encoder Representations from Transformers (BERT), ALBERT, and Embeddings from Language Models. These neural word embeddings were applied using Keras and Scikit-Learn. The analysis focuses on Cornell subjectivity review data within the restaurant domain, and metrics evaluating performance, such as accuracy, F1-score, recall, and precision, are assessed on a dataset containing subjective reviews. A wide range of conventional vector models and deep learning-based word embeddings are utilized for subjective review classification, frequently in combination with deep learning architectures like Long Short-Term Memory (LSTM). Notably, pre-trained BERT-base word embeddings exhibited exceptional accuracy of 96.4%, surpassing the performance of all other models considered in this study. It has been observed that BERT-base is expensive because of its larger structure.

Automatic construction and optimization method of enterprise data asset knowledge graph based on graph attention network

Traditional knowledge graph construction methods often rely on a large amount of human intervention and specialized knowledge, which seems inefficient and error-prone when dealing with massive, multi-source, and heterogeneous data assets of enterprises. To address this issue, a Multi Neighborhood Awareness Network (MNAN) model was proposed, which combines a multi language pre training model based on Bidirectional Encoder Representations from Transformers (BERT) and a Graph Convolutional Neural Network (GCN). By introducing an improved attention mechanism to deeply explore the neighborhood characteristics of entities, the accuracy and efficiency of entity matching are enhanced. In addition, to improve the integrity of the knowledge graph, a dynamic graph attention network (RDGAT) model based on graph attention network fusion relationship features is proposed. Through dynamic attention mechanism and relationship fusion strategy, deep learning of relationship features is carried out to optimize the completion performance of the knowledge graph spectrum. The performance test results show that the Hits@1 metric of the multi-neighborhood perceptual network model is close to 90% as the proportion of aligned entity seeds increases to 50%; the average inverse rank of the dynamic graph attention network on the test dataset improves by 6.3%. The experimental results show that the research-proposed knowledge graph fusion and complementation model exhibits significant effectiveness in automatically identifying and integrating large-scale, multi-source heterogeneous data at the enterprise level, and can significantly improve the automation level and accuracy of knowledge graph construction. The research method has achieved significant results in the data asset knowledge graph of petroleum enterprises and other fields, providing strong support for applications such as intelligent search, recommendation systems, and semantic analysis.

Early detection of pediatric health risks using maternal and child health data

Machine learning (ML)-driven diagnosis systems are particularly relevant in pediatrics given the well-documented impact of early-life health conditions on later-life outcomes. Yet, early identification of diseases and their subsequent impact on length of hospital stay for this age group has so far remained uncharacterized, likely because access to relevant health data is severely limited. Thanks to a confidential data use agreement with the California Department of Health Care Access and Information, we introduce Ped-BERT: a state-of-the-art deep learning model that accurately predicts the likelihood of 100+ conditions and the length of stay in a pediatric patient’s next medical visit. We link mother-specific pre- and postnatal period health information to pediatric patient hospital discharge and emergency room visits. Our data set comprises 513.9K mother–baby pairs and contains medical diagnosis codes, length of stay, as well as temporal and spatial pediatric patient characteristics, such as age and residency zip code at the time of visit. Following the popular bidirectional encoder representations from the transformers (BERT) approach, we pre-train Ped-BERT via the masked language modeling objective to learn embedding features for the diagnosis codes contained in our data. We then continue to fine-tune our model to accurately predict primary diagnosis outcomes and length of stay for a pediatric patient’s next visit, given the history of previous visits and, optionally, the mother’s pre- and postnatal health information. We find that Ped-BERT generally outperforms contemporary and state-of-the-art classifiers when trained with minimum features. We also find that incorporating mother health attributes leads to significant improvements in model performance overall and across all patient subgroups in our data. Our most successful Ped-BERT model configuration achieves an area under the receiver operator curve (ROC AUC) of 0.927 and an average precision score (APS) of 0.408 for the diagnosis prediction task, and a ROC AUC of 0.855 and APS of 0.815 for the length of hospital stay task. Further, we examine Ped-BERT’s fairness by determining whether prediction errors are evenly distributed across various subgroups of mother–baby demographics and health characteristics, or if certain subgroups exhibit a higher susceptibility to prediction errors.

Pedagogical sentiment analysis based on the BERT-CNN-BiGRU-attention model in the context of intercultural communication barriers.

Amid the wave of globalization, the phenomenon of cultural amalgamation has surged in frequency, bringing to the fore the heightened prominence of challenges inherent in cross-cultural communication. To address these challenges, contemporary research has shifted its focus to human-computer dialogue. Especially in the educational paradigm of human-computer dialogue, analysing emotion recognition in user dialogues is particularly important. Accurately identify and understand users' emotional tendencies and the efficiency and experience of human-computer interaction and play. This study aims to improve the capability of language emotion recognition in human-computer dialogue. It proposes a hybrid model (BCBA) based on bidirectional encoder representations from transformers (BERT), convolutional neural networks (CNN), bidirectional gated recurrent units (BiGRU), and the attention mechanism. This model leverages the BERT model to extract semantic and syntactic features from the text. Simultaneously, it integrates CNN and BiGRU networks to delve deeper into textual features, enhancing the model's proficiency in nuanced sentiment recognition. Furthermore, by introducing the attention mechanism, the model can assign different weights to words based on their emotional tendencies. This enables it to prioritize words with discernible emotional inclinations for more precise sentiment analysis. The BCBA model has achieved remarkable results in emotion recognition and classification tasks through experimental validation on two datasets. The model has significantly improved both accuracy and F1 scores, with an average accuracy of 0.84 and an average F1 score of 0.8. The confusion matrix analysis reveals a minimal classification error rate for this model. Additionally, as the number of iterations increases, the model's recall rate stabilizes at approximately 0.7. This accomplishment demonstrates the model's robust capabilities in semantic understanding and sentiment analysis and showcases its advantages in handling emotional characteristics in language expressions within a cross-cultural context. The BCBA model proposed in this study provides effective technical support for emotion recognition in human-computer dialogue, which is of great significance for building more intelligent and user-friendly human-computer interaction systems. In the future, we will continue to optimize the model's structure, improve its capability in handling complex emotions and cross-lingual emotion recognition, and explore applying the model to more practical scenarios to further promote the development and application of human-computer dialogue technology.

Detecting review fraud using metaheuristic graph neural networks

Although online reviews play a critical role in shaping consumer behavior and establishing institutional trust, this reliance on online reviews has given rise to a proliferation of fake reviews. Consequently, there is an urgent need for the development of robust mechanisms to identify and mitigate these inauthentic reviews. In this study, we introduce an innovative approach for the detection of fake reviews, leveraging a metaheuristic graph neural network model. Our methodology begins by extracting distinctive features from review text, comprising of term frequency—inverse document frequency (TF-IDF) vectors and bidirectional encoder representation transformer (BERT) embedding vectors. We feed these features into a graph convolutional network (GCN) that uses message passing to label each review. Furthermore, we address the optimization of our graph neural network models using Harris Hawk and Cat Swarm optimization techniques. Our experiments reveal that optimization of the GNN with Harris Hawk Optimization yields a significantly enhanced model performance, further validating the efficacy of our detection framework. This research contributes to the ongoing efforts in the realm of online review authenticity and showcases the potential of advanced techniques to combat the proliferation of fraudulent reviews.

Performance of Artificial Intelligence Models (Bidirectional Encoder Representations from Transformers + TextCNN) in Detecting Eight Psychiatric Diagnoses from Unstructured Texts Chinese Electronic Medical Records

Objectives: Advances in artificial intelligence (AI) have revolutionized various industries, including health care. In this study, we intended to explore the capability of AI assistants in psychiatric diagnoses. To achieve this goal, we proposed a series of deep active learning models, namely bidirectional encoder representations from transformers (BERT) – TextCNN. These models combine the strengths of two powerful techniques: BERT and convolutional neural network (CNN) for the text. Methods: We collected 21,003 Chinese psychiatry electronic medical records (EMRs) and developed two types of models: a multi-diagnosis classifier and eight single-diagnosis classifiers for schizophrenia (SCZ), major depressive disorder (MDD), manic state (MANIA), adjustment disorder (ADJ), substance use disorder (SUD), personality disorder (PD), attention-deficit/hyperactivity disorder (ADHD), and autistic spectrum disorder (ASD). Their performance was compared through plotting receiver operating characteristic curves and assessing the performance, area under curve (AUC) using the DeLong test. Results: This study showed the excellent performance of our BERT + TextCNN models in detecting almost all eight psychiatric diagnoses, achieving AUCs being greater than 0.9, except for the single-diagnosis classifier for ADHD (AUC = 0.83). Conclusion: This study highlights the promising applicability of the BERT + TextCNN model as a diagnostic assistant for psychiatry diagnoses derived from EMRs. Being consistent with previous findings, the single-diagnosis classifiers generally outperform the multi-diagnosis classifier in predicting most diagnoses, though not all. Further studies are warranted to confirm whether the specific characteristics of illnesses contribute to the performance gap between multi- and single-diagnosis classifiers.

Advancing smart tourism destinations: A case study using bidirectional encoder representations from transformers‐based occupancy predictions in torrevieja (Spain)

AbstractTourism represents a crucial socio‐economic pillar globally, yet the multifaceted challenges it poses necessitate innovative management approaches. The paradigm of smart tourism harnesses advanced data analytics tools to promote both profitability and sustainability in tourist destinations, leading to new levels of destination smartness. Accurate tourist occupancy prediction, particularly in areas dominated by second‐home accommodations where traditional hospitality data may be insufficient, plays a key role in optimising tourism management. To address this data gap, our prior research employed ARIMA modelling on Airbnb booking time series and analysed tourism‐related Twitter conversations to forecast occupancy levels in Torrevieja (Alicante); a prominent second‐home tourism destination in Southeastern Spain. In this extended study, we delve deeper into the realm of social sensing by utilising bidirectional encoder representations from transformers (BERT) for topic modelling. Our methodology involves the processing and analysis of Twitter data to identify prominent themes related to Torrevieja. The findings not only reveal nuanced perceptions and discussions about the destination but also underscore the effectiveness of BERT in capturing intricate topic dynamics. Importantly, this work highlights how the alignment of specific topics with booking patterns can further enhance predictive accuracy for tourist occupancy, presenting a robust toolkit for stakeholders in the tourism sector.