Accelerate Literature Icon
Want to do a literature review? Try our new Literature Review workflow

Comparison of Artificial Intelligence Tools With Human Coding for Sentiment, Topic, and Thematic Analysis Tasks of Public Health Datasets During the COVID-19 Pandemic in Australia: Case Study.

  • Abstract
  • Literature Map
  • Similar Papers
Abstract
Translate article icon Translate Article Star icon

Public opinion, which may be influenced by personal experiences, news, and social media, can impact compliance with public health measures (PHMs) during health emergencies. Artificial intelligence (AI) tools offer opportunities to analyze public opinion in real time during health emergencies. However, their performance in accurately identifying sentiment and themes in health-related online content remains unclear. This study aimed to evaluate the performance of natural language processing-based and large language model (LLM)-based AI tools when compared to human coding for sentiment analysis, topic modeling, and thematic analysis of public health datasets. Tools were selected to reflect those available to public health analysts and decision-makers. Data were collected via Google Alerts (GA) and social media posts from X (formerly known as Twitter) relevant to COVID-19 mitigation PHMs from December 2022 to February 2023. Following relevance screening, the sentiment of the complete datasets was analyzed by a human rater, with descriptive statistics used to summarize the overall sentiment profile. Subsets of 400 GA articles and 400 tweets were manually coded for sentiment by 2 human raters. Results were compared with outputs from 5 AI tools, including VADER (Valence Aware Dictionary and Sentiment Reasoner), SentimentGI, SentimentQDAP, Microsoft Azure, and OpenAI's ChatGPT-4. Topic modeling of the GA and X datasets was conducted using latent Dirichlet allocation in R and zero-shot prompting in ChatGPT-4 and compared with manual topic summaries. Thematic analysis of positive and negative sentiment datasets was conducted by a human rater and ChatGPT-4, with outputs evaluated for proficiency and reasonableness. The sentiment of the entire datasets was analyzed by a human rater, and descriptive statistics were calculated. Of 2227 GA results and 3484 tweets, 58% (n=1238) and 71% (n=2473), respectively, were relevant to PHMs. Human-coded sentiment analysis showed mostly neutral reporting in the news media, while social media expressed more polarized views. Across both datasets, AI tools demonstrated poor concordance with human-coded sentiment (Cohen κ <0.5 for all tools and sentiment categories). Topic modeling with ChatGPT-4 aligned more closely with human-rated topics than latent Dirichlet allocation, and of the 20 LLM-generated thematic outputs, 13 were rated proficient, and 7 were rated partially proficient. LLM outputs provided coherent, high-level summaries but lacked contextual insight. Human and LLM thematic analyses both identified themes of vaccine effectiveness, debate regarding PHMs, and public trust. Accessible AI tools demonstrate limited reliability for sentiment classification of health-related online text but show promise for rapid thematic exploration when combined with human oversight. These tools could complement traditional qualitative research in the context of health emergencies; however, they require human review to enhance the accuracy of interpretation. Further research is needed for non-English datasets.

Similar Papers
  • Research Article
  • Cite Count Icon 73
  • 10.1016/j.jclinepi.2025.111746
Large language models for conducting systematic reviews: on the rise, but not yet ready for use-a scoping review.
  • May 1, 2025
  • Journal of clinical epidemiology
  • Judith-Lisa Lieberum + 8 more

Machine learning promises versatile help in the creation of systematic reviews (SRs). Recently, further developments in the form of large language models (LLMs) and their application in SR conduct attracted attention. We aimed at providing an overview of LLM applications in SR conduct in health research. We systematically searched MEDLINE, Web of Science, IEEEXplore, ACM Digital Library, Europe PMC (preprints), Google Scholar, and conducted an additional hand search (last search: February 26, 2024). We included scientific articles in English or German, published from April 2021 onwards, building upon the results of a mapping review that has not yet identified LLM applications to support SRs. Two reviewers independently screened studies for eligibility; after piloting, 1 reviewer extracted data, checked by another. Our database search yielded 8054 hits, and we identified 33 articles from our hand search. We finally included 37 articles on LLM support. LLM approaches covered 10 of 13 defined SR steps, most frequently literature search (n = 15, 41%), study selection (n = 14, 38%), and data extraction (n = 11, 30%). The mostly recurring LLM was Generative Pretrained Transformer (GPT) (n = 33, 89%). Validation studies were predominant (n = 21, 57%). In half of the studies, authors evaluated LLM use as promising (n = 20, 54%), one-quarter as neutral (n = 9, 24%) and one-fifth as nonpromising (n = 8, 22%). Although LLMs show promise in supporting SR creation, fully established or validated applications are often lacking. The rapid increase in research on LLMs for evidence synthesis production highlights their growing relevance. Systematic reviews are a crucial tool in health research where experts carefully collect and analyze all available evidence on a specific research question. Creating these reviews is typically time- and resource-intensive, often taking months or even years to complete, as researchers must thoroughly search, evaluate, and synthesize an immense number of scientific studies. For the present article, we conducted a review to understand how new artificial intelligence (AI) tools, specifically large language models (LLMs) like Generative Pretrained Transformer (GPT), can be used to help create systematic reviews in health research. We searched multiple scientific databases and finally found 37 relevant articles. We found that LLMs have been tested to help with various parts of the systematic review process, particularly in 3 main areas: searching scientific literature (41% of studies), selecting relevant studies (38%), and extracting important information from these studies (30%). GPT was the most commonly used LLM, appearing in 89% of the studies. Most of the research (57%) focused on testing whether these AI tools actually work as intended in this context of systematic review production. The results were mixed: about half of the studies found LLMs promising, a quarter were neutral, and one-fifth found them not promising. While LLMs show potential for making the systematic review process more efficient, there is still a lack of fully tested and validated applications. However, the increasing number of studies in this field suggests that these AI tools are becoming increasingly important in creating systematic reviews.

  • Preprint Article
  • 10.2196/preprints.70128
Patient Voices in Dialysis Care: Sentiment Analysis and Topic Modeling Study of Social Media Discourse (Preprint)
  • Dec 16, 2024
  • Ravi Shankar + 2 more

BACKGROUND Patients with end-stage kidney disease undergoing dialysis face significant physical, psychological, and social challenges that impact their quality of life. Social media platforms such as X (formerly known as Twitter) have become important outlets for these patients to share experiences and exchange information. OBJECTIVE This study aimed to uncover key themes, emotions, and challenges expressed by the dialysis community on X from April 2006 to August 2024 by leveraging natural language processing techniques, specifically sentiment analysis and topic modeling. METHODS We collected 12,976 publicly available X posts related to dialysis using the platform’s application programming interface version 2 and Python’s Tweepy library. After rigorous preprocessing, 58.13% (7543/12,976) of the posts were retained for analysis. Sentiment analysis using the Valence Aware Dictionary and Sentiment Reasoner (VADER) model, which is a rule-based sentiment analyzer specifically attuned to social media content, classified the emotional tone of posts. VADER uses a human-curated lexicon that maps lexical features to sentiment scores, considering punctuation, capitalization, and modifiers. For topic modeling, posts with &amp;lt;50 tokens were removed, leaving 53.81% (4059/7543) of the posts, which were analyzed using latent Dirichlet allocation with coherence score optimization to identify the optimal number of topics (&lt;i&gt;k&lt;/i&gt;=8). The analysis pipeline was implemented using Python’s Natural Language Toolkit, Gensim, and scikit-learn libraries, with hyperparameter tuning to maximize model performance. RESULTS Sentiment analysis revealed 49.2% (3711/7543) positive, 26.2% (1976/7543) negative, and 24.7% (1863/7543) neutral sentiment posts. Latent Dirichlet allocation topic modeling identified 8 key thematic clusters: medical procedures and outcomes (722/4059, 17.8% prevalence), daily life impact (666/4059, 16.4%), risks and complications (621/4059, 15.3%), patient education and support (544/4059, 13.4%), health care access and costs (499/4059, 12.3%), symptoms and side effects (442/4059, 10.9%), patient experiences and socioeconomic challenges (406/4059, 10%), and diet and fluid management (162/4059, 4%). Cross-analysis of topics and sentiment revealed that negative sentiment was highest for daily life impact (580/666, 87.1%) and socioeconomic challenges (145/406, 35.8%), whereas the education and support topic exhibited more positive sentiment (250/544, 46%). Topic coherence scores ranged from 0.38 to 0.52, with the medical procedures topic showing the highest semantic coherence. Intertopic distance mapping via multidimensional scaling revealed conceptual relationships between identified themes, with lifestyle impact and socioeconomic challenges clustering closely. Our longitudinal analysis demonstrated evolving discourse patterns, with technology-related discussions increasing by 24% in recent years, whereas financial concerns remained consistently prominent. CONCLUSIONS This study provides a comprehensive, data-driven understanding of the complex lived experiences of patients undergoing dialysis shared on social media. The findings underscore the need for more holistic, patient-centered care models and policies that address the multidimensional challenges illuminated by patients’ voices.

  • PDF Download Icon
  • Research Article
  • Cite Count Icon 29
  • 10.2196/47826
Using Natural Language Processing to Explore Social Media Opinions on Food Security: Sentiment Analysis and Topic Modeling Study
  • Mar 21, 2024
  • Journal of Medical Internet Research
  • Annika Molenaar + 4 more

BackgroundSocial media has the potential to be of great value in understanding patterns in public health using large-scale analysis approaches (eg, data science and natural language processing [NLP]), 2 of which have been used in public health: sentiment analysis and topic modeling; however, their use in the area of food security and public health nutrition is limited.ObjectiveThis study aims to explore the potential use of NLP tools to gather insights from real-world social media data on the public health issue of food security.MethodsA search strategy for obtaining tweets was developed using food security terms. Tweets were collected using the Twitter application programming interface from January 1, 2019, to December 31, 2021, filtered for Australia-based users only. Sentiment analysis of the tweets was performed using the Valence Aware Dictionary and Sentiment Reasoner. Topic modeling exploring the content of tweets was conducted using latent Dirichlet allocation with BigML (BigML, Inc). Sentiment, topic, and engagement (the sum of likes, retweets, quotations, and replies) were compared across years.ResultsIn total, 38,070 tweets were collected from 14,880 Twitter users. Overall, the sentiment when discussing food security was positive, although this varied across the 3 years. Positive sentiment remained higher during the COVID-19 lockdown periods in Australia. The topic model contained 10 topics (in order from highest to lowest probability in the data set): “Global production,” “Food insecurity and health,” “Use of food banks,” “Giving to food banks,” “Family poverty,” “Food relief provision,” “Global food insecurity,” “Climate change,” “Australian food insecurity,” and “Human rights.” The topic “Giving to food banks,” which focused on support and donation, had the highest proportion of positive sentiment, and “Global food insecurity,” which covered food insecurity prevalence worldwide, had the highest proportion of negative sentiment. When compared with news, there were some events, such as COVID-19 support payment introduction and bushfires across Australia, that were associated with high periods of positive or negative sentiment. Topics related to food insecurity prevalence, poverty, and food relief in Australia were not consistently more prominent during the COVID-19 pandemic than before the pandemic. Negative tweets received substantially higher engagement across 2019 and 2020. There was no clear relationship between topics that were more likely to be positive or negative and have higher or lower engagement, indicating that the identified topics are discrete issues.ConclusionsIn this study, we demonstrated the potential use of sentiment analysis and topic modeling to explore evolution in conversations on food security using social media data. Future use of NLP in food security requires the context of and interpretation by public health experts and the use of broader data sets, with the potential to track dimensions or events related to food security to inform evidence-based decision-making in this area.

  • Research Article
  • Cite Count Icon 1
  • 10.12688/mep.19911.3
Developing a clinician-friendly rubric for assessing history-taking skills in medical undergraduates speaking English as a foreign language.
  • Jan 1, 2024
  • MedEdPublish (2016)
  • Takayuki Oshimi

ChatGPT is a large language model that uses deep learning techniques to generate human-like texts. ChatGPT has the potential to revolutionize medical education as it acts as an interactive virtual tutor and personalized learning assistant. We assessed the use of ChatGPT and other Artificial Intelligence (AI) tools among medical faculty in Uganda. We conducted a descriptive cross-sectional study among medical faculty at four public universities in Uganda from November to December 2023. Participants were recruited consecutively. We used a semi-structured questionnaire to collect data on participants' sociodemographics and the use of AI tools such as ChatGPT. Our outcome variable was the use of ChatGPT and other AI tools. Data were analyzed in Stata version 17.0. We recruited 224 medical faculty, majority [75% (167/224)] were male. The median age (interquartile range) was 41 years (34-50). Almost all medical faculty [90% (202/224)] had ever heard of AI tools such as ChatGPT. Over 63% (120/224) of faculty had ever used AI tools. The most commonly used AI tools were ChatGPT (56.3%) and Quill Bot (7.1%). Fifty-six faculty use AI tools for research writing, 37 for summarizing information, 28 for proofreading work, and 28 for setting exams or assignments. Forty faculty use AI tools for nonacademic purposes like recreation and learning new skills. Faculty older than 50 years were 40% less likely to use AI tools compared to those aged 24 to 35 years (Adjusted Prevalence Ratio (aPR):0.60; 95% Confidence Interval (CI): [0.45, 0.80]). The use of ChatGPT and other AI tools was high among medical faculty in Uganda. Older faculty (>50 years) were less likely to use AI tools compared to younger faculty. Training on AI use in education, formal policies, and guidelines are needed to adequately prepare medical faculty for the integration of AI in medical education.

  • Research Article
  • Cite Count Icon 1
  • 10.21956/mep.21327.r36689
Developing a clinician-friendly rubric for assessing history-taking skills in medical undergraduates speaking English as a foreign language
  • May 14, 2024
  • MedEdPublish
  • Alexandra Lopez Vera

BackgroundChatGPT is a large language model that uses deep learning techniques to generate human-like texts. ChatGPT has the potential to revolutionize medical education as it acts as an interactive virtual tutor and personalized learning assistant. We assessed the use of ChatGPT and other Artificial Intelligence (AI) tools among medical faculty in Uganda.MethodsWe conducted a descriptive cross-sectional study among medical faculty at four public universities in Uganda from November to December 2023. Participants were recruited consecutively. We used a semi-structured questionnaire to collect data on participants’ sociodemographics and the use of AI tools such as ChatGPT. Our outcome variable was the use of ChatGPT and other AI tools. Data were analyzed in Stata version 17.0.ResultsWe recruited 224 medical faculty, majority [75% (167/224)] were male. The median age (interquartile range) was 41 years (34–50). Almost all medical faculty [90% (202/224)] had ever heard of AI tools such as ChatGPT. Over 63% (120/224) of faculty had ever used AI tools. The most commonly used AI tools were ChatGPT (56.3%) and Quill Bot (7.1%). Fifty-six faculty use AI tools for research writing, 37 for summarizing information, 28 for proofreading work, and 28 for setting exams or assignments. Forty faculty use AI tools for nonacademic purposes like recreation and learning new skills. Faculty older than 50 years were 40% less likely to use AI tools compared to those aged 24 to 35 years (Adjusted Prevalence Ratio (aPR):0.60; 95% Confidence Interval (CI): [0.45, 0.80]).ConclusionsThe use of ChatGPT and other AI tools was high among medical faculty in Uganda. Older faculty (>50 years) were less likely to use AI tools compared to younger faculty. Training on AI use in education, formal policies, and guidelines are needed to adequately prepare medical faculty for the integration of AI in medical education.

  • Research Article
  • Cite Count Icon 2
  • 10.12688/mep.20554.3
Utilisation of ChatGPT and other Artificial Intelligence tools among medical faculty in Uganda: a cross-sectional study.
  • Apr 28, 2025
  • MedEdPublish (2016)
  • David Mukunya + 18 more

ChatGPT is a large language model that uses deep learning techniques to generate human-like texts. ChatGPT has the potential to revolutionize medical education as it acts as an interactive virtual tutor and personalized learning assistant. We assessed the use of ChatGPT and other Artificial Intelligence (AI) tools among medical faculty in Uganda. We conducted a descriptive cross-sectional study among medical faculty at four public universities in Uganda from November to December 2023. Participants were recruited consecutively. We used a semi-structured questionnaire to collect data on participants' socio-demographics and the use of AI tools such as ChatGPT. Our outcome variable was the use of ChatGPT and other AI tools. Data were analyzed in Stata version 17.0. We recruited 224 medical faculty, majority [75% (167/224)] were male. The median age (interquartile range) was 41 years (34-50). Almost all medical faculty [90% (202/224)] had ever heard of AI tools such as ChatGPT. Over 63% (120/224) of faculty had ever used AI tools. The most commonly used AI tools were ChatGPT (56.3%) and Quill Bot (7.1%). Fifty-six faculty use AI tools for research writing, 37 for summarizing information, 28 for proofreading work, and 28 for setting exams or assignments. Forty faculty use AI tools for non-academic purposes like recreation and learning new skills. Faculty older than 50 years were 40% less likely to use AI tools compared to those aged 24 to 35 years (Adjusted Prevalence Ratio (aPR):0.60; 95% Confidence Interval (CI): [0.45, 0.80]). The use of ChatGPT and other AI tools was high among medical faculty in Uganda. Older faculty (>50 years) were less likely to use AI tools compared to younger faculty. Training on AI use in education, formal policies, and guidelines are needed to adequately prepare medical faculty for the integration of AI in medical education.

  • Research Article
  • 10.12688/mep.20554.2
Utilisation of ChatGPT and other Artificial Intelligence tools among medical faculty in Uganda: a cross-sectional study.
  • Jan 23, 2025
  • MedEdPublish (2016)
  • David Mukunya + 18 more

ChatGPT is a large language model that uses deep learning techniques to generate human-like texts. ChatGPT has the potential to revolutionize medical education as it acts as an interactive virtual tutor and personalized learning assistant. We assessed the use of ChatGPT and other Artificial Intelligence (AI) tools among medical faculty in Uganda. We conducted a descriptive cross-sectional study among medical faculty at four public universities in Uganda from November to December 2023. Participants were recruited consecutively. We used a semi-structured questionnaire to collect data on participants' socio-demographics and the use of AI tools such as ChatGPT. Our outcome variable was the use of ChatGPT and other AI tools. Data were analyzed in Stata version 17.0. We recruited 224 medical faculty, majority [75% (167/224)] were male. The median age (interquartile range) was 41 years (34-50). Almost all medical faculty [90% (202/224)] had ever heard of AI tools such as ChatGPT. Over 63% (120/224) of faculty had ever used AI tools. The most commonly used AI tools were ChatGPT (56.3%) and Quill Bot (7.1%). Fifty-six faculty use AI tools for research writing, 37 for summarizing information, 28 for proofreading work, and 28 for setting exams or assignments. Forty faculty use AI tools for non-academic purposes like recreation and learning new skills. Faculty older than 50 years were 40% less likely to use AI tools compared to those aged 24 to 35 years (Adjusted Prevalence Ratio (aPR):0.60; 95% Confidence Interval (CI): [0.45, 0.80]). The use of ChatGPT and other AI tools was high among medical faculty in Uganda. Older faculty (>50 years) were less likely to use AI tools compared to younger faculty. Training on AI use in education, formal policies, and guidelines are needed to adequately prepare medical faculty for the integration of AI in medical education.

  • Research Article
  • Cite Count Icon 56
  • 10.5204/mcj.3004
ChatGPT Isn't Magic
  • Oct 2, 2023
  • M/C Journal
  • Tama Leaver + 1 more

Introduction Author Arthur C. Clarke famously argued that in science fiction literature “any sufficiently advanced technology is indistinguishable from magic” (Clarke). On 30 November 2022, technology company OpenAI publicly released their Large Language Model (LLM)-based chatbot ChatGPT (Chat Generative Pre-Trained Transformer), and instantly it was hailed as world-changing. Initial media stories about ChatGPT highlighted the speed with which it generated new material as evidence that this tool might be both genuinely creative and actually intelligent, in both exciting and disturbing ways. Indeed, ChatGPT is part of a larger pool of Generative Artificial Intelligence (AI) tools that can very quickly generate seemingly novel outputs in a variety of media formats based on text prompts written by users. Yet, claims that AI has become sentient, or has even reached a recognisable level of general intelligence, remain in the realm of science fiction, for now at least (Leaver). That has not stopped technology companies, scientists, and others from suggesting that super-smart AI is just around the corner. Exemplifying this, the same people creating generative AI are also vocal signatories of public letters that ostensibly call for a temporary halt in AI development, but these letters are simultaneously feeding the myth that these tools are so powerful that they are the early form of imminent super-intelligent machines. For many people, the combination of AI technologies and media hype means generative AIs are basically magical insomuch as their workings seem impenetrable, and their existence could ostensibly change the world. This article explores how the hype around ChatGPT and generative AI was deployed across the first six months of 2023, and how these technologies were positioned as either utopian or dystopian, always seemingly magical, but never banal. We look at some initial responses to generative AI, ranging from schools in Australia to picket lines in Hollywood. We offer a critique of the utopian/dystopian binary positioning of generative AI, aligning with critics who rightly argue that focussing on these extremes displaces the more grounded and immediate challenges generative AI bring that need urgent answers. Finally, we loop back to the role of schools and educators in repositioning generative AI as something to be tested, examined, scrutinised, and played with both to ground understandings of generative AI, while also preparing today’s students for a future where these tools will be part of their work and cultural landscapes. Hype, Schools, and Hollywood In December 2022, one month after OpenAI launched ChatGPT, Elon Musk tweeted: “ChatGPT is scary good. We are not far from dangerously strong AI”. Musk’s post was retweeted 9400 times, liked 73 thousand times, and presumably seen by most of his 150 million Twitter followers. This type of engagement typified the early hype and language that surrounded the launch of ChatGPT, with reports that “crypto” had been replaced by generative AI as the “hot tech topic” and hopes that it would be “‘transformative’ for business” (Browne). By March 2023, global economic analysts at Goldman Sachs had released a report on the potentially transformative effects of generative AI, saying that it marked the “brink of a rapid acceleration in task automation that will drive labor cost savings and raise productivity” (Hatzius et al.). Further, they concluded that “its ability to generate content that is indistinguishable from human-created output and to break down communication barriers between humans and machines reflects a major advancement with potentially large macroeconomic effects” (Hatzius et al.). Speculation about the potentially transformative power and reach of generative AI technology was reinforced by warnings that it could also lead to “significant disruption” of the labour market, and the potential automation of up to 300 million jobs, with associated job losses for humans (Hatzius et al.). In addition, there was widespread buzz that ChatGPT’s “rationalization process may evidence human-like cognition” (Browne), claims that were supported by the emergent language of ChatGPT. The technology was explained as being “trained” on a “corpus” of datasets, using a “neural network” capable of producing “natural language“” (Dsouza), positioning the technology as human-like, and more than ‘artificial’ intelligence. Incorrect responses or errors produced by the tech were termed “hallucinations”, akin to magical thinking, which OpenAI founder Sam Altman insisted wasn’t a word that he associated with sentience (Intelligencer staff). Indeed, Altman asserts that he rejects moves to “anthropomorphize” (Intelligencer staff) the technology; however, arguably the language, hype, and Altman’s well-publicised misgivings about ChatGPT have had the combined effect of shaping our understanding of this generative AI as alive, vast, fast-moving, and potentially lethal to humanity. Unsurprisingly, the hype around the transformative effects of ChatGPT and its ability to generate ‘human-like’ answers and sophisticated essay-style responses was matched by a concomitant panic throughout educational institutions. The beginning of the 2023 Australian school year was marked by schools and state education ministers meeting to discuss the emerging problem of ChatGPT in the education system (Hiatt). Every state in Australia, bar South Australia, banned the use of the technology in public schools, with a “national expert task force” formed to “guide” schools on how to navigate ChatGPT in the classroom (Hiatt). Globally, schools banned the technology amid fears that students could use it to generate convincing essay responses whose plagiarism would be undetectable with current software (Clarence-Smith). Some schools banned the technology citing concerns that it would have a “negative impact on student learning”, while others cited its “lack of reliable safeguards preventing these tools exposing students to potentially explicit and harmful content” (Cassidy). ChatGPT investor Musk famously tweeted, “It’s a new world. Goodbye homework!”, further fuelling the growing alarm about the freely available technology that could “churn out convincing essays which can't be detected by their existing anti-plagiarism software” (Clarence-Smith). Universities were reported to be moving towards more “in-person supervision and increased paper assessments” (SBS), rather than essay-style assessments, in a bid to out-manoeuvre ChatGPT’s plagiarism potential. Seven months on, concerns about the technology seem to have been dialled back, with educators more curious about the ways the technology can be integrated into the classroom to good effect (Liu et al.); however, the full implications and impacts of the generative AI are still emerging. In May 2023, the Writer’s Guild of America (WGA), the union representing screenwriters across the US creative industries, went on strike, and one of their core issues were “regulations on the use of artificial intelligence in writing” (Porter). Early in the negotiations, Chris Keyser, co-chair of the WGA’s negotiating committee, lamented that “no one knows exactly what AI’s going to be, but the fact that the companies won’t talk about it is the best indication we’ve had that we have a reason to fear it” (Grobar). At the same time, the Screen Actors’ Guild (SAG) warned that members were being asked to agree to contracts that stipulated that an actor’s voice could be re-used in future scenarios without that actor’s additional consent, potentially reducing actors to a dataset to be animated by generative AI technologies (Scheiber and Koblin). In a statement issued by SAG, they made their position clear that the creation or (re)animation of any digital likeness of any part of an actor must be recognised as labour and properly paid, also warning that any attempt to legislate around these rights should be strongly resisted (Screen Actors Guild). Unlike the more sensationalised hype, the WGA and SAG responses to generative AI are grounded in labour relations. These unions quite rightly fear the immediate future where human labour could be augmented, reclassified, and exploited by, and in the name of, algorithmic systems. Screenwriters, for example, might be hired at much lower pay rates to edit scripts first generated by ChatGPT, even if those editors would really be doing most of the creative work to turn something clichéd and predictable into something more appealing. Rather than a dystopian world where machines do all the work, the WGA and SAG protests railed against a world where workers would be paid less because executives could pretend generative AI was doing most of the work (Bender). The Open Letter and Promotion of AI Panic In an open letter that received enormous press and media uptake, many of the leading figures in AI called for a pause in AI development since “advanced AI could represent a profound change in the history of life on Earth”; they warned early 2023 had already seen “an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control” (Future of Life Institute). Further, the open letter signatories called on “all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4”, arguing that “labs and independent experts should use this pause to jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts” (Future of Life Institute). Notably, many of the signatories work for the very companies involved in the “out-of-control race”. Indeed, while this letter could be read as a moment of ethical clarity for the AI industry, a more cynical reading might just be that in warning that their AIs could effectively destroy the w

  • Research Article
  • Cite Count Icon 293
  • 10.1007/s13278-019-0568-8
Topic modeling and sentiment analysis of global climate change tweets
  • Jun 10, 2019
  • Social Network Analysis and Mining
  • Biraj Dahal + 2 more

Social media websites can be used as a data source for mining public opinion on a variety of subjects including climate change. Twitter, in particular, allows for the evaluation of public opinion across both time and space because geotagged tweets include timestamps and geographic coordinates (latitude/longitude). In this study, a large dataset of geotagged tweets containing certain keywords relating to climate change is analyzed using volume analysis and text mining techniques such as topic modeling and sentiment analysis. Latent Dirichlet allocation was applied for topic modeling to infer the different topics of discussion, and Valence Aware Dictionary and sEntiment Reasoner was applied for sentiment analysis to determine the overall feelings and attitudes found in the dataset. These techniques are used to compare and contrast the nature of climate change discussion between different countries and over time. Sentiment analysis shows that the overall discussion is negative, especially when users are reacting to political or extreme weather events. Topic modeling shows that the different topics of discussion on climate change are diverse, but some topics are more prevalent than others. In particular, the discussion of climate change in the USA is less focused on policy-related topics than other countries.

  • Research Article
  • Cite Count Icon 3
  • 10.2196/70128
Patient Voices in Dialysis Care: Sentiment Analysis and Topic Modeling Study of Social Media Discourse
  • May 15, 2025
  • Journal of Medical Internet Research
  • Ravi Shankar + 2 more

BackgroundPatients with end-stage kidney disease undergoing dialysis face significant physical, psychological, and social challenges that impact their quality of life. Social media platforms such as X (formerly known as Twitter) have become important outlets for these patients to share experiences and exchange information.ObjectiveThis study aimed to uncover key themes, emotions, and challenges expressed by the dialysis community on X from April 2006 to August 2024 by leveraging natural language processing techniques, specifically sentiment analysis and topic modeling.MethodsWe collected 12,976 publicly available X posts related to dialysis using the platform’s application programming interface version 2 and Python’s Tweepy library. After rigorous preprocessing, 58.13% (7543/12,976) of the posts were retained for analysis. Sentiment analysis using the Valence Aware Dictionary and Sentiment Reasoner (VADER) model, which is a rule-based sentiment analyzer specifically attuned to social media content, classified the emotional tone of posts. VADER uses a human-curated lexicon that maps lexical features to sentiment scores, considering punctuation, capitalization, and modifiers. For topic modeling, posts with <50 tokens were removed, leaving 53.81% (4059/7543) of the posts, which were analyzed using latent Dirichlet allocation with coherence score optimization to identify the optimal number of topics (k=8). The analysis pipeline was implemented using Python’s Natural Language Toolkit, Gensim, and scikit-learn libraries, with hyperparameter tuning to maximize model performance.ResultsSentiment analysis revealed 49.2% (3711/7543) positive, 26.2% (1976/7543) negative, and 24.7% (1863/7543) neutral sentiment posts. Latent Dirichlet allocation topic modeling identified 8 key thematic clusters: medical procedures and outcomes (722/4059, 17.8% prevalence), daily life impact (666/4059, 16.4%), risks and complications (621/4059, 15.3%), patient education and support (544/4059, 13.4%), health care access and costs (499/4059, 12.3%), symptoms and side effects (442/4059, 10.9%), patient experiences and socioeconomic challenges (406/4059, 10%), and diet and fluid management (162/4059, 4%). Cross-analysis of topics and sentiment revealed that negative sentiment was highest for daily life impact (580/666, 87.1%) and socioeconomic challenges (145/406, 35.8%), whereas the education and support topic exhibited more positive sentiment (250/544, 46%). Topic coherence scores ranged from 0.38 to 0.52, with the medical procedures topic showing the highest semantic coherence. Intertopic distance mapping via multidimensional scaling revealed conceptual relationships between identified themes, with lifestyle impact and socioeconomic challenges clustering closely. Our longitudinal analysis demonstrated evolving discourse patterns, with technology-related discussions increasing by 24% in recent years, whereas financial concerns remained consistently prominent.ConclusionsThis study provides a comprehensive, data-driven understanding of the complex lived experiences of patients undergoing dialysis shared on social media. The findings underscore the need for more holistic, patient-centered care models and policies that address the multidimensional challenges illuminated by patients’ voices.

  • Research Article
  • Cite Count Icon 4
  • 10.1007/s13278-025-01447-6
Longitudinal analysis of Covid-19 vaccine-related tweets in india: linking sentiment fluctuations with topic modeling
  • Apr 1, 2025
  • Social Network Analysis and Mining
  • Seba Susan + 3 more

Twitter (now known as “X”) is a popular medium for Covid-19 related discussions. This paper presents a novel case study on sentiment analysis and topic modeling of Covid-19 vaccine-related tweets of users geo-located in India in the duration of 12 December 2020 to 11 November 2021 in the course of which more than half of the country’s 1.3 billion population got vaccinated. The sentiment analysis was performed, on day-wise basis, using unsupervised lexicon-driven sentiment analysis tools AFINN and Valence Aware Dictionary and sEntiment Reasoner, as well as BERTweet and Covid-Twitter-BERT transformer models pre-trained on Covid-19 tweets. The models were comparatively evaluated on a smaller annotated dataset of Covid-19 vaccine-related tweets prior to the longitudinal analysis. AFINN was ultimately chosen due to its better performance and ease of use for large unannotated data. AFINN analysis revealed that 51.38% of the 44,130 tweets were neutral, while 38.84% were positive, and 9.78% negative. Latent Dirichlet Allocation was used for topic modeling at the peak points corresponding to large sentiment fluctuations in the positive and negative longitudinal graphs derived using AFINN. The positive and negative vocabularies at peak points were scrutinized to derive insights on national/international events that triggered a change in public opinion. These findings could guide policy makers in gathering intelligence on misinformation and associated sentiments, and planning counter-measures to combat anti-vaccine campaigns. This study informs future strategies to counter vaccine hesitancy through targeted communication aimed at vulnerable groups that experience high anxiety and psychosocial burden during the pandemic.

  • Journal Issue
  • 10.57234/jiimc.december24.2361
A Framework for Ethical Integration of Artificial Intelligence Tools in Medical Research and Writing
  • Dec 27, 2024
  • Journal of Islamic International Medical College
  • Fareeha Farooq , Farooq Azam Rathore

The global landscape of medical research and writing over the last decade has been evolving rapidly.It got a major boost by the public launch of ChatGPT in November 2022.It has been further accelerated due to the widespread availability and easy access to a variety of artificial intelligence (AI) tools based on 1 Large Language Models (LLMs).The use of AI tools in medical research and writing is on the rise.However, many students, faculty members and researchers particularly in developing countries are still unclear on the transparent and ethical integration of the AI tools in research and writing.We propose a framework for the appropriate integration of AI in medical research and writing in context of a developing country like Pakistan.It highlights responsible and transparent use of AI tools, discuss authorship in the era of AI tools and how to ensure scientific integrity. How AI Tools can Enhance Research and WritingAI tools can help researchers with various steps of research and writing like brainstorming and refining research questions, creating questionnaires, detailed data analysis, creating outlines of the manuscripts and improving the readability by enhancing the language, grammar and syntax of the 2 text.This can potentially improve the efficiency and productivity of the researchers.However, there are some ethical challenges unique to the use of these AI tools that need consideration.It is important to establish and follow a framework for responsible and ethical use of AI in research and writing that promotes transparency, upholds the integrity of the research and maintains public trust in research 3 findings. Brainstorming Ideas and Creating Research QuestionsThe first step in a research project is conceptualizing research goals and brainstorming ideas.Before using AI tools, researchers must understand questions they aim to answer, and the potential impact it can create.This clarity helps in ensuring that AI tools will complement, not dictate, the direction of the research.This step should not be totally delegated to an AI tool.Any LLM can be used to generate and brainstorm a research question using appropriate p ro m p t s .A I t o o l s l i ke " Re s e a rc h k i c k " (https://www.researchkick.com/) are specifically created to brainstorm ideas and refine research questions.

  • Research Article
  • Cite Count Icon 30
  • 10.1177/27526461231215083
To use or not to use ChatGPT and assistive artificial intelligence tools in higher education institutions? The modern-day conundrum – students’ and faculty’s perspectives
  • Nov 11, 2023
  • Equity in Education &amp; Society
  • Charmaine Bissessar

Students’ use of Artificial Intelligence tools to complete assignments spawns issues in academic integrity. The purpose of this study was to explore students’ and faculty’s perspectives on the benefits and challenges of using ChatGPT and assistive Artificial intelligence (AI) tools to complete assignments. This descriptive phenomenological qualitative methodology study encompassed interviews with eight students who used Large Language Models (LLMs) AI tools to complete their assignments and nine students who did not. It also contains interviews with six Faculty and their perspectives on students’ use of Large Language Models (LLMs) AI tools to complete their assignments and their thoughts on the benefits and challenges. The participants were purposively selected. The data were coded based on Braun and Clarke’s (2013) six steps in thematic analysis. Descriptive, in vivo, and evaluative coding were used. Additionally, data were examined semantically and latently using reductionist analysis to determine the final themes. Five components of the Unified Theory of Acceptance and Use of Technology (UTTAUT) were applied to the data collected and provided the framework for the study. Behavioural intention served as the foundation. Effort and Performance Expectancies, and facilitating conditions were exemplified in participants’ responses about the use of ChatGPT, Grammarly, and other AI assistive tools, plagiarism/academic integrity, and social influence were indicated when participants (both Students and Faculty) suggested the need for the development of policies and procedures toward the appropriate use of AI tools. Effort and performance expectancies and habits were found in the data collected in the form of consideration of the pros of using AI tools such as ChatGPT and assistive tools. These include the time saved by generating information, examples for both students and Faculty, and help in the teaching/learning process, and one participant found that it motivated her. The cons cited were students’ lack of creativity and the inability to think critically, the cost of the AI assistive tools (related to the component Price), the bandwidth needed to use them, the digital divide, and the false information generated. This study has significance for the use of ChatGPT and assistive AI tools in education and the ethical implications. It is recommended that specific policies be established and enacted to ensure the appropriate use of assistive and Artificial Intelligence (LLMs) tools.

  • Research Article
  • Cite Count Icon 13
  • 10.1002/cesm.70036
Using Artificial Intelligence Tools as Second Reviewers for Data Extraction in Systematic Reviews: A Performance Comparison of Two AI Tools Against Human Reviewers.
  • Jul 1, 2025
  • Cochrane evidence synthesis and methods..
  • T Helms Andersen + 4 more

Systematic reviews are essential but time-consuming and expensive. Large language models (LLMs) and artificial intelligence (AI) tools could potentially automate data extraction, but no comprehensive workflow has been tested for different review types. To evaluate Elicit's and ChatGPT's abilities to extract data from journal articles as a replacement for one of two human data extractors in systematic reviews. Human-extracted data from three systematic reviews (30 articles in total) was compared to data extracted by Elicit and ChatGPT. The AI tools extracted population characteristics, study design, and review-specific variables. Performance metrics were calculated against human double-extracted data as the gold standard, followed by a detailed error analysis. Precision, recall and F1-score were all 92% for Elicit and 91%, 89% and 90% for ChatGPT. Recall was highest for study design (Elicit: 100%; ChatGPT: 90%) and population characteristics (Elicit: 100%; ChatGPT: 97%), while review-specific variables achieved 77% in Elicit and 80% in ChatGPT. Elicit had four instances of confabulation while ChatGPT had three. There was no significant difference between the two AI tools' performance (recall difference: 3.3% points, 95% CI: -5.2%-11.9%, p = 0.445). AI tools demonstrated high and similar performance in data extraction compared to human reviewers, particularly for standardized variables. Error analysis revealed confabulations in 4% of data points. We propose adopting AI-assisted extraction to replace the second human extractor, with the second human instead focusing on reconciling discrepancies between AI and the primary human extractor.

  • Research Article
  • Cite Count Icon 1
  • 10.58600/eurjther1880
Should We Wait for Major Frauds to Unveil to Plan an AI Use License?
  • Dec 22, 2023
  • European Journal of Therapeutics
  • Istemihan Coban

Should We Wait for Major Frauds to Unveil to Plan an AI Use License?

Save Icon
Up Arrow
Open/Close
Notes

Save Important notes in documents

Highlight text to save as a note, or write notes directly

You can also access these Documents in Paperpal, our AI writing tool

Powered by our AI Writing Assistant