Word Count Research Articles

Evaluating the readability of online patient information for cholangiocarcinoma.

264 Background: The digital age has facilitated the dissemination of free comprehensive information regarding the diagnosis and treatment of various cancers. Especially for rarer cancers, such as cholangiocarcinoma, patients heavily rely on online patient information (OPI) as a primary source of information. However, the usefulness of these websites is dependent on the readability of the information. The National Institute of Health (NIH) recommends OPI have a readability level of grade 8 or lower. We sought to assess the readability of OPI for cholangiocarcinoma. Methods: We utilized Google to identify the top ten websites utilizing the search query "cholangiocarcinoma". The readability score of each website was defined as the median of ten popular standardized readability scales - Flesch–Kincaid, Gunning Fog, Coleman–Liau, Simple Measure of Gobbledygook, Automated Readability Index, FORCAST, Raygor, Fry Graph, and New Dale Chall. These readability scales were obtained using www.readable.com. Group comparisons of readability score were performed for affiliation (i.e., academic, non-profit, or government) and website content (i.e., treatment modalities, complications, clinical trials, supportive care, and survivorship). Statistical analysis was done in RStudio. Pairwise comparisons were performed using the Wilcoxon rank sum test. Results: A total of 10 websites were analyzed with 4/10 (40%) affiliated with academic institutions, 3/10 (30%) affiliated with non-profit organization, and 3/10 (30%) affiliated with government websites. The median readability score for all 10 websites was 9.95 (IQR 9.15-10.75). The majority of the websites (9/10, 90%) websites had readability scores that did not satisfy the NIH recommendation for readability. Readability scores did not differ significantly between academic, government, and nonprofit websites (p=0.68). Readability also did not differ by website content. The correlation between word count and readability score was not statistically significant (r = 0.03, p = 0.92). Conclusions: We performed a first of its kind analysis to assess the readability of OPI for cholangiocarcinoma. The median readability of OPI for cholangiocarcinoma was at a 10th grade level and the majority of the websites did not satisfy the NIH guideline for readability. Surprisingly, government websites were not simpler to read than their academic and nonprofit counterparts. Our study underscores the need to develop OPI for cholangiocarcinoma that is easily readable for patients and their caregivers. Organizations publishing OPI may use readability scores to ensure their content is accessible for the general population.

Overuse of familiar phrases by individuals with Williams syndrome masks differences in language processing.

We investigated whether individuals with Williams Syndrome (WS) produce language with a bias towards statistical properties of word combinations rather than grammatical rules, resulting in an overuse of holistically stored, familiar phrases. We analysed continuous speech samples from English children with WS (n = 12), typically developing (TD) controls matched on chronological age (n = 15) and TD controls matched on language age (n = 14). Alongside word count, utterance length, grammatical complexity, and morphosyntactic errors, we measured familiarity of expressions by computing collocation strength of each word combination. The WS group produced stronger collocations than both control groups. Moreover, the WS group produced fewer complex sentences, shorter utterances, and more frequent function words than chronological-age matched controls. Language in WS may appear more typical than it is because familiar, holistically processed expressions mask grammatical and other difficulties.

Is Information About Musculoskeletal Malignancies From Large Language Models or Web Resources at a Suitable Reading Level for Patients?

Patients and caregivers may experience immense distress when receiving the diagnosis of a primary musculoskeletal malignancy and subsequently turn to internet resources for more information. It is not clear whether these resources, including Google and ChatGPT, offer patients information that is readable, a measure of how easy text is to understand. Since many patients turn to Google and artificial intelligence resources for healthcare information, we thought it was important to ascertain whether the information they find is readable and easy to understand. The objective of this study was to compare readability of Google search results and ChatGPT answers to frequently asked questions and assess whether these sources meet NIH recommendations for readability. (1) What is the readability of ChatGPT-3.5 as a source of patient information for the three most common primary bone malignancies compared with top online resources from Google search? (2) Do ChatGPT-3.5 responses and online resources meet NIH readability guidelines for patient education materials? This was a cross-sectional analysis of the 12 most common online questions about osteosarcoma, chondrosarcoma, and Ewing sarcoma. To be consistent with other studies of similar design that utilized national society frequently asked questions lists, questions were selected from the American Cancer Society and categorized based on content, including diagnosis, treatment, and recovery and prognosis. Google was queried using all 36 questions, and top responses were recorded. Author types, such as hospital systems, national health organizations, or independent researchers, were recorded. ChatGPT-3.5 was provided each question in independent queries without further prompting. Responses were assessed with validated reading indices to determine readability by grade level. An independent t-test was performed with significance set at p < 0.05. Google (n = 36) and ChatGPT-3.5 (n = 36) answers were recorded, 12 for each of the three cancer types. Reading grade levels based on mean readability scores were 11.0 ± 2.9 and 16.1 ± 3.6, respectively. This corresponds to the eleventh grade reading level for Google and a fourth-year undergraduate student level for ChatGPT-3.5. Google answers were more readable across all individual indices, without differences in word count. No difference in readability was present across author type, question category, or cancer type. Of 72 total responses across both search modalities, none met NIH readability criteria at the sixth-grade level. Google material was presented at a high school reading level, whereas ChatGPT-3.5 was at an undergraduate reading level. The readability of both resources was inadequate based on NIH recommendations. Improving readability is crucial for better patient understanding during cancer treatment. Physicians should assess patients' needs, offer them tailored materials, and guide them to reliable resources to prevent reliance on online information that is hard to understand. Level III, prognostic study.

A comparative analysis of the ethics of gene editing: ChatGPT vs. Bard

AbstractRecently, there has been a growing trend in using large language models (LLMs) to develop diverse applications suitable for a wide range of tasks. These tasks range from solving programming bugs to helping teach elementary school students how to enhance their writing. Even with all these beneficial use cases, researchers worry about the potential bias these tools could produce and their effect on society. In this research, we compared responses that resulted from prompting two chatbots, namely OpenAI ChatGPT and Google Bard, about the issue of gene editing. Twelve prompts that are part of two subgroups were used to generate responses (text) about the issue of gene editing when the political affiliation (Democrat, Republican, and Communist) or geographical areas (United States, China, and Europe) of the prompter is provided. The Twelve responses were then analyzed semantically using three dictionary-based tools, i.e., Linguistic Inquiry and Word Count, the Moral Foundation Theory and Biblical Ethics dictionary, and Google’s Perspective API, to test and analyze the semantic and linguistic differences (measured via the Mann–Whitney U test) in the responses returned from the two chatbots. The results suggest that there are semantic and linguistic differences in responses per chatbots and prompts.

Satisfiers and Dissatisfiers for Las Vegas Casino Buffet Restaurants: A Google Review and LIWC Analysis

This study aims to identify satisfiers and dissatisfiers by analyzing Google Reviews on Las Vegas Casino Buffet Restaurants. It also compared its results to the nascent literature regarding satisfiers and dissatisfiers for restaurants, as revealed in online review by analyzing 14,064 reviews using Nvivo and Excel, with content analysis and text analysis with Linguistic Inquiry and Word Count to gain insights into reviewers' emotional expressions. It found that food, option/variety, price/value, employees, and atmosphere were important satisfiers/dissatisfiers. For casino buffets, the theme of options was present as both satisfier and dissatisfier, indicating it is important to this type of restaurant.

Abstract B049: Building rapport in navigator-delivered health insurance literacy interventions: Engagement and tone used by parents of children with cancer and patient navigators

Abstract Purpose: To explore the linguistic elements of engagement and tone used by parents of children with cancer and a patient navigator (PN) during a pilot study of a health insurance literacy (HIL) intervention. Methods: A trained PN delivered a four-session Zoom-based HIL intervention for parents of children with cancer that contained content on: Session 1 (S1) health insurance terms, (S2) using health insurance, (S3) insurance appeals/legislation, and (S4) budgeting and financial strategies. We applied Linguistic Inquiry Word Count (LIWC-22) software to analyze transcriptions of each session regarding engagement and tone. We report, in aggregate, differences in language: analytical language (e.g., formal, logical), clout language (e.g., confidence, leadership), authenticity language (e.g., degree of self-monitoring), and emotional tone (e.g., positive and negative emotion words) (all range 1-99) between S1 and S4 using paired t-tests. We also analyzed language similarities between the PN and the parent using Language Style Matching, which represents the degree to which both speakers are engaged in the conversation. Results: Of 10 participants, 9 completed 4 sessions, and 1 parent completed 3 (39 sessions total); sessions were on average 35.1 (SD=6.4) minutes. Most participants were women (80%), married (90%), white (100%), and non-Hispanic (90%); half had a college degree. The PN spoke more than parents across all four sessions (90% PN vs 8.8% parent); however, parent engagement, as represented by word count, increased as the sessions went on (mean S1: 414 words, S4: 603.8 words, p=0.21). S1 contained the most language exchange between parents and the PN (median word count: 5,533.5). The discussions became increasingly authentic (mean by session: S1:24.2; S4:28.6; p=0.02), held more clout (S1:77.8, S4: 85.8; p=0.009), and increased in emotional tone (S1:49; S4:65, p=0.002). Analytical language decreased over time (S1:24.4; S4:19.8, p=0.05). Language style matching was consistently high and trending upward across sessions (S1:0.80; S4: 0.81, p=0.3). Conclusions: In a pilot test of a HIL intervention, we saw an increase in authentic language, emotional tone, and parent word count coupled with a decrease in analytical language, indicating increased rapport between the PN and parent over the sessions. This suggests there is emotional and supportive value in PN-led interventions as compared to self-directed interventions. PNs can build trust and promote equity by deconstructing and personalizing health insurance information. Future work should explore how rapport built between parents and PNs relates to health insurance literacy and financial toxicity outcomes. Citation Format: Mary K. Killela, Anne C. Kirchhoff, Perla Vaca Lopez, Amy Chevrier, Monique Stefanou, Katie Rogers, Douglas Fair, Echo L. Warner. Building rapport in navigator-delivered health insurance literacy interventions: Engagement and tone used by parents of children with cancer and patient navigators [abstract]. In: Proceedings of the 17th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2024 Sep 21-24; Los Angeles, CA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2024;33(9 Suppl):Abstract nr B049.

Abstract A164: Rural and religious differences in online news outlet headlines about HPV and HPV vaccines in Utah

Abstract Purpose: To evaluate the penetration of online HPV vaccine information in Utah, where HPV vaccination and completion rates are low, and uncover potential disparities by religiosity and rurality in the types of language used by news headlines in the social media ecosystem. Methods: We described the language used in local news outlets' headlines about HPV and HPV vaccines. We included headlines posted on Facebook from 23 news outlets within 6 counties (including high and low vaccination, rurality, and religiosity) from 2011-2023 to represent the public online discourse about HPV vaccination. We searched each news outlet’s Facebook page for eligible posts using 11 key terms (e.g., HPV, HPV vaccination) that were identified through literature review and refined with content experts, resulting in N=851 headlines. After removing duplicates, there were a total of n=531 unique headlines for analysis. Area-level (county) measures of rurality (2013 Rural and Urban Continuum Codes) and religiosity (Public Religion Research Institute 2020 Census) were coded for each post title based on the originating news outlet. We analyzed the language used in each headline. Differences in the valence of online discourse about HPV and HPV vaccination by rurality and religiosity were compared using summary statistics and t-tests. Linguistic analyses were conducted using Linguistic Inquiry Word Count and Stata 18. Results: Overall, urban areas were less religious than rural areas (28% unaffiliated in urban areas vs. 13-26% unaffiliated in rural areas, p&amp;lt;0.001). There were more headlines about HPV and HPV vaccination from news outlets in urban than rural areas (69.7% vs. 30.3%). Nearly 70% of HPV and HPV vaccine headlines came from the least religious areas—counties whose religiously unaffiliated populations ranged from 28%-32%. News outlets differed in how they disseminated information about HPV and HPV vaccines by rurality and religiosity. On average, news outlets in urban areas used more positive language than in rural areas (mean 2.04 vs. 1.28 words, p&amp;lt;0.01). Rural news outlets used more emotion-laden terms than those in urban areas (mean 1.2 vs. 0.6 words, p&amp;lt;0.01) and negative emotion was more prevalent among rural than urban headlines (mean 0.73 vs. 0.42 words, p=0.02). These observations were mirrored for areas with higher religiously unaffiliated populations demonstrating titles with more positive words (mean 2.05 vs. 1.27, p&amp;lt;0.01), less emotional words (mean 0.63 vs. 1.17 words, p&amp;lt;0.01), and less negative language (mean 0.42 vs. 0.73 words, p=0.01) compared to headlines in areas with lower proportions of religiously unaffiliated, respectively. Conclusions: The penetration of HPV and HPV vaccine information in rural and religious areas is less positive, more emotionally charged, and uses more negative emotional language than in urban areas. This may, in part, perpetuate the low HPV vaccine uptake in these regions. Citation Format: Echo L. Warner, Ida Tovar, Kaila Christini, Stephen Gubler, Becca Slagle, Jennifer Smith, Michelle Litchman, Wendy Birmingham, Deanna Kepka. Rural and religious differences in online news outlet headlines about HPV and HPV vaccines in Utah [abstract]. In: Proceedings of the 17th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2024 Sep 21-24; Los Angeles, CA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2024;33(9 Suppl):Abstract nr A164.

Establishing the reliability of metrics extracted from long-form recordings using LENA and the ACLEW pipeline.

Long-form audio recordings are increasingly used to study individual variation, group differences, and many other topics in theoretical and applied fields of developmental science, particularly for the description of children's language input (typically speech from adults) and children's language output (ranging from babble to sentences). The proprietary LENA software has been available for over a decade, and with it, users have come to rely on derived metrics like adult word count (AWC) and child vocalization counts (CVC), which have also more recently been derived using an open-source alternative, the ACLEW pipeline. Yet, there is relatively little work assessing the reliability of long-form metrics in terms of the stability of individual differences across time. Filling this gap, we analyzed eight spoken-language datasets: four from North American English-learning infants, and one each from British English-, French-, American English-/Spanish-, and Quechua-/Spanish-learning infants. The audio data were analyzed using two types of processing software: LENA and the ACLEW open-source pipeline. When all corpora were included, we found relatively low to moderate reliability (across multiple recordings, intraclass correlation coefficient attributed to the child identity [Child ICC], was < 50% for most metrics). There were few differences between the two pipelines. Exploratory analyses suggested some differences as a function of child age and corpora. These findings suggest that, while reliability is likely sufficient for various group-level analyses, caution is needed when using either LENA or ACLEW tools to study individual variation. We also encourage improvement of extant tools, specifically targeting accurate measurement of individual variation.

The “How many?” task inadequately assesses the understanding of the cardinality principle

Once children have acquired the cardinality principle, they understand that the last number-word used in counting represents the total number of objects in a set. This principle is often assessed using the “How many?” task, which consists in asking “How many?” objects there are in a set after children have counted them. However, we show in this study that out of 188 kindergarteners (mean age: 4 ½ years), 42 (22,3 %) succeeded in repeating the last count word in the “How many?” task but failed to correctly apply the one-to-one correspondence principle during counting. Even when only the easiest countable sets were considered (i.e., linear and homogeneous collections or sets with a very limited number of objects), still more than 10 % of children from our sample repeated the last count word but failed to apply the one-to-one correspondence principle. Such developmental profile, in which children understand that the last word used in counting represents the total number of objects in a set but fail to grasp that each individual object must be associated with a single number word to determine this total, is not psychologically plausible. We conclude that the “How many?” task leads to an inaccurate assessment of the cardinality principle, in both its basic and more meaningful conception, in a non-negligible number of young children.

Peering into the team role kaleidoscope: the interplay of personal characteristics and verbal interactions in collaborative problem solving.

The objective of this study is to explore the relationship between personality and peer-rated team role behavior on the one hand and team role behavior and verbal behavior on the other hand. To achieve this, different data types were collected in fifteen professional teams of four members (N = 60) from various private and public organizations in Flanders, Belgium. Participants' personalities were assessed using a workplace-contextualized personality questionnaire based on the Big Five, including domains and facets. Typical team role behavior was assessed by the team members using the Team Role Experience and Orientation peer rating system. Verbal interactions of nine of the teams (n = 36) were recorded in an educational lab setting, where participants performed several collaborative problem-solving tasks as part of a training. To process these audio data, a coding scheme for collaborative problem solving and linguistic inquiry and word count were used. We identified robust links and logical correlation patterns between personality traits and typical team role behaviors, complementing prior research that only focused on self-reported team behavior. For instance, a relatively strong correlation was found between Altruism and the Team builder role. Next, the study reveals that role taking within teams is associated with specific verbal interaction patterns. For example, members identified as Organizers were more engaged in responding to others' ideas and monitoring execution.

Language sentiment predicts changes in depressive symptoms

The prevalence of depression is a major societal health concern, and there is an ongoing need to develop tools that predict who will become depressed. Past research suggests that depression changes the language we use, but it is unclear whether language is predictive of worsening symptoms. Here, we test whether the sentiment of brief written linguistic responses predicts changes in depression. Across two studies (N = 467), participants provided responses to neutral open-ended questions, narrating aspects of their lives relevant to depression (e.g., mood, motivation, sleep). Participants also completed the Patient Health Questionnaire (PHQ-9) to assess depressive symptoms and a risky decision-making task with periodic measurements of momentary happiness to quantify mood dynamics. The sentiment of written responses was evaluated by human raters (N = 470), Large Language Models (LLMs; ChatGPT 3.5 and 4.0), and the Linguistic Inquiry and Word Count (LIWC) tool. We found that language sentiment evaluated by human raters and LLMs, but not LIWC, predicted changes in depressive symptoms at a three-week follow-up. Using computational modeling, we found that language sentiment was associated with current mood, but language sentiment predicted symptom changes even after controlling for current mood. In summary, we demonstrate a scalable tool that combines brief written responses with sentiment analysis by AI tools that matches human performance in the prediction of future psychiatric symptoms.

A Retrospective, Longitudinal Registry Study on the Long-Term Durability of Ivacaftor Treatment in People with Cystic Fibrosis.

Ivacaftor (IVA) has been shown to change the trajectory of cystic fibrosis (CF) disease progression by slowing the rate of lung function decline in clinical studies. Long-term real-world data help to confirm the durability of this response. This non-interventional, longitudinal study used data from the US CF Foundation Patient Registry to describe the annualized rate of change in lung function in people with CF receiving IVA. The IVA-treated cohort included people with CF aged ≥ 6years who had ≥ 1 CF transmembrane conductance regulator (CFTR)-gating mutation and initiated IVA between 31 January 2012 and 31 December 2018. An age-matched comparator cohort included people with CF heterozygous for the F508del-CFTR mutation and a minimal function mutation (R117H excluded) and had not received CFTR modulator therapy. Baseline characteristics were balanced using standardized mortality ratio (SMR) weights computed from estimated propensity scores. The annualized rate of change in percent predicted forced expiratory volume in 1s (ppFEV1) was estimated over 5years and used to calculate the relative annualized rate of change in lung function in the IVA-treated versus comparator cohorts. In the 5-year follow-up period, 548 people were in the IVA-treated and 541 in the comparator cohorts after SMR weighting. The annualized rate of change in ppFEV1 over 5years was -1.23 (95% CI -1.45, -1.03) and -2.03 (-2.16, -1.90) percentage points in the IVA-treated and comparator cohorts, respectively. There was a 39% reduction (95% CI: 28, 50) in the rate of lung function decline in the IVA-treated versus comparator cohort over 5years. Findings were generally consistent with those of shorter follow-up periods. IVA showed a durable clinical benefit by slowing the rate of lung function decline over 5years. Results support a sustained and consistent impact of IVA on lung function trajectory in people with CF. Word count: 300 (limit: 300 words).

Examining the relationship between parents' self-reported mindfulness and observed language use in attachment-relevant communication.

Parents' language use is an important context for early socialization. We examined the relationship between parents' self-reported mindfulness and observed language use in two forms of attachment-relevant communication. Sixty-three parents of 6-18-month-old infants from Australia (n=32) and New Zealand (n=31) completed the five facets of mindfulness-short form (FFMQ-SF) questionnaire, the adult attachment interview (AAI), and a 10-min play session with their infant. We examined parents' frequency of word usage within the categories of the linguistic inquiry word count (LIWC) text analysis program to explore the relationship between mindfulness and language use. Mindfulness was associated with cognitive, affective, perceptual, and time orientation language use in the AAI. However, fewer associations were identified between mindfulness and language use in the parent-infant play session. Results are discussed in terms of their relevance to mindfulness and attachment.

Emotions and Goal Progress: A Linguistic Analysis of Goal Narratives and its Relation To Goal Progress

Individuals express a wide range of emotions in their writing beyond positive and negative emotions. In the current study, we set out to examine how positive, negative, and epistemic (curiosity and anxiety) emotions are related to individual goal progress. Instead of relying solely on self-reported emotions, we employed an indirect qualitative analysis using the Linguistic Inquiry and Word Count (LIWC) to examine student goal narratives (N = 201) and self-reported goal progress. Our results suggest that students use various emotions when describing their goals, especially positive emotions and epistemic curiosity. Despite the frequent use, we found no statistical association between emotions and progress toward achieving goals. In order to gain a realistic insight into how multiple emotions are utilized in goal narratives and goal progress, we suggest that future research use a more sophisticated method, such as a longitudinal study or combination of self-report, human coders and LIWC.

Are numerical scores important for grant assessment? A cross-sectional study

Background: In the evaluation of research proposals, reviewers are often required to provide their opinions using various forms of quantitative and qualitative criteria. In 2020, the European Commission removed, for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) funding scheme, the numerical scores from the individual evaluations but retained them in the consensus report. This study aimed to assess whether there were any differences in reviewer comments’ linguistic characteristics after the numerical scoring was removed, compared to comments from 2019 when numerical scoring was still present. Methods: This was an observational study and the data were collected for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) evaluation reports from the calls of 2019 and 2020, for both individual and consensus comments and numerical scores about the quality of the research proposal on three evaluation criteria: Excellence, Impact and Implementation. All comments were analyzed using the Linguistic Inquiry and Word Count (LIWC) program. Results: For both years, the comments for proposal’s strengths were written in a style that reflects objectivity, clout, and positive affect, while in weaknesses cold and objective style dominated, and that pattern remained stable across proposal status and research domains. Linguistic variables explained a very small proportion of the variance of the differences between 2019 and 2020 (McFadden R2=0.03). Conclusions: Removing the numerical scores was not associated with the differences in linguistic characteristics of the reviewer comments. Future studies should adopt a qualitative approach to assess whether there are conceptual changes in the content of the comments.

Pengaruh Disiplin Kerja dan Lingkungan Kerja terhadap Kinerja Karyawan dengan Kompensasi sebagai Variabel Intervening pada PT Jayamandiri Gemasejati Bogor

This is a new author's manual and a template for Jurnal Dirosah Islamiyah. Published every May and November since 2019. Articles must start with Article Title followed by Author Name and Affiliate Address and abstract. This part of the abstract must be typed in 150-250 word counts. Especially for the abstract, use the Cambria font, measuring 10 pt. Single space per line in this article. If the article is written in Indonesian, the abstract should be typed in English and Indonesian. Meanwhile, if the article is written in English, the abstract must be typed in English only. The abstract is typed as concisely as possible and consists of: research objectives, methods, results and discussion and brief conclusions. 250 words maximum. Abstract should only be typed in one paragraph and in one column format.

Honesty repeats itself: comparing manual and automated coding on the veracity cues total details and redundancy

AbstractLie detection research comparing manual and automated coding of linguistic cues is limited. In Experiment 1, we attempted to extend this line of research by directly comparing the veracity differences in manual coding and two coding software programs (Text Inspector and Linguistic Inquiry and Word Count [LIWC]) on the linguistic cue “total details” across eight published datasets. Mixed model analyses revealed that LIWC showed larger veracity differences in total details than Text Inspector and manual coding. Follow-up classification analyses showed that both automated coding and manual coding could accurately classify honest and false accounts. In Experiment 2, we examined if LIWC’s sensitivity to veracity differences was the result of honest accounts including more redundant (repeated) words than false accounts as LIWC—but not Text Inspector or manual coding—accounts for redundancy. Our prediction was supported, and the most redundant words were function words. The results implicated that automated coding can detect veracity differences in total details and redundancy, but it is not necessarily better than manual coding at accurately classifying honest and false accounts.

Gender Differences in Letters of Recommendations and Personal Statements for Neurotology Fellowship over 10 Years: A Deep Learning Linguistic Analysis.

Personal statements (PSs) and letters of recommendation (LORs) are critical components of the neurotology fellowship application process but can be subject to implicit biases. This study evaluated general and deep learning linguistic differences between the applicant genders over a 10-year span. Retrospective cohort. Two institutions. PSs and LORs were collected from 2014 to 2023 from two institutions. The Valence Aware Dictionary and Sentiment Reasoner (VADER) natural language processing (NLP) package was used to compare the positive or negative sentiment in LORs and PSs. Next, the deep learning tool, Empath, categorized the text into scores, and Wilcoxon rank sum tests were performed for comparisons between applicant gender. Among 177 applicants over 10 years, 120 were males and 57 were females. There were no differences in word count or VADER sentiment scores between genders for both LORs and PSs. However, among Empath sentiment categories, male applicants had more words of trust ( p = 0.03) and leadership ( p = 0.002) in LORs. Temporally, the trends show a consistently higher VADER sentiment and Empath "trust" and "leader" in male LORs from 2014 to 2019, after which there was no statistical significance in sentiment scores between genders, and females even have higher scores of trust and leadership in 2023. Linguistic content overall favored male applicants because they were more frequently described as trustworthy and leaders. However, the temporal analysis of linguistic differences between male and female applicants found an encouraging trend suggesting a reduction of gender bias in recent years, mirroring an increased composition of women in neurotology over time.

An Observational Study to Evaluate Readability and Reliability of AI-Generated Brochures for Emergency Medical Conditions.

Introduction The study assesses the readability of AI-generated brochures for common emergency medical conditions like heart attack, anaphylaxis, and syncope. Thus, the study aims to compare the AI-generated responses for patient information guides of common emergency medical conditions using ChatGPT and Google Gemini. Methodology Brochures for each condition were created by both AI tools. Readability was assessed using the Flesch-Kincaid Calculator, evaluating word count, sentence count and ease of understanding. Reliability was measured using the Modified DISCERN Score. The similarity between AI outputs was determined using Quillbot. Statistical analysis was performed with R (v4.3.2). Results ChatGPT and Gemini produced brochures with no statistically significant differences in word count (p= 0.2119), sentence count (p=0.1276), readability (p=0.3796), or reliability (p=0.7407). However, ChatGPT provided more detailed content with 32.4% more words (582.80 vs. 440.20) and 51.6% more sentences (67.00 vs. 44.20). In addition, Gemini's brochures were slightly easier to read with a higher ease score (50.62 vs. 41.88). Reliability varied by topic with ChatGPT scoring higher for Heart Attack (4 vs. 3) and Choking (3 vs. 2), while Google Gemini scored higher for Anaphylaxis (4 vs. 3) and Drowning (4 vs. 3), highlighting the need for topic-specific evaluation. Conclusions Although AI-generated brochures from ChatGPT and Gemini are comparable in readability and reliability for patient information on emergency medical conditions, this study highlights that there is no statistically significant difference in the responses generated by the two AI tools.

Generative artificial intelligence as a source of breast cancer information for patients: Proceed with caution.

This study evaluated the accuracy, clinical concordance, and readability of the chatbot interface generative pretrained transformer (ChatGPT) 3.5 as a source of breast cancer information for patients. Twenty questions that patients are likely to ask ChatGPT were identified by breast cancer advocates. These were posed to ChatGPT 3.5 in July 2023 and were repeated three times. Responses were graded in two domains: accuracy (4-point Likert scale, 4=worst) and clinical concordance (information is clinically similar to physician response; 5-point Likert scale, 5=not similar at all). The concordance of responses with repetition was estimated using intraclass correlation coefficient (ICC) of word counts. Response readability was calculated using the Flesch Kincaid readability scale. References were requested and verified. The overall average accuracy was 1.88 (range 1.0-3.0; 95% confidence interval [CI], 1.42-1.94), and clinical concordance was 2.79 (range 1.0-5.0; 95% CI, 1.94-3.64). The average word count was 310 words per response (range, 146-441 words per response) with high concordance (ICC, 0.75; 95% CI, 0.59-0.91; p<.001). The average readability was poor at 37.9 (range, 18.0-60.5) with high concordance (ICC, 0.73; 95% CI, 0.57-0.90; p<.001). There was a weak correlation between ease of readability and better clinical concordance (-0.15; p=.025). Accuracy did not correlate with readability (0.05; p=.079). The average number of references was 1.97 (range, 1-4; total, 119). ChatGPT cited peer-reviewed articles only once and often referenced nonexistent websites (41%). Because ChatGPT 3.5 responses were incorrect 24% of the time and did not provide real references 41% of the time, patients should be cautioned about using ChatGPT for medical information.