Abstract
Causal interpretation of correlational findings from observational studies has been a major type of misinformation in science communication. Prior studies on identifying inappropriate use of causal language relied on manual content analysis, which is not scalable for examining a large volume of science publications. In this study, we first annotated a corpus of over 3,000 PubMed research conclusion sentences, then developed a BERT-based prediction model that classifies conclusion sentences into “no relationship”, “correlational”, “conditional causal”, and “direct causal” categories, achieving an accuracy of 0.90 and a macro-F1 of 0.88. We then applied the prediction model to measure the causal language use in the research conclusions of about 38,000 observational studies in PubMed. The prediction result shows that 21.7% studies used direct causal language exclusively in their conclusions, and 32.4% used some direct causal language. We also found that the ratio of causal language use differs among authors from different countries, challenging the notion of a shared consensus on causal language use in the global science community. Our prediction model could also be used to help identify the inappropriate use of causal language in science publications.
Highlights
Establishing causality is one of the most important goals and concerns of science
Causal interpretation of correlational findings from observational studies has been a major type of misinformation in science communication
Our prediction model could be used to help identify the inappropriate use of causal language in science publications
Summary
Establishing causality is one of the most important goals and concerns of science. The language that describes causal relationships plays a crucial role in communicating science findings among scientists and with the general public (Kleinberg and Hripcsak, 2011). Scientists and journalists have been found to inappropriately use causal language in research publications and news articles. Causal interpretation of correlational findings from observational studies has been a major type of misinformation in science communication (Cofield et al, 2010; Sumner et al, 2014; Chiu et al, 2017; Boutron and Ravaud, 2018). Observational studies (non-intervention) are designed for testing association/correlation between variables, while intervention studies, such as clinical trials, are for testing causal relations (Buhse et al, 2018). Misinterpreting correlations as causations can lead to serious consequences, such as wrong medical decisions (Buhse et al, 2018) or harmful misperception of certain groups of people in society (Richardson et al, 2014)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
