Differences in information accessed in a pharmacologic knowledge base using a conversational agent vs traditional search methods

Abstract

IntroductionClinicians rely on pharmacologic knowledge bases to answer medication questions and avoid potential adverse drug events. In late 2018, an artificial intelligence-based conversational agent, Watson Assistant (WA), was made available to online subscribers to the pharmacologic knowledge base, Micromedex®. WA allows users to ask medication-related questions in natural language. This study evaluated search method-dependent differences in the frequency of information accessed by traditional methods (keyword search and heading navigation) vs conversational agent search. Materials and methodsWe compared the proportion of information types accessed through the conversational agent to the proportion of analogous information types accessed by traditional methods during the first 6 months of 2020. ResultsAddition of the conversational agent allowed early adopters to access 22 different information types contained in the ‘quick answers’ portion of the knowledge base. These information types were accessed 117,550 times with WA during the study period, compared to 33,649,651 times using traditional search methods. The distribution across information types differed by method employed (c2 test, P < .0001). Single drug/dosing, FDA/non-FDA uses, adverse effects, and drug administration emerged as 4 of the top 5 information types accessed by either method. Intravenous compatibility was accessed more frequently using the conversational agent (7.7% vs. 0.6% for traditional methods), whereas dose adjustments were accessed more frequently via traditional methods (4.8% vs. 1.4% for WA). ConclusionIn a widely used pharmacologic knowledge base, information accessed through conversational agents versus traditional methods differed. User-centered studies are needed to understand these differences.