Computer-interpretable Guidelines Research Articles

A Digital Tool for Clinical Evidence-Driven Guideline Development by Studying Properties of Trial Eligible and Ineligible Populations: Development and Usability Study.

Clinical guideline development preferentially relies on evidence from randomized controlled trials (RCTs). RCTs are gold-standard methods to evaluate the efficacy of treatments with the highest internal validity but limited external validity, in the sense that their findings may not always be applicable to or generalizable to clinical populations or population characteristics. The external validity of RCTs for the clinical population is constrained by the lack of tailored epidemiological data analysis designed for this purpose due to data governance, consistency of disease or condition definitions, and reduplicated effort in analysis code. This study aims to develop a digital tool that characterizes the overall population and differences between clinical trial eligible and ineligible populations from the clinical populations of a disease or condition regarding demography (eg, age, gender, ethnicity), comorbidity, coprescription, hospitalization, and mortality. Currently, the process is complex, onerous, and time-consuming, whereas a real-time tool may be used to rapidly inform a guideline developer's judgment about the applicability of evidence. The National Institute for Health and Care Excellence-particularly the gout guideline development group-and the Scottish Intercollegiate Guidelines Network guideline developers were consulted to gather their requirements and evidential data needs when developing guidelines. An R Shiny (R Foundation for Statistical Computing) tool was designed and developed using electronic primary health care data linked with hospitalization and mortality data built upon an optimized data architecture. Disclosure control mechanisms were built into the tool to ensure data confidentiality. The tool was deployed within a Trusted Research Environment, allowing only trusted preapproved researchers to conduct analysis. The tool supports 128 chronic health conditions as index conditions and 161 conditions as comorbidities (33 in addition to the 128 index conditions). It enables 2 types of analyses via the graphic interface: overall population and stratified by user-defined eligibility criteria. The analyses produce an overview of statistical tables (eg, age, gender) of the index condition population and, within the overview groupings, produce details on, for example, electronic frailty index, comorbidities, and coprescriptions. The disclosure control mechanism is integral to the tool, limiting tabular counts to meet local governance needs. An exemplary result for gout as an index condition is presented to demonstrate the tool's functionality. Guideline developers from the National Institute for Health and Care Excellence and the Scottish Intercollegiate Guidelines Network provided positive feedback on the tool. The tool is a proof-of-concept, and the user feedback has demonstrated that this is a step toward computer-interpretable guideline development. Using the digital tool can potentially improve evidence-driven guideline development through the availability of real-world data in real time.

Provision and evaluation of explanations within an automated planning-based approach to solving the multimorbidity problem

The multimorbidity problem involves the identification and mitigation of adverse interactions that occur when multiple computer interpretable guidelines are applied concurrently to develop a treatment plan for a patient diagnosed with multiple diseases. Solving this problem requires decision support approaches which are difficult to comprehend for physicians. As such, the rationale for treatment plans generated by these approaches needs to be provided. Objective:To develop an explainability component for an automated planning-based approach to the multimorbidity problem, and to assess the fidelity and interpretability of generated explanations using a clinical case study. Methods:The explainability component leverages the task-network model for representing computer interpretable guidelines. It generates post-hoc explanations composed of three aspects that answer why specific clinical actions are in a treatment plan, why specific revisions were applied, and how factors like medication cost, patient’s adherence, etc. influence the selection of specific actions. The explainability component is implemented as part of MitPlan, where we revised our planning-based approach to support explainability. We developed an evaluation instrument based on the system causability scale and other vetted surveys to evaluate the fidelity and interpretability of its explanations using a two dimensional comparison study design. Results:The explainability component was implemented for MitPlan and tested in the context of a clinical case study. The fidelity and interpretability of the generated explanations were assessed using a physician-focused evaluation study involving 21 participants from two different specialties and two levels of experience. Results show that explanations provided by the explainability component in MitPlan are of acceptable fidelity and interpretability, and that the clinical justification of the actions in a treatment plan is important to physicians. Conclusion:We created an explainability component that enriches an automated planning-based approach to solving the multimorbidity problem with meaningful explanations for actions in a treatment plan. This component relies on the task-network model to represent computer interpretable guidelines and as such can be ported to other approaches that also use the task-network model representation. Our evaluation study demonstrated that explanations that support a physician’s understanding of the clinical reasons for the actions in a treatment plan are useful and important.

GLARE-SSCPM: an Intelligent Systemto Support the Treatment of Comorbid Patients

The development of software tools supporting physicians in the treatment of comorbid patients is a challenging goal and a “hot topic” in medical informatics and artificial intelligence. Computer interpretable guidelines (CIGs) are consolidated tools to support physicians with evidence-based recommendations in the treatment of patients affected by a specific disease. However, the application of two or more CIGs on comorbid patients is critical, since dangerous interactions between actions from different CIGs may arise. GLARE-SSCPM is the first tool supporting, in an integrated way, the knowledge-based detection of interactions, the management of the interactions, and the final “merge” of (part of) the CIGs operating on the patient. GLARE-SSCPM is characterized by being very supportive to physicians, providing them support for focusing, interaction detection, and for a “hypothesize and test” approach to manage the detected interactions. To achieve such goals, it provides advanced artificial intelligence techniques. Preliminary tests in the educational context, within the RoPHS project, have provided encouraging results.

Computer-interpretable guidelines: electronic tools to enhance the utility of thyroid nodule clinical practice guidelines and risk stratification tools.

Clinicians seeking guidance for evaluating and managing thyroid nodules currently have several resources. The principal ones are narrative clinical guidelines and clinical risk calculators. This paper will review the strengths and weaknesses of both. The paper will introduce a concept of computer interpretable guideline, a novel way of transforming narrative guidelines in to a clinical decision support tool that can provide patient specific recommendations at the point of care. The paper then describes an experience of developing an interactive web based computer interpretable guideline for thyroid nodule management, called Thyroid Nodule Management App (TNAPP). The advantages of this approach and the potential barriers for widespread adaptation are discussed.

META-GLARE: A Computer-Interpretable Guideline System Shell

Computer-interpretable Guideline (CIG) systems are important tools for ensuring healthcare practice quality and standardization. They usually provide a tool to acquire CIGs, and one to execute them on specific patients. Current CIG systems rely on their own formalism to represent clinical guidelines, so moving to new phenomena/domains may require substantial extensions. We propose an innovative approach, providing a “shell” that facilitates system designers to define new CIG systems (or to update an existing one) through the definition of a new CIG representation formalism, based on the Task-Network model. We based it on our previous work on META-GLARE, and we extend it with a general execution tool, able to operate on any CIG representation formalism acquired through the META-GLARE acquisition tool. Developed with modularity and compositionality principles, the tool exploits an open library of basic execution methods. It offers a general execution mechanism supporting various CIG formalisms. We successfully applied our approach to three practical case studies. We have identified a reference CIG formalism (the one currently supported by the META-GLARE library) and compared its expressiveness to benchmark approaches. META-GLARE constitutes the first shell in the literature to facilitate the (formalism-based) design and development of CIG systems, considering both acquisition and execution.

A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support

ObjectiveThe study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. Materials and methodsOur methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods—research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. ResultsSix research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DiscussionThe proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. ConclusionWe posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Representation of evidence-based clinical practice guideline recommendations on FHIR

BackgroundVarious formalisms have been developed to represent clinical practice guideline recommendations in a computer-interpretable way. However, none of the existing formalisms leverage the structured and computable information that emerge from the evidence-based guideline development process. Thus, we here propose a FHIR-based format that uses computer-interpretable representations of the knowledge artifacts that emerge during the process of evidence-based guideline development to directly serve as the basis of evidence-based recommendations. MethodsWe identified the information required to represent evidence-based clinical practice guideline recommendations and reviewed the knowledge artifacts emerging during the evidence-based guideline development process. We then conducted a consensus-based design process with domain experts to develop an information model for guideline recommendation representation that is structurally aligned to the evidence-based guideline recommendation development process and a corresponding representation based on FHIR resources developed for evidence-based medicine (EBMonFHIR). The resulting recommendations were modelled and represented in conformance with the FHIR Clinical Guidelines (CPG-on-FHIR) implementation guide. ResultsThe information model of evidence-based clinical guideline recommendations and its EBMonFHIR-/CPG-on-FHIR-based representation contain the clinical contents of individual guideline recommendations, a set of metadata for the recommendations, the ratings for the recommendations (e.g., strength of recommendation, certainty of overall evidence), the ratings of certainty of evidence for individual outcomes (e.g., risk of bias) and links to the underlying evidence (systematic reviews based on primary studies). We created profiles and an implementation guide for all FHIR resources required to represent the knowledge artifacts generated during evidence-based guideline development and their re-use as the basis for recommendations and used the profiles to implement an exemplary clinical guideline recommendation. ConclusionsThe FHIR implementation guide presented here can be used to directly link the evidence assessment process of evidence-based guideline recommendation development, i.e. systematic reviews and evidence grading, and the underlying evidence from primary studies to the resulting guideline recommendations. This not only allows the evidence on which recommendations are based on to be evaluated transparently and critically, but also enables guideline developers to leverage computable evidence in a more direct way to facilitate the generation of computer-interpretable guideline recommendations.

A model-driven transformation approach for the modelling of processes in clinical practice guidelines

Clinical Practice Guidelines (CPGs) include recommendations aimed at optimising patient care, informed by a review of the available clinical evidence. To achieve their potential benefits, CPG should be readily available at the point of care. This can be done by translating CPG recommendations into one of the languages for Computer-Interpretable Guidelines (CIGs). This is a difficult task for which the collaboration of clinical and technical staff is crucial. However, in general CIG languages are not accessible to non-technical staff. We propose to support the modelling of CPG processes (and hence the authoring of CIGs) based on a transformation, from a preliminary specification in a more accessible language into an implementation in a CIG language. In this paper, we approach this transformation following the Model-Driven Development (MDD) paradigm, in which models and transformations are key elements for software development. To demonstrate the approach, we implemented and tested an algorithm for the transformation from the BPMN language for business processes to the PROforma CIG language. This implementation uses transformations defined in the ATLAS Transformation Language. Additionally, we conducted a small experiment to assess the hypothesis that a language such as BPMN can facilitate the modelling of CPG processes by clinical and technical staff.

Supporting physicians in the coordination of distributed execution of CIGs to treat comorbid patients

Clinical Practice Guidelines (CPGs) encode the “best” medical practices to treat patients affected by a specific disease and are widely used in the medical practice. Starting from the ‘90s', several Computer-Interpretable Guideline (CIG) systems have been devised to provide physicians with CPG-based decision support. CPGs (and CIGs) are devoted to provide evidence-based recommendations for one specific disease. In order to support the treatment of patients affected by multiple diseases (i.e., comorbid patients), challenging additional tasks have to be addressed, such as (i) the detection of the interactions between CIG actions, (ii) their management, and, finally, (iii) the “merge” or conciliation of the CIGs. Several CIG approaches have been recently extended in order to face (at least one of) such challenging problems, and one of them is GLARE. However, besides the solutions to tasks (i)-(iii) above, the “run-time” support to physicians treating a comorbid patient requires additional capabilities, to support the distribution of the management of interactions and of the execution of CIGs among different physicians. In this paper, we propose a general framework, based on GLARE and GLARE-SSCPM, to provide such additional capabilities.

System for Context-Specific Visualization of Clinical Practice Guidelines (GuLiNav): Concept and Software Implementation.

BackgroundClinical decision support systems often adopt and operationalize existing clinical practice guidelines leading to higher guideline availability, increased guideline adherence, and data integration. Most of these systems use an internal state-based model of a clinical practice guideline to derive recommendations but do not provide the user with comprehensive insight into the model.ObjectiveHere we present a novel approach based on dynamic guideline visualization that incorporates the individual patient’s current treatment context.MethodsWe derived multiple requirements to be fulfilled by such an enhanced guideline visualization. Using business process and model notation as the representation format for computer-interpretable guidelines, a combination of graph-based representation and logical inferences is adopted for guideline processing. A context-specific guideline visualization is inferred using a business rules engine.ResultsWe implemented and piloted an algorithmic approach for guideline interpretation and processing. As a result of this interpretation, a context-specific guideline is derived and visualized. Our implementation can be used as a software library but also provides a representational state transfer interface. Spring, Camunda, and Drools served as the main frameworks for implementation. A formative usability evaluation of a demonstrator tool that uses the visualization yielded high acceptance among clinicians.ConclusionsThe novel guideline processing and visualization concept proved to be technically feasible. The approach addresses known problems of guideline-based clinical decision support systems. Further research is necessary to evaluate the applicability of the approach in specific medical use cases.

Bottom-Up Natural Language Processing Based Evaluation of the Fitness of UMLS as a Semantic Source for a Computer Interpretable Guidelines Ontology.

CIGs languages consist of approach specific concepts. More widely used concepts, such as those in UMLS are not typically used. An evaluation of UMLS concept sufficiency for CIG definition. A popular guideline is mapped to UMLS concepts with NLP. Results are reviewed to evaluate gaps, and appropriateness. A significant number of the guideline text mapped to UMLS concepts. The approach has shown promise and highlighted further challenges.

Temporal reasoning and query answering with preferences and probabilities for medical decision support

Knowledge-based decision support systems have a long tradition within the medical area. In particular, in the last decades, many Computer-Interpretable Guidelines (CIG) systems have been built to provide evidence-based and knowledge-based support to physicians. Since CIGs are, by definition, devoted to the management of specific diseases, the treatment of comorbid patients constitutes a challenging task in the area, involving (i) the detection of the possible interactions between (the effects of) the actions recommended by multiple CIGs (one for each disease of the patient), (ii) the management of such interactions and, finally, (iii) the conciliation of (the recommendations of) different CIGs. This paper focuses on issue (i) above, and specifically, on an innovative approach to support interaction detection along the temporal dimension. Practically, interactions can only occur between effects that intersect in time. Therefore, interaction detection involves the representation of temporal information (temporal constraints), and temporal reasoning (to propagate such constraints). Additionally, query answering facilities are important to support physicians in the investigation of the results of temporal reasoning. Current CIG approaches that face such issues take into account only “crisp” temporal constraints, i.e., they consider all temporal constraints as equally probable\\preferred. However, preferences about the temporal constraints between CIGs actions may be available, as well as knowledge about the probabilistic distribution of the effects of CIGs actions in time. Considering such additional pieces of information can provide crucial advantages, in term of the flexibility and informativeness of the support provided by the CIG system to physicians. In this paper, we propose the first homogeneous approach to represent and reason with (propagate) temporal constraints with both preferences and probabilities. We ground our approach on the widely-used Simple Temporal Problem (STP) framework, which supports temporal reasoning on temporal constraints about possible distances between events. We extend (i) the representation formalism to associate preferences and\\or probabilities to the possible distances, and (ii) the operations to propagate the constraints to combine also preferences and probabilities. We also (iii) provide an experimental evaluation of our approach, and (iv) propose a wide range of query-answering supports, to facilitate physicians in the analysis of the results of temporal reasoning in general, and in the temporal detection of possible interactions in particular. Finally, (v) we also show how such a temporal framework is integrated in GLARE-SSCPM, a CIG system to treat comorbid patients, and show the advantages of our approach considering a running example.

Process Model Metrics for Quality Assessment of Computer-Interpretable Guidelines in PROforma

Background: Clinical Practice Guidelines (CPGs) include recommendations to optimize patient care and thus have the potential to improve the quality and outcomes of healthcare. To achieve this, CPG recommendations are usually formalized in terms of Computer-Interpretable Guideline (CIG) languages. However, a clear understanding of CIG models may prove complicated, due to the inherent complexity of CPGs and the specificities of CIG languages. Drawing a parallel with the Business Process Management (BPM) and the Software Engineering fields, understandability and modifiability of CIG models can be regarded as primary quality attributes, in order to facilitate their validation, as well as their adaptation to accommodate evolving clinical evidence, by modelers (typically teams made up of clinical and IT experts). This constitutes a novel approach in this area of CIG development, where understandability and modifiability aspects have not been considered to date. Objective: In this paper, we define a comprehensive set of process model metrics for CIGs described in the PROforma CIG language, with the main objective of providing tools for quality assessment of CIG models in this language. Methods: To this end, we first reinterpret a set of metrics from the BPM field in terms of PROforma and then we define new metrics to capture the singularities of PROforma models. Additionally, we report on a set of experiments to assess the relationship between the structural and logical properties of CIG models, as measured by the proposed metrics, and their understandability and modifiability from the point of view of modelers, both clinicians and IT staff. For the analysis of the experiment results, we perform statistical analysis based on a generalized linear mixed model with binary logistic regression. Results: Our contribution includes the definition of a comprehensive set of metrics that allow measuring model quality aspects of PROforma CIG models, the implementation of tools and algorithms to assess the metrics for PROforma models, and the empirical validation of the proposed metrics as quality indicators. Conclusions: In light of the results, we conclude that the proposed metrics can be of great value, as they capture the PROforma-specific features in addition to those inspired by the general-purpose BPM metrics in the literature. In particular, the newly defined metrics for PROforma prevail as statistically significant when the whole CIG model is considered, which means that they better characterize its complexity. Consequently, the proposed metrics can be used as quality indicators of the understandability, and thereby maintainability, of PROforma CIGs.

XML Data and Knowledge-Encoding Structure for a Web-Based and Mobile Antenatal Clinical Decision Support System: Development Study.

BackgroundDispleasure with the functionality of clinical decision support systems (CDSSs) is considered the primary challenge in CDSS development. A major difficulty in CDSS design is matching the functionality to the desired and actual clinical workflow. Computer-interpretable guidelines (CIGs) are used to formalize medical knowledge in clinical practice guidelines (CPGs) in a computable language. However, existing CIG frameworks require a specific interpreter for each CIG language, hindering the ease of implementation and interoperability.ObjectiveThis paper aims to describe a different approach to the representation of clinical knowledge and data. We intended to change the clinician’s perception of a CDSS with sufficient expressivity of the representation while maintaining a small communication and software footprint for both a web application and a mobile app. This approach was originally intended to create a readable and minimal syntax for a web CDSS and future mobile app for antenatal care guidelines with improved human-computer interaction and enhanced usability by aligning the system behavior with clinical workflow.MethodsWe designed and implemented an architecture design for our CDSS, which uses the model-view-controller (MVC) architecture and a knowledge engine in the MVC architecture based on XML. The knowledge engine design also integrated the requirement of matching clinical care workflow that was desired in the CDSS. For this component of the design task, we used a work ontology analysis of the CPGs for antenatal care in our particular target clinical settings.ResultsIn comparison to other common CIGs used for CDSSs, our XML approach can be used to take advantage of the flexible format of XML to facilitate the electronic sharing of structured data. More importantly, we can take advantage of its flexibility to standardize CIG structure design in a low-level specification language that is ubiquitous, universal, computationally efficient, integrable with web technologies, and human readable.ConclusionsOur knowledge representation framework incorporates fundamental elements of other CIGs used in CDSSs in medicine and proved adequate to encode a number of antenatal health care CPGs and their associated clinical workflows. The framework appears general enough to be used with other CPGs in medicine. XML proved to be a language expressive enough to describe planning problems in a computable form and restrictive and expressive enough to implement in a clinical system. It can also be effective for mobile apps, where intermittent communication requires a small footprint and an autonomous app. This approach can be used to incorporate overlapping capabilities of more specialized CIGs in medicine.

A Computer-Interpretable Guideline for COVID-19: Rapid Development and Dissemination

BackgroundCOVID-19 is a global pandemic that is affecting more than 200 countries worldwide. Efficient diagnosis and treatment are crucial to combat the disease. Computer-interpretable guidelines (CIGs) can aid the broad global adoption of evidence-based diagnosis and treatment knowledge. However, currently, no internationally shareable CIG exists.ObjectiveThe aim of this study was to establish a rapid CIG development and dissemination approach and apply it to develop a shareable CIG for COVID-19.MethodsA 6-step rapid CIG development and dissemination approach was designed and applied. Processes, roles, and deliverable artifacts were specified in this approach to eliminate ambiguities during development of the CIG. The Guideline Definition Language (GDL) was used to capture the clinical rules. A CIG for COVID-19 was developed by translating, interpreting, annotating, extracting, and formalizing the Chinese COVID-19 diagnosis and treatment guideline. A prototype application was implemented to validate the CIG.ResultsWe used 27 archetypes for the COVID-19 guideline. We developed 18 GDL rules to cover the diagnosis and treatment suggestion algorithms in the narrative guideline. The CIG was further translated to object data model and Drools rules to facilitate its use by people who do not employ the non-openEHR archetype. The prototype application validated the correctness of the CIG with a public data set. Both the GDL rules and Drools rules have been disseminated on GitHub.ConclusionsOur rapid CIG development and dissemination approach accelerated the pace of COVID-19 CIG development. A validated COVID-19 CIG is now available to the public.

Towards a goal-oriented methodology for clinical-guideline-based management recommendations for patients with multimorbidity: GoCom and its preliminary evaluation.

Patients with chronic multimorbidity are becoming more common as life expectancy increases, making it necessary for physicians to develop complex management plans. We are looking at the patient management process as a goal-attainment problem. Hence, our aim is to develop a goal-oriented methodology for providing decision support for managing patients with multimorbidity continuously, as the patient's health state is progressing and new goals arise (e.g., treat ulcer, prevent osteoporosis). Our methodology allows us to detect and mitigate inconsistencies among guideline recommendations stemming from multiple clinical guidelines, while consulting medical ontologies and terminologies and relying on patient information standards. This methodology and its implementation as a decision-support system, called GoCom, starts with computer-interpretable clinical guidelines (CIGs) for single problems that are formalized using the PROforma CIG language. We previously published the architecture of the system as well as a CIG elicitation guide for enriching PROforma tasks with properties referring to vocabulary codes of goals and physiological effects of management plans. In this paper, we provide a formalization of the conceptual model of GoCom that generates, for each morbidity of the patient, a patient-specific goal tree that results from the PROforma engine's enactment of the CIG with the patient's data. We also present the "Controller" algorithm that drives the GoCom system. Given a new problem that a patient develops, the Controller detects inconsistencies among goals pertaining to different comorbid problems and consults the CIGs to generate alternative non-conflicted and goal-oriented management plans that address the multiple goals simultaneously. In this stage of our research, the inconsistencies that can be detected are of two types - starting vs. stopping medications that belong to the same medication class hierarchy, and detecting opposing physiological effect goals that are specified in concurrent CIGs (e.g., decreased blood pressure vs. increased blood pressure). However, the design of GoCom is modular and generic and allows the future introduction of additional interaction detection and mitigation strategies. Moreover, GoCom generates explanations of the alternative non-conflicted management plans, based on recommendations stemming from the clinical guidelines and reasoning patterns. GoCom's functionality was evaluated using three cases of multimorbidity interactions that were checked by our three clinicians. Usefulness was evaluated with two studies. The first evaluation was a pilot study with ten 6th year medical students and the second evaluation was done with 27 6th medical students and interns. The participants solved complex realistic cases of multimorbidity patients: with and without decision-support, two cases in the first evaluation and 6 cases in the second evaluation. Use of GoCom increased completeness of the patient management plans produced by the medical students from 0.44 to 0.71 (P-value of 0.0005) in the first evaluation, and from 0.31 to 0.78 (P-value<0.0001) in the second evaluation. Correctness in the first evaluation was very high with (0.98) or without the system (0.91), with non-significant difference (P-value≥0.17). In the second evaluation, use of GoCom increased correctness from 0.68 to 0.83 (P-value of 0.001). In addition, GoCom's explanation and visualization were perceived as useful by the vast majority of participants. While GoCom's detection of goal interactions is currently limited to detection of starting vs. stopping the same medication or medication subclasses and detecting conflicting physiological effects of concurrent medications, the evaluation demonstrated potential of the system for improving clinical decision-making for multimorbidity patients.

A Domain-Independent Semantically Validated Authoring Tool for Formalizing Clinical Practice Guidelines.

Clinical Practice Guidelines (CPGs) are promoted as a powerful tool for standardization of the medical care quality and improvement of patients' outcomes. However, CPGs need to be formalized in a computer interpretable format (i.e. as Computer Interpretable Guidelines or CIGs) for their implementation within Clinical Decision Support Systems (CDSS). But, maintaining the reliability of these guidelines when deploying them in different clinical settings is still a challenge. On the one hand, the complexity of the medical language complicates the adoption of the guidelines in different clinical institutions. On the other hand, the continuous discovery of new evidence needs to be included within CPGs, updating their contents and providing tools for evidence assessment. Furthermore, although nowadays' clinical decision-making tends towards a personalized process, guidelines are designed for a general population. In this paper, we present an Authoring Tool (AT) that allows clinicians to take an active role in the process of CPG formalization. This AT enables them to introduce new clinical knowledge and create personalized CIGs for their local application, which best fits their clinical needs. The proposed system also allows the use of ontologies to facilitate the standardization and interoperability of the created guidelines. Finally, the content included in the CIGs can be evaluated using standard systems for grading clinical evidence.

Conformance analysis for comorbid patients in Answer Set Programming.

The treatment of comorbid patients is a hot problem in Medical Informatics, since the plain application of multiple Computer-Interpretable Guidelines (CIGs) can lead to interactions that are potentially dangerous for the patients. The specialized literature has mostly focused on the "a priori" or "execution-time" analysis of the interactions between multiple Computer-Interpretable Guidelines (CIGs), and/or CIG "merge". In this paper, we face a complementary problem, namely, the a posteriori analysis of the treatment of comorbid patients. Given the CIGs, the history of the status of the patient, and the log of the clinical actions executed on her, we try to explain the actions executed on the patient (i.e., the log) in terms of the actions recommended by the CIGs, of their potential interactions, and of the possible ways of managing each such interaction, pointing out (i) deviations from CIG recommendations not explained in terms of interaction management (if any) and (ii) unmanaged interactions (if any). Our approach is based on Answer Set Programming, and, to face realistic problems, devotes specific attention to the temporal dimension.

Comprehensive analysis of rule formalisms to represent clinical guidelines: Selection criteria and case study on antibiotic clinical guidelines.

The over-use of antibiotics in clinical domains is causing an alarming increase in bacterial resistance, thus endangering their effectiveness as regards the treatment of highly recurring severe infectious diseases. Whilst Clinical Guidelines (CGs) focus on the correct prescription of antibiotics in a narrative form, Clinical Decision Support Systems (CDSS) operationalize the knowledge contained in CGs in the form of rules at the point of care. Despite the efforts made to computerize CGs, there is still a gap between CGs and the myriad of rule technologies (based on different logic formalisms) that are available to implement CDSSs in real clinical settings. To helpCDSS designers to determine the most suitable rule-based technology (medical-oriented rules, production rules and semantic web rules) with which to model knowledge from CGs for the prescription of antibiotics. We propose a framework of criteria for this purpose that is extensible to more generic CGs. Our proposal is based on the identification of core technical requirements extracted from both literature and the analysis of CGs for antibiotics, establishing three dimensions for analysis: language expressivity, interoperability and industrial aspects. We present a case study regarding the John Hopkins Hospital (JHH) Antibiotic Guidelines for Urinary Tract Infection (UTI), a highly recurring hospital acquired infection. We have adopted our framework of criteria in order to analyse and implement these CGs using various rule technologies: HL7 Arden Syntax, general-purpose Production Rules System (Drools), HL7 standard Rule Interchange Format (RIF), Semantic Web Rule Language (SWRL) and SParql Inference Notation (SPIN) rule extensions (implementing our own ontology for UTI). We have identified the main criteria required to attain a maintainable and cost-affordable computable knowledge representation for CGs. We have represented the JHH UTI CGs knowledge in a total of 12 Arden Syntax MLMs, 81 Drools rules and 154 ontology classes, properties and individuals. Our experiments confirm the relevance of the proposed set of criteria and show the level of compliance of the different rule technologies with the JHH UTI CGs knowledge representation. The proposed framework of criteria may help clinical institutions to select the most suitable rule technology for the representation of CGs in general, and for the antibiotic prescription domain in particular, depicting the main aspects that lead to Computer Interpretable Guidelines (CIGs), such as Logic expressivity (Open/Closed World Assumption, Negation-As-Failure), Temporal Reasoning and Interoperability with existing HIS and clinical workflow. Future work will focus on providing clinicians with suggestions regarding new potential steps for CGs, considering process mining approaches and CGs Process Workflows, the use of HL7 FHIR for HIS interoperability and the representation of Knowledge-as- a-Service (KaaS).

Mobile-deployed care management based on standardized ePRO and computer-interpretable guideline engine for Chinese lymphoma patients.

1 Background: To effectively assist lymphoma patients and their care-givers after their hospital discharge, a standard follow-up guideline for lymphoma patients was designed and encoded through a computer-interpret-able-guideline (CiG) engine and deployed through mobile-Web and mobile-Apps to ensure continuous care throughout the treatment and recovery processes. Methods: Scheduled tasks including ePRO questionnaires and requests for test results were sent for completion, and a virtual care management team was assigned to ensure compliance and respond to automatic triggered alerts. Attending oncologists also were given mobile-Apps to access the patients’ securely uploaded records and consult remotely to manage side-effects after discharge. The out-of-hospital record management and care coordination enabled the adjustment of treatment plans and the optimized unitilization of clinical resources. Results: During 2018, 1012 patients or their caregivers interacted with our team through the mobile platform after discharge. 856 individual patients uploaded enough information to be initiated in the mobile follow-up program. Approximately 30% of these patients were Beijing residents while the rest came from all over China. About half of these patients had follow-up tests done in other 387 hospitals or 52 clinics. Based on ePRO questionnaires submitted, 218 patients had 616 side-effect-symptom alerts triggered and followed up, in addition to patient-initiated mobile consultation with their attending oncologists. Lab results especially white blood cell counts, platelet counts, and neutrophil counts, contributed to the majority of test alerts managed by our team. Analysis of 8000+ anonymized questions identified clinical processes in need of optimization. A new patient education manual addressing frequently asked questions, and new processes such as chemo-regime scheduling were put in place. Conclusions: We have demonstrated the feasibility of a mobile-deployed care management process after discharge for lymphoma patients and their caregivers. We have initiated retrospective studies of impacts on our patients’ treatment compliance and quality of life.