Referral Decision Support Research Articles

Evaluating machine learning classifiers for glaucoma referral decision support in primary care settings

Several artificial intelligence algorithms have been proposed to help diagnose glaucoma by analyzing the functional and/or structural changes in the eye. These algorithms require carefully curated datasets with access to ocular images. In the current study, we have modeled and evaluated classifiers to predict self-reported glaucoma using a single, easily obtained ocular feature (intraocular pressure (IOP)) and non-ocular features (age, gender, race, body mass index, systolic and diastolic blood pressure, and comorbidities). The classifiers were trained on publicly available data of 3015 subjects without a glaucoma diagnosis at the time of enrollment. 337 subjects subsequently self-reported a glaucoma diagnosis in a span of 1–12 years after enrollment. The classifiers were evaluated on the ability to identify these subjects by only using their features recorded at the time of enrollment. Support vector machine, logistic regression, and adaptive boosting performed similarly on the dataset with F1 scores of 0.31, 0.30, and 0.28, respectively. Logistic regression had the highest sensitivity at 60% with a specificity of 69%. Predictive classifiers using primarily non-ocular features have the potential to be used for identifying suspected glaucoma in non-eye care settings, including primary care. Further research into finding additional features that improve the performance of predictive classifiers is warranted.

Impact of a digital Modified Checklist for Autism in Toddlers-Revised on likelihood and age of autism diagnosis and referral for developmental evaluation.

This was a project in primary care for young children (1-2 years old). We tested a parent questionnaire on a tablet. This tablet questionnaire asked questions to see whether the child may have autism. We compared the paper and pencil version of the questionnaire to the tablet questionnaire. We read the medical charts for the children until they were 4 years old to see whether they ended up having autism. We found that doctors were more likely to recommend an autism evaluation when a parent used the tablet questionnaire. We think that the tablet's automatic scoring feature helped the doctors. We also think that the doctors benefited from the advice the tablet gave them.

Primary care perspectives on implementation of clinical trial recruitment.

Poor clinical trial (CT) recruitment is a significant barrier to translating basic science discoveries into medical practice. Improving support for primary care provider (PCP) referral of patients to CTs may be an important part of the solution. However, implementing CT referral support in primary care is not only technically challenging, but also presents challenges at the person and organization levels. The objectives of this study were (1) to characterize provider and clinical supervisor attitudes and perceptions regarding CT research, recruitment, and referrals in primary care and (2) to identify perceived workflow strategies and facilitators relevant to designing a technology-supported primary care CT referral program. Focus groups were conducted with PCPs, directors, and supervisors. Analysis indicated widespread support for the intrinsic scientific value of CTs, while at the same time deep concerns regarding protecting patient well-being, perceived loss of control when patients participate in trials, concern about the impact of point-of-care referrals on clinic workflow, the need for standard processes, and the need for CT information that enables referring providers to quickly confirm that the burdens are justified by the benefits at both patient and provider levels. PCP suggestions pertinent to implementing a CT referral decision support system are reported. The results from this work contribute to developing an implementation approach to support increased referral of patients to CTs.

Using Hospital Claim Data to Develop Referral Decision Support Systems: Improving Patient Flow from The Primary Care

The decision to make a referral for a primary physician is influenced by several factors such as the need for specialist consultation and requesting support for diagnosis or disease management using advanced equipment. Unfortunately, primary care physicians are faced with a number of hospitals that have different resources. This study aims to improve the quality of referral services by facilitating primary care physicians in determining the right referral hospital or specialty care. Prototyping method was used to extend rule-reasoning decision support in an electronic referral system where hospital coordinates and real-time availability of bed exist. The research was conducted in DI Yogyakarta province, Indonesia. Appropriate referral hospital was inferred based on the presence of differential diagnosis and or required procedure. The history of medical diagnosis and procedures from hospital claim data was used as knowledge-based that connect hospital master data. IF ... AND (OR) ... THEN rule-reasoning was used to recommend referral hospitals based on a differential diagnosis and or required medical procedure determined by a primary care physician in the electronic referral system. To narrow down the referral hospitals, the Google Maps API was used to estimate the distance from primary health facilities to several available hospital options. Information on the availability of beds was also visualized for preference in determining the choice of a referral hospital. A decision support system can help primary care physicians to find the right referral hospital or specialty care for a desired patient before making a referral to a higher level of health facility.

A Patient-based Hospital Referral Decision Support System

A Patient-based Hospital Referral Decision Support System

Referral decision support in patients with subacute brain injury: evaluation of the Rehabilitation Complexity Scale – Extended

Purpose: To test if the Rehabilitation Complexity Scale Extended (RCS-E) can be used as decision support for patient referral to primary rehabilitation as either complex specialized services (CSS) or district specialist services (DSS).Method: Two independent expert teams analyzed medical records on 299 consecutive patients admitted for CSS or DSS rehabilitation. One team provided a golden standard for the patient referrals, and the other team provided RCS-E scores. Models for predicting referrals from RCS-E scores were developed on data for 149 patients and tested on the remaining 150 patients.Results: The optimal RCS-E sum score threshold for referral prediction was 11, predicting the golden standard for patient referral with sensitivity 88%, specificity 78% and correct classification rate 81%. Improved referral prediction performance was achieved by using RCS-E item-wise score thresholds (sensitivity 81%, specificity 89%, correct classification rate 87%). The RCS-E sum score range for patients referred CSS and DSS by the item-wise model was, respectively, 0–12 and 2–22 suggesting strong non-linear interaction of the RCS-E items.Conclusions: We found excellent referral decision support in the RCS-E and the item specific threshold model, when patients with acquired brain injury are to be referred to CSS or DSS as their primary rehabilitation.Implications for RehabilitationEfficient rehabilitation after acquired brain injury requires rehabilitation settings that meet patient needs.Validated tools for referral decision support make the process more transparent.Patient rehabilitation complexity can be stratified by the RCS-E with high sensitivity, specificity and predictive value of positive test.RCS-E is an excellent tool for referral decision support.

Improving referral activity on primary–secondary care interface using an electronic decision support system

ObjectiveOur main aim was to evaluate the ability of electronic feedback and of an electronic referral decision support system (ERSS) to enhance agreement between primary care physicians (PCPs) and specialists in priority assignment in clinics. Method62 PCPs used a ranking system of waiting times based on different categories of clinical priority called ‘Homogeneous Waiting Groups’ (HWGs), which was also used by specialists to assign a priority category for each patient. From the year 2001, all PCPs had to use a paper-based manual (group 1); instead from 2008, specialists began to use a computer-based tool, whereby the priority category reassigned to each patient by specialists promptly appeared on PCPs’ computers (group 2). During the course of 2010, the manual was incorporated in ERSS and was used by a subgroup of PCPs (group 3). Agreement between PCPs and specialists’ priority assignments was evaluated by the kappa statistic. ResultsIn group 1, the kappa statistic was 0.564 (95% CI=0.526–0.602); in group 2, the kappa statistic was 0.668 (95% CI=0.619–0.716); whereas in group 3 (that used ERSS) a very high kappa statistic emerged of 0.883 (95% CI=0.854–0.912). There was a significant difference in the proportion of agreement among the three groups (χ2=182.5, 2df, p<.0001). A significant difference in statistics was also observed in the proportion of priority levels used by PCPs over the years. ConclusionsOur results emphasize the positive effect of feedback and ERSS for improving referral activity and agreement between PCPs and specialists.