Provide Performance Assessment Research Articles

Performance assessment and comparison of online food delivery service providers based upon the aggregated perspectives of restaurant operators

Online food delivery businesses have grown rapidly during the past few years. To attract customers, food delivery service providers are forming partnerships with as many restaurants as possible. For restaurant entrepreneurs, the selection of the best service provider is a critical decision that influences their sales. However, many qualitative criteria make this problem difficult. Although rating scales can convert subjective judgments into quantitative analysis, linguistic scales may make a comparison of alternatives’ performances vague and unstandardised, especially for group decision making. The prioritisation of the criteria importance is also questionable, and reaching a consensus among decision makers might be difficult. To address such complications, this study proposes the use of fuzzy logic in determining the importance weights of the criteria in the direct rating approach, based on the perspectives of 60 restaurant operators in Thailand. The fuzzy technique for order preference by similarity to ideal solution (fuzzy TOPSIS) is then used to rank alternatives, as demonstrated by a case of three food delivery providers in Thailand. Fuzzy simple additive weighting (fuzzy SAW), two rank-based weighting methods (rank sum and rank-order centroid), and two objective weighting methods (entropy and mean) are used to compare results. The results show that user-friendly apps, tracking systems, payment options, marketing campaigns, and service areas are among the important criteria that delivery service providers should address and improve in order to attract more restaurants and customers. Guidelines for restaurant operators to choose the appropriate weighting methodology and to rationally assess and compare the service providers’ performances are provided.

Innovative Logistics Practices: Performance Assessment of Third-party Logistics Services Providers in Malaysia

In the contemporary business global sphere, it is so problematic, if not impossible, for an organization to be competitively deprived of working in alliance, closely with external associates. The idea of Supply Chain Management, (SCM), arose in this track and strives to optimally manage the physical goods and/or services. Logistics is the key competitive factor in the manufacturing sector because of the innovative models’ variants and options. For instance, in Malaysia, the manufacturing sector is a major contributor to the annual GDP, hence, there is a need for sustainable technology that can project significant business operations and supply chain management impacts. But this is far from being the case in the contemporary. Based on the increasing logistics benefits, the importance of the assessment of logistics efficiency and impact is attracting increased consideration. Green Logistics (GL) involves not only the delivery of green products or services to customers but also the inclusive logistics flow of items from cradle to grave in addition to reverse logistics. Numerous green events and operations have been instigated, such as manufacture scheduling and network construction. This article aimed at investigating the innovative logistics practices relationship with the operations of third-party logistics service providers in Malaysia. To achieve this, the study engages a structured questionnaire survey. 256 questionnaire was retrieved out of 333 questionnaires distributed electronically and 27 questionnaires were rejected based on some parallel errors and 229 questionnaires was eventually analyzed This is 69% of the questionnaire. In the analysis, SPSS version 20 was utilized. The outcome showed that selection of vehicle type and procurement on quality, cost, and environmental impacts as the main feature of innovation in sustainability. Innovative green logistics serve as a moderating factor in the relationship between 3PLs service provider’s key performance factors and green logistics in the Manufacturing industry of Malaysia. Hence, it fulfills one of the key economic growth activities or operations under strategic thrust 2 of SPV2030.

Exploring the potential of Data Envelopment Analysis for enhancing pay-for-performance programme design in primary health care

In recent years, implementation of pay-for-performance (P4P) programmes in health care has become a worldwide initiative. However, most P4P programmes incorporate systems of Performance Indicators (PI) without accounting for trade-offs between indicators. This article has two objectives: (1) to develop a Data Envelopment Analysis (DEA) methodology for performance assessment of primary care providers; and (2) to propose an innovative methodology for P4P contracting based on the DEA assessment results. To achieve the first objective, we modify the standard DEA model to account for the necessary relations between the weights attributed to each PI and domain of performance and to account for the effect of the relevant environmental variables. To achieve the second objective, we combine relative and absolute performance assessments in the elaboration of several bases for reward, and then we link these assessments to a system of graduated rewards. A benchmarking programme is also proposed to contribute to the dissemination of best practices. This article contributes to the literature by proposing an enhanced methodology for performance assessment of primary care providers which can form the basis for P4P rewards planning. The applicability and advantages of the proposed methodology are illustrated with data from Portugal, but it can easily be adapted to different sets of PIs or domains, making it relevant for performance assessment and for P4P reward setting in other contexts and countries.

Evaluation of Alternative Navigation, Positioning and Timing Systems Based on Their Performance and Additional Characteristics

The aim of this paper is to provide performance assessment of Alternative Navigation Positioning and Timing (APNT) systems considered possible GNSS backup in European region. The performance of the APNT systems is assessed based on their performance characteristics such as accuracy, integrity, and by additional beneficial abilities, that could help by implementation or could provide different added value in terms of solving certain issues in the Communication, Navigation and Surveillance (CNS) domain.

Factors Affecting University Choice Behaviour in the UK Higher Education

Although regulations and established practices in academia have focused on a data-rich model of performance information, both to evidence operational capability and to support recruitment, it is considered that this approach has been largely ineffective in addressing student choice behaviour. Historical studies, business, psychology, and technology theories have pointed to the oversimplification that a data-led strategy can result in for mapping human behaviour. In this case, decision processes of students are understood to be nuanced by a vast range of factors with variable relevance for everyone. The effort of the study is to address how Systems Thinking, related policy development, and associated enabling techniques can be applied to the field to provide both a deeper understanding of the dictates of student behaviour and, by extension, the appropriate foci for data provision, enabling comparative business performance assessment of Higher Education Providers. This research has followed the Design Science Research (DSR) methodology; the developed model has been successfully evaluated against the understanding of education practitioners in an interview consultation process of the methodology. The analysis of interview feedback and the development and refinement of the proposed model generate the principle findings of the study. The model outlines the factors that might affect students’ choices in the UK Higher Education.

Using a Choquet integral-based approach for incorporating decision-maker’s preference judgments in a Data Envelopment Analysis model

In a world in permanent (r)evolution that revolves around money, seeking new ways to contain costs, better allocate resources, and, overall, improve performance is a constant across all fields. Hence, the use of computational methods based on operational research and statistical science is crucial for achieving an appropriate combination of efficiency and effectiveness, especially in domains where the decision-making process is a complex task. This is where Data Envelopment Analysis (DEA) comes in. However, as a non-parametric and, usually, purely objective technique, DEA makes up for what it lacks in incorporating preference information with flexibility and adaptability, which is particularly important in areas where the decision-makers’ judgments are crucial. This work proposes a cutting-edge and original approach to fill in this knowledge gap by linking DEA and multiple criteria decision-making with an additive DEA model that takes into account criteria interactivity, by using an inference methodology to determine their weights, and decision-makers’ preference information incorporation, by taking advantage of the Choquet multiple criteria preference aggregation model. Thus, this approach was applied to a case study of performance assessment of Portuguese National Healthcare Service secondary healthcare providers across robustness-testing perspectives, generating credible weights stemmed from the decision-maker’s judgments and yielding acceptable and valid results.

3025 Individual Anesthesia Provider Performance Assessment

OBJECTIVES/SPECIFIC AIMS: We developed a multilevel hierarchical statistical model which describes the association of prophylactic interventions to patient PONV risk, and provides an intuitive summary for anesthesiologists to understand how well they are adhering to PONV guidelines. METHODS/STUDY POPULATION: Accepted PONV risk factors as well as preventative interventions to reduce the PONV risk, (e.g. total intravenous anesthesia or pharmacological prophylaxis) are retrieved from the electronic medical record (EMR). Risk is regressed against interventions. Fig 1, Panel A visualizes adherence for an individual provider by plotting anesthesia cases, with PONV risk in the x-axis and the number of interventions in the y-axis. Fig 1, Panel B shows a “Jitterplot”, jittering individual cases, which would otherwise plot onto the same coordinates (Panel A). The distribution of the number of interventions in each risk category is better summarized in Fig 1 Panel C by overlaying a violin plot onto the “Jitterplot”. Finally, a fitted regression line provides a summary measure for the individual provider’s risk-adjusted utilization of PONV prophylaxis in Fig 1, Panel D. The model can control for confounders and interactions, such as patient or procedure characteristics, such as supervision by attending physicians, institutional culture, and surgical procedure. RESULTS/ANTICIPATED RESULTS: Fig. 2, Panel A demonstrates good adherence. The provider responded to increased risk with additional interventions leading to a steep regression line. Less discriminate administration of prophylaxis is shown in Fig 2, Panel B. The graphical representation of our proposed measure of individual provider performance is intuitive, allowing us to compare adherence of two distinct groups of providers (light lines) and institutional averages (dark lines) as shown in Fig 2, Panel C. Controlling for known risk factors and potential confounders renders the assessment irrepudiable. The rigorous statistical approach allows for multi-level modeling and comparative effectiveness research, realistically evaluating process changes and interventions like CDS in the hierarchical structure of contemporary healthcare delivery. DISCUSSION/SIGNIFICANCE OF IMPACT: The strength of our novel measure of individual provider performance is its generalizability to other care settings, as well as the intuitive graphical representation of risk-adjusted individual performance. However, accuracy, precision and validity, sensitivity to system perturbations (like the implementation of CDS), and acceptance among providers remain to be evaluated. Fig 1. Risk-Adjusted Utilization of Antiemetic Prophylaxis Fig 2. Comparing Performance between Provider Groups

Performance assessment of health care providers

Performance assessment of health care providers

Case-Mix for Performance Management: A Risk Algorithm Based on ICD-10-CM.

Accurate risk adjustment is the key to a reliable comparison of cost and quality performance among providers and hospitals. However, the existing case-mix algorithms based on age, sex, and diagnoses can only explain up to 50% of the cost variation. More accurate risk adjustment is desired for provider performance assessment and improvement. To develop a case-mix algorithm that hospitals and payers can use to measure and compare cost and quality performance of their providers. All 6,048,895 patients with valid diagnoses and cost recorded in the US Veterans health care system in fiscal year 2016 were included in this study. The dependent variable was total cost at the patient level, and the explanatory variables were age, sex, and comorbidities represented by 762 clinically homogeneous groups, which were created by expanding the 283 categories from Clinical Classifications Software based on ICD-10-CM codes. The split-sample method was used to assess model overfitting and coefficient stability. The predictive power of the algorithms was ascertained by comparing the R, mean absolute percentage error, root mean square error, predictive ratios, and c-statistics. The expansion of the Clinical Classifications Software categories resulted in higher predictive power. The R reached 0.72 and 0.52 for the transformed and raw scale cost, respectively. The case-mix algorithm we developed based on age, sex, and diagnoses outperformed the existing case-mix models reported in the literature. The method developed in this study can be used by other health systems to produce tailored risk models for their specific purpose.

Comparing the performance of English mental health providers in achieving patient outcomes

Evidence on provider payment systems that incorporate patient outcomes is limited for mental health care. In England, funding for mental health care services is changing to a prospective payment system with a future objective of linking some part of provider payment to outcomes. This research examines performance of mental health providers offering hospital and community services, in order to investigate if some are delivering better outcomes. Outcomes are measured using the Health of the Nation Outcome Scales (HoNOS) – a clinician-rated routine outcome measure (CROM) mandated for national use. We use data from the Mental Health Minimum Data Set (MHMDS) – a dataset on specialist mental health care with national coverage – for the years 2011/12 and 2012/13 with a final estimation sample of 305,960 observations with follow-up HoNOS scores. A hierarchical ordered probit model is used and outcomes are risk adjusted with independent variables reflecting demographic, need, severity and social indicators. A hierarchical linear model is also estimated with the follow-up total HoNOS score as the dependent variable and the baseline total HoNOS score included as a risk-adjuster. Provider performance is captured by a random effect that is quantified using Empirical Bayes methods. We find that worse outcomes are associated with severity and better outcomes with older age and social support. After adjusting outcomes for various risk factors, variations in performance are still evident across providers. This suggests that if the intention to link some element of provider payment to outcomes becomes a reality, some providers may gain financially whilst others may lose. The paper contributes to the limited literature on risk adjustment of outcomes and performance assessment of providers in mental health in the context of prospective activity-based payment systems.

The Society of Thoracic Surgeons Adult Cardiac Surgery Database Version 2.73: More Is Better

With the introduction of version 2.73, several new patient risk factors are now captured in The Society of Thoracic Surgeons' (STS) Adult Cardiac Surgery Database. We sought to evaluate the potential association of these risk factors with mortality. We reviewed all patients with an STS predicted risk of mortality in our statewide quality collaborative database from July 2011 to September 2013 (N= 19,743). Univariate analyses were used to determine significant associations between mortality and the new risk factors in version 2.73. We then performed multivariable analysis, incorporating the STS predicted risk of mortality into our regression. In the univariate model, patients with illicit drug use, syncope, unresponsive neurologic state, cancer within the last 5 years, current smoking history, other tobacco use, or sleep apnea had no significant difference in mortality (p > 0.05). Patients with liver disease, elevated Model for End-Stage Liver Disease score, mediastinal radiation, prolonged 5-meter walk test, home oxygen use, inhaled medications or bronchodilator therapy, decreased forced expiratory volume, and history of recent pneumonia had significant increases in operative mortality (p < 0.05). In multivariable analysis incorporating the STS predicted risk models, liver disease, elevated Model for End-Stage Liver Disease score, prolonged 5-meter walk test, home oxygen use, bronchodilator therapy, and abnormal pulmonary function tests were independently predictive of mortality. Several of the new STS data variables were significantly associated with operative mortality after cardiac surgery. The addition of these patient factors improves our understanding of evolving patient demographics and comorbid conditions and their impact on perioperative risk. This will improve both shared decision making and assessments of provider performance.

Implementation of the Standards-Based Management and Recognition approach to quality improvement in maternal, newborn, and child health programs in low-resource countries

The Standards-Based Management and Recognition (SBM-R; Jhpiego, Baltimore, MD, USA) approach to quality improvement was developed by Jhpiego to respond to common challenges faced by health systems in low-resource settings, including poor pre-service education, lack of resources for conventional supervisory models, and weak health information systems. Since its introduction in Brazil in 1997, SBM-R has been implemented in approximately 30 countries and continues expanding to new places and service delivery areas. The present article: (1) describes key steps in the SBM-R methodology focusing on provider performance assessment using evidence-based standards; and (2) presents examples of improvements in provider performance in maternal, newborn, and child health care following SBM-R implementation derived from routine program data, quasi-experimental evaluations, and in-depth case studies. SBM-R incorporates evidence-based methods that are known to have positive effects on healthcare quality, including audit and feedback, educational outreach visits, and checklist usage; however, further rigorous research is needed to document the population-level impacts of the SBM-R approach.

OPMDC: Architecture Design and Implementation of a New Optical Pyramid Data Center Network

In this paper, we propose a novel optical pyramid data center network architecture (OPMDC), achieving scalable and high bandwidth, low latency, and reduced power consumption and wiring complexity. Based on an incremental and modular design, a full-scale OPMDC is built on three types of WSS-based optical switching nodes in three tiers. These optical nodes are recursively interconnected according to a pyramid structure parameterized by the number of nodes ( B ) at the base. Such a pyramid-based topology facilitates horizontal mesh connections that are tailored to achieve flexible optical packet-based transport. To this aim, we design a wavelength scheduling algorithm, called most-contentious-first, augmented with source and destination relay and aggregation (SDRA) via the horizontal mesh connections in the source and destination pods. Simulation results show that employing SDRA results in a vast improvement of throughput from 42.5% to 87.9% at full load, at a cost of no more than two additional hops of latency. Further, we present our OPMDC ( $B = 7$ ) prototype and provide performance assessment and the measurements of packet latency. Finally, experimental results show that after travelling the longest possible path with six optical nodes in OPMDC, the signal suffers no accumulated distortion and noise from the cascades of WSS and EDFA modules.

Statistical Benchmarks for Health Care Provider Performance Assessment: A Comparison of Standard Approaches to a Hierarchical Bayesian Histogram‐Based Method

Examine how widely used statistical benchmarks of health care provider performance compare with histogram-based statistical benchmarks obtained via hierarchical Bayesian modeling. Publicly available data from 3,240 hospitals during April 2009-March 2010 on two process-of-care measures reported on the Medicare Hospital Compare website. Secondary data analyses of two process-of-care measures comparing statistical benchmark estimates and threshold exceedance determinations under various combinations of hospital performance measure estimates and benchmarking approaches. Statistical benchmarking approaches for determining top 10 percent performance varied with respect to which hospitals exceeded the performance benchmark; such differences were not found at the 50 percent threshold. Benchmarks derived from the histogram of provider performance under hierarchical Bayesian modeling provide a compromise between benchmarks based on direct (raw) estimates, which are overdispersed relative to the true distribution of provider performance and prone to high variance for small providers, and posterior mean provider performance, for which over-shrinkage and under-dispersion relative to the true provider performance distribution is a concern. Given the rewards and penalties associated with characterizing top performance, the ability of statistical benchmarks to summarize key features of the provider performance distribution should be examined.

Benchmarking urban acute care hospitals

Over the last couple of decades, hospitals in the United States are facing pressures to maximize performance in terms of production efficiency and quality. An increasing emphasis on value-based purchasing on the part of third-party payers as well as the prevalence of pay for performance initiatives create an imperative for more accurate assessments of health care provider performance. The objectives of this study were to measure hospital performance in terms of both technical efficiency and quality using data envelopment analysis (DEA) models in urban acute care hospitals. In this observational cross-sectional study of a nationally representative sample of 371 urban acute care hospitals, hospital performance was assessed using slack-based additive DEA models. The technical inputs included in the DEA models were total number of beds setup and staffed, nonphysician full-time equivalent staffing, and nonpayroll operating expenses. The technical outputs were adjusted patient days, total number of outpatient visits, and training full-time equivalent, obtained from the American Hospital Association 2008 database. The quality measures used for the quality of care dimension of performance were survival rates for acute myocardial infarction, congestive heart failure, and pneumonia obtained from the Nationwide Inpatient Sample 2008 data. Less than 20% of the sample hospitals were optimally performing for both quality and efficiency. Tobit regression analysis of the DEA scores found that public, small, teaching hospitals had higher DEA efficiency and quality scores. DEA is a promising tool for benchmarking both aspects of performance: efficiency and quality of hospitals. Because quality is a multidimensional construct, the choice of an appropriate composite quality measure has to be addressed in future research. However, incorporating quality into the DEA models would be a better reflection of the hospital product.

Is patient satisfaction in primary care dependent on structural and organizational characteristics among providers? Findings based on data from the national patient survey in Sweden

In parallel to market-like reforms in Swedish primary care, the gathering and compilation of comparative information about providers, for example through survey tools, has been improved. Such information is increasingly being used to guide individuals' choice of provider and payers' assessments of provider performance, often without critically reflecting about underlying factors affecting the results. The purpose of this study was to analyze variation in patient satisfaction, with respect to organizational and structural factors, including the mix of registered individuals, among primary care providers, based on information from a national patient survey in primary care and register data in three Swedish county councils. Systematic variation in patient satisfaction was found with respect to both organizational and structural factors, including characteristics of registered individuals. Smaller practices and practices where a high proportion of all visits were with a doctor were associated with higher patient satisfaction. Also practices where registered individuals had a low level of social deprivation and a high overall illness on average were associated with higher patient satisfaction. Factors that are of relevance for how well providers perform according to patient surveys are more or less possible to control for providers. This adds to the complexity for the use of such information by individuals and payers to assess provider performance.

Development of Peer‐Group‐Classification Criteria for the Comparison of Cost Efficiency among General Hospitals under the Korean NHI Program

To classify general hospitals into homogeneous systematic-risk groups in order to compare cost efficiency and propose peer-group-classification criteria. Health care institution registration data and inpatient-episode-based claims data submitted by the Korea National Health Insurance system to the Health Insurance Review and Assessment Service from July 2007 to December 2009. Cluster analysis was performed to classify general hospitals into peer groups based on similarities in hospital characteristics, case mix complexity, and service-distribution characteristics. Classification criteria reflecting clustering were developed. To test whether the new peer groups better adjusted for differences in systematic risks among peer groups, we compared the R(2) statistics of the current and proposed peer groups according to total variations in medical costs per episode and case mix indices influencing the cost efficiency. A total of 1,236,471 inpatient episodes were constructed for 222 general hospitals in 2008. New criteria were developed to classify general hospitals into three peer groups (large general hospitals, small and medium general hospitals treating severe cases, and small and medium general hospitals) according to size and case mix index. This study provides information about using peer grouping to enhance fairness in the performance assessment of health care providers.

An in-depth assessment of a diagnosis-based risk adjustment model based on national health insurance claims: the application of the Johns Hopkins Adjusted Clinical Group case-mix system in Taiwan

BackgroundDiagnosis-based risk adjustment is becoming an important issue globally as a result of its implications for payment, high-risk predictive modelling and provider performance assessment. The Taiwanese National Health Insurance (NHI) programme provides universal coverage and maintains a single national computerized claims database, which enables the application of diagnosis-based risk adjustment. However, research regarding risk adjustment is limited. This study aims to examine the performance of the Adjusted Clinical Group (ACG) case-mix system using claims-based diagnosis information from the Taiwanese NHI programme.MethodsA random sample of NHI enrollees was selected. Those continuously enrolled in 2002 were included for concurrent analyses (n = 173,234), while those in both 2002 and 2003 were included for prospective analyses (n = 164,562). Health status measures derived from 2002 diagnoses were used to explain the 2002 and 2003 health expenditure. A multivariate linear regression model was adopted after comparing the performance of seven different statistical models. Split-validation was performed in order to avoid overfitting. The performance measures were adjusted R2 and mean absolute prediction error of five types of expenditure at individual level, and predictive ratio of total expenditure at group level.ResultsThe more comprehensive models performed better when used for explaining resource utilization. Adjusted R2 of total expenditure in concurrent/prospective analyses were 4.2%/4.4% in the demographic model, 15%/10% in the ACGs or ADGs (Aggregated Diagnosis Group) model, and 40%/22% in the models containing EDCs (Expanded Diagnosis Cluster). When predicting expenditure for groups based on expenditure quintiles, all models underpredicted the highest expenditure group and overpredicted the four other groups. For groups based on morbidity burden, the ACGs model had the best performance overall.ConclusionsGiven the widespread availability of claims data and the superior explanatory power of claims-based risk adjustment models over demographics-only models, Taiwan's government should consider using claims-based models for policy-relevant applications. The performance of the ACG case-mix system in Taiwan was comparable to that found in other countries. This suggested that the ACG system could be applied to Taiwan's NHI even though it was originally developed in the USA. Many of the findings in this paper are likely to be relevant to other diagnosis-based risk adjustment methodologies.

Online wind turbine fault detection through automated SCADA data analysis

AbstractThis paper describes a set of anomaly‐detection techniques and their applicability to wind turbine fault identification. It explains how the anomaly‐detection techniques have been adapted to analyse supervisory control and data acquisition data acquired from a wind farm, automating and simplifying the operators' analysis task by interpreting the volume of data available. The techniques are brought together into one system to collate their output and provide a single decision support environment for an operator. The framework used is a novel multi‐agent system architecture that offers the opportunity to corroborate the output of the various interpretation techniques in order to improve the accuracy of fault detection. The results presented demonstrate that the interpretation techniques can provide performance assessment and early fault identification, thereby giving the operators sufficient time to make more informed decisions regarding the maintenance of their machines. Copyright © 2009 John Wiley &amp; Sons, Ltd.

Key Issues in Assessing the Performance of Microfinance Institutions

The microfinance revolution and the tremendous growth of the industry have drawn attention to performance assessment of microfinance providers, the operations of which are often supported by substantial public funds. In this paper we discuss five related issues: a) the search for an adequate measure of financial sustainability, b) the trade-off between outreach and sustainability, c) the question of how to determine the opportunity cost of concessionary borrowings, d) the conceptual framework of performance assessment, and e) the integration of saving services into performance assessment and subsidy dependence measures.