Abstract
BackgroundThe Care Quality Commission regulates, inspects, and rates general practice providers in England. Inspections are costly and infrequent, and are supplemented by a system of routine quality indicators, measuring patient satisfaction and the management of chronic conditions. These indicators can be used to prioritise or target inspections.AimTo determine whether this set of indicators can be used to predict the ratings awarded in subsequent inspections.Design and settingThis cross-sectional study was conducted using a dataset of 6860 general practice providers in England.MethodThe indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system (‘outstanding’, ‘good’, ‘requires improvement’, and ‘inadequate’) for domain ratings and the ‘overall’ rating. Predictive accuracy was assessed using the percentage of correct predictions and a measure of agreement (weighted κ).ResultsThe model correctly predicted 79.7% of the ‘overall’ practice ratings. However, 78.8% of all practices were rated ‘good’ on ‘overall’, and the weighted κ measure of agreement was very low (0.097); as such, predictions were little more than chance. This lack of predictive power was also found for each of the individual domain ratings.ConclusionThe poor power of performance of these indicators to predict subsequent inspection ratings may call into question the validity and reliability of the indicators, inspection ratings, or both. A number of changes to the way data relating to performance indicators are collected and used are suggested to improve the predictive value of indicators. It is also recommended that assessments of predictive power be undertaken prospectively when sets of indicators are being designed and selected by regulators.
Highlights
MethodsThe indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system (‘outstanding’, ‘good’, ‘requires improvement’, and ‘inadequate’) for domain ratings and the ‘overall’ rating
Background The Care QualityCommission regulates, inspects, and rates general practice providers in England
The poor power of performance of these indicators to predict subsequent inspection ratings may call into question the validity and reliability of the indicators, inspection ratings, or both
Summary
The indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system (‘outstanding’, ‘good’, ‘requires improvement’, and ‘inadequate’) for domain ratings and the ‘overall’ rating. Predictive accuracy was assessed using the percentage of correct predictions and a measure of agreement (weighted κ). The IM dataset was obtained from the CQC. There were two releases of IM, with indicator values being updated as new data were published. The second release[13] was used; this covered the time period from April 2013 until December 2014 and, corresponded to the majority (90%) of inspections. There were no further updates to IM, so the indicator values for practices inspected later in the cycle were more than 2 years old
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
