Abstract
ObjectiveTo determine whether a large set of care performance indicators (‘Intelligent Monitoring (IM)’) can be used to predict the Care Quality Commission’s (CQC) acute hospital trust provider ratings.DesignThe IM dataset and first-inspection ratings were used to build linear and ordered logistic regression models for the whole dataset (all trusts). This was repeated for subsets of the trusts, with these models then applied to predict the inspection ratings of the remaining trusts.SettingThe United Kingdom Department of Health and Social Care’s Care Quality Commission is the regulator for all health and social care services in England. We consider their first-inspection cycle of acute hospital trusts (2013–2016).ParticipantsAll 156 English NHS acute hospital trusts.Intervention(s)None.Main Outcome Measure(s)Percentage of correct predictions and weighted kappa.ResultsOnly 24% of the predicted overall ratings for the test sample were correct and the weighted kappa of 0.01 indicates very poor agreement between predicted and actual ratings. This lack of predictive power is also found for each of the rating domains.ConclusionWhile hospital inspections draw on a much wider set of information, the poor power of performance indicators to predict subsequent inspection ratings may call into question the validity of indicators, ratings or both. We conclude that a number of changes to the way performance indicators are collected and used could improve their predictive value, and suggest that assessing predictive power should be undertaken prospectively when the sets of indicators are being designed and selected by regulators.
Highlights
Healthcare regulators use a variety of methods to assess the performance of healthcare providers
Most use some form of inspection or survey visits, at which regulatory agency staff and other experts gather a range of data and assess performance, often against a set of regulatory standards or requirements
In 2013, the Care Quality Commission (CQC) developed and began to implement a new model for regulating NHS acute hospitals [4], and it has recently completed its first cycle of inspections
Summary
Healthcare regulators (in some jurisdictions termed inspectorates, or accreditation bodies) use a variety of methods to assess the performance of healthcare providers. Most use some form of inspection or survey visits, at which regulatory agency staff and other experts gather a range of data and assess performance, often against a set of regulatory standards or requirements. These inspections are used to arrive at a judgement or rating of performance [1, 2]. The Francis Inquiry [8] examined the systems for oversight, scrutiny and regulation in the NHS, which had permitted the failures in care at Stafford Hospital, and its many detailed recommendations reinforced the need for change in how regulators identify and respond to variations in performance
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
