Inconsistent prescribing behaviour by physicians: Its effect on the validity of written case simulations

Abstract

Objectives: The prescribing behaviour of doctors can be assessed using various methods, such as record review or simulated case evaluation. The validity of a method is often assessed by comparing the behaviour measured by it with the best external measure of this behaviour available. Such validation using repeated measurements requires that the behaviour measured is consistent over a certain period of time. A study has been conducted to establish the extent to which possible inconsistencies in prescribing behaviour of general practitioners may affect the validity assessment of written case simulations.Methods: We have reanalysed the material from a previous validity study in which data on prescribing in uncomplicated urinary tract infections were collected from 25 general practitioners. Two methods were used for data collection: real consultations with undetected standardised patients and written cases of similar patients. To test the validity of the written cases, the agreement was calculated between the prescribing decisions for the written cases and the real consultations. This measure of agreement was adjusted for the inconsistency in repeated measurement of prescribing behaviour, which was assessed by conducting a test-retest comparison of the written case simulation in another sample of 20 general practitioners.Results: The agreement between the treatment choices for the standardised patient and the written case was 65.2% (kappa 0.57). The agreement between the test and the retest of the written case was 63.2% (kappa 0.51). Adjusting for the apparent inconsistency, there seems to be no actual disagreement between the drug choices made for written cases and during real consultations with standardised patients.Conclusions: The observed disagreement between prescribing behaviour for a real case and a written case, which in the past has been fully attributed to a lack of validity, can also be explained by inconsistent prescribing behaviour of physicians. In general, the large variation observed in drug selection for identical cases implies that one should be careful when drawing conclusions from differences found in repeated measurements of prescribing behaviour.