REPORTING BIAS AND SELF‐REPORTED DRUG USE

Abstract

Percy and colleagues report that 17% of participants reporting cannabis use at recruitment to a large prospective study later denied that they had ever used cannabis [1]. ‘Recanting’ of reports of use of most other illicit drugs was even higher and the strongest predictor of recanting was interim exposure to drugs education. We agree with the suggestion of Fendrich that this casts doubt on the strength of evidence for the effectiveness of most drugs prevention interventions [2]. But what these data illustrate is reporting bias, and this has wider implications for epidemiological studies on correlates of illicit drug use—particularly studies on cannabis, usually the only illicit drug whose use is common enough in general population samples for effects to be estimated [3]. Reporting of drug use will be influenced by perceptions of social desirability. In the extreme scenario, teaching young people that drugs use is undesirable leads some of them to deny previously reported use. Effects may be subtler than this. Drug use attracts official disapproval. This may lead some to play down (rather than completely deny) their own use; for others—those attracted to the sort of outlaw cachet that can be seen as being associated with drug use—it may lead to exaggerated reports. The perceptions that influence reports of drug use are also likely to influence reports of any phenomenon carrying social stigma. Thus reports of all these constructs are likely to be deflated or inflated depending on whether the informant values the appearance of social conformity or prefers badges of rebellion or outsider status. In this situation where the same reporting bias influences measures of both exposure and outcome in the same direction then automatic, yet spurious, effects will appear. Indeed, even where only exposure assessments are influenced effect estimates may still be biased if the outcome in question, educational attainment for example, is associated with notions of social acceptability. Most evidence on the health and social consequences of illicit drug use rests on uncorroborated self-reports of use, often in relation to self-reported, socially stigmatised outcomes [3]. Reporting bias can generate apparently strong and robust effects. For example, amongst 6000 middle-aged men, self-reported psychological stress more than doubled the incidence of self-reported angina [4]. Yet in the same men, stress was not associated with increased risk of any objective index of heart disease. The stress–angina association probably arose through the influence of reporting bias on both exposure and outcome assessments. This wasn’t an issue of measure validity; the stress and angina measures were highly valid by conventional criteria [5]. Because of this, whilst we agree with Fendrich's suggestions on improving the validity of adolescent drug use measurement, we doubt these strategies will overcome the issue of reporting bias, other than in the most extreme cases [2]. Instead what we need are objective measures—mainly of drug use but also, where possible, of the outcomes drug use may be related to [6]. These measures should be collected alongside self-reports so that effects suggested by both can be compared. The objective measures are unlikely to be more valid (they may be less valid) but since reporting bias does not influence them, effects suggested by them are unlikely to be artefacts of bias. Objective measures of drug use are seldom used in epidemiological studies of adolescents, perhaps because of ethical concerns. However, since greater validity is not the rationale for their use the associated ethical issues are the same as those faced by any study measuring adolescent drug use in any way. Toxicology has limitations; for example it can only be used to corroborate reports of recent drug use [6]. Since regular, rather than ‘ever’, use is likely to be of greater epidemiological significance, this drawback may not be critical. As long as we remain unable to discount the influence of reporting bias on the available evidence of the causes and consequences of illicit drug use the meaning of much of this evidence will remain uncertain.