Self selection and sample selection in a treatment study of social phobia

Abstract

The external validity or generalizability of a treatment outcome study may be influenced by the choices patients make (self-selection) and by study requirements intended to increase internal validity or protect subject welfare (e.g. sample selection). Although these effects are well-known, they have rarely been studied, and little is known about the impact they may have on generalizability of findings. In this study, subjects accepting random assignment to a larger study examining pharmacologic and cognitive-behavioral treatments for social phobia were compared with subjects refusing random assignment (i.e. self-selected) and excluded subjects (i.e. “sample-selected”). “Acceptors” differed from “refusers” on some measures suggesting that they have a lower annual income and may have fewer social supports. Therefore, they may have fewer available resources and might be more willing to accept help wherever it is offered. Despite these differences, acceptors differed from refusers on just one pretreatment measure of clinical functioning. Excluded subjects did not differ from refusers or acceptors on demographic or pretreatment clinical measures. Refusers and excluded subjects were treated with the same cognitive-behavioral treatment used in the comparative outcome study, and their posttreatment data were compared with the subgroup of acceptors who were randomly assigned to that treatment. Again, there were few significant differences. These results suggest that while self-selection and sample selection may have influenced some characteristics of the sample in this comparative outcome study, the three groups of subjects were clinically similar and responded similarly to cognitive-behavioral treatment of social phobia.