Do “real world” studies on antipsychotics tell us the real truth?

Abstract

In recent years, the so-called “effectiveness” or “real world” studies (“pragmatic trials”) have gained increasing importance, claiming that they can give a better answer to questions related to efficacy and side effects of psychopharmacological treatment than phase III studies. However, the actual advantage of these “effectiveness” studies on antipsychotics remains questionable 1. This does not deny, though, that effectiveness studies, as well as other kinds of phase IV studies, can give a complementary view of the results of phase III studies. Some effectiveness studies appear to have a different kind of selection of patients than phase III trials, but they are not at all representative for average clinical samples. Often, patients with milder or more chronic symptoms may be selected than is the case in phase III studies, thus making it more difficult to demonstrate drug effects, and in particular differences between drugs, because a relevant subgroup of patients might be partially unresponsive to a drug. In contrast to phase III studies, the “real world” approach allows more comorbidity, comedication, etc., so that a broader range of information may be obtained than from the respective phase III studies. However, there is often no differentiated analysis of the influence of these variables. Thus, no advantage is taken of the chance to learn more about these “confounders”. On the other hand, the inclusion of such “confounders” (from the perspective of a phase III trial) increases the variance and results in a reduced signal-to-noise ratio, which makes it more difficult to find differences between two groups (beta error problem), even if these factors are adequately considered in the statistical analysis. It might sometimes even be difficult to judge without placebo conditions whether there is a real drug effect, especially if the pre-post difference is unexpectedly low and if there are no differences between two active comparators. It should be questioned whether so-called pragmatic primary outcome criteria such as “discontinuation”, or similar categorical endpoints like “level of caring”, really are ideal outcome criteria, given the fact that they can easily be influenced by the investigators (who may be biased by their expectations if they are not blinded) and are of poorer psychometric value than dimensional ones. Another measure of global outcome used as a primary outcome criterion in effectiveness studies is “quality of life”. There is no doubt that this is an important outcome criterion, which reflects the subjective dimension of the patient’s experience. The classical approach assesses quality of life using a self-rating scale in order to guarantee the subjective perspective. There are pros and cons for the use of self-rating scales. They give a complementary view to the observer rating of the same construct/dimension 1. The correlation between the observer ratings and self ratings might not be high and may be quite changeable, depending on the psychopathological state in terms of severity and type of symptoms. It is often unclear what exactly self ratings of quality of life reflect. If such a scale is used as the primary outcome criterion of a study, it is doubtful whether it is sensitive enough to detect inter-group differences of treatment-induced changes, given the high variance of self rating in general and of self ratings of quality of life in particular. In summary, because of the less restrictive methodology, effectiveness studies are not able to falsify the results of carefully designed phase III studies, but they can only give a complementary view. Despite the amount of attention being paid to them, we should not start to doubt earlier findings from phase III studies on antipsychotics, but should continue to consider the full array of evidence and use it to guide an evidence-based approach to treatment 2.