We must do a better job of cumulating knowledge.

Abstract

CPA Award for Distinguished Contributions in the Application of Psychology, 2001 Prix de la SCP pour contributions remarquables a l'application de la psychologie Abstract It is argued that our attempts at knowledge cumulation have been flawed in four ways. They are the eroding of empiricism in clinical practice, the tendency towards paradigm passion and ethnocentricism, the failure to attend to simple measures of effect size, and the misuse of significance testing. It is recommended that speciality designations, the replacement of significance testing with point estimates and confidence intervals, the use of practical effect size statistics, the establishment of data repositories, and a renewed focus on replication would help resolve some of these problems. On occasions such as this, it is usually the case, and quite appropriate I might add, that those so honoured summarize their research accomplishments. I am going to resist doing that, however, because the confidence interval about my point estimate worth as a scientist is embarrassingly wide. As testament to this realization (Gendreau, 2000), I remember making what I thought were trenchant criticisms of the then-fashionable addiction prone theory (circa 1960s) only to realize later that I had used the same fallacious logic in presenting my case as I had previously accused my opponents of doing. This was followed by my generating a recidivism index for Canada that was upside down and then, when I became a clinical administrator in corrections, I took over a successful token economy and introduced 300% inflation in two weeks. Of course, being a psychologist, none of the above had any negative consequences for either my citation index or career in general. Rather, what I wish to address is an issue that I am confident I am correct about because, for one good reason, none of the ideas originated with me. In my view, we must do a better job of cumulating knowledge in order to generate helpful policies for people in need. I suggest that this is best done by: (1) reaffirming the scientist-practitioner model; (2) avoiding paradigm passion and ethnocentrism; (3) reporting statistics suited to knowledge cumulation; and (4) most importantly, giving significance testing a delay of game penalty. To illustrate, I will draw upon examples from my own field of study. Four Antidotes for Bad Knowledge Cumulation COULD WE HAVE A LITTLE MORE SCIENCE IN OUR CLINICAL PRACTICE? I am on shaky ground when grousing about the sad state of affairs regarding the decline of the scientist-- practitioner model as I have no hard data. In asserting this claim, I am influenced by occasional protestations penned by senior citizen psychologists of my generation and rumours about cadres of clinical graduate students enchanted by the siren call of grounded theory and the bizarre belief in the law of small numbers (cf. Potter & Wetherell, 1987) wherein unstructured interviews with 10 souls provides sufficient data to generalize the conclusions to all and sundry. I have also been periodically informed by some clinicians that it is not their responsibility to generate knowledge that informs their practice. Having supervised clinicians for a number of years, and been one myself, I am somewhat empathetic with that position given the nature of their jobs. If clinicians, however, carry this position to its logical extension and disclaim all responsibility in this regard, I fear the next step will be not even bothering to keep up with their relevant literatures. The substance of my case rests on two examples: the continued use of the MMPI in corrections and the generally poor quality of correctional programming. One would think that 50 years of MMPI administration (is there an offender anywhere in North America who has not had to complete one?) would have generated hundreds of predictive validities in the literature attesting to the utility of the measure, particularly the Pd scale, which is the granddaddy measure for assessing offenders' criminal propensity. …