What is actually being done in most addiction treatment programs? The truth is that no one really knows, including the administrators and supervisors of those programs. One thing I have learnt in four decades of treatment outcome research is that what therapists actually do behind closed doors, even with specific training and with video or audiotape recorders running, often bears surprisingly little resemblance to what they say and believe they are doing. Self-reported skill in a specific treatment method is at best related modestly to actual behavioral proficiency and fidelity in delivering it [1]. Training does not ensure the acquisition or maintenance of proficiency. To know what is actually being done in treatment, there is quite simply no alternative to direct observation of practice. This indicates a need for a sea change in the culture of addiction service delivery, where treatment is done currently in private and often without any real clinical supervision. Medical care in general is much more observable and accountable than counseling. Primary care providers are expected to document in detail what they do, and their work interacts with a steady stream of other health care professionals (and students in teaching facilities) coming in and out of the treatment room. Surgery is not usually performed one-to-one in private, but is witnessed and assisted by others around the table, or observing live or via videotape. Addiction treatment needs to come out of the closet, and become an enterprise of open professional collaboration for continuous quality improvement. Competent supervision without observation is an oxymoron. The quality of treatment-as-usual can be quite good. We compared head-to-head the drinking outcomes of clients treated by the closely supervised manual-guided therapies in Project MATCH with those for clients admitted during the same period to normal treatment at the Albuquerque clinical site, who received whatever treatment their assigned therapist delivered. The outcomes for these two groups were virtually identical [2]. As Carroll & Rounsaville observe in this issue [3], it is not a foregone conclusion that retraining clinical staff in evidence-based treatment (EBT) methods will necessarily improve client outcomes. Site × treatment interactions in multi-site clinical trials indicate that such training improves outcomes in some programs but not others. Similarly, therapist training needs to come out of the closet. Training by reading books, watching videotapes and attending workshops usually has little or no impact on actual practice [4]. Rather than a fixed dose of instruction, training should be to a criterion of proficiency in observed practice. Counselors will vary widely in the length and amount of training needed to reach and maintain proficiency in an EBT. Meanwhile, there are several practical things to be undertaken to advance the standard of delivering EBTs in addiction services. Clearly, we ought to be training the next generation of providers to be proficient in EBTs. Shaping new practice is easier than retraining and unlearning established practice habits. It is simply unethical to continue training new providers in unproven or (worse) discredited treatment methods. In clinical policy, we should at a minimum call a moratorium on the use of treatment methods that have been shown to be generally ineffective or harmful, as one would hope and expect happens in health care more generally. In hiring new treatment staff, priority can be given to selecting those who demonstrate the characteristics of effective therapists (such as accurate empathy; [5,6]) and proficiency in EBTs. All this depends on the systemic change of making addiction treatment practices more observable and accountable, which in turn offers providers the opportunity, supervision and feedback needed to become better at what they do.