Reliability of online self-report versions of the Australian Treatment Outcome Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people in residential treatment for substance use problems

Abstract

BackgroundOnline self-report measures are resource-efficient and widely used for monitoring substance use, yet few studies have assessed their reliability. This study assessed the reliability of online self-report versions of the Australian Treatment Outcomes Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people seeking treatment for substance use problems. ProceduresOne-hundred and five clients entering residential treatment for substance use problems (Mage = 33.34, 65% male) were recruited from two facilities in Queensland, Australia. Using a repeated-measures design, we compared online self-report with the original interview versions of the ASSIST, which measured (i) lifetime substance use and (ii) past 3-month substance-use and related harms, and the ATOP, which measured (i) past month frequency of substance use and (ii) the typical quantity used per day. Assessments were administered 1–7 days apart. FindingsThe ATOP demonstrated moderate-excellent inter-rater reliability for the past month use (yes/no) for all substance types, but had poor reliability for alcohol and cannabis. ATOP reliability was high-excellent for the total number of days used in the past month for all substances. The ASSIST demonstrated moderate-excellent inter-rater reliability for substance-use and related harms for all substances except tobacco, however was poor for lifetime use for most substances due to greater reporting in the interview assessment. ConclusionsReliable responding was observed for the frequency of substance use in the past month on the ATOP, and past 3-month substance-use and related harms on the ASSIST. These findings support use of online-self report measures a resource-efficient method to monitor substance use. Underreporting of lifetime use was found in self-report online version of the ASSIST, highlighting the need for improved instruction or interviewer/clinician assistance for lifetime use.