Agreement between Computerized and Human Assessment of Performance on the Ruff Figural Fluency Test

Martin F Elderson,Johan Kok,Joris P J Slaets,Bruce H R Wolffenbuttel,Oliver Tucha,Ron T Gansevoort,Marlise E A Van Eersel,Lifelines Cohort Study ,Melanie M Van Der Klauw,Sander Pham,Gerbrand J Izaks

doi:10.1371/journal.pone.0163286

Martin F Elderson, Johan Kok + Show 9 more

Open Access

https://doi.org/10.1371/journal.pone.0163286

Copy DOI

Abstract

The Ruff Figural Fluency Test (RFFT) is a sensitive test for nonverbal fluency suitable for all age groups. However, assessment of performance on the RFFT is time-consuming and may be affected by interrater differences. Therefore, we developed computer software specifically designed to analyze performance on the RFFT by automated pattern recognition. The aim of this study was to compare assessment by the new software with conventional assessment by human raters. The software was developed using data from the Lifelines Cohort Study and validated in an independent cohort of the Prevention of Renal and Vascular End Stage Disease (PREVEND) study. The total study population included 1,761 persons: 54% men; mean age (SD), 58 (10) years. All RFFT protocols were assessed by the new software and two independent human raters (criterion standard). The mean number of unique designs (SD) was 81 (29) and the median number of perseverative errors (interquartile range) was 9 (4 to 16). The intraclass correlation coefficient (ICC) between the computerized and human assessment was 0.994 (95%CI, 0.988 to 0.996; p<0.001) and 0.991 (95%CI, 0.990 to 0.991; p<0.001) for the number of unique designs and perseverative errors, respectively. The mean difference (SD) between the computerized and human assessment was -1.42 (2.78) and +0.02 (1.94) points for the number of unique designs and perseverative errors, respectively. This was comparable to the agreement between two independent human assessments: ICC, 0.995 (0.994 to 0.995; p<0.001) and 0.985 (0.982 to 0.988; p<0.001), and mean difference (SD), -0.44 (2.98) and +0.56 (2.36) points for the number of unique designs and perseverative errors, respectively. We conclude that the agreement between the computerized and human assessment was very high and comparable to the agreement between two independent human assessments. Therefore, the software is an accurate tool for the assessment of performance on the RFFT.

Highlights

Cognitive decline is a common chronic condition in old age
All Ruff Figural Fluency Test (RFFT) protocols were assessed by the new software and two independent human raters
For the number of unique designs, the intraclass correlation coefficient between two human assessments was 0.995 (95%CI, 0.994 to 0.995; p

Summary

Introduction

Cognitive decline is a common chronic condition in old age. An estimated 36 million people live with dementia and it is expected that this number will double every twenty years to approximately 115 million in 2050 [1,2]. It is generally believed that dementia is the result of a long-term pathological process that spans at least two to three decades. This is supported by the recent finding that cognitive decline is already evident at the age of 45 years [3]. Cognitive decline is an important outcome in life course epidemiology and prospective cohort studies. Few cognitive test are sensitive to cognitive changes across life span, from young adulthood to old age

Objectives

Methods

Results

Discussion

Conclusion