Spatio-Semantic Graphs From Picture Description: Applications to Detection of Cognitive Impairment.

Pranav S Ambadi,Julie M Liss,Kimberly D Mueller,Kristin Basche,Rebecca L Koscik,Visar Berisha

doi:10.3389/fneur.2021.795374

Abstract

Clinical assessments often use complex picture description tasks to elicit natural speech patterns and magnify changes occurring in brain regions implicated in Alzheimer's disease and dementia. As The Cookie Theft picture description task is used in the largest Alzheimer's disease and dementia cohort studies available, we aimed to create algorithms that could characterize the visual narrative path a participant takes in describing what is happening in this image. We proposed spatio-semantic graphs, models based on graph theory that transform the participants' narratives into graphs that retain semantic order and encode the visuospatial information between content units in the image. The resulting graphs differ between Cognitively Impaired and Unimpaired participants in several important ways. Cognitively Impaired participants consistently scored higher on features that are heavily associated with symptoms of cognitive decline, including repetition, evidence of short-term memory lapses, and generally disorganized narrative descriptions, while Cognitively Unimpaired participants produced more efficient narrative paths. These results provide evidence that spatio-semantic graph analysis of these tasks can generate important insights into a participant's cognitive performance that cannot be generated from semantic analysis alone.

Highlights

Asking patients to describe a complex picture is a mainstay of clinical assessment tasks in aphasia, and increasingly so in the context of cognitive decline and dementia [1]
The Cookie Theft picture description task has been widely studied for its ability to elicit symptoms associated with early dementia and Alzheimer’s disease [6, 25]
We developed an approach to extract novel additional information from transcriptions of The Cookie Theft picture descriptions using graph theory and spatio-semantic graphs

Summary

Introduction

Asking patients to describe a complex picture is a mainstay of clinical assessment tasks in aphasia, and increasingly so in the context of cognitive decline and dementia [1]. Transcripts of the spoken picture descriptions are coded by hand by trained individuals to tag parts of speech, content information units (CIUs) (content units, semantic relevance), empty speech, repetitions, among others; as well as acoustic measures extracted from speech recordings [1, 5,6,7]. These data have been used to detect preclinical changes in cognitive-linguistics and differentiate among dementia etiologies such as Alzheimer’s (AD), frontotemporal dementia (FTD), dementia due to Parkinson’s disease (PD) and dementia with Lewy bodies (DLB) [8,9,10]

Objectives

Methods

Results

Conclusion