Multi‐modal inadvertent signals enabling the capture of the learning process in real‐time. A new opportunity for preclinical intervention?

Abstract

AbstractBackgroundGold standard episodic memory tests used to detect AD‐related cognitive decline are insensitive to subjective cognitive decline evident in preclinical AD. We believe a breakdown in the healthy learning process may be a cornerstone in preclinical AD cognitive impairment. We demonstrate the ability to capture the learning process in real‐time using strategic test design, inadvertent eye‐movements and digital ink behavior. We show early evidence of previously unseen learning strategies using data from three participants.MethodWe used the Digital Symbol Digit Learning Test (dSDLT) that we designed to explore learning. One condition presents a key that pairs elementary geometric symbols with numbers and a section of symbols alone, and is administered twice. The participant writes the appropriate digit beneath each symbol using a digitizing pen (providing time‐stamped digital ink coordinates) while wearing a lightweight PupilLabs eye‐tracker (permitting unconstrained head movement, reporting gaze position 240 times/sec to ±2 mm). Time‐synchronized signals measure gaze time in the key. Heat maps show fixation points in the key and response boxes.ResultFigure 1 shows gaze time in the key for each administration (by item, in sequence for one subject), revealing no time in key for some items and less total time in the key for the second administration. We interpret no time in key as paired‐learning. Fixation heat maps showed an expected gaze pattern of looking from the stimulus symbol up to and scanning the key. Unexpected gaze patterns also emerged: Targeted gazes from the stimulus symbol directly to the corresponding symbol in the key, Figure 2, (suggesting spatial placement learning); Within‐key symbol comparisons, indicated by looking back‐and‐forth between items, suggesting figure discrimination learning (Figure3); and Referring back to a recently completed response box containing the same symbol, suggesting working memory (Figure 4).ConclusionMultimodal behavior capture combining inadvertent eye‐tracking signals with digital ink enables the capture of real‐time learning and provides insights into what participants are thinking in addition to what they are doing. The identification of unexpectedly rich, previously unseen learning behaviors provides a glimpse of the complex healthy learning process, provides opportunity to identify strategies heralding preclinical AD and potential preclinical intervention targets.