Eye-tracking during concurrent scene and sentence presentation

Abstract

Eye-tracking has a long tradition in psycholinguistic research, for example helping to investigate human language comprehension in the presence of non-linguistic context. We present a concurrent scene and written sentence presentation study, where we manipulate congruence and sentence type of structurally ambiguous German sentences. The corresponding displayed scenes show role-ambiguous characters involved in depicted actions. We investigate the effects of depicted events on ambiguity resolution in the context of a congruence decision task. Our concurrent presentation experiment combines aspects of several recent studies on scenesentence integration. We put our results in context and present a comprehensive analysis of our data. The detailed analysis allows direct comparison with various aspects of existing and future results of other studies. We replicate some results from related studies and discuss parallels and differences in the results. We present some interesting interactions between sentence type and congruence effects. Another goal of our study is to evaluate the new open source eye-tracking environment NEWTRACK (running on FreeDOS) and our custom portable DPIFilter post-processing and visualization software. NEWTRACK is more suitable for our study than for example PCEXPT. We give an overview of the features of NEWTRACK (e.g. support for near-standard hardware and automatic stimulus image analysis) and assess the current performance. We also review the current limitations of NEWTRACK and make suggestions for future enhancements.