How symbol and text combine to promote sign comprehension: Evidence from eye-tracking

Abstract

In practical scenarios, numerous symbolic signs may not be rapidly and accurately comprehended due to poor design or unfamiliarity regarding their intended meanings. Previous studies have shown that combining symbols with text effectively enhances the comprehension of signs. However, the underlying factors and cognitive mechanisms for this phenomenon have remained unexplored. To gain a deeper understanding of these factors, we categorized symbols into three comprehension levels: low, medium, and high, based on a standard comprehension test. Subsequently, we displayed signs using three different conditions: text-only, symbol-only, and symbol with text, making it a 3 (level of comprehension) x 3 (display condition) two-factor experiment. In the eye-tracking experiment, the Average Fixation Duration indicator suggests that, among the three display conditions, text-only exhibits the lowest cognitive processing difficulty, and the symbol-only display condition has the highest cognitive processing difficulty. Additionally, compared to the symbol-only display condition, the symbol with text display condition significantly reduces comprehension reaction time and substantially improves correct comprehension scores; the percentage of time spent on the text area of interest reveals the underlying reasons: participants allocate their visual attention to the text area according to the comprehension level of symbols. Furthermore, participants consistently direct their initial attention to the symbols in the symbol with text display condition. All these findings provide valuable insights for designing and displaying signs in practical applications.