Multiple Fixations Research Articles

How Do Consumers Read and Encode a Price?

Abstract Do consumers really read a price from left to right, as assumed in past research? Or does price reading operate like word reading, with a single fixation toward the middle? Three eye-tracking lab studies reject both theories, revealing instead a distinct reading pattern: multiple fixations, with the first located on average between the first third and middle of the price; the first eye movement is usually to the left; and subsequent eye movements are as often to the left as to the right. Overall, consumers pay as much attention to cents as euros, with the cents part influencing how prices are encoded in memory, as evidenced by an in-store price-recall survey. The reading process identifies whether to encode a price verbally as is or replace it with a shorter substitute that is easier to memorize and turns out to be well correlated with the actual price (r = 0.952). When consumers compare two prices, eye movements and the subsequent subjective estimation of the price difference depend on whether or not the prices have identical integer parts. The combined findings of four studies suggest that consumers have developed a reliable, efficient ability to read and encode prices, despite limitations of their visual span and working memory.

Visual Short-Term Memory Persists Across Multiple Fixations: An n-Back Approach to Quantifying Capacity in Infants and Adults

Visual short-term memory (STM) is a foundational component of general cognition that develops rapidly during the first year of life. Although previous research has revealed important relations between overt visual fixation and memory formation, it is unknown whether infants can maintain distinct memories for sequentially fixated items or remember nonfixated array items. Participants (5-month-olds, 11-month-olds, and adults; n = 24 at each age) from the United States were tested in a passive change-detection paradigm with an n-back manipulation to examine memory for the last fixated item (one-back), second-to-last fixated item (two-back), or nonfixated item (change-other). Eye tracking was used to measure overt fixation while participants passively viewed arrays of colored circles. Results for all ages revealed convergent evidence of memory for up to two sequentially fixated objects (i.e., one-back, two-back), with moderate evidence for nonfixated array items (change-other). A permutation analysis examining change preference over time suggested that differences could not be explained by perseverative looking or location biases.

A Foveated Vision-Transformer Model for Scene Classification

Introduction: Humans can rapidly categorize scenes (R. VanRullen & S. Thorpe, 2001), even using peripheral vision (Larson & Loschky, 2009). Various computational models have been proposed for rapid scene categorization in terms of low-level properties such as spatial envelopes (Oliva & Torralba, 2001) and texture summary statistics (TTM, Rosenholtz et al., 2012). Yet, these models do not explicitly model the foveated properties of the visual system nor the interaction between eye movements and the scene category task. We propose a model with a foveated visual system and eye movements that can predict the dependence of human categorization performance across fixations. The model combines square pooling regions with the computer vision-transformer architecture (Dosovitskiy et al., 2020, Touvron et al., 2020) and makes multiple fixations to maximize classification using the technique of self-attention (Parikh et al., 2016, Bahdanau et al., 2015). Methods: Twenty-two participants classified 360 images (Places365 database, places2.csail.mit.edu) into 30 classes. Images subtended a viewing angle of 22.7 degrees. A gaze-contingent display was used to randomly interrupt the display after 1, 2, 3, or 4 fixations with initial forced-fixation at bottom-center or top-center. Results: We show that there is no significant improvement in performance after the 2nd fixation (Δ correct categorization=0.015; p=0.4729), unlike performance for object search (Koehler and Eckstein, 2017). The model correctly predicts modest classification improvements for free-viewing fixations (Δ=0.016). The model-human correlation in classification choices was not significantly lower than human-human correlations. Our findings suggest that human categorization of scenes within a single fixation can be explained by the spatially global distribution of the visual information in the scene and their availability even through the bottlenecks of the visual periphery. The newly proposed hybrid approach using biologically based modeling and Transformers can flexibly be applied to various naturalistic tasks and stimuli.

Quantifying the Predictability of Visual Scanpaths Using Active Information Storage.

Entropy-based measures are an important tool for studying human gaze behavior under various conditions. In particular, gaze transition entropy (GTE) is a popular method to quantify the predictability of a visual scanpath as the entropy of transitions between fixations and has been shown to correlate with changes in task demand or changes in observer state. Measuring scanpath predictability is thus a promising approach to identifying viewers’ cognitive states in behavioral experiments or gaze-based applications. However, GTE does not account for temporal dependencies beyond two consecutive fixations and may thus underestimate the actual predictability of the current fixation given past gaze behavior. Instead, we propose to quantify scanpath predictability by estimating the active information storage (AIS), which can account for dependencies spanning multiple fixations. AIS is calculated as the mutual information between a processes’ multivariate past state and its next value. It is thus able to measure how much information a sequence of past fixations provides about the next fixation, hence covering a longer temporal horizon. Applying the proposed approach, we were able to distinguish between induced observer states based on estimated AIS, providing first evidence that AIS may be used in the inference of user states to improve human–machine interaction.

Seeing Beyond Salience and Guidance: The Role of Bias and Decision in Visual Search.

Visual search is a popular tool for studying a range of questions about perception and attention, thanks to the ease with which the basic paradigm can be controlled and manipulated. While often thought of as a sub-field of vision science, search tasks are significantly more complex than most other perceptual tasks, with strategy and decision playing an essential, but neglected, role. In this review, we briefly describe some of the important theoretical advances about perception and attention that have been gained from studying visual search within the signal detection and guided search frameworks. Under most circumstances, search also involves executing a series of eye movements. We argue that understanding the contribution of biases, routines and strategies to visual search performance over multiple fixations will lead to new insights about these decision-related processes and how they interact with perception and attention. We also highlight the neglected potential for variability, both within and between searchers, to contribute to our understanding of visual search. The exciting challenge will be to account for variations in search performance caused by these numerous factors and their interactions. We conclude the review with some recommendations for ways future research can tackle these challenges to move the field forward.

The Scene Perception & Event Comprehension Theory (SPECT) Applied to Visual Narratives.

Understanding how people comprehend visual narratives (including picture stories, comics, and film) requires the combination of traditionally separate theories that span the initial sensory and perceptual processing of complex visual scenes, the perception of events over time, and comprehension of narratives. Existing piecemeal approaches fail to capture the interplay between these levels of processing. Here, we propose the Scene Perception & Event Comprehension Theory (SPECT), as applied to visual narratives, which distinguishes between front‐end and back‐end cognitive processes. Front‐end processes occur during single eye fixations and are comprised of attentional selection and information extraction. Back‐end processes occur across multiple fixations and support the construction of event models, which reflect understanding of what is happening now in a narrative (stored in working memory) and over the course of the entire narrative (stored in long‐term episodic memory). We describe relationships between front‐ and back‐end processes, and medium‐specific differences that likely produce variation in front‐end and back‐end processes across media (e.g., picture stories vs. film). We describe several novel research questions derived from SPECT that we have explored. By addressing these questions, we provide greater insight into how attention, information extraction, and event model processes are dynamically coordinated to perceive and understand complex naturalistic visual events in narratives and the real world.

ADOLESCENT IDIOPATHIC SCOLIOSIS AND SIMILAR CONDITIONS: NEW TOOL TO MEASURE THE CORONAL PLANE

ABSTRACT Objective: To create a method to measure the overall coronal plane of the spine, called the sacral clavicular angle (SCA). Methods: A line is drawn at the base of the sacrum; a second central line is drawn perpendicular to the first one in the proximal extension of the spine. A third line is drawn passing through the intersections of the superior points of the clavicles with the two second ribs, forming two angles, the greater of which is measured. Therefore, the degrees exceeding 90° are the SCA values. This tool was tested retrospectively in a study of 46 patients with idiopathic scoliosis who underwent short, apical, single or multiple fixations. Third generation instrumentation was used and the SCA was evaluated in the pre- and postoperative periods, which were compared with another group of 46 patients treated with the traditional technique. Results: Patients submitted to the traditional fixation method presented a median SCA of 3° in the preoperative period, and in the postoperative period, the SCA remained in 3°. Therefore, there was a 0% reduction. Patients submitted to short, apical, single or multiple fixation method presented a median preoperative SCA of 6°, and a postoperative median SCA of 1°, a reduction of approximately 83%. Conclusions: The group treated with short, apical, single or multiple instrumentation presented a reduction in the SCA of approximately 83% in the postoperative period compared to the preoperative period. The difference between preoperative and postoperative values was considered statistically significant. Level of Evidence II; Retrospective study.

ADOLESCENT IDIOPATHIC SCOLIOSIS AND SIMILAR CONDITIONS: SHORT, APICAL, SINGLE OR MULTIPLE FIXATION

ABSTRACT Objective: To create a new instrumentation principle for the treatment of adolescent idiopathic scoliosis (AIS) and similar conditions. Methods: A new fixation format was created for the treatment of AIS using 3rd generation instrumentation in short, apical and single or multiple fixations. In patients presenting one scoliotic curve, one fixation is performed. In patients with two curves, two fixations are performed, and in those with three curves, three fixations are performed. To evaluate the new method, a retrospective study of 54 patients who had already undergone surgery for AIS and similar conditions using this innovation was conducted. Results: In this series, average corrections of 72% in the proximal curve, of 83% in the thoracic curve, and of 85% in the thoracolumbar curve were verified. Conclusions: It was concluded that the new fixation principle in the treatment of AIS and similar conditions using short, apical and multiple fixations presented excellent correction, better still in the curves that were more distal in relation to the spine. There was a significant statistical difference between the percentages of curvature correction. Level of Evidence III; Retrospective study.

Examining the Role of Task Requirements in the Magnitude of the Vigilance Decrement.

The vigilance decrement in sustained attention tasks is a prevalent example of cognitive fatigue in the literature. A critical challenge for current theories is to account for differences in the magnitude of the vigilance decrement across tasks that involve memory (successive tasks) and those that do not (simultaneous tasks). The empirical results described in this paper examine this issue by comparing performance, including eye movement data, between successive and simultaneous tasks that require multiple fixations to encode the stimulus for each trial. The findings show that differences in the magnitude of the vigilance decrement between successive and simultaneous tasks were observed only when a response deadline was imposed in the analysis of reaction times. This suggests that memory requirements did not exacerbate the deleterious impacts of time on task on the ability to accurately identify the critical stimuli. At the same time, eye tracking data collected during the study provided evidence for disruptions in cognitive processing that manifested as increased delays between fixations on stimulus elements and between encoding the second stimulus element and responding. These delays were particularly pronounced in later stages of encoding and responding. The similarity of the findings for both tasks suggests that the vigilance decrement may arise from common mechanisms in both cases. Differences in the magnitude of the decrement arise as a function of how degraded cognitive processing interacts with differences in the information processing requirements and other task characteristics. The findings are consistent with recent accounts of the vigilance decrement, which integrate features of prior theoretical perspectives.

How do birds look at their world? A novel avian visual fixation strategy

A central assumption in behavioral research is that the observer knows where an animal is looking; however, establishing when an animal is gazing (i.e., visually fixating on an object) has been challenging in species with laterally placed eyes. We quantitatively tested three fixation strategies proposed in the literature for birds, using European starlings (Sturnus vulgaris). We did not find strong support for any of the three strategies, despite high statistical a priori power (93%). However, we did observe a new visual fixation strategy that we labeled monocular alternating fixation. In this strategy, starlings moved their heads to make multiple fixations with a single eye before switching to the other eye and repeating the same process. Additionally, we established that individuals favored using the left over the right eye, supporting that laterality in starlings is left-eye dominant. The newly observed fixation strategy may be associated with the high level of intra-retinal variation (density of photoreceptors, overall sensitivity of visual pigments, etc.) in the starling retina. From a functional perspective, this monocular alternating fixation strategy may be beneficial to integrate the different types of information gathered by the different portions of each retina more quickly. We discuss the implications of our results for designing and interpreting behavioral experiments that require an understanding of where a bird is looking. This is the first study to quantitatively test three hypotheses in the literature about how animals with laterally placed eyes look at objects. We found that there was not strong support for any of these three strategies, but found support for a newly described strategy for birds to look at objects (i.e., multiple looks with a single eye before switching to the other eye).

Object detection through search with a foveated visual system.

Humans and many other species sense visual information with varying spatial resolution across the visual field (foveated vision) and deploy eye movements to actively sample regions of interests in scenes. The advantage of such varying resolution architecture is a reduced computational, hence metabolic cost. But what are the performance costs of such processing strategy relative to a scheme that processes the visual field at high spatial resolution? Here we first focus on visual search and combine object detectors from computer vision with a recent model of peripheral pooling regions found at the V1 layer of the human visual system. We develop a foveated object detector that processes the entire scene with varying resolution, uses retino-specific object detection classifiers to guide eye movements, aligns its fovea with regions of interest in the input image and integrates observations across multiple fixations. We compared the foveated object detector against a non-foveated version of the same object detector which processes the entire image at homogeneous high spatial resolution. We evaluated the accuracy of the foveated and non-foveated object detectors identifying 20 different objects classes in scenes from a standard computer vision data set (the PASCAL VOC 2007 dataset). We show that the foveated object detector can approximate the performance of the object detector with homogeneous high spatial resolution processing while bringing significant computational cost savings. Additionally, we assessed the impact of foveation on the computation of bottom-up saliency. An implementation of a simple foveated bottom-up saliency model with eye movements showed agreement in the selection of top salient regions of scenes with those selected by a non-foveated high resolution saliency model. Together, our results might help explain the evolution of foveated visual systems with eye movements as a solution that preserves perceptual performance in visual search while resulting in computational and metabolic savings to the brain.

The Honeycomb illusion: Uniform textures not perceived as such

We present a series of patterns, in which texture is perceived differently at fixation in comparison to the periphery, such that a physically uniform stimulus yields a nonuniform percept. We call this the Honeycomb illusion, and we discuss it in relation to the similar Extinction illusion (Ninio & Stevens, 2000). The effect remains strong despite multiple fixations, dynamic changes, and manipulations of the size of texture elements. We discuss the phenomenon in relation to how vision achieves a detailed and stable representation of the environment despite changes in retinal spatial resolution and dramatic changes across saccades. The Honeycomb illusion complements previous related observations in suggesting that this representation is not necessarily based on multiple fixations (i.e., memory) or on extrapolation from information available to central vision.

Incidental memory for parts of scenes from eye movements

Incidental memory for parts of scenes was examined in two search experiments and one memory control experiment. Eye movements were recorded during the search experiments and used to select gaze-contingent sections from search scenes for a surprise memory recognition task. Results from the recognition task showed incidental memory was better for sections viewed longer and with multiple fixations. Sections not fixated during search were still recognized above chance as well. Differences in sections did not affect memory performance in a control experiment when viewing time was held constant. These results show that memory for parts of scenes can occur incidentally during search and encoding of tested sections is better with longer viewing time and with multiple fixations.

Reading transposed text: effects of transposed letter distance and consonant-vowel status on eye movements.

Two experiments were conducted to investigate the flexibility of letter-position encoding in word identification during reading. In both experiments, two tasks were used. First, participants' eye movements were measured as they read sentences containing transposed letter (TL) strings. Second, participants were presented with the TL strings in isolation and were asked to discriminate them from nonwords. In Experiment 1, we manipulated the distance between transposed letters (ligament vs. liagment vs. limagent vs. lieamgnt). Reading/response times increased with the distance between TLs. In Experiment 2, we manipulated whether the TLs were consonants, vowels, or one of each (ssytem vs. faeture vs. fromat). Reading/response times showed that CV transpositions were the most disruptive. In both experiments, response accuracy was particularly poor for words presented in isolation when there was an intervening letter between TLs. These data show that processing across multiple fixations, and the presence of a meaningful sentence context, are important for flexible letter position encoding in lexical identification.

Predictive saccade target selection in superior colliculus during visual search.

Searching for a visual object naturally involves sequences of gaze fixations, during which the current foveal image is analyzed and the next object to inspect is selected as a saccade target. Fixation durations during such sequences are short, suggesting that saccades may be concurrently processed. Therefore, the selection of the next saccade target may occur before the current saccade target is acquired. To test this hypothesis, we trained four female rhesus monkeys (Macaca mulatta) to perform a multiple-fixation visual conjunction search task. We simultaneously recorded the activity of sensorimotor neurons in the midbrain superior colliculus (SC) in two monkeys. In this task, monkeys made multiple fixations before foveating the target. Fixation durations were significantly shorter than the latency of the initial responses to the search display, with approximately one-quarter being shorter than the shortest response latencies. The time at which SC sensorimotor activity discriminated the target from distracters occurred significantly earlier for the selection of subsequent fixations than for the selection of the first fixation. Target selection during subsequent fixations occurred even before the visual afferent delay in more than half of the neuronal sample, suggesting that the process of selection can encompass at least two future saccade targets. This predictive selection was present even when differences in saccade latencies were taken into account. Altogether, these findings demonstrate how neural representations on the visual salience map are processed in parallel, thus facilitating visual search.

Landing Position Effects of Coordinate and Attributive Structure Compound Words

For alphabetic language scripts, the decision about where to move the eyes next during reading is strongly influenced by low-level visual variables such as word length, which is provided by spacing information. In contrast, high-level linguistic variables, such as word frequency and word predictability do not influence where readers move their eyes(Rayner, 2009). Similar to the alphabetic writing systems, there has been evidence that word frequency and word predictability do not influence initial landing positions during Chinese reading(Wu et al., 2011; Guo, 2012). However, written Chinese is a kind of ideographic writing system, which differs from alphabetic writing systems in many dimensions. There are different constructions of two-character compound words according to the semantic relationship between the two constituent characters of a compound word, such as coordinate and attributive structure. Some researchers found that the structure of compound words influenced vocabulary recognition processing(Feng, 2003; Gan, 2009; Zhang, 1993). In this study, two experiments were conducted to explore whether the high-level lexical properties(the semantic relationship between the two constituent characters of a compound word) influenced landing positions. In Experiment 1, we manipulated the constructions of 2-character compound words(coordinate and attributive structure). In Experiment 2, compound words pairs shared the same first constituent character but different constructions(coordinate or attributive) were used as target words to further investigate the influence on landing positions of compound words' constructions. 32 undergraduates were asked to read the experimental sentences which included target words in Experiment 1, and another 28 undergraduates took part in Experiment 2. Their eye movements were recorded by an SR Research EyeLink II eyetracker(sampling rate = 500 Hz) that monitored the position of the right eye every two milliseconds. The results showed that there were different eye movement patterns in different fixation cases. When there was only one fixation on a target word, the first fixation mostly landed on the centre of the word. When there were multiple fixations, readers first fixated at the beginning of the target word. There was a preferred viewing location in single-fixation cases during Chinese reading. In multiple fixation cases, if the first fixation landed at the beginning of a target word, the probability of refixating this word was highest, and refixations tended to land at the end of the word. There were similar fixation patterns when readers fixated the target words with different construction of compound words. We argue that Chinese children use the "strategy-tactics" approach during reading.

Fixating picture boundaries does not eliminate boundary extension: Implications for scene representation

Observers frequently remember seeing more of a scene than was shown (boundary extension). Does this reflect a lack of eye fixations to the boundary region? Single-object photographs were presented for 14-15 s each. Main objects were either whole or slightly cropped by one boundary, creating a salient marker of boundary placement. All participants expected a memory test, but only half were informed that boundary memory would be tested. Participants in both conditions made multiple fixations to the boundary region and the cropped region during study. Demonstrating the importance of these regions, test-informed participants fixated them sooner, longer, and more frequently. Boundary ratings (Experiment 1) and border adjustment tasks (Experiments 2-4) revealed boundary extension in both conditions. The error was reduced, but not eliminated, in the test-informed condition. Surprisingly, test knowledge and multiple fixations to the salient cropped region, during study and at test, were insufficient to overcome boundary extension on the cropped side. Results are discussed within a traditional visual-centric framework versus a multisource model of scene perception.

Optimal and Preferred Eye Landing Positions in Objects and Scenes

Viewing position effects are commonly observed in reading, but they have only rarely been investigated in object perception or in the realistic context of a natural scene. In two experiments, we explored where people fixate within photorealistic objects and the effects of this landing position on recognition and subsequent eye movements. The results demonstrate an optimal viewing position-objects are processed more quickly when fixation is in the centre of the object. Viewers also prefer to saccade to the centre of objects within a natural scene, even when making a large saccade. A central landing position is associated with an increased likelihood of making a refixation, a result that differs from previous reports and suggests that multiple fixations within objects, within scenes, occur for a range of reasons. These results suggest that eye movements within scenes are systematic and are made with reference to an early parsing of the scene into constituent objects.

Assemblage and Icon in Perception and Art

In The Problem of Form (1893), the German sculptor Adolf Hildebrand distinguishes categorically between perception obtained from multiple fixations or vantage points (G.: Bewegungsvorstellungen; we call these ‘assemblages’), and from purely ‘iconic’ imagery (G.: Fernbilder). Only the latter he considers properly ‘artistic’. Hildebrand finds the reason for this ontological distinction in the microgenesis of visual awareness. What to make of this? We analyze the various ‘modes of seeing’ in some detail. The conceptual issues involved are fundamental, and relevant to both vision science and the visual arts.

Getting the gist of events: recognition of two-participant actions from brief displays.

Unlike rapid scene and object recognition from brief displays, little is known about recognition of event categories and event roles from minimal visual information. In 3 experiments, we displayed naturalistic photographs of a wide range of 2-participant event scenes for 37 ms and 73 ms followed by a mask, and found that event categories (the event gist; e.g., "kicking," "pushing") and event roles (i.e., Agent and Patient) can be recognized rapidly, even with various actor pairs and backgrounds. Norming ratings from a subsequent experiment revealed that certain physical features (e.g., outstretched extremities) that correlate with Agent-hood could have contributed to rapid role recognition. In a final experiment, using identical twin actors, we then varied these features in 2 sets of stimuli, in which Patients had Agent-like features or not. Subjects recognized the roles of event participants less accurately when Patients possessed Agent-like features, with this difference being eliminated with 2-s durations. Thus, given minimal visual input, typical Agent-like physical features are used in role recognition, but with sufficient input from multiple fixations, people categorically determine the relationship between event participants.