Eye-tracking Experiment Research Articles

Enhancing auditors’ professional skepticism through nudges: an eye-tracking experiment

This article studies whether nudges – that is, gentle alterations of people’s behaviour – increase audit quality. Although the utility of nudges is well-established in behavioural sciences, their applicability and efficacy have been less studied in accounting and auditing. To bridge this knowledge gap, the study extends nudge theory to financial audits, offering experimental evidence of the impact of social norms and justification nudges on auditor behaviour. A factorial 2 × 2 between-subject experiment (social norms and justification) shows that nudges amplify professional skepticism, a critical indicator of audit quality. A follow-up eye-tracking experiment involving an audit task identifies the underlying cognitive mechanism of this effect; nudges heighten auditors’ visual attention to pertinent information, thereby refining their evaluations of audit evidence and increasing their professional skepticism.

Approximating facial expression effects on diagnostic accuracy via generative AI in medical genetics.

Artificial intelligence (AI) is increasingly used in genomics research and practice, and generative AI has garnered significant recent attention. In clinical applications of generative AI, aspects of the underlying datasets can impact results, and confounders should be studied and mitigated. One example involves the facial expressions of people with genetic conditions. Stereotypically, Williams (WS) and Angelman (AS) syndromes are associated with a "happy" demeanor, including a smiling expression. Clinical geneticists may be more likely to identify these conditions in images of smiling individuals. To study the impact of facial expression, we analyzed publicly available facial images of approximately 3500 individuals with genetic conditions. Using a deep learning (DL) image classifier, we found that WS and AS images with non-smiling expressions had significantly lower prediction probabilities for the correct syndrome labels than those with smiling expressions. This was not seen for 22q11.2 deletion and Noonan syndromes, which are not associated with a smiling expression. To further explore the effect of facial expressions, we computationally altered the facial expressions for these images. We trained HyperStyle, a GAN-inversion technique compatible with StyleGAN2, to determine the vector representations of our images. Then, following the concept of InterfaceGAN, we edited these vectors to recreate the original images in a phenotypically accurate way but with a different facial expression. Through online surveys and an eye-tracking experiment, we examined how altered facial expressions affect the performance of human experts. We overall found that facial expression is associated with diagnostic accuracy variably in different genetic conditions.

Application to combine questioning, stimulus presentation and external measurements in a real-life music eye tracking experiment

Studying people in real-life situations, such as making music in a vocal group, requires flexible and integrative measurement technology. Therefore, a digital browser-based survey instrument was developed for this study. It was designed to seamlessly introduce questions on participants' mobile devices through external control immediately after relevant events, aiming to achieve high accuracy in self-administered situational questions. In addition, chronological synchronization with supplementary measurements (here eye-tracking and audio recording) was incorporated. Digital features of this web app offer convenient integration into everyday situations, synchronous interviewing of multiple people, and gathering time-based data. Due to the numerous possibilities of the browser-based development interface, various other application areas can be opened up. The contribution of this article is:-App - explanation and offer for use-Feasibility report on the implementation of the app in an eye-tracking study with vocal groups

Evaluating computer science students reading comprehension of educational multimedia-enhanced text using scalable eye-tracking methodology

In this research, a mixed-method approach was employed to conduct large-scale eye-tracking measurements, traditionally associated with high costs and extensive time commitments. Utilizing consumer-grade webcams in conjunction with open-source software, data was collected from an expansive cohort of students, thereby demonstrating the scalability and cost-effectiveness of this innovative methodology. The primary objective of this research was to discern the disparities in reading behaviour when students were presented with standard text accompanied by illustrations, compared to the same text with highlighted key terms. The participants, comprised of first-year university students, completed a questionnaire and an introductory test to ascertain their knowledge level. Subsequently, they were segregated into two groups and participated in two reading sessions, during which their ocular movements were recorded. The amassed data underwent both qualitative analyses, facilitated by visualizations, and quantitative analysis, employing statistical measures on the data and test results. Notably, no significant difference was observed in the gaze patterns or test results between the experimental and control groups. However, a significant divergence in gaze patterns was identified between high-achieving students and those experiencing difficulties, as evidenced by the averaged composite heatmaps generated from the data. The findings underscore two pivotal points. Firstly, the feasibility of conducting large-scale eye-tracking experiments is demonstrated. Traditional studies in this field often employ small population samples due to the time and financial constraints associated with methods that utilize specialized eye-tracking hardware. In contrast, our methodology is scalable, relying on low-end hardware and enabling students to record data on their personal devices. Secondly, while eye-tracking may not provide substantial benefits for fine-tuning text already optimized for readability, it could serve as a valuable tool for identifying and assisting learners who are struggling. This mixed-method approach holds significant potential to revolutionize the conduct and interpretation of eye-tracking studies within educational settings.

The effect of expertise in music reading: cross-modal competence

We hypothesize that the fundamental difference between expert and learner musicians isthe capacity to efficiently integrate cross-modal information. This capacity might be anindex of an expert memory using both auditory and visual cues built during many years oflearning and extensive practice. Investigating this issue through an eye-tracking experiment, two groups of musicians, experts and non-experts, were required to report whether afragment of classical music, successively displayed both auditorily and visually on a computer screen (cross-modal presentation) was same or different. An accent mark, associated on a particular note, was located in a congruent or incongruent way according to musical harmony rules, during the auditory and reading phases. The cross-modal competence of experts was demonstrated by shorter fixation durations and less errors. Accent mark appeared for non-experts as interferences and lead to incorrect judgments. Results are discussed in terms of amodal memory for expert musicians that can be supported within the theoretical framework of Long-Term Working Memory (Ericsson and Kintsch, 1995).

Investigating the Readability of School Geographic Map Symbols Using Eye-Tracking Technology

Geography atlas is an educational material that is very important in geography education. For the eye tracking experiment, six mineral maps were selected from different geography atlases for high school students. The aim of this study was to analyze which design of conventional symbols is more understandable for high school students. We hope that the presented methodology and the case study will help mapmakers to create more effective school geographic maps.

The influence of Chinese typography on information dissemination in graphic design: based on eye-tracking data

The arrangement of Chinese characters has a significant impact on the visual effect and information dissemination in graphic design. In traditional Chinese layout, vertical arrangement of characters is predominant, but in recent times, there has been a gradual transition towards horizontal arrangement. To compare the influence of different character arrangement forms on visual meaning generation and information dissemination, This study employed an eye-tracking experiment to investigate two common Chinese character layouts in posters—horizontal and vertical, and collected data such as eye-tracking heatmap, pupil diameter and eye-tracking trajectory map. Based on objective eye-tracking data, combined with post-test interviews and questionnaire surveys, it was found that vertical character arrangement in Chinese typography is more effective in attracting visual attention and facilitating the expression and stimulating interest in viewing/reading under the premise of meeting formal requirements, which may provide guidance and inspiration for the practical application of Chinese characters in layout design, advertising design, packaging design, exhibition design, UI design, and other related fields.

Exploring the impact of nonverbal social behavior on learning outcomes in instructional video design

Online education has become increasingly popular in recent years, and video lectures have emerged as a common instructional format. While the importance of instructors’ nonverbal social cues such as gaze, facial expression, and gestures for learning progress in face-to-face teaching is well-established, their impact on instructional videos is not fully understood. Most studies on nonverbal social cues in instructional videos focus on isolated cues rather than considering multimodal nonverbal behavior patterns and their effects on the learning progress. This study examines the role of instructors’ nonverbal immediacy (a construct capturing multimodal nonverbal behaviors that reduce psychological distance) in video lectures with respect to learners’ cognitive, affective, and motivational outcomes. We carried out an eye-tracking experiment with 87 participants (Mage = 24.11, SD = 4.80). Results of multilevel path analyses indicate that high nonverbal immediacy substantially increases learners’ state motivation and enjoyment, but does not affect cognitive learning. Analyses of learners’ eye movements show that learners allocate more attention to the instructor than to the learning material with increasing levels of nonverbal immediacy displayed by the instructor. The study highlights the importance of considering the role of multimodal nonverbal behavior patterns in online education and provides insights for effective video lecture design.

Effects of visual risk indicators on phishing detection behavior: An eye-tracking experiment

Cybersecurity vulnerability ranks among the foremost global business risks. Phishing attempts, in particular through email, persistently challenge organizations despite substantial investments in IT security and awareness training. Recognizing the limitations of unilateral technology- or human-centered approaches, this study explores how visual risk indication can support employees in detecting phishing attempts. To do so, we conducted an eye-tracking lab experiment in which participants rated the trustworthiness of emails with varying levels of credibility. Our analysis focuses on human information processing in identifying phishing attempts, indicating that the availability of a visual risk indicator can significantly influence trust and response behavior, without incapacitating implicit phishing cues (such as conspicuous senders or anonymous recipients). Our findings suggest that organizations should appropriately calibrate visual risk indicators to achieve the intended guiding effects. However, the calibration remains a trade-off and depends on the organization’s environment. We discuss implications for integrative cybersecurity approaches to mitigate phishing attempts more effectively.

Visual Experimentation Research on Traditional Residential Courtyard Spaces Using Virtual Eye-Tracking Technology

Courtyard space serves as the soul of traditional residences in Southern China, representing a gem in traditional Chinese architecture. Courtyards play a multifaceted role in architecture, including functions such as lighting, ventilation, water collection, solar radiation, and environmental beautification. This paper, building upon previous research and theoretical foundations, takes three types of courtyards in "Kong House" as the research subjects. Through methods like data collection, literature analysis, questionnaire surveys, and eye-tracking experiments, the study investigates the spatial experience of courtyard spaces in traditional residences. It quantitatively validates the design theory of courtyard spaces, analyzes the similarities and differences in spatial experiences among comprehensive courtyards, landscape courtyards, and functional courtyards, and concludes the study. The experiments indicate that incorporating courtyard spaces in traditional residences can effectively influence human behavior patterns and perceptual choices. Comprehensive courtyards typically stimulate more exploration and lingering behaviors. The combination of virtual reality and eye-tracking experiments provides valuable visual information and statistical data for analyzing human behavior and evaluating traditional residential design schemes.

Visual Experimentation Research on Traditional Residential Courtyard Spaces Using Virtual Eye-Tracking Technology

Courtyard space serves as the soul of traditional residences in Southern China, representing a gem in traditional Chinese architecture. Courtyards play a multifaceted role in architecture, including functions such as lighting, ventilation, water collection, solar radiation, and environmental beautification. This paper, building upon previous research and theoretical foundations, takes three types of courtyards in "Kong House" as the research subjects. Through methods like data collection, literature analysis, questionnaire surveys, and eye-tracking experiments, the study investigates the spatial experience of courtyard spaces in traditional residences. It quantitatively validates the design theory of courtyard spaces, analyzes the similarities and differences in spatial experiences among comprehensive courtyards, landscape courtyards, and functional courtyards, and concludes the study. The experiments indicate that incorporating courtyard spaces in traditional residences can effectively influence human behavior patterns and perceptual choices. Comprehensive courtyards typically stimulate more exploration and lingering behaviors. The combination of virtual reality and eye-tracking experiments provides valuable visual information and statistical data for analyzing human behavior and evaluating traditional residential design schemes.

Examining Cross-linguistic Influence of Japanese Word Order: An Eye-Tracking Study on L2 English Learners’ Text Comprehension

Cross-linguistic influence is a systematic and unavoidable feature of language learning. This study aimed to investigate whether prepositive word order in Japanese noun clauses can serve as a cross-linguistic influence element in the acquisition of relative clauses (a form of a postpositive modifier) in English. To test this hypothesis, this study conducted an eye-tracking experiment with native Japanese speakers and showed the following results statistically. First, Japanese participants generally paid more attention to modifier clauses than the noun phrases they modified when reading English sentences with relative clauses. Second, when interpreting accusative relative clauses, longer fixations were observed both in the modifier clauses and in the antecedents, likely due to their complexity for Japanese L2 learners of English, as evidenced by the lowest accuracy rate. Finally, and most notably, the findings revealed that the participants who had not fully acquired the relative clause construction focused more on the left side of the relative pronoun than those who had. This tendency was most prominent when interpreting subjective relative clauses, a result that supports the hypothesis that prepositive L1 modifiers could be a CLI element in the acquisition of English postpositive modifiers.

Using Synchronized Eye Movements to Predict Attention in Online Video Learning

Concerns persist about attentional engagement in online learning. The inter-subject correlation of eye movements (ISC) has shown promise as an accessible and effective method for attention assessment in online learning. This study extends previous studies investigating ISC of eye movements in online learning by addressing two research questions. Firstly, can ISC predict students’ attentional states at a finer level beyond a simple dichotomy of attention states (e.g., attending and distracted states)? Secondly, whether learners’ learning styles affect ISC’s prediction rate of attention assessment in video learning? Previous studies have shown that learners of different learning styles have different eye movement patterns when viewing static materials. However, limited research has explored the impact of learning styles on viewing patterns in video learning. An eye tracking experiment with participants watching lecture videos demonstrated a connection between ISC and self-reported attention states at a finer level. We also demonstrated that learning styles did not significantly affect ISC’s prediction rate of attention assessment in video learning, suggesting that ISC of eye movements can be effectively used without considering learners’ learning styles. These findings contribute to the ongoing discourse on optimizing attention assessment in the evolving landscape of online education.

From Eye Movements to Personality Traits: A Machine Learning Approach in Blood Donation Advertising

Blood donation heavily depends on voluntary involvement, but the problem of motivating and retaining potential blood donors remains. Understanding the personality traits of donors can assist in this case, bridging communication gaps and increasing participation and retention. To this end, an eye-tracking experiment was designed to examine the viewing behavior of 75 participants as they viewed various blood donation-related advertisements. The purpose of these stimuli was to elicit various types of emotions (positive/negative) and message framings (altruistic/egoistic) to investigate cognitive reactions that arise from donating blood using eye-tracking parameters such as the fixation duration, fixation count, saccade duration, and saccade amplitude. The results indicated significant differences among the eye-tracking metrics, suggesting that visual engagement varies considerably in response to different types of advertisements. The fixation duration also revealed substantial differences in emotions, logo types, and emotional arousal, suggesting that the nature of stimuli can affect how viewers disperse their attention. The saccade amplitude and saccade duration were also affected by the message framings, thus indicating their relevance to eye movement behavior. Generalised linear models (GLMs) showed significant influences of personality trait effects on eye-tracking metrics, including a negative association between honesty–humility and fixation duration and a positive link between openness and both the saccade duration and fixation count. These results indicate that personality traits can significantly impact visual attention processes. The present study broadens the current research frontier by employing machine learning techniques on the collected eye-tracking data to identify personality traits that can influence donation decisions and experiences. Participants’ eye movements were analysed to categorize their dominant personality traits using hierarchical clustering, while machine learning algorithms, including Support Vector Machine (SVM), Random Forest, and k-Nearest Neighbours (KNN), were employed to predict personality traits. Among the models, SVM and KNN exhibited high accuracy (86.67%), while Random Forest scored considerably lower (66.67%). This investigation reveals that computational models can infer personality traits from eye movements, which shows great potential for psychological profiling and human–computer interaction. This study integrates psychology research and machine learning, paving the way for further studies on personality assessment by eye tracking.

Using Eye tracking to Examine Young Adults' Visual Attention to E-cigarette Advertising Features and Associated Positive E-cigarette Perceptions.

Little is known about the influence of e-cigarette marketing features on the antecedents of e-cigarette use. Using an eye-tracking experiment, we examined visual attention to common features in e-cigarette ads and its associations with positive e-cigarette perceptions among young adults. Young adults (ages 18-29) who smoke cigarettes (n = 40) or do not use tobacco (n = 71) viewed 30 e-cigarette ads on a computer screen. Eye-tracking technology measured dwell time (fixation duration) and entry time (time to first fixation) for 14 pre-defined ad features. Participants then completed a survey about perceptions of e-cigarettes shown in the ads. We used regression models to examine the associations between ad features and standardized attention metrics among all participants and by tobacco-use status and person-aggregated standardized attention for each ad feature and positive e-cigarette perceptions. Dwell time was the longest for smoker-targeted claims, positive experience claims, and price promotions. Entry time was the shortest for multiple flavor descriptions, nicotine warnings, and people. Those who do not use tobacco had a longer dwell time for minor sales restrictions and longer entry time for purchasing information than those who smoke. Longer dwell time for multiple flavor descriptions was associated with e-cigarette appeal. A shorter entry time for fruit flavor description was associated with positive e-cigarette-use expectancies. Young adults allocated attention differently to various e-cigarette ad features, and such viewing patterns were largely similar by tobacco-use statuses. Multiple or fruit flavors may be the features that contribute to the positive influence of e-cigarette marketing among young adults.

The impact of virtual influencer advertising on consumer response: a moderated chain mediation model based on mind perception and parasocial interaction

ABSTRACT This study explore how virtual influencer advertisements affect consumer response. Based on the SOR theory, an eyetracking experiment with 76 participants and a questionnaire survey with 298 participants were conducted to construct a moderated chain mediation model. The results show that emotional virtual influencers attract more advertisement attention and exert more positive impacts on consumer response than professional ones. Mind perception and parasocial interaction act as chain mediators, while the impact of parasocial interaction on unfamiliar products endorsed is more significant. These findings open up the “black box” of virtual influencer reactions and provide guidance for virtual emotional marketing.

Not all consumer-generated images are attractive and persuasive: A heuristic cue perspective

This study explored how the features of consumer-generated images (CGIs) influence consumers' attention and purchase intention in both browsing and buying stages of online shopping, as well as the mediation of these effects. We consider the common features of image reviews (e.g. brightness, clarity, product displaying proportion and consistency) as heuristic cues evaluated by consumers. We posit that image brightness, clarity and product displaying proportion are product irrelevant cues associated with CGI attractiveness in the browsing stage, whereas product consistency is a product relevant cue associated with CGI attractiveness and purchase intention during the buying stage. Eye-tracking experiments with 127 undergraduates using Parka products support our hypotheses. The results indicate a positive correlation between the quality of product-irrelevant cues and CGI attractiveness in browsing, and a similar positive association with product-relevant cues during buying. The results also show that both product relevant and irrelevant cues are positively associated with consumers’ purchase intention, mediated by eliciting emotional arousal rather than visual attention. This study extends the literature by shifting the focus from assessing the overall aesthetic quality of CGIs to the importance of specific features in different online shopping stages. The study provides important implications for e-commerce platforms to strategically encourage users to submit CGIs that maintain consistency with the merchant-provided images and exhibit high image quality attributes such as brightness and clarity. Future research should explore CGIs across different product types to understand their varying roles.

A study of spatial cognition in the rural heritage based on VR 3D eye-tracking experiments

Traditional Chinese villages act as the essential carriers of China’s rural heritage. Such villages are considered to retain living practices of intangible heritage through know-how, artistic and folk customs, as well as a well-preserved architectural appearance. However, their conservation is at stake due to degradation, abandonment, and urbanization. Spatial cognition is one way of bringing people to the matter of what is important to conserve (attributes) and why they are important (values). This is done by analysing how people perceive and value rural heritage. Traditional methods of gathering spatial cognition data, such as laborious fieldwork or subjective qualitative analyses, often fall short of providing a holistic representation of real-world experiences. This article presents an innovative method for analysing cognitive features in traditional village spaces using an immersive virtual reality equipped with an eye-tracking system. We selected Cheng and Jitou, two traditional Chinese villages, as our case studies. In two virtual reality environments, we captured accurate 3D cognitive data, including participants’ locomotion, gaze point, and sightline. The results indicate that the accessibility of village roads affects the distribution of people’s locomotion, which in turn affects the distribution of people’s areas of interest; the changes in the rhythm of the sightline scale reflect the rhythm of spatial changes in the villages. By broadening the understanding of spatial cognition in traditional Chinese villages related to people’s values, this paper sheds light on an alternative approach to assess the cultural significance of such villages for higher conservation and sustainability.

Assessing two methods of webcam-based eye-tracking for child language research.

We assess the feasibility of conducting web-based eye-tracking experiments with children using two methods of webcam-based eye-tracking: automatic gaze estimation with the WebGazer.js algorithm and hand annotation of gaze direction from recorded webcam videos. Experiment 1 directly compares the two methods in a visual-world language task with five to six year-old children. Experiment 2 more precisely investigates WebGazer.js' spatiotemporal resolution with four to twelve year-old children in a visual-fixation task. We find that it is possible to conduct web-based eye-tracking experiments with children in both supervised (Experiment 1) and unsupervised (Experiment 2) settings - however, the webcam eye-tracking methods differ in their sensitivity and accuracy. Webcam video annotation is well-suited to detecting fine-grained looking effects relevant to child language researchers. In contrast, WebGazer.js gaze estimates appear noisier and less temporally precise. We discuss the advantages and disadvantages of each method and provide recommendations for researchers conducting child eye-tracking studies online.

Form generative approach for front face design of electric vehicle under female aesthetic preferences

Vehicles are the most representative product of both transportation and industry. Fueled by the growing popularity of energy-saving and environmentally friendly ideas and policies, new energy technologies and their supporting industries are experiencing constant development and refinement. As a result, electric vehicles (EVs) are steadily displacing traditional fuel-powered vehicles as the dominant force in the vehicle market. The experience economy is driving a significant shift in the direction of EV development, which is transitioning from improving functionality to optimizing the emotional connection consumers have with their vehicles. This means the form factor of the vehicle itself is becoming a key element in attracting consumers and influencing purchasing decisions. Over half of China's EV consumer market is comprised of females. However, many companies continue to develop products solely from a male perspective, missing out on a vast market opportunity. Therefore, incorporating design elements that cater to female aesthetic preferences into EVs can undoubtedly play a crucial role in unlocking this significant segment of the market. Based on this, the paper proposes a form generative design approach that specifically caters to the preferences of female consumers within the theoretical framework of Kansei engineering.The research process begins by obtaining Kansei imagery through the collection and analysis of big data. Subsequently, eye-tracking experiments are conducted to define the key design elements of the automobiles. Concurrently, appropriate aesthetic measurement indicator (AMI) system is established. Next, AMI values are computed for the sample set. This data is then leveraged to create a mapping relationship with the Kansei imagery, enabling the objective reflection of subjective aesthetic evaluations. Finally, an integrated Kansei evaluation equation serves as the fitness function within a generative approach that utilizes genetic algorithms and visual AI to produce form design solutions for electric vehicles. The evaluation confirms that the proposed design approach successfully generates a substantial number of high-quality creative solutions that align with female aesthetic preferences, while maintaining low cost and high efficiency.