Visual Expertise Research Articles

Differences in biological motion perception associated with hearing status and age of signed language exposure.

This study investigates how American Sign Language (ASL) fluency and hearing status influence the perception of biological motion, using three point-light display (PLD) tasks. Prior research indicates that early exposure to ASL among deaf signers results in more rapid and effortless recognition of biological motion than hearing nonsigners, potentially due to the expertise in deciphering complex human movements or possibly due to neuroplasticity in deaf brains. However, it remains uncertain whether this advantage stems from signed language proficiency or the experience of being deaf. To explore this, we designed three PLD tasks involving viewing randomly moving dots, identifying a person from biological motion PLDs, and determining whether right-side up and inverted PLDs depict actions involving a ball. A diverse cohort of participants (N = 224) with varying ASL fluencies and hearing statuses completed the tasks online, providing us with reaction time and accuracy data. Our results demonstrate that earlier ASL exposure is associated with accuracy, especially on complex action identification tasks. Furthermore, we discovered robust evidence for a speed-accuracy trade-off in deaf participants, in which they performed more quickly but less accurately. The speed-accuracy trade-off was evident in the most difficult task, the action identification task. Further analysis of this deaf group revealed that earlier signed language acquisition led to higher accuracy in action identification task. We conclude that age of ASL exposure and hearing status both significantly contribute to variations in biological motion perception, with implications for understanding visual expertise and cognitive processing in both deaf and signing populations. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Preserving text space integrity for robust compositional zero-shot learning via mixture of pretrained experts

In the current landscape of Compositional Zero-Shot Learning (CZSL) methods that leverage CLIP, the predominant approach is based on prompt learning paradigms. These methods encounter significant computational complexity when dealing with a large number of categories. Additionally, when confronted with new classification tasks, there is a necessity to learn the prompts again, which can be both time-consuming and resource-intensive. To address these challenges, We present a new methodology, named the Mixture of Pretrained Expert (MoPE), for enhancing Compositional Zero-shot Learning through Logit-Level Fusion with Mutile Expert Fusion Module. The MoPE skillfully blends the benefits of extensive pre-trained models like CLIP, Bert, GPT-3 and Word2Vec for effectively tackling Compositional Zero-shot Learning. Firstly, we extract the text label space for each language model individually, then map the visual feature vectors to their respective text spaces. This maintains the integrity and structure of the original text space. During this process, the pre-trained expert parameters are kept frozen. The mapping of visual features to the corresponding text spaces is subject to learning and could be considered as multiple learnable visual experts. In the model fusion phase, we propose a new fusion strategy that features a gating mechanism that adjusts the contributions of various models dynamically. This enables our approach to adapt more effectively to a range of tasks and data sets. The method’s robustness is demonstrated by the fact that the language model is not tailored to specific downstream task datasets or losses. This preserves the larger model’s topology and expands the potential for application. Preliminary experiments conducted on the UT-Zappos, AO-Clever, and C-GQA datasets indicate that MoPE performs competitively when compared to existing techniques.

Artistic Expertise and Free Viewing of Modern Art

Abstract Visual art and vision science had long been intertwined, both revealing and sharing countless insights on human vision over their respective histories. Deep artistic experiences and extensive artistic knowledge are often associated with “artistic expertise,” which in vision science traditionally echoes visual expertise from other professional domains (e.g., medicine, sports, driving). However, unlike other visual domains, artistic experiences involve more spontaneous and imaginative aspects, and no explicit success parameters. We hypothesized that viewing of artworks would lead to nuanced effects of expertise that may differ from other types of visual expertise. In our study, art experts and non-experts observed modern paintings and were free to view them within a relatively long time window. Higher expertise in art was associated with viewing modern paintings for significantly longer durations, and liking the paintings significantly more. However, contrary to other visual expertise studies, eye movement patterns exhibited little to no effect of artistic expertise. Based on these findings, we suggest a recontextualization of visual artistic expertise and discuss potential research directions, alongside methodological challenges in the experimental design.

RSVP for VPSA : A Meta Design Study on Rapid Suggestive Visualization Prototyping for Visual Parameter Space Analysis.

Visual Parameter Space Analysis (VPSA) enables domain scientists to explore input-output relationships of computational models. Existing VPSA applications often feature multi-view visualizations designed by visualization experts for a specific scenario, making it hard for domain scientists to adapt them to their problems without professional help. We present RSVP, the Rapid Suggestive Visualization Prototyping system encoding VPSA knowledge to enable domain scientists to prototype custom visualization dashboards tailored to their specific needs. The system implements a task-oriented, multi-view visualization recommendation strategy over a visualization design space optimized for VPSA to guide users in meeting their analytical demands. We derived the VPSA knowledge implemented in the system by conducting an extensive meta design study over the body of work on VPSA. We show how this process can be used to perform a data and task abstraction, extract a common visualization design space, and derive a task-oriented VisRec strategy. User studies indicate that the system is user-friendly and can uncover novel insights.

Gradual change of cortical representations with growing visual expertise for synthetic shapes

Abstract Objective: Visual expertise for particular categories of objects (e.g., mushrooms, birds, flowers, minerals, and so on) is known to enhance cortical responses in parts of the ventral occipitotemporal cortex. How is such additional expertise integrated into the prior cortical representation of life-long visual experience? To address this question, we presented synthetic visual objects rotating in three dimensions and recorded multivariate BOLD responses as initially unfamiliar objects gradually became familiar. Main results: An analysis of pairwise distances between multivariate BOLD responses (“representational similarity analysis,” RSA) revealed that visual objects were linearly discriminable in large parts of the ventral occipital cortex, including the primary visual cortex, as well as in certain parts of the parietal and frontal cortex. These cortical representations were present from the start, when objects were still unfamiliar, and even though objects were shown from different sides. As shapes became familiar with repeated viewing, the distribution of responses expanded to fill more of the available space. In contrast, the distribution of responses to novel shapes (which appeared only once) contracted and shifted to the margins of the available space. Conclusion: Our results revealed cortical representations of object shape and gradual changes in these representations with learning and consolidation. The cortical representations of once-viewed shapes that remained novel diverged dramatically from repeatedly viewed shapes that became familiar. This disparity was evident in both the similarity and the diversity of multivariate BOLD responses.

A framework to improve causal inferences from visualizations using counterfactual operators

Exploratory data analysis of high-dimensional datasets is a crucial task for which visual analytics can be especially useful. However, the ad hoc nature of exploratory analysis can also lead users to draw incorrect causal inferences. Previous studies have demonstrated this risk and shown that integrating counterfactual concepts within visual analytics systems can improve users’ understanding of visualized data. However, effectively leveraging counterfactual concepts can be challenging, with only bespoke implementations found in prior work. Moreover, it can require expertise in both counterfactual subset analysis and visualization to implement the functionalities practically. This paper aims to help address these challenges in two ways. First, we propose an operator-based conceptual model for the use of counterfactuals that is informed by prior work in visualization research. Second, we contribute the Co-op library, an open and extensible reference implementation of this model that can support the integration of counterfactual-based subset computation with visualization systems. To evaluate the effectiveness and generalizability of Co-op, the library was used to construct two different visual analytics systems each supporting a distinct user workflow. In addition, expert interviews were conducted with professional visual analytics researchers and engineers to gain more insights regarding how Co-op could be leveraged. Finally, informed in part by these evaluation results, we distil a set of key design implications for effectively leveraging counterfactuals in future visualization systems.

On Network Structural and Temporal Encodings: A Space and Time Odyssey.

The dynamic network visualization design space consists of two major dimensions: network structural and temporal representation. As more techniques are developed and published, a clear need for evaluation and experimental comparisons between them emerges. Most studies explore the temporal dimension and diverse interaction techniques supporting the participants, focusing on a single structural representation. Empirical evidence about performance and preference for different visualization approaches is scattered over different studies, experimental settings, and tasks. This paper aims to comprehensively investigate the dynamic network visualization design space in two evaluations. First, a controlled study assessing participants' response times, accuracy, and preferences for different combinations of network structural and temporal representations on typical dynamic network exploration tasks, with and without the support of standard interaction methods. Second, the best-performing combinations from the first study are enhanced based on participants' feedback and evaluated in a heuristic-based qualitative study with visualization experts on a real-world network. Our results highlight node-link with animation and playback controls as the best-performing combination and the most preferred based on ratings. Matrices achieve similar performance to node-link in the first study but have considerably lower scores in our second evaluation. Similarly, juxtaposition exhibits evident scalability issues in more realistic analysis contexts.

Are pie charts evil? An assessment of the value of pie and donut charts compared to bar charts

Many data visualization experts recommend the use of bar charts over pie charts because they consider comparing the area or angle of segments to be less accurate than comparing bars on a bar chart. However, many studies show that when the pie chart is used to estimate proportions (arguably its main function) it is as accurate as the bar chart. A major issue is that most previous studies have only looked at one method of extracting information from pie charts, for example either by comparing the segment to the circle (the part-whole relationship) or one segment to another (relative magnitude estimation). Therefore, in this study I test multiple metrics to provide a more holistic assessment of the pie and donut chart against the bar chart. I also measured cognitive load through pupillometry. In summary, bar charts were more precise than pie and donut charts for ranking elements, but all charts were equally accurate for extracting the part-whole relationship. There was little difference in cognitive load between chart types, although bar charts were consistently faster to use on average. Overall, the bar chart was more flexible, but where there were statistically significant differences between charts, the effect sizes were often small, and unlikely to prevent effective extraction of quantitative information. That is, as long as they were used appropriately, all chart types were arguably acceptable for displaying simple, categorical data.

AutoVizuA11y: A Tool to Automate Screen Reader Accessibility in Charts

AbstractCharts remain widely inaccessible on the web for users of assistive technologies like screen readers. This is, in part, due to data visualization experts still lacking the experience, knowledge, and time to consistently implement accessible charts. As a result, screen reader users are prevented from accessing information and are forced to resort to tabular alternatives (if available), limiting the insights that they can gather. We worked with both groups to develop AutoVizuA11y, a tool that automates the addition of accessible features to web‐based charts. It generates human‐like descriptions of the data using a large language model, calculates statistical insights from the data, and provides keyboard navigation between multiple charts and underlying elements. Fifteen screen reader users interacted with charts made accessible with AutoVizuA11y in a usability test, thirteen of which praised the tool for its intuitive design, short learning curve, and rich information. On average, they took 66 seconds to complete each of the eight analytical tasks presented and achieved a success rate of 89%. Through a SUS questionnaire, the participants gave AutoVizuA11y an “Excellent” score — 83.5/100 points. We also gathered feedback from two data visualization experts who used the tool. They praised the tool availability, ease of use and functionalities, and provided feedback to add AutoVizuA11y support for other technologies in the future.

‘How I edit my Instagram images’: investigating skilled vision in the work of YouTube’s lifestyle-content creators

Woman social media content creators’ visual expertise is ill understood and arguably overlooked. Through the example of lifestyle-creators, this article explores the ways they plan, produce, and look at imagery on Instagram and YouTube, and shows that creators have specific ways to learn, communicate, and take part into social practices that are key in formulating their skilled vision and visual professionality. The study is part of a larger nethnographic project in which lifestyle-YouTube channels and related social media platforms have been studied for three years. In this article, online observation of 13 creators and content analysis of instructional ‘how I edit’ – videos show, that creators have trained sense of vision that involves everyday work practices common both for amateur and professional photography practitioners. The findings suggest that creators’ skilled vision is produced through four processes: (1) sharing expertise; (2) the use of specific technology and software; (3) routine; (4) creativity. Finally, it is argued that investigating creators’ skilled vision adds to our understanding of the mechanisms that have excluded a great deal of ‘feminine’ creativity from our historical and present accounts. I situate lifestyle creators within a longer tradition of women engaging with photography, photographic technologies, and product advice, and offer ways for understanding digitally networked ‘influencing’ and visual practices on social media today.

The State of the Art in Visual Analytics for 3D Urban Data

AbstractUrbanization has amplified the importance of three‐dimensional structures in urban environments for a wide range of phenomena that are of significant interest to diverse stakeholders. With the growing availability of 3D urban data, numerous studies have focused on developing visual analysis techniques tailored to the unique characteristics of urban environments. However, incorporating the third dimension into visual analytics introduces additional challenges in designing effective visual tools to tackle urban data's diverse complexities. In this paper, we present a survey on visual analytics of 3D urban data. Our work characterizes published works along three main dimensions (why, what, and how), considering use cases, analysis tasks, data, visualizations, and interactions. We provide a fine‐grained categorization of published works from visualization journals and conferences, as well as from a myriad of urban domains, including urban planning, architecture, and engineering. By incorporating perspectives from both urban and visualization experts, we identify literature gaps, motivate visualization researchers to understand challenges and opportunities, and indicate future research directions.

Classification images for aerial images capture visual expertise for binocular disparity and a prior for lighting from above.

Using a novel approach to classification images (CIs), we investigated the visual expertise of surveyors for luminance and binocular disparity cues simultaneously after screening for stereoacuity. Stereoscopic aerial images of hedges and ditches were classified in 10,000 trials by six trained remote sensing surveyors and six novices. Images were heavily masked with luminance and disparity noise simultaneously. Hedge and ditch images had reversed disparity on around half the trials meaning hedges became ditch-like and vice versa. The hedge and ditch images were also flipped vertically on around half the trials, changing the direction of the light source and completing a 2 × 2 × 2 stimulus design. CIs were generated by accumulating the noise textures associated with "hedge" and "ditch" classifications, respectively, and subtracting one from the other. Typical CIs had a central peak with one or two negative side-lobes. We found clear differences in the amplitudes and shapes of perceptual templates across groups and noise-type, with experts prioritizing binocular disparity and using this more effectively. Contrariwise, novices used luminance cues more than experts meaning that task motivation alone could not explain group differences. Asymmetries in the luminance CIs revealed individual differences for lighting interpretation, with experts less prone to assume lighting from above, consistent with their training on aerial images of UK scenes lit by a southerly sun. Our results show that (i) dual noise in images can be used to produce simultaneous CI pairs, (ii) expertise for disparity cues does not depend on stereoacuity, (iii) CIs reveal the visual strategies developed by experts, (iv) top-down perceptual biases can be overcome with long-term learning effects, and (v) CIs have practical potential for directing visual training.

Visual Exploratory Analysis for Designing Large-Scale Network-on-Chip Architectures: A Domain Expert-Led Design Study.

Visualization design studies bring together visualization researchers and domain experts to address yet unsolved data analysis challenges stemming from the needs of the domain experts. Typically, the visualization researchers lead the design study process and implementation of any visualization solutions. This setup leverages the visualization researchers' knowledge of methodology, design, and programming, but the availability to synchronize with the domain experts can hamper the design process. We consider an alternative setup where the domain experts take the lead in the design study, supported by the visualization experts. In this study, the domain experts are computer architecture experts who simulate and analyze novel computer chip designs. These chips rely on a Network-on-Chip (NOC) to connect components. The experts want to understand how the chip designs perform and what in the design led to their performance. To aid this analysis, we develop Vis4Mesh, a visualization system that provides spatial, temporal, and architectural context to simulated NOC behavior. Integration with an existing computer architecture visualization tool enables architects to perform deep-dives into specific architecture component behavior. We validate Vis4Mesh through a case study and a user study with computer architecture researchers. We reflect on our design and process, discussing advantages, disadvantages, and guidance for engaging in a domain expert-led design studies.

Data visualization guidance using a software product line approach

Data visualization aims to convey quantitative and qualitative information effectively by determining which techniques and visualizations are most appropriate for different situations and why. Various software solutions can produce numerous visualizations of the same data set. However, data visualization encompasses a wide range of visual configurations that depend on factors such as the type of data being displayed, the different displays (e.g., scatter plots, line graphs, and pie charts), the visual components used to represent the data (e.g., lines, dots, and bars), and the specific visual attributes of those components (e.g., color, shape, size, and length). A similar problem arises when designing data tables, where the dimensionality of the data and its complexity influence the choice of the most appropriate structure (e.g., unidirectional, bidirectional). Often, this broad spectrum of configurations requires a visualization expert who knows which techniques are best for which type of data source and what is to be conveyed. Typically, researchers and developers lack knowledge of data visualization best practices and must learn the design principles that enable effective communication and the technical details of the specific software tool they use to generate visualizations. This paper proposes a software product line approach to model and realize the variability of the visualization design process, using feature models to encode knowledge about design best practices in graphs and charts. Our approach involves solving visualization design variability through a stepwise configuration process and evaluating the proposal for a specific software visualization tool. Our solution facilitates effective communication of quantitative results by helping researchers and developers select and generate the most effective visualizations for each case. This approach opens up new opportunities for research at the intersection of data visualization and variability.

Examining Voice Community Use

Visual online communities can present accessibility challenges to older adults or people with vision and motor disabilities. Motivated by this challenge, accessibility and HCI researchers have called for voice-based communities to support aging and disability. This paper extends prior work on voice community design and short-term use by providing empirical data on how people interact with voice communities over time and intentional instances of non-use. We conducted a one-year study with 43 blind and low vision older adults, of whom 21 used a voice-based community. We use vignettes to unpack five different voice community member roles - the obligatory poster, routine poster, cross-platform lurker, busy socialite, and visual expertise seeker - and discuss community interactions over time. Findings show how participation varied based on engagement in other communities and ways that participants sought interaction. We discuss (1) how to design voice communities for member roles and (2) the implications of synchronous and asynchronous voice community interaction in voice-only communities.

A personalized semi-automatic sleep spindle detection (PSASD) framework

BackgroundSleep spindles are distinct electroencephalogram (EEG) patterns of brain activity that have been posited to play a critical role in development, learning, and neurological disorders. Manual scoring for sleep spindles is labor-intensive and tedious but could supplement automated algorithms to resolve challenges posed with either approaches alone. New methodsA Personalized Semi-Automatic Sleep Spindle Detection (PSASD) framework was developed to combine the strength of automated detection algorithms and visual expertise of human scorers. The underlying model in the PSASD framework assumes a generative model for EEG sleep spindles as oscillatory components, optimized to EEG amplitude, with remaining signals distributed into transient and low-frequency components. ResultsA single graphical user interface (GUI) allows both manual scoring of sleep spindles (model training data) and verification of automatically detected spindles. A grid search approach allows optimization of parameters to balance tradeoffs between precision and recall measures. Comparison with existing methodsPSASD outperformed DETOKS in F1-score by 19% and 4% on the DREAMS and P-DROWS-E datasets, respectively. It also outperformed YASA in F1-score by 25% in the P-DROWS-E dataset. Further benchmarking analysis showed that PSASD outperformed four additional widely used sleep spindle detectors in F1-score in the P-DROWS-E dataset. Titration analysis revealed that four 30-second epochs are sufficient to fine-tune the model parameters of PSASD. Associations of frequency, duration, and amplitude of detected sleep spindles matched those previously reported with automated approaches. ConclusionsOverall, PSASD improves detection of sleep spindles in EEG data acquired from both younger healthy and older adult patient populations.

The development of visual expertise in a virtual environment: A case of maritime pilots in training

This study connects to an ongoing discussion about the limits and affordances of simulators as realistic and relevant contexts for professional learning, in this case in the development of visual expertise. Earlier studies of simulator-based maritime pilot training conclude that there are risks associated with so-called negative skills transfer due to a lack of photorealism in simulator environments. The aim of this study is to carefully examine how visual expertise develops in and through training in a simulated environment. Through a practice-based approach to the development of visual expertise, and by using qualitative interaction analysis of video recorded training sessions, the analytical focus is directed towards maritime pilot trainees’ talk about imperfections and inconsistencies in the virtual environment during exercises in a high-fidelity bridge simulator. Considering the multi-layered nature of the maritime pilot’s visual expertise, findings show that the maritime pilots in training noticed and adapted to the specific methodological and technological challenges when manoeuvring a simulated vessel. During such reflection-in-action, they also commented on and explored the differences between, navigating in a simulator, on the one hand, and, on the other hand, navigating on board a ship. Instead of concluding that there is a risk for negative skills transfer that follows from the differences between the two contexts of navigating, we argue that the challenges introduced by representations encountered when training in a virtual environment may add to the expertise of the trainees and lead to enriched conceptual, methodological, and technical knowledge regarding the specificities of visually demanding and ambiguous navigation situations. In this way, this study contributes to advance our understanding of learning in virtual environments to the frontline of learning research.

UnDRground Tubes: Exploring Spatial Data With Multidimensional Projections and Set Visualization.

In various scientific and industrial domains, analyzing multivariate spatial data, i.e., vectors associated with spatial locations, is common practice. To analyze those datasets, analysts may turn to methods such as Spatial Blind Source Separation (SBSS). Designed explicitly for spatial data analysis, SBSS finds latent components in the dataset and is superior to popular non-spatial methods, like PCA. However, when analysts try different tuning parameter settings, the amount of latent components complicates analytical tasks. Based on our years-long collaboration with SBSS researchers, we propose a visualization approach to tackle this challenge. The main component is UnDRground Tubes (UT), a general-purpose idiom combining ideas from set visualization and multidimensional projections. We describe the UT visualization pipeline and integrate UT into an interactive multiple-view system. We demonstrate its effectiveness through interviews with SBSS experts, a qualitative evaluation with visualization experts, and computational experiments. SBSS experts were excited about our approach. They saw many benefits for their work and potential applications for geostatistical data analysis more generally. UT was also well received by visualization experts. Our benchmarks show that UT projections and its heuristics are appropriate.

Characteristics of visual perception of patients’ faces and the formation of “observation” in the process of training cosmetologists

Professional experience assists specialists in processing information more quickly and effectively making precise decisions. Visual expertise, as a crucial component of experience, refers to the capability to rapidly perceive visual information within a specific professional domain, acquired through systematic education, professional development, and practice. Numerous studies utilize eye-tracking methods to analyze the relationship between professional experience and gaze patterns. For instance, experienced dermatologists have developed optimal scanning patterns, allowing them to efficiently allocate attention and accurately identify key diagnostic features within images amidst numerous secondary details. The purpose of this study is to analyze changes in oculomotor patterns after completing the author’s training program for doctors, aimed at versatile aesthetic analytics of the face, building individual treatment protocols and increasing the patient’s adherence to aesthetic medicine in order to assess the possibility of developing observation in the conditions of short-term training. Thirty cosmetic doctors participated in the study, which utilized eye-tracking methods with the Neuroburo program-hardware complex. The study was conducted in two phases – before and after training – with 15 patient facial images exhibiting various dermatological issues presented on a computer screen to the cosmetic doctors. Participants were tasked with identifying signs of dermatological disorders of the patients. Following training, a decrease in the number of fixations, an increase in saccadic amplitude, and a reduction in the overall scanning path length were observed among the cosmetic doctors. These findings indicate a more optimized perception strategy and suggest a more systematic and directed approach to diagnostic image analysis. The results suggest an enhanced global perception ability among the doctors, which may aid in recognizing broader patterns and structures and understanding the wider context and relationships between details.

Promises and Pitfalls: Using Large Language Models to Generate Visualization Items.

Visualization items-factual questions about visualizations that ask viewers to accomplish visualization tasks-are regularly used in the field of information visualization as educational and evaluative materials. For example, researchers of visualization literacy require large, diverse banks of items to conduct studies where the same skill is measured repeatedly on the same participants. Yet, generating a large number of high-quality, diverse items requires significant time and expertise. To address the critical need for a large number of diverse visualization items in education and research, this paper investigates the potential for large language models (LLMs) to automate the generation of multiple-choice visualization items. Through an iterative design process, we develop the VILA (Visualization Items Generated by Large LAnguage Models) pipeline, for efficiently generating visualization items that measure people's ability to accomplish visualization tasks. We use the VILA pipeline to generate 1,404 candidate items across 12 chart types and 13 visualization tasks. In collaboration with 11 visualization experts, we develop an evaluation rulebook which we then use to rate the quality of all candidate items. The result is the VILA bank of ∼1,100 items. From this evaluation, we also identify and classify current limitations of the VILA pipeline, and discuss the role of human oversight in ensuring quality. In addition, we demonstrate an application of our work by creating a visualization literacy test, VILA-VLAT, which measures people's ability to complete a diverse set of tasks on various types of visualizations; comparing it to the existing VLAT, VILA-VLAT shows moderate to high convergent validity (R = 0.70). Lastly, we discuss the application areas of the VILA pipeline and the VILA bank and provide practical recommendations for their use. All supplemental materials are available at https://osf.io/ysrhq/.