Human-computer Interaction Community Research Articles

A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?

Explainable artificial intelligence (XAI) has gained significant attention, especially in AI-powered autonomous and adaptive systems (AASs). However, a discernible disconnect exists among research efforts across different communities. The machine learning community often overlooks “explaining to whom,” while the human-computer interaction community has examined various stakeholders with diverse explanation needs without addressing which XAI methods meet these requirements. Currently, no clear guidance exists on which XAI methods suit which specific stakeholders and their distinct needs. This hinders the achievement of the goal of XAI: providing human users with understandable interpretations. To bridge this gap, this paper presents a comprehensive XAI roadmap. Based on an extensive literature review, the roadmap summarizes different stakeholders, their explanation needs at different stages of the AI system lifecycle, the questions they may pose, and existing XAI methods. Then, by utilizing stakeholders’ inquiries as a conduit, the roadmap connects their needs to prevailing XAI methods, providing a guideline to assist researchers and practitioners to determine more easily which XAI methodologies can meet the specific needs of stakeholders in AASs. Finally, the roadmap discusses the limitations of existing XAI methods and outlines directions for future research.

Zooming In: A Review of Designing for Photo Taking in Human-Computer Interaction and Future Prospects

Photography has been pivotal in culture for decades and its importance has increased with the rise of digital technology. However, the exploration of picture taking within the Human-Computer Interaction (HCI) community does not seem to match its cultural and technological significance. Recognizing this discrepancy, we sought to understand areas of interest in photography as a conscious creative process. To address this issue, we performed a systematic literature review using the PRISMA methodology. From our research, we identified 62 pertinent papers spanning from 2005 to 2022. Our examination revealed six primary dimensions, further classified into study type, design goal, photo-taking style, device type, interaction style, and number of users. In-depth analysis showed the dominant role of exploratory and functional research, the balance between qualitative and quantitative methods, and a strong focus on smartphone cameras. Our review has highlighted significant gaps in the existing literature, offering valuable insights for future research on photo taking.

PACMHCI, VI, MHCI, September 2024 Editorial

Welcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, which brings together contributions from the Mobile Human-Computer Interaction (MHCI) community. This issue showcases innovations in research focused on mobile, wearable, and personal devices. Research in MHCI encompasses both technical innovations and social considerations. Mobile technologies accompany us throughout our daily lives: at home, at work, in traffic, and out in the wild. However, while mobile technologies provide the capacity for constant connectivity, they also pose risks of harm and unintended consequences. There has never been a more pressing time to debate and explore the meaning of digital culture, and to understand how mobile technologies can and should be used to connect us, our data, and mobile applications and services in meaningful ways - true to the conference theme of 'Connecting Cultures'.

A Functional Usability Analysis of Appearance-Based Gaze Tracking for Accessibility.

Appearance-based gaze tracking algorithms, which compute gaze direction from user face images, are an attractive alternative to infrared-based external devices. Their accuracy has greatly benefited by using powerful machine-learning techniques. The performance of appearance-based algorithms is normally evaluated on standard benchmarks typically involving users fixating at points on the screen. However, these metrics do not easily translate into functional usability characteristics. In this work, we evaluate a state-of-the-art algorithm, FAZE, in a number of tasks of interest to the human-computer interaction community. Specifically, we study how gaze measured by FAZE could be used for dwell-based selection and reading progression (line identification and progression along a line) - key functionalities for users facing motor and visual impairments. We compared the gaze data quality from 7 participants using FAZE against that from an infrared tracker (Tobii Pro Spark). Our analysis highlights the usability of appearance-based gaze tracking for such applications.

Communicating the Privacy-Utility Trade-off: Supporting Informed Data Donation with Privacy Decision Interfaces for Differential Privacy

Data collections, such as those from citizen science projects, can provide valuable scientific insights or help the public to make decisions based on real demand. At the same time, the collected data might cause privacy risks for their volunteers, for example, by revealing sensitive information. Similar but less apparent trade-offs exist for data collected while using social media or other internet-based services. One approach to addressing these privacy risks might be to anonymize the data, for example, by using Differential Privacy (DP). DP allows for tuning and, consequently, communicating the trade-off between the data contributors' privacy and the resulting data utility for insights. However, there is little research that explores how to communicate the existing trade-off to users. % We contribute to closing this research gap by designing interactive elements and visualizations that specifically support people's understanding of this privacy-utility trade-off. We evaluated our user interfaces in a user study (N=378). Our results show that a combination of graphical risk visualization and interactive risk exploration best supports the informed decision, \ie the privacy decision is consistent with users' privacy concerns. Additionally, we found that personal attributes, such as numeracy, and the need for cognition, significantly influence the decision behavior and the privacy usability of privacy decision interfaces. In our recommendations, we encourage data collectors, such as citizen science project coordinators, to communicate existing privacy risks to their volunteers since such communication does not impact donation rates. %Understanding such privacy risks can also be part of typical training efforts in citizen science projects. %DP allows volunteers to balance their privacy concerns with their wish to contribute to the project. From a design perspective, we emphasize the complexity of the decision situation and the resulting need to design with usability for all population groups in mind. % We hope that our study will inspire further research from the human-computer interaction community that will unlock the full potential of DP for a broad audience and ultimately contribute to a societal understanding of acceptable privacy losses in specific data contexts.

Social connectedness and the role of virtual reality: experiences and perceptions of people living with dementia and their caregivers

Purpose People living with dementia are often at increased risk of becoming socially disconnected due to dementia-related challenges. In recent years, digital technology has been designed to help address the social health of people living with dementia and provide opportunities to promote or maintain their social connectedness. This paper presents the findings from phase two of a participatory action research project, which explored people living with dementia and their caregiver’s experiences and perceptions of social connectedness and the potential role of Virtual Reality (VR) in promoting or maintaining same. Materials and Methods People living with dementia (n = 8) and their informal caregivers (n = 8) participated in an individual, 1:1 online interview. Data analysis was guided by reflexive thematic analysis. Results The findings presented four themes: social connectedness: lived experiences and insights, facilitating social connectedness, barriers to social connectedness and the potential of multi-user VR for social connectedness. People living with dementia experienced a range of personal, community and societal connectedness. Facilitators of social connectedness included supportive, non-judgemental, and reciprocal relationships, technology adoption, and personal and contextual facilitators. Dementia-related difficulties and periods of disruption or change were considered barriers to social connectedness. Multi-user VR was perceived as useful for promoting and maintaining social connectedness. Conclusions The perceived usefulness of multi-user VR for social connectedness indicates its potential for use with this population. Understanding the lived experiences, barriers, and facilitators of social connectedness will assist researchers and the human-computer interaction community to inform the design of future multi-user VR for social connectedness outcomes with people living with dementia and their caregivers.

Designing technology to support greater participation of people living with dementia in daily and meaningful activities.

People living with dementia should be at the center of decision-making regarding their plans and goals for daily living and meaningful activities that help promote health and mental well-being. The human-computer interaction community has recently begun to recognize the need to design technologies where the person living with dementia is an active rather than a passive user of technology in the management of their care. Data collection comprised semi-structured interviews and focus groups held with dyads of people with early-stage dementia (n = 5) and their informal carers (n = 4), as well as health professionals (n = 5). This article discusses findings from the thematic analysis of this qualitative data. Analysis resulted in the construction of three main themes: (1) maintaining a sense of purpose and identity, (2) learning helplessness and (3) shared decision-making and collaboration. Within each of the three main themes, related sub-themes were also constructed. There is a need to design technologies for persons living with dementia/carer dyads that can support collaborative care planning and engagement in meaningful activities while also balancing persons living with dementia empowerment and active engagement in self-management with carer support.

The effects of non-fungible token platform affordances on customer loyalty: A Buyer–Creator duality perspective

Prior academic research on non-fungible tokens (NFTs), an application of blockchain technology in the metaverse environment, has focused primarily on the market demand side (i.e., buyers), with limited attention given to the supply side (i.e., creators). To address this gap, we seek to understand creators' and buyers’ perceptions of platform affordances that can aid NFT platforms in developing functions that address the needs and expectations of both parties. To do this, we draw on affordance theory and psychological contract theory and use a mixed-methods approach consisting of text mining, structural equation modeling, and importance-performance map analysis. The results show that platform affordances (transaction, aesthetics, status, and user interface affordance) facilitate the creation of psychological contracts, which in turn promotes customer loyalty. Moreover, the relationships identified differ between creators and buyers. Overall, these results point to the value of using a user-centric human–computer interaction perspective to study emerging blockchain-based systems. They can inform design principles and guidelines for NFT platforms to effectively support creator–buyer interactions, transactions, and relationships through enhanced affordances and user-centric experience. This study advances knowledge of design in the nascent NFT domain relevant to the cross-disciplinary human–computer interaction community.

Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature Review

Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing. Designers, as a special user group interacting with AI, have received more attention in the Human-Computer Interaction community. Prior work has discussed emerging challenges that persist in designing for AI. However, few systematic reviews focus on AI for design to understand how designers and AI can augment each other's complementary strengths in design collaboration. In this work, we conducted a landscape analysis of AI for design, via a systematic literature review of 93 papers. The analysis first provides a bird's eye view of overall patterns in this area. The analysis also reveals three themes interpreted from the paper corpus associated with AI for design, including AI assisting designers, designers assisting AI, and characterizing designer-AI collaboration. We discuss the implications of our findings and suggested methodological proposals to guide HCI toward research and practices that center on collaborative creativity.

Eye-tracking in map use, map user and map usability research: what are we looking for?

ABSTRACT This overview paper summarises the ‘state of the art and of the science’ of eye-tracking, and its applications in map use research. Cartographic research is introduced, and its contemporary direction, which indicates that the main areas of such research are now focussed on human beings and their interaction with maps and geospatial displays, is stressed. A brief outline of several different methodologies for map use research is presented: observation, thinking loud, keyboard analysis, eye-tracking, and questionnaires. The role of eye-tracking as a major methodology for use, user, and usability investigation is explored; along with the possible choices for the researcher in the important areas of participant selection, eye-tracking equipment, set-up and use of the testing environment, and analysis of output data. Typical outcomes from eye tracking research are considered, with an assessment of its value in cartographic research in general. Future directions are suggested, along with the need for cartography to promote the valuable work done by researchers using eye-tracking for map use studies to the wider human-computer interaction community, expanding the scope of the geospatial-based stimuli in such experiments beyond maps, making use of the significant expertise and enthusiasm of cartographic researchers.

Human-Computer Integration as an Extension of Interaction: Understanding Its State-of-the-Art and the Next Challenges

The Interaction Era is evolving and extending to the Human-Computer Integration Era, in which the relationship between humans and technologies is characterized as a partnership. Motivated by this evolution, we conducted a Systematic Literature Review to present the current state of knowledge about the emerging paradigm of Human-Computer Integration (HInt), as well as its opportunities and challenges for the Human-Computer Interaction (HCI) area. As a result, we organize the existing knowledge and research on HInt which describes it as a new interaction paradigm. Our findings indicate that HInt is a topic of growing interest in the HCI Community, and provide an overview of existing initiatives and key research gaps on HInt within the HCI scope. As contributions, our results are useful to (1) expand the HCI community’s knowledge regarding this emerging paradigm and (2) stimulate new initiatives and contributions to the consolidation and evolution of the HCI with a lens on HInt.

Dead Users’ Profiles on Facebook: Limited Interaction Beyond Human Existence

Abstract Death has become increasingly visible on social networks, especially after COVID-19, and Facebook addresses that with double standards: while some profiles remain active, others turn into memorials. This article investigates how Facebook’s system deals with dead users’ profiles either to support or restrict interactions concerning users’ deaths. Our qualitative analysis of data from 54 public profiles of people who died between June 2020 and March 2021 showed that (i) Facebook fails to communicate the criteria for transforming profiles into memorials; (ii) no information about their contacts of deceased users’ profiles is given; (iii) the frequency of interaction with memorials and with active profiles is different; (iv) profiles’ privacy settings shape interaction. Our results exemplify how a sociotechnical system influences people’s interactions with dead users’ profiles. We herein highlight implications for interaction design and evaluation, besides the need to consider interaction as existence, which raises big challenges to the Human Computer Interaction (HCI) community.

Subject-Specific Human Modeling for Human Pose Estimation

3-D human pose estimation or human tracking has always been the focus of research in the human–computer interaction community. As the calibration step of human pose estimation, subject-specific modeling is crucially important to the subsequent pose estimation process. It not only provides a priori knowledge but also clearly defines the tracking target. This article presents a fully automatic subject modeling framework to reconstruct human pose, shape, as well as the body texture in a challenging optimization scenario. By integrating powerful differentiable rendering into the subject-specific modeling pipeline, the proposed method transforms the texture reconstruction problem into analysis by synthesis minimization and solves it efficiently by a gradient-based method. Furthermore, a novel covariance matrix adaptation annealing algorithm is proposed to attack the high-dimensional multimodal optimization problem in an adaptive manner. The domain knowledge of hierarchical human anatomy is seamlessly injected to the annealing optimization process by using a soft covariance matrix mask. All together contributes to the novel algorithm robust to the temptation of local minima. Experiments on the Human3.6 M dataset and the People-Snapshot dataset demonstrate the competitive results to the state of the art both qualitatively and quantitatively.

Lessons Learned from Modeling the Interaction with Conversational Agents

Intelligent conversational agents have become widespread. Inspired by conversations in natural language, they present different degrees of intelligence and autonomy, bringing challenges for Human-Computer Interaction (HCI). One such challenge concerns design languages for modeling user-agent interaction. We focus here on MoLIC, a design-phase dialogue model based on Semiotic Engineering theory devised to represent user-system interactions as conversations. We performed two case studies with MoLIC interaction diagrams representing two conversational agents – the ANA chatbot and Samsung Bixby. We examined how the interactive aspects of these agents could be expressed in MoLIC. Although it was possible to express the general interaction, our results showed limitations related to the language expressiveness or its inadequacy to represent these systems. We identified limitations in the applicability of MoLIC in modeling and pondered on how to extend or adapt it; directing the HCI community to issues and initiatives that can help design and model these technologies.

One Design Rule to Rule Them All: Towards a Universal Golden Rule for Designers of Human-Technology Interaction

The aim of this article is to raise awareness and discussion of the ethical aspects of the human-technology design of socio-technical systems in general, as well as the Human-Computer Interaction (HCI) designer responsibility towards users, stakeholders, and society. The dark side of HCI design is rising and the HCI community must respond to it in order to maintain its credibility and legitimacy. This article identifies and discusses four dimensions in human-technology interaction design ethics. To sum up these ethical dimensions of design and the responsibilities of a designer, this position article concludes with a proposed universal golden rule for designing human-technology interactions: Design as easy to use, honest, sustainable, and safe human-technology interactions as you would want others to design for you.

Connecting Twitter With Scholarly Networks: Exploring HCI Scholars’ Interactions From an SNA Approach

<roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Background:</b></roman> Building a reputable network on Twitter is viewed as impactful in several scholarly disciplines, but little is known about the professional and interdisciplinary human-computer interaction (HCI) community. This study combined two approaches from scholarly communication and technical communication to capture the static and dynamic features of the HCI scholar Twitter network. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Literature review:</b></roman> Related studies that described the scholarly reputation built through Twitter and social networking in the field of HCI were reviewed and discussed. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Research questions:</b></roman> 1. In which countries are HCI scholars more likely to follow their peers in the same country? 2. What are the characteristics (country, reputation) and actions (reciprocity) of HCI scholars who are more likely to build HCI scholarly network profiles on Twitter? <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Research methodology:</b></roman> The network analysis method of the exponential random graph model (ERGM) was adopted to trace and visualize current follower networks on Twitter. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Results and discussion:</b></roman> We found that 22.9% of HCI scholars use Twitter and that reciprocity and country of current employment best drive the Twitter connections of scholars. Characteristics of HCI scholars’ tie formation online are also illustrated and discussed. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Implications for practice:</b></roman> This study contributes to field studies of professional networks by identifying the structural properties and factors that influence scholars’ search for professional development on Twitter. The empirical findings should be a helpful reference for HCI professional societies and individual scholars in operating online professional networks.

May the Patterns Be with You: A Framework for HCI Patterns Development

By the end of the nineties, the concept of design patterns became a hot topic among the human-computer interaction community and many workshops have been held on the subject within international HCI conferences. After more than twenty years, HCI patterns continue to attract the attention of researchers around the world, but still struggle to be more widely adopted as a practical design tool. To better understand this context, we conducted a systematic literature mapping including papers from the ACM CHI Conference and from other five reputed HCI conference series. Through the analysis of 50 papers, we were able to elicit regional aspects, common terminology, and best practices for the research of patterns in HCI. Finally, based on the findings of the literature mapping, we propose an HCI pattern development framework that can assist researchers and professionals in the process of developing practical and useful pattern languages in a structured way.

Designing technologies for self-care: Describing the lived experiences of individuals with rheumatoid arthritis

Self-care of a chronic illness is a lifelong management process that includes taking medications, monitoring symptoms, and coping with emotional and lifestyle changes. Human-Computer Interaction (HCI) research has often responded to these needs by developing technologies that help an individual quantify aspects of their chronic illness, like daily pain, flare ups, or personal behaviours like diet and exercise. But this quantification fails to account for the ongoing needs of understanding ones disease and maintaining a balanced lifestyle. To understand these needs, we interviewed 12 people about their lived experience with Rheumatoid Arthritis (RA). We performed a thematic analysis of collected data and identified three types of support currently lacking for RA: (1) psychosocial care (2) patient agency, and (3) lifestyle adaptations. Our results highlight the need to support long term uncertainty while living with a chronic illness and identify needs the HCI community should consider when developing self-care technologies.

Two-stream fusion model using 3D-CNN and 2D-CNN via video-frames and optical flow motion templates for hand gesture recognition.

Hand gestures are useful tools for many applications in the human-computer interaction community. Here, the objective is to track the movement of the hand irrespective of the shape, size and color of the hand. And, for this, a motion template guided by optical flow (OFMT) is proposed. OFMT is a compact representation of the motion information of a gesture encoded into a single image. Recently, deep networks have shown impressive improvements as compared to conventional hand-crafted feature-based techniques. Moreover, it is seen that the use of different streams with informative input data helps to increase the recognition performance. This work basically proposes a two-stream fusion model for hand gesture recognition. The two-stream network consists of two layers—a 3D convolutional neural network (C3D) that takes gesture videos as input and a 2D-CNN that takes OFMT images as input. C3D has shown its efficiency in capturing spatiotemporal information of a video, whereas OFMT helps to eliminate irrelevant gestures providing additional motion information. Though each stream can work independently, they are combined with a fusion scheme to boost the recognition results. We have shown the efficiency of the proposed two-stream network on two databases.

Exploring the Experience of Traveling to Familiar Places in VR: An Empirical Study Using Google Earth VR

Considered to be an emerging topic in tourism research, traveling to familiar places benefits both the travelers and destinations. However, the current development of familiar tourism is severely affected by the epidemic. Recognizing the potential of virtual reality (VR), this research aims to explore VR’s relevance in the application of visiting familiar places and the empirical features of VR beyond on-site familiar tourism. We first reviewed the literature and developed an experience framework. Secondly, we conducted an exploratory activity in which participants (N = 16) used Google Earth VR to travel to their familiar places with two conditions (free exploration and task-oriented travel). In the activity, we employed think-aloud protocols, a scale, and semi-structured interviews. With our framework, we categorized findings into five dimensions and indicated the similarities and differences between familiar tourism in VR and that in actual places. Particularly, three empirical features (the sense of distance, multi-view space, and montage time) were recognized and formed VR’s opportunity to outperform on-site travel. We conclude by discussing the impacts of VR on familiar tourism and providing implications, including designing the timeline, sounds, tasks, and virtual guides. These suggestions inspire designers to acknowledge VR limitations and identify directions for future tourism applications. As a beginning of empirical investigation on VR familiar tourism, this study opens up a new field of discussion around VR tourism experience. We invite the Human-Computer Interaction (HCI) community to collectively probe into the VR experience and design of visiting familiar places. Efforts in this area will stretch beyond the current understandings of various forms of tourism and contribute to crafting rich travel experience by immersive technologies.