Human-computer Interaction Community Research Articles

Communicating the Privacy-Utility Trade-off: Supporting Informed Data Donation with Privacy Decision Interfaces for Differential Privacy

Data collections, such as those from citizen science projects, can provide valuable scientific insights or help the public to make decisions based on real demand. At the same time, the collected data might cause privacy risks for their volunteers, for example, by revealing sensitive information. Similar but less apparent trade-offs exist for data collected while using social media or other internet-based services. One approach to addressing these privacy risks might be to anonymize the data, for example, by using Differential Privacy (DP). DP allows for tuning and, consequently, communicating the trade-off between the data contributors' privacy and the resulting data utility for insights. However, there is little research that explores how to communicate the existing trade-off to users. % We contribute to closing this research gap by designing interactive elements and visualizations that specifically support people's understanding of this privacy-utility trade-off. We evaluated our user interfaces in a user study (N=378). Our results show that a combination of graphical risk visualization and interactive risk exploration best supports the informed decision, \ie the privacy decision is consistent with users' privacy concerns. Additionally, we found that personal attributes, such as numeracy, and the need for cognition, significantly influence the decision behavior and the privacy usability of privacy decision interfaces. In our recommendations, we encourage data collectors, such as citizen science project coordinators, to communicate existing privacy risks to their volunteers since such communication does not impact donation rates. %Understanding such privacy risks can also be part of typical training efforts in citizen science projects. %DP allows volunteers to balance their privacy concerns with their wish to contribute to the project. From a design perspective, we emphasize the complexity of the decision situation and the resulting need to design with usability for all population groups in mind. % We hope that our study will inspire further research from the human-computer interaction community that will unlock the full potential of DP for a broad audience and ultimately contribute to a societal understanding of acceptable privacy losses in specific data contexts.

Social connectedness and the role of virtual reality: experiences and perceptions of people living with dementia and their caregivers

Purpose People living with dementia are often at increased risk of becoming socially disconnected due to dementia-related challenges. In recent years, digital technology has been designed to help address the social health of people living with dementia and provide opportunities to promote or maintain their social connectedness. This paper presents the findings from phase two of a participatory action research project, which explored people living with dementia and their caregiver’s experiences and perceptions of social connectedness and the potential role of Virtual Reality (VR) in promoting or maintaining same. Materials and Methods People living with dementia (n = 8) and their informal caregivers (n = 8) participated in an individual, 1:1 online interview. Data analysis was guided by reflexive thematic analysis. Results The findings presented four themes: social connectedness: lived experiences and insights, facilitating social connectedness, barriers to social connectedness and the potential of multi-user VR for social connectedness. People living with dementia experienced a range of personal, community and societal connectedness. Facilitators of social connectedness included supportive, non-judgemental, and reciprocal relationships, technology adoption, and personal and contextual facilitators. Dementia-related difficulties and periods of disruption or change were considered barriers to social connectedness. Multi-user VR was perceived as useful for promoting and maintaining social connectedness. Conclusions The perceived usefulness of multi-user VR for social connectedness indicates its potential for use with this population. Understanding the lived experiences, barriers, and facilitators of social connectedness will assist researchers and the human-computer interaction community to inform the design of future multi-user VR for social connectedness outcomes with people living with dementia and their caregivers.

Designing technology to support greater participation of people living with dementia in daily and meaningful activities.

People living with dementia should be at the center of decision-making regarding their plans and goals for daily living and meaningful activities that help promote health and mental well-being. The human-computer interaction community has recently begun to recognize the need to design technologies where the person living with dementia is an active rather than a passive user of technology in the management of their care. Data collection comprised semi-structured interviews and focus groups held with dyads of people with early-stage dementia (n = 5) and their informal carers (n = 4), as well as health professionals (n = 5). This article discusses findings from the thematic analysis of this qualitative data. Analysis resulted in the construction of three main themes: (1) maintaining a sense of purpose and identity, (2) learning helplessness and (3) shared decision-making and collaboration. Within each of the three main themes, related sub-themes were also constructed. There is a need to design technologies for persons living with dementia/carer dyads that can support collaborative care planning and engagement in meaningful activities while also balancing persons living with dementia empowerment and active engagement in self-management with carer support.

The effects of non-fungible token platform affordances on customer loyalty: A Buyer–Creator duality perspective

Prior academic research on non-fungible tokens (NFTs), an application of blockchain technology in the metaverse environment, has focused primarily on the market demand side (i.e., buyers), with limited attention given to the supply side (i.e., creators). To address this gap, we seek to understand creators' and buyers’ perceptions of platform affordances that can aid NFT platforms in developing functions that address the needs and expectations of both parties. To do this, we draw on affordance theory and psychological contract theory and use a mixed-methods approach consisting of text mining, structural equation modeling, and importance-performance map analysis. The results show that platform affordances (transaction, aesthetics, status, and user interface affordance) facilitate the creation of psychological contracts, which in turn promotes customer loyalty. Moreover, the relationships identified differ between creators and buyers. Overall, these results point to the value of using a user-centric human–computer interaction perspective to study emerging blockchain-based systems. They can inform design principles and guidelines for NFT platforms to effectively support creator–buyer interactions, transactions, and relationships through enhanced affordances and user-centric experience. This study advances knowledge of design in the nascent NFT domain relevant to the cross-disciplinary human–computer interaction community.

Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature Review

Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing. Designers, as a special user group interacting with AI, have received more attention in the Human-Computer Interaction community. Prior work has discussed emerging challenges that persist in designing for AI. However, few systematic reviews focus on AI for design to understand how designers and AI can augment each other's complementary strengths in design collaboration. In this work, we conducted a landscape analysis of AI for design, via a systematic literature review of 93 papers. The analysis first provides a bird's eye view of overall patterns in this area. The analysis also reveals three themes interpreted from the paper corpus associated with AI for design, including AI assisting designers, designers assisting AI, and characterizing designer-AI collaboration. We discuss the implications of our findings and suggested methodological proposals to guide HCI toward research and practices that center on collaborative creativity.

Eye-tracking in map use, map user and map usability research: what are we looking for?

ABSTRACT This overview paper summarises the ‘state of the art and of the science’ of eye-tracking, and its applications in map use research. Cartographic research is introduced, and its contemporary direction, which indicates that the main areas of such research are now focussed on human beings and their interaction with maps and geospatial displays, is stressed. A brief outline of several different methodologies for map use research is presented: observation, thinking loud, keyboard analysis, eye-tracking, and questionnaires. The role of eye-tracking as a major methodology for use, user, and usability investigation is explored; along with the possible choices for the researcher in the important areas of participant selection, eye-tracking equipment, set-up and use of the testing environment, and analysis of output data. Typical outcomes from eye tracking research are considered, with an assessment of its value in cartographic research in general. Future directions are suggested, along with the need for cartography to promote the valuable work done by researchers using eye-tracking for map use studies to the wider human-computer interaction community, expanding the scope of the geospatial-based stimuli in such experiments beyond maps, making use of the significant expertise and enthusiasm of cartographic researchers.

Human-Computer Integration as an Extension of Interaction: Understanding Its State-of-the-Art and the Next Challenges

The Interaction Era is evolving and extending to the Human-Computer Integration Era, in which the relationship between humans and technologies is characterized as a partnership. Motivated by this evolution, we conducted a Systematic Literature Review to present the current state of knowledge about the emerging paradigm of Human-Computer Integration (HInt), as well as its opportunities and challenges for the Human-Computer Interaction (HCI) area. As a result, we organize the existing knowledge and research on HInt which describes it as a new interaction paradigm. Our findings indicate that HInt is a topic of growing interest in the HCI Community, and provide an overview of existing initiatives and key research gaps on HInt within the HCI scope. As contributions, our results are useful to (1) expand the HCI community’s knowledge regarding this emerging paradigm and (2) stimulate new initiatives and contributions to the consolidation and evolution of the HCI with a lens on HInt.

Dead Users’ Profiles on Facebook: Limited Interaction Beyond Human Existence

Abstract Death has become increasingly visible on social networks, especially after COVID-19, and Facebook addresses that with double standards: while some profiles remain active, others turn into memorials. This article investigates how Facebook’s system deals with dead users’ profiles either to support or restrict interactions concerning users’ deaths. Our qualitative analysis of data from 54 public profiles of people who died between June 2020 and March 2021 showed that (i) Facebook fails to communicate the criteria for transforming profiles into memorials; (ii) no information about their contacts of deceased users’ profiles is given; (iii) the frequency of interaction with memorials and with active profiles is different; (iv) profiles’ privacy settings shape interaction. Our results exemplify how a sociotechnical system influences people’s interactions with dead users’ profiles. We herein highlight implications for interaction design and evaluation, besides the need to consider interaction as existence, which raises big challenges to the Human Computer Interaction (HCI) community.

Subject-Specific Human Modeling for Human Pose Estimation

3-D human pose estimation or human tracking has always been the focus of research in the human–computer interaction community. As the calibration step of human pose estimation, subject-specific modeling is crucially important to the subsequent pose estimation process. It not only provides a priori knowledge but also clearly defines the tracking target. This article presents a fully automatic subject modeling framework to reconstruct human pose, shape, as well as the body texture in a challenging optimization scenario. By integrating powerful differentiable rendering into the subject-specific modeling pipeline, the proposed method transforms the texture reconstruction problem into analysis by synthesis minimization and solves it efficiently by a gradient-based method. Furthermore, a novel covariance matrix adaptation annealing algorithm is proposed to attack the high-dimensional multimodal optimization problem in an adaptive manner. The domain knowledge of hierarchical human anatomy is seamlessly injected to the annealing optimization process by using a soft covariance matrix mask. All together contributes to the novel algorithm robust to the temptation of local minima. Experiments on the Human3.6 M dataset and the People-Snapshot dataset demonstrate the competitive results to the state of the art both qualitatively and quantitatively.

Lessons Learned from Modeling the Interaction with Conversational Agents

Intelligent conversational agents have become widespread. Inspired by conversations in natural language, they present different degrees of intelligence and autonomy, bringing challenges for Human-Computer Interaction (HCI). One such challenge concerns design languages for modeling user-agent interaction. We focus here on MoLIC, a design-phase dialogue model based on Semiotic Engineering theory devised to represent user-system interactions as conversations. We performed two case studies with MoLIC interaction diagrams representing two conversational agents – the ANA chatbot and Samsung Bixby. We examined how the interactive aspects of these agents could be expressed in MoLIC. Although it was possible to express the general interaction, our results showed limitations related to the language expressiveness or its inadequacy to represent these systems. We identified limitations in the applicability of MoLIC in modeling and pondered on how to extend or adapt it; directing the HCI community to issues and initiatives that can help design and model these technologies.

One Design Rule to Rule Them All: Towards a Universal Golden Rule for Designers of Human-Technology Interaction

The aim of this article is to raise awareness and discussion of the ethical aspects of the human-technology design of socio-technical systems in general, as well as the Human-Computer Interaction (HCI) designer responsibility towards users, stakeholders, and society. The dark side of HCI design is rising and the HCI community must respond to it in order to maintain its credibility and legitimacy. This article identifies and discusses four dimensions in human-technology interaction design ethics. To sum up these ethical dimensions of design and the responsibilities of a designer, this position article concludes with a proposed universal golden rule for designing human-technology interactions: Design as easy to use, honest, sustainable, and safe human-technology interactions as you would want others to design for you.

Connecting Twitter With Scholarly Networks: Exploring HCI Scholars’ Interactions From an SNA Approach

<roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Background:</b></roman> Building a reputable network on Twitter is viewed as impactful in several scholarly disciplines, but little is known about the professional and interdisciplinary human-computer interaction (HCI) community. This study combined two approaches from scholarly communication and technical communication to capture the static and dynamic features of the HCI scholar Twitter network. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Literature review:</b></roman> Related studies that described the scholarly reputation built through Twitter and social networking in the field of HCI were reviewed and discussed. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Research questions:</b></roman> 1. In which countries are HCI scholars more likely to follow their peers in the same country? 2. What are the characteristics (country, reputation) and actions (reciprocity) of HCI scholars who are more likely to build HCI scholarly network profiles on Twitter? <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Research methodology:</b></roman> The network analysis method of the exponential random graph model (ERGM) was adopted to trace and visualize current follower networks on Twitter. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Results and discussion:</b></roman> We found that 22.9% of HCI scholars use Twitter and that reciprocity and country of current employment best drive the Twitter connections of scholars. Characteristics of HCI scholars’ tie formation online are also illustrated and discussed. <roman xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>Implications for practice:</b></roman> This study contributes to field studies of professional networks by identifying the structural properties and factors that influence scholars’ search for professional development on Twitter. The empirical findings should be a helpful reference for HCI professional societies and individual scholars in operating online professional networks.

May the Patterns Be with You: A Framework for HCI Patterns Development

By the end of the nineties, the concept of design patterns became a hot topic among the human-computer interaction community and many workshops have been held on the subject within international HCI conferences. After more than twenty years, HCI patterns continue to attract the attention of researchers around the world, but still struggle to be more widely adopted as a practical design tool. To better understand this context, we conducted a systematic literature mapping including papers from the ACM CHI Conference and from other five reputed HCI conference series. Through the analysis of 50 papers, we were able to elicit regional aspects, common terminology, and best practices for the research of patterns in HCI. Finally, based on the findings of the literature mapping, we propose an HCI pattern development framework that can assist researchers and professionals in the process of developing practical and useful pattern languages in a structured way.

Designing technologies for self-care: Describing the lived experiences of individuals with rheumatoid arthritis

Self-care of a chronic illness is a lifelong management process that includes taking medications, monitoring symptoms, and coping with emotional and lifestyle changes. Human-Computer Interaction (HCI) research has often responded to these needs by developing technologies that help an individual quantify aspects of their chronic illness, like daily pain, flare ups, or personal behaviours like diet and exercise. But this quantification fails to account for the ongoing needs of understanding ones disease and maintaining a balanced lifestyle. To understand these needs, we interviewed 12 people about their lived experience with Rheumatoid Arthritis (RA). We performed a thematic analysis of collected data and identified three types of support currently lacking for RA: (1) psychosocial care (2) patient agency, and (3) lifestyle adaptations. Our results highlight the need to support long term uncertainty while living with a chronic illness and identify needs the HCI community should consider when developing self-care technologies.

Two-stream fusion model using 3D-CNN and 2D-CNN via video-frames and optical flow motion templates for hand gesture recognition.

Hand gestures are useful tools for many applications in the human-computer interaction community. Here, the objective is to track the movement of the hand irrespective of the shape, size and color of the hand. And, for this, a motion template guided by optical flow (OFMT) is proposed. OFMT is a compact representation of the motion information of a gesture encoded into a single image. Recently, deep networks have shown impressive improvements as compared to conventional hand-crafted feature-based techniques. Moreover, it is seen that the use of different streams with informative input data helps to increase the recognition performance. This work basically proposes a two-stream fusion model for hand gesture recognition. The two-stream network consists of two layers—a 3D convolutional neural network (C3D) that takes gesture videos as input and a 2D-CNN that takes OFMT images as input. C3D has shown its efficiency in capturing spatiotemporal information of a video, whereas OFMT helps to eliminate irrelevant gestures providing additional motion information. Though each stream can work independently, they are combined with a fusion scheme to boost the recognition results. We have shown the efficiency of the proposed two-stream network on two databases.

Exploring the Experience of Traveling to Familiar Places in VR: An Empirical Study Using Google Earth VR

Considered to be an emerging topic in tourism research, traveling to familiar places benefits both the travelers and destinations. However, the current development of familiar tourism is severely affected by the epidemic. Recognizing the potential of virtual reality (VR), this research aims to explore VR’s relevance in the application of visiting familiar places and the empirical features of VR beyond on-site familiar tourism. We first reviewed the literature and developed an experience framework. Secondly, we conducted an exploratory activity in which participants (N = 16) used Google Earth VR to travel to their familiar places with two conditions (free exploration and task-oriented travel). In the activity, we employed think-aloud protocols, a scale, and semi-structured interviews. With our framework, we categorized findings into five dimensions and indicated the similarities and differences between familiar tourism in VR and that in actual places. Particularly, three empirical features (the sense of distance, multi-view space, and montage time) were recognized and formed VR’s opportunity to outperform on-site travel. We conclude by discussing the impacts of VR on familiar tourism and providing implications, including designing the timeline, sounds, tasks, and virtual guides. These suggestions inspire designers to acknowledge VR limitations and identify directions for future tourism applications. As a beginning of empirical investigation on VR familiar tourism, this study opens up a new field of discussion around VR tourism experience. We invite the Human-Computer Interaction (HCI) community to collectively probe into the VR experience and design of visiting familiar places. Efforts in this area will stretch beyond the current understandings of various forms of tourism and contribute to crafting rich travel experience by immersive technologies.

Peluang Penelitian UI/UX pada Pengembangan Aplikasi Mobile: Systematic Literature Review

Mobile application is one form of information technology development that continues to increase from year to year. In the development of the digital world, it is increasingly encouraging to increase the affordability of the use of mobile devices (smartphones) to generate the most sustainable technological growth. This enormous growth in its development has given impetus to smartphone manufacturers to produce new applications on mobile devices to meet user needs. A mobile application development must have user interface and user experience aspects as part of the human-computer interaction (HCI) discipline. Mobile applications under development can run on Android and iOS platforms. This study aims to identify research opportunities in the UI/UX aspect of mobile application development whose data was obtained from related journals in 2017-2021 to provide an overview of the latest studies on research related to UI/UX mobile applications and provide analysis on topics and areas that do not have enough information and what factors to focus on. This study uses the Systematic Literature Review (SLR) method. The result of this research is to provide a systematic literature review of existing studies on UI/UX mobile applications. This research is expected to be useful for the HCI community in seeing the UI/UX description in the development of mobile applications to shape the direction of future research.

Joint modelling of audio-visual cues using attention mechanisms for emotion recognition

Emotions play a crucial role in human-human communications with complex socio-psychological nature. In order to enhance emotion communication in human-computer interaction, this paper studies emotion recognition from audio and visual signals in video clips, utilizing facial expressions and vocal utterances. Thereby, the study aims to exploit temporal information of audio-visual cues and detect their informative time segments. Attention mechanisms are used to exploit the importance of each modality over time. We propose a novel framework that consists of bi-modal time windows spanning short video clips labeled with discrete emotions. The framework employs two networks, with each one being dedicated to one modality. As input to a modality-specific network, we consider a time-dependent signal deriving from the embeddings of the video and audio modalities. We employ the encoder part of the Transformer on the visual embeddings and another one on the audio embeddings. The research in this paper introduces detailed studies and meta-analysis findings, linking the outputs of our proposition to research from psychology. Specifically, it presents a framework to understand underlying principles of emotion recognition as functions of three separate setups in terms of modalities: audio only, video only, and the fusion of audio and video. Experimental results on two datasets show that the proposed framework achieves improved accuracy in emotion recognition, compared to state-of-the-art techniques and baseline methods not using attention mechanisms. The proposed method improves the results over baseline methods by at least 5.4%. Our experiments show that attention mechanisms reduce the gap between the entropies of unimodal predictions, which increases the bimodal predictions’ certainty and, therefore, improves the bimodal recognition rates. Furthermore, evaluations with noisy data in different scenarios are presented during the training and testing processes to check the framework’s consistency and the attention mechanism’s behavior. The results demonstrate that attention mechanisms increase the framework’s robustness when exposed to similar conditions during the training and the testing phases. Finally, we present comprehensive evaluations of emotion recognition as a function of time. The study shows that the middle time segments of a video clip are essential in the case of using audio modality. However, in the case of video modality, the importance of time windows is distributed equally.

How do people experience the temporality of everyday life changes? Towards the exploration of existential time in HCI

The idea of time as an existential concern has found scarce attention in the Human-Computer Interaction (HCI) community. Still, time in our life is mostly experienced in connection with important existential matters. Life changes are emblematic examples of aspects of our existence that are fundamentally temporal. In this article, I explore how people experience the temporalities of “change”, by asking 16 participants to report on life changes they consider relevant and their temporal aspects. I found that temporal features like velocity, direction, and synchronization deeply impact the process of change, by affecting the participants’ opportunities to make sense of and act upon their own transformations. The study findings point to a conception of existential time that contributes to both HCI research on time and life changing events, by highlighting the relativity of temporal slowness and speediness, the malleability of time, and the fundamental openness of our time experiences.

Ask the Users

Just-in-Time Adaptive Interventions (JITAIs) are envisioned to harness rich data on users' contexts. However, many JITAIs fall short in leveraging the value of the data while sculpting the interventions. Investigating the literature reveals a lack of user-centered design (UCD) methods in designing JITAIs. Our case study of applying a UCD process revealed that even without deploying a JITAI, UCD could uncover user interactions that inform critical design decisions of a JITAI's components. We reflect on our experiences engaging in a user-centered JITAI design process and urge the broader human-computer interaction (HCI) community to devise concrete design guidelines for JITAIs. We take the first step toward that goal by proposing a checklist of key design considerations for future JITAIs. Together, this case study contributes insights on applying UCD to the design of all components of a JITAI, which can help capture users' needs and provide actionable interventions.