HCI Research Research Articles

Moving Towards Mobility Justice: Challenges and Considerations for Supporting Advocacy

In response to climate change and continued urbanization, urban transportation systems around the world are undergoing transitions to promote lower-emission vehicles, public transit, biking and walking. However, mobility is a complex issue that raises important questions of social justice as a result of its connections to numerous aspects of everyday life and the broader social and political contexts. Drawing on interviews with transportation advocates across Canada, we identify four ways in which the design and use of existing mobility tools and technologies perpetuate mobility injustices, and deepen the divide between urban planners and the public. Looking across these arguments, we note path-dependence in transportation knowledge infrastructures as a common barrier to mobility justice advocacy that can be difficult to recognize or overcome. Finally, we consider tactics that research in HCI and CSCW might pursue as part of efforts to unsettle path-dependence and reorient transportation planning towards mobility justice.

Security Patchworking in Lebanon: Infrastructuring Across Failing Infrastructures

In this paper we bring to light the infrastructuring work carried out by people in Lebanon to establish and maintain everyday security in response to multiple simultaneously failing infrastructures. We do so through interviews with 13 participants from 12 digital and human rights organisations and two weeks of ethnographically informed fieldwork in Beirut, Lebanon, in July 2022. Through our analysis we develop the notion of security patchworking that makes visible the infrastructuring work necessitated to secure basic needs such as electricity provision, identity authentication and financial resources. Such practices are rooted in differing mechanisms of protection that often result in new forms of insecurity. We discuss the implications for CSCW and HCI researchers and point to security patchworking as a lens to be used when designing technologies to support infrastructuring, while advocating for collaborative work across CSCW and security research.

The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines

Pilot studies are an essential cornerstone of the design of crowdsourcing campaigns, yet they are often only mentioned in passing in the scholarly literature. A lack of details surrounding pilot studies in crowdsourcing research hinders the replication of studies and the reproduction of findings, stalling potential scientific advances. We conducted a systematic literature review on the current state of pilot study reporting at the intersection of crowdsourcing and HCI research. Our review of ten years of literature included 171 articles published in the proceedings of the Conference on Human Computation and Crowdsourcing (AAAI HCOMP) and the ACM Digital Library. We found that pilot studies in crowdsourcing research (i.e., crowd pilot studies) are often under-reported in the literature. Important details, such as the number of workers and rewards to workers, are often not reported. On the basis of our findings, we reflect on the current state of practice and formulate a set of best practice guidelines for reporting crowd pilot studies in crowdsourcing research. We also provide implications for the design of crowdsourcing platforms and make practical suggestions for supporting crowd pilot study reporting.

Missed Opportunities for Human-Centered AI Research: Understanding Stakeholder Collaboration in Mental Health AI Research

In the mental health domain, patient engagement is key to designing human-centered technologies. CSCW and HCI researchers have delved into various facets of collaboration in AI research; however, previous research neglects the individuals who both produce the data and will be most impacted by the resulting technologies, such as patients. This study examines how interdisciplinary researchers and mental health patients who donate their data for AI research collaborate and how we can improve human-centeredness in mental health AI research. We interviewed patient participants, AI researchers, and clinical researchers in a federally funded mental health AI research project. We used the concept of boundary objects to understand stakeholder collaboration. Our findings reveal that the social media data provided by patient participants functioned as boundary objects that facilitated stakeholder collaboration. Although the collaboration appeared to be successful, we argue that building consensus, or understanding each other's perspectives, can improve the human-centeredness of mental health AI research. Based on the findings, we provide suggestions for human-centered mental health AI research, working with data donors as domain experts, making invisible work visible, and privacy implications.

Better Living Through Creepy Technology? Exploring Tensions Between a Novel Class of Well-Being Apps and Affective Discomfort in App Culture

Well-being apps promise to improve people's lives. Yet evidence shows that the data-hungry app culture that contextualizes well-being apps normalizes the user experience of affective discomfort. This apparent contradiction raises a difficult question: Is it responsible to ask people to improve their well-being by engaging further with an app culture that normalizes affective discomfort? We approached this question by deploying an online, scenario-based survey (n=688) about a fictional, but realistic well-being app called "Thalia." Thalia represents a novel class of well-being apps that are envisioned to: (i) utilize AI-driven facial recognition and analysis; and (ii) collect data for use in medical contexts. We found that people perceived Thalia to be affectively discomfiting even as they judged Thalia to be beneficial. Such findings imply a trade-off between 'better living through technology' and the negative affective implications of surveillance capitalistic app culture. Such a trade-off necessitates high-level analysis of just what "well-being" means in the context of contemporary app culture. Through analysis and discussion, we explore a troubling interplay between novel well-being apps and affective discomfort that requires careful attention from HCI researchers if human-centered well-being -- flourishing -- is truly what our products are intended to foster.

Mapping the Landscape of Independent Food Delivery Platforms in the United States

Beyond the well-known giants like Uber Eats and DoorDash, there are hundreds of independent food delivery platforms in the United States. However, little is known about the sociotechnical landscape of these "indie'' platforms. In this paper, we analyzed these platforms to understand why they were created, how they operate, and what technologies they use. We collected data on 495 indie platforms and detailed survey responses from 29 platforms. We found that personalized, timely service is a central value of indie platforms, as is a sense of responsibility to the local community they serve. Indie platforms are motivated to provide fair rates for restaurants and couriers. These alternative business practices differentiate them from mainstream platforms. Though indie platforms have plans to expand, a lack of customizability in off-the-shelf software prevents independent platforms from personalizing services for their local communities. We show that these platforms are a widespread and longstanding fixture of the food delivery market. We illustrate the diversity of motivations and values to explain why a one-size-fits-all support is insufficient, and we discuss the siloing of technology that inhibits platforms' growth. Through these insights, we aim to promote future HCI research into the potential development of public-interest technologies for local food delivery.

NewsGuesser: Using Curiosity to Reduce Selective Exposure

Selective exposure has long been a concern of HCI researchers as it can lead to ideological polarization and distrust in society. Efforts have tried to reduce selective exposure online by serving diversified news content, but their effectiveness has been limited by users' lack of motivation to engage with the diverse content offered. To address this, we design the NewsGuesser system, which leverages the insight that curiosity can prompt motivation and engagement, by asking readers to guess the source of their news. In interviews with 40 participants, balanced for partisan affiliation, we use NewsGuesser as a probe tool to explore how guessing affects their perceptions of selective exposure. Participants struggled with the guessing game, which revealed a misalignment between users' expectations of different news sources and reality. Faced with the visualizations of the (often inaccurate) guessing results, participants were able to reflect on their own biases and selective exposure. In a number of cases, the guessing process changed participants' impressions of news organizations and some expressed an interest in engaging with more diverse news sources. While many also found the guessing game frustrating, the system and interview results suggest a number of new directions for designing social media and news media platforms.

Socio-digital Rural Resilience: An Exploration of Information Infrastructures Within and Across Rural Villages During Covid-19

Access to the internet and digital technology are reported determinants and enablers of social capital and community resilience. However, despite rural areas facing ecological, social, and economic uncertainties across the world, communication technologies and the design of civic technologies in HCI are not designed with the rural in mind. In this paper, we explore information infrastructures within and across rural villages in the UK focusing on the interplay between human, social, and technical infrastructures. We report on the existing communication practices, underlining the impacts of transitioning to digital communication technologies during the COVID-19 pandemic. Our findings point to the fragility of information infrastructure, the "civic duty" of bridging information and skills gaps, and the role of technology as a threat to social norms and disrupter of the 'politics of the rural'. The work presented here has implications for Rural HCI researchers and designers interested in design from the rural.

Toward Trauma-Informed Research Practices with Youth in HCI: Caring for Participants and Research Assistants When Studying Sensitive Topics

Research involving sensitive data often leads to valuable human-centered insights. Yet, the effects of participating in and conducting research about sensitive data with youth are poorly understood. We conducted meta-level research to improve our understanding of these effects. We did the following: (i) asked youth (aged 13-21) to share their private Instagram Direct Messages (DMs) and flag their unsafe DMs; (ii) interviewed 30 participants about the experience of reflecting on this sensitive data; (iii) interviewed research assistants (RAs, n=12) about their experience analyzing youth's data. We found that reflecting about DMs brought discomfort for participants and RAs, although both benefited from increasing their awareness about online risks, their behavior, and privacy and social media practices. Participants had high expectations for safeguarding their private data while their concerns were mitigated by the potential to improve online safety. We provide implications for ethical research practices and the development of reflective practices among participants and RAs through applying trauma-informed principles to HCI research.

Critical pedagogy and disability in participatory research: a review

PurposeThe literature review explores how multidisciplinary approaches based on critical pedagogy and participatory research can provide frameworks for equitable partnerships and genuine participation in educational design and research practices. Additionally, the essay aims to expand understandings of equitable engagement within educational research and design based on principles from critical pedagogy.Design/methodology/approachThe essay draws from diverse literature in the learning sciences, health informatics, industrial design, disability studies, ethnic studies, rehabilitation science, and to a lesser extent HCI research to understand how critical pedagogy and participatory research methods can provide useful frameworks for disabled peoples' equitable engagement and genuine participation in educational research and design. The literature reviewed in the paper concern topics such as participatory approaches to community development with disabled adults, the implementation of university-initiated community partnerships, participatory research with students and disabled people, and the importance of culturally-responsive research practices. The design literature in this review explores various arenas such as the co-design of assistive technologies with disabled children and adults and the design of curricula for students with and without disabilities. This review focuses on research practices that engender disabled peoples' participation in educational research and design, with focus on developing multidisciplinary frameworks for such research.FindingsThe literature review concludes that participatory research methods and critical pedagogy provide useful frameworks for disabled peoples’ participation in educational design and research practices. Critical pedagogy and participatory design allow for the genuine participation of disabled people in the research process.Social implicationsEmphases on collaboration and collective knowledge-building in social transformation are present in scholarship concerning critical pedagogy, participatory research, and disability studies. However, these connections have been routinely underexplored in the literature. This paper aims to underscore these integral connections as a means to build solidarity between disabled and other marginalized people.Originality/valueThe connections between participatory research methods, critical pedagogy, and disability studies have been previously underexplored. The literature review proposes a combined approach, which has the potential to radically transform multiple realms of research beyond the learning and information sciences.

Mitigating Epistemic Injustice: The Online Construction of a Bisexual Culture

People participating in online groups often co-construct knowledge of what they believe and, sometimes, co-construct their understanding of who they are . Through participant observation and semi-structured interviews with 13 members of the online forum r/bisexual on Reddit, we found participants collaboratively constructing an understanding of bisexuality. We found this knowledge-building fills an epistemic gap resulting from bisexuality often being poorly understood. When individuals do not possess knowledge key to understanding their own lives, this can be seen as hermeneutical injustice – a type of epistemic injustice. We use the lens of hermeneutical injustice to shed light on participants’ experiences on r/bisexual. Our work contributes to recent research on epistemic injustice in HCI by looking at how members of r/bisexual mitigate epistemic injustice by reclaiming residuality – the space outside the gay-straight binary. We also discuss considerations for hermeneutical injustice to inform the design of online communities and HCI research practice.

Examining Voice Community Use

Visual online communities can present accessibility challenges to older adults or people with vision and motor disabilities. Motivated by this challenge, accessibility and HCI researchers have called for voice-based communities to support aging and disability. This paper extends prior work on voice community design and short-term use by providing empirical data on how people interact with voice communities over time and intentional instances of non-use. We conducted a one-year study with 43 blind and low vision older adults, of whom 21 used a voice-based community. We use vignettes to unpack five different voice community member roles - the obligatory poster, routine poster, cross-platform lurker, busy socialite, and visual expertise seeker - and discuss community interactions over time. Findings show how participation varied based on engagement in other communities and ways that participants sought interaction. We discuss (1) how to design voice communities for member roles and (2) the implications of synchronous and asynchronous voice community interaction in voice-only communities.

The Realities of Evaluating Educational Technology in School Settings

HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [ 18 , 53 ]. Through the analysis of the authors’ contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process. We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

Evaluating Interpretive Research in HCI

Evaluating Interpretive Research in HCI

Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms

Machine learning (ML) models that achieve high average accuracy can still underperform on semantically coherent subsets ("slices") of data. This behavior can have significant societal consequences for the safety or bias of the model in deployment, but identifying these underperforming slices can be difficult in practice, especially in domains where practitioners lack access to group annotations to define coherent subsets of their data. Motivated by these challenges, ML researchers have developed new slice discovery algorithms that aim to group together coherent and high-error subsets of data. However, there has been little evaluation focused on whether these tools help humans form correct hypotheses about where (for which groups) their model underperforms. We conduct a controlled user study (N = 15) where we show 40 slices output by two state-of-the-art slice discovery algorithms to users, and ask them to form hypotheses about an object detection model. Our results provide positive evidence that these tools provide some benefit over a naive baseline, and also shed light on challenges faced by users during the hypothesis formation step. We conclude by discussing design opportunities for ML and HCI researchers. Our findings point to the importance of centering users when creating and evaluating new tools for slice discovery.

Understanding the Effects of Movement Direction on 2D Touch Pointing Tasks

HCI researchers have long recognized the significant effects of movement direction on human performance, and this factor has been carefully addressed to benefit user interface design. According to our previous study (2012), the weights of the two target dimensions, width W and height H, in the extended index of difficulty (ID) for 2D pointing tasks are asymmetric and appear to vary periodically based on movement direction (θ), following a cosine function. However, this periodic effect of movement direction is uncertain for direct 2D touch pointing tasks, and a thorough understanding of the effects of movement direction on direct pointing tasks, such as on touch input surfaces, is still lacking. In this paper, we conducted two experiments on a 24-inch touch screen, with tilted and horizontal orientations respectively, to confirm the periodic effect in the context of direct pointing and illustrate its variations across different pointing tasks. At the same time, we propose a quantification formula to measure the real differences in task difficulty caused by the direction factor. To the best of our knowledge, this is the first study to do so. Using this formula, the ID values in different directions can be unified to the same scale and compared, providing a new perspective for understanding and evaluating human performance in different interaction environments.

Using Virtual Objects With Hand-Tracking: The Effects of Visual Congruence and Mid-Air Haptics on Sense of Agency.

Virtual reality expands the possibilities of human action. With hand-tracking technology, we can directly interact with these environments without the need for a mediating controller. Much previous research has looked at the user-avatar relationship. Here we explore the avatar-object relationship by manipulating the visual congruence and haptic feedback of the virtual object of interaction. We examine the effect of these variables on the sense of agency (SoA), which refers to the feeling of control over our actions and their effects. This psychological variable is highly relevant to user experience and is attracting increased interest in the field. Our results showed that implicit SoA was not significantly affected by visual congruence and haptics. However, both of these manipulations significantly affected explicit SoA, which was strengthened by the presence of mid-air haptics and was weakened by the presence of visual incongruence. We propose an explanation of these findings that draws on the cue integration theory of SoA. We also discuss the implications of these findings for HCI research and design.

Congratulations! You are severely underweight—Exploring negative consequences of persuasive system design

Personal health and wellbeing have grown as an object of study within HCI research and many individuals undergoing weight loss journeys use persuasive systems, such as weight loss apps, to guide them. These systems rely on various principles to persuade users toward their health goal and have in much research been considered having positive effects. There are however also parallel findings indicating unwanted effects. To explore the effects of persuasive system design, we conduct a longitudinal exploratory expert evaluation of 10 free weight loss tracking apps by emulating drastic weight loss for an extended period of time. The objective of the exploration is to unravel potential negative effects of using such systems. Our findings highlight risks of negative effects as some apps take a passive stance toward drastic weight loss, while others actively encourage it through design. We contribute to a growing strand of health related HCI research by mapping out the relation between design and negative consequences of persuasive system use.

“An Adapt-or-Die Type of Situation”: Perception, Adoption, and Use of Text-to-Image-Generation AI by Game Industry Professionals

Text-to-image generation (TTIG) models, a recent addition to creative AI, can generate images based on a text description. These models have begun to rival the work of professional creatives, and sparked discussions on the future of creative work, loss of jobs, and copyright issues, amongst other important implications. To support the sustainable adoption of TTIG, we must provide rich, reliable and transparent insights into how professionals perceive, adopt and use TTIG. Crucially though, the public debate is shallow, narrow and lacking transparency, while academic work has focused on studying the use of TTIG in a general artist population, but not on the perceptions and attitudes of professionals in a specific industry. In this paper, we contribute a qualitative, exploratory interview study on TTIG in the Finnish videogame industry. Through a Template Analysis on semi-structured interviews with 14 game professionals, we reveal 12 overarching themes, structured into 39 sub-themes on professionals' perception, adoption and use of TTIG in games industry practice. Experiencing (yet another) change of roles and creative processes, our participants' reflections can inform discussions within the industry, be used by policymakers to inform urgently needed legislation, and support researchers in games, HCI and AI to support the sustainable, professional use of TTIG, and foster games as cultural artefacts.

A Scoping Review of Heuristics in Videos Games Research: Definitions, Development, Application, and Operationalisation

Heuristics present a cheap and effective way of evaluating usability. However, in video games, evaluating unique player experiences that are dependent on individual preferences and abilities presents a challenge that goes beyond usability. Video games are more than just functional software, so games heuristics have been adapted to help examine functionality and experience. This paper reports on how papers published in the ACM Digital Library between 2012 and 2022 develop and apply heuristics in video games research. We found that heuristics are often used outside their intended purpose of being used in an expert evaluation. Instead, they are used as survey instruments, interview guides, codes for thematic analysis, and as design guidelines. This research contributes to HCI and video games research by distinguishing the terms design guidelines and design principles from heuristics. We make recommendations for researchers around developing heuristics and conducting video game heuristic evaluations. We propose a method for operationalising heuristics and make recommendations for the implementation of heuristics to improve the quality of video game heuristic reviews.