Human-computer Interaction Research Research Articles

Theory-driven Visual Design to Support Reflective Dietary Practice via mHealth: A Design Science Approach

Design for reflection in human-computer interaction (HCI) has evolved from focusing on an abstract and outcome-driven design subject towards exposing procedural or structural reflection characteristics. Although HCI research has recognized that an individual's reflection is a long-lasting, multi-layered process that can be supported by meaningful design, researchers have made few efforts to derive insights from a theoretical perspective about appropriate translation into end-user visual means. Therefore, we synthesize theoretical knowledge from reflective practice and learning and argue for a differentiation between time contexts of reflection that design needs to address differently. In an interdisciplinary design-science-research project in the mHealth nutrition promotion context, we developed theory-driven guidelines for “reflection-in-action” and “reflection-on-action”. Our final design guidelines emerged from prior demonstrations and a final utility evaluation with mockup artifacts in a laboratory experiment with 64 users. Our iterative design and the resulting design guidelines offer assistance for addressing reflection design by answering reflective practice’s respective contextual requirements. Based on our user study, we show that reflection in terms of “reflection- in-action” benefits from offering actionable choice criteria in an instant timeframe, while “reflection-on-action” profits from the structured classification of behavior-related criteria from a longer, still memorable timeframe.

Special Issue on Human–Computer Interactions 2.0

Human–computer interaction (HCI) research involves the design and use of computer technology, focusing in particular on the interfaces between people (users) and computers. HCI researchers observe the ways in which humans interact with computers and design technologies that allow them to interact in novel ways. As HCI evolves into HCI 2.0, user experiences and feedback become ever more relevant. This Special Issue, “Human Computer Interactions 2.0”, presents 11 excellent papers about topics related to human–computer interactions. It aims to provide a broad international forum for world researchers, engineers and professionals in human–computer interaction research for the discussion and exchange of various scientific, technical and management discoveries across the world.

An Underdeveloped Metaphor: The Mismatched Designs and Motivations of Digital Picture Interactions

Picture interactions are key to daily and long-term social connections between families and communities, especially through reminiscence. Across the nearly 200-year history of domestic photography, this social reminiscence has been accomplished largely through photo albums. However, in the now common digital setting, albums are pushed aside for the endless film roll metaphor. In this article, we explore this metaphor through the 20-year history of proposed digital picture interactions from Human–Computer Interaction research, and compare this to ongoing interactions with modern picture tools. Through these, we reveal that this prominent design metaphor does not create space for social reminiscence, but does fit the novel use of immediate sharing seen across social networking. Furthermore, the endless and ever-growing nature of digital film rolls are not meaningfully browsable for either intended use. We close by reconnecting to the past works that explore the broad potential of digital pictures.

Designing for Emotion Regulation Interventions: An Agenda for HCI Theory and Research

There is a growing interest in human-computer interaction (HCI) to envision, design, and evaluate technology-enabled interventions that support users’ emotion regulation. This interest stems in part from increased recognition that the ability to regulate emotions is critical to mental health, and that a lack of effective emotion regulation is a transdiagnostic factor for mental illness. However, the potential to combine innovative HCI designs with the theoretical grounding and state-of-the-art interventions from psychology has yet to be fully realised. In this article, we synthesise HCI work on emotion regulation interventions and propose a three-part framework to guide technology designers in making: (i) theory-informed decisions about intervention targets; (ii) strategic decisions regarding the technology-enabled intervention mechanisms to be included in the system; and (iii) practical decisions around previous implementations of the selected intervention components. We show how this framework can both systematise HCI work to date and suggest a research agenda for future work.

Process Mining IPTV Customer Eye Gaze Movement Using Discrete-Time Markov Chains

Human-Computer Interaction (HCI) research has extensively employed eye-tracking technologies in a variety of fields. Meanwhile, the ongoing development of Internet Protocol TV (IPTV) has significantly enriched the TV customer experience, which is of great interest to researchers across academia and industry. A previous study was carried out at the BT Ireland Innovation Centre (BTIIC), where an eye tracker was employed to record user interactions with a Video-on-Demand (VoD) application, the BT Player. This paper is a complementary and subsequent study of the analysis of eye-tracking data in our previously published introductory paper. Here, we propose a method for integrating layout information from the BT Player with mining the process of customer eye movement on the screen, thereby generating HCI and Industry-relevant insights regarding user experience. We incorporate a popular Machine Learning model, a discrete-time Markov Chain (DTMC), into our methodology, as the eye tracker records each gaze movement at a particular frequency, which is a good example of discrete-time sequences. The Markov Model is found suitable for our study, and it helps to reveal characteristics of the gaze movement as well as the user interface (UI) design on the VoD application by interpreting transition matrices, first passage time, proposed ‘most likely trajectory’ and other Markov properties of the model. Additionally, the study has revealed numerous promising areas for future research. And the code involved in this study is open access on GitHub.

Diagnostic advertisements: The phantom disabilities created by social media surveillance

This theoretical essay builds on existing literature to draw out the consequences of dehumanizing and disseminating autism discourses within the field of human-computer interaction (HCI). Focusing mainly on narratives in HCI that frame autistic people as or like machines, we explore how dominant constructions of autism in HCI work to normalize the field’s complicity in violent autism intervention paradigms, despite HCI researchers’ well-meaning intentions. We work towards developing crip-cyborgs as an alternative framework for understanding autistic people (as opposed to computers or robots) and suggest crip technoscience as a framework for research based on this alternative understanding. In doing so, we hope to enroll misguided but well-intentioned researchers in dismantling anti-autistic ableism, both in and beyond HCI.

Not robots; Cyborgs — Furthering anti-ableist research in human-computer interaction

This theoretical essay builds on existing literature to draw out the consequences of dehumanizing and disseminating autism discourses within the field of human-computer interaction (HCI). Focusing mainly on narratives in HCI that frame autistic people as or like machines, we explore how dominant constructions of autism in HCI work to normalize the field’s complicity in violent autism intervention paradigms, despite HCI researchers’ well-meaning intentions. We work towards developing crip-cyborgs as an alternative framework for understanding autistic people (as opposed to computers or robots) and suggest crip technoscience as a framework for research based on this alternative understanding. In doing so, we hope to enroll misguided but well-intentioned researchers in dismantling anti-autistic ableism, both in and beyond HCI.

Seeing the faces is so important—Experiences from online team meetings on commercial virtual reality platforms

During the COVID-19 pandemic, online meetings became common for daily teamwork in the home office. To understand the opportunities and challenges of meeting in virtual reality (VR) compared to videoconferences, we conducted the weekly team meetings of our human-computer interaction research lab on five off-the-shelf online meeting platforms over 4 months. After each of the 12 meetings, we asked the participants (N = 32) to share their experiences, resulting in 200 completed online questionnaires. We evaluated the ratings of the overall meeting experience and conducted an exploratory factor analysis of the quantitative data to compare VR meetings and video calls in terms of meeting involvement and co-presence. In addition, a thematic analysis of the qualitative data revealed genuine insights covering five themes: spatial aspects, meeting atmosphere, expression of emotions, meeting productivity, and user needs. We reflect on our findings gained under authentic working conditions, derive lessons learned for running successful team meetings in VR supporting different kinds of meeting formats, and discuss the team’s long-term platform choice.

Appearance Based Dynamic Hand Gesture Recognition Using 3D Separable Convolutional Neural Network

Appearance-based dynamic Hand Gesture Recognition (HGR) remains a prominent area of research in Human-Computer Interaction (HCI). Numerous environmental and computational constraints limit its real-time deployment. In addition, the performance of a model decreases as the subject’s distance from the camera increases. This study proposes a 3D separable Convolutional Neural Network (CNN), considering the model’s computational complexity and recognition accuracy. The 20BN-Jester dataset was used to train the model for six gesture classes. After achieving the best offline recognition accuracy of 94.39%, the model was deployed in real-time while considering the subject’s attention, the instant of performing a gesture, and the subject’s distance from the camera. Despite being discussed in numerous research articles, the distance factor remains unresolved in real-time deployment, which leads to degraded recognition results. In the proposed approach, the distance calculation substantially improves the classification performance by reducing the impact of the subject’s distance from the camera. Additionally, the capability of feature extraction, degree of relevance, and statistical significance of the proposed model against other state-of-the-art models were validated using t-distributed Stochastic Neighbor Embedding (t-SNE), Mathew’s Correlation Coefficient (MCC), and the McNemar test, respectively. We observed that the proposed model exhibits state-of-the-art outcomes and a comparatively high significance level.

Human-Computer Interaction: Enhancing User Experience in Interactive Systems

Enhancing user experience (UX) in interactive systems requires effective human-computer interaction (HCI). The relationship between people and computers has grown in significance as technology progresses, having an impact on many areas of our life. The main ideas and tactics used in HCI to enhance the user experience in interactive systems are examined in this abstract. Understanding the capabilities and constraints of both humans and computers forms the basis of HCI. HCI researchers and designers may develop interactive systems that complement users' mental models and cognitive processes by researching human behavior, cognition, and psychology. Additionally, taking into account the features of the computer system, such as its responsiveness, processing capacity, and interface design, enables the development of systems that are more user-friendly and effective. As a result, the discipline of human-computer interaction works to improve the user experience in interactive systems. Researchers and designers in the field of human-computer interaction (HCI) can produce interactive systems that are simple, effective, entertaining, and satisfying for users by comprehending human capabilities, applying user-centered design principles, utilizing interactive techniques and interfaces, integrating multimodal interfaces, and embracing emerging technologies. The continued development of HCI will continue to influence and enhance how people use computers, enhancing user experiences and creating new opportunities for interactive systems in the future.

A High-Precision Real-Time System for Pupil Size Measurement using an Infrared Camera and LED Light Stimulation

This research is dedicated to the exploration of sophisticated methodologies and the creation of a high-precision system for pupil size measurement that is capable of displaying real-time results. The comprehensive system encompasses an infrared camera, an infrared LED light, an LED light box designed for pupil stimulation, and robust control software that enables analysis, measurement, and real-time documentation of results. Our study yielded promising results with a high degree of precision in pupil size measurements, thereby opening new possibilities for intricate vision-related research and applications. With its potential impact, the system can offer profound contributions to areas including eye-health diagnosis, human-computer interaction research, and studies focusing on light-induced pupil response.

Advancements in VDL and CPDLC: revolutionizing European aviation communication

The paper is focused on the transformative impact of VDL and CPDLC on aviation safety, efficiency, and reliability. Tracing the historical evolution from radio systems to the adoption of VDL and its integration within Europe's unified airspace initiatives, this paper deals with key theories including VDL Mode 2, ACARS, Aeronautical Telecommunication Network, ADS-B, and CPDLC. Emphasizing interdisciplinary collaboration to tackle communication challenges, the paper spotlights CPDLC's evolution integrating AI, Machine Learning, cybersecurity protocols, and technological integrations like 5G and satellites. Human-Computer Interaction research underscores user-friendly CPDLC interface designs. Methodologies encompass operational analyses, communication protocols, avionics technologies, and usability testing. Aligning with the Single European Sky initiative, the integration of VDL and CPDLC anticipates emerging trends while addressing challenges like interoperability, training complexities, and cybersecurity vulnerabilities through proposed solutions. The paper offers a comprehensive understanding of advancements shaping the modern aviation communication landscape.

EEG-induced Fear-type Emotion Classification Through Wavelet Packet Decomposition, Wavelet Entropy, and SVM

Among the most significant characteristics of human beings is their ability to feel emotions. In recent years, human-machine interface (HM) research has centered on ways to empower the classification of emotions. Mainly, human-computer interaction (HCI) research concentrates on methods that enable computers to reveal the emotional states of humans. In this research, an emotion detection system based on visual IAPPS pictures through EMOTIV EPOC EEG signals was proposed. We employed EEG signals acquired from channels (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) for individuals in a visual induced setting (IAPS fear and neutral aroused pictures). The wavelet packet transform (WPT) combined with the wavelet entropy algorithm was applied to the EEG signals. The entropy values were extracted for every two classes. Finally, these feature matrices were fed into the SVM (Support Vector Machine) type classifier to generate the classification model. Also, we evaluated the proposed algorithm as area under the ROC (Receiver Operating Characteristic) curve, or simply AUC (Area under the curve) was utilized as an alternative single-number measure. Overall classification accuracy was obtained at 91.0%. For classification, the AUC value given for SVM was 0.97. The calculations confirmed that the proposed approaches are successful for the detection of the emotion of fear stimuli via EMOTIV EPOC EEG signals and that the accuracy of the classification is acceptable.

Empathy and exclusion in the design process

Designers are now taught that empathy with users is crucial to technology design. We offer a warning that this dictum and its implementation, despite admirable intentions, can promote exclusion in design: Empathy will not bring the desired benefit to the design process if it is naively construed and understood as a feminine trait, if shortcuts are used to allegedly take the effort out of the empathic process, or if the social situation in which empathy is taking place is not considered. We show that these issues are closely coupled in design practices. Using personas—fictitious descriptions of people used to make users visible in the design process—as an example, we argue that the danger of reifying gendered assumptions might be inherent in those methods and tools in human-computer interaction research that are supposed to enable and strengthen empathy.

HCI Research Experiences During the Pandemic: Lessons Learned for the Road Ahead

Abstract The social interaction restrictions imposed by COVID-19 required academic researchers to rethink the design of their research projects, especially those that involved human interaction. Considering this scenario, we report here the experience of conducting four studies in Human–Computer Interaction research projects that were adapted to be executed remotely: a focus group study, a communicability evaluation study, a speculative design study and a user observation study. Considering how the studies were planned, executed and analysed, we reflect on the adaptations needed, and lessons related to our experiences. From these lessons, we believe that, even after all the pandemic restrictions are lifted, we may continue conducting at least part of our research studies remotely to increase our geographical reach, reduce costs and facilitate data capture.

Interactive skin through a social- sensory speculative lens

ABSTRACT This paper uses a speculative lens to explore the social and sensory trajectories of Interactive Skin, a class of skin-worn epidermal devices that augment the human body in ways that are significant for affective techno-touch. The paper presents and discusses the use of a speculative narrative on Interactive Skin futures produced through an exploratory research-collaboration with a Human–Computer Interaction (HCI) lab, combining data from speculative methods (cultural probe returns and a future-orientated workshop) with an ethnographic sensitivity to writing. The speculative narrative is in the form of a found archive of fictional fragments that are research provocations in their own right. We discuss their potentials, including the ability to foster interdisciplinary dialogue between social and HCI researchers and to agitate the socio-technological space of interactive skin futures, as well as their limitations. The paper concludes that a socially orientated speculative approach can provide useful insights on the interconnection between the senses, society, and technology in the context of emergent affective techno-touch technologies.

Imperceptible, designable, and scalable braided electronic cord

Flexible sensors, friendly interfaces, and intelligent recognition are important in the research of novel human-computer interaction and the development of smart devices. However, major challenges are still encountered in designing user-centered smart devices with natural, convenient, and efficient interfaces. Inspired by the characteristics of textile-based flexible electronic sensors, in this article, we report a braided electronic cord with a low-cost, and automated fabrication to realize imperceptible, designable, and scalable user interfaces. The braided electronic cord is in a miniaturized form, which is suitable for being integrated with various occasions in life. To achieve high-precision interaction, a multi-feature fusion algorithm is designed to recognize gestures of different positions, different contact areas, and different movements performed on a single braided electronic cord. The recognized action results are fed back to varieties of interactive terminals, which show the diversity of cord forms and applications. Our braided electronic cord with the features of user friendliness, excellent durability and rich interaction mode will greatly promote the development of human-machine integration in the future.

Time related changes of affective dimensions and distinct emotions in the interaction with a tablet PC

AbstractExamining peoples’ affect and emotions over time and their effects on peoples’ behavior are ongoing endeavors in human-computer-interaction (HCI) research. This paper reports an experiment in which participants watched either positive or negative film clips on a tablet PC to enter a positive or negative affective state. Successively, they accomplished four basic system interaction tasks like changing fonts of an app on the same device. Results show that, in line with previous studies, peoples’ general valence ratings quickly reverted to neutral when starting the task accomplishment. At the level of distinct positive emotions, participants’ ratings of hope, joy, and serenity decreased after watching negative film clips. Moreover, amusement, love, and serenity decreased during the interaction with the tablet PC. Amongst the negative emotions, only ratings of sadness increased after watching negative film clips and decreased again after the interaction. Also, participants in the positive film group were slower in executing one of the basic tasks than participants in the negative film group. The findings suggest that only few emotions may be causal for peoples’ ratings of general affect. Results also indicate that negative emotions may help people executing standard tasks, in contrast to positive emotions. Implications for HCI design and research are discussed.

Optimal Feedback Control for Modeling Human–Computer Interaction

Optimal feedback control (OFC) is a theory from the motor control literature that explains how humans move their body to achieve a certain goal, e.g., pointing with the finger. OFC is based on the assumption that humans aim at controlling their body optimally, within the constraints imposed by body, environment, and task. In this article, we explain how this theory can be applied to understanding Human-Computer Interaction (HCI) in the case of pointing. We propose that the human body and computer dynamics can be interpreted as a single dynamical system. The system state is controlled by the user via muscle control signals, and estimated from observations. Between-trial variability arises from signal-dependent control noise and observation noise. We compare four different models from optimal control theory and evaluate to what degree these models can replicate movements in the case of mouse pointing. We introduce a procedure to identify parameters that best explain observed user behavior. To support HCI researchers in simulating, analyzing, and optimizing interaction movements, we provide the Python toolboxOFC4HCI. We conclude that OFC presents a powerful framework for HCI to understand and simulate motion of the human body and of the interface on a moment-by-moment basis.

The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT

The topic of algorithmic fairness is of increasing importance to the Human–Computer Interaction research community following accumulating concerns regarding the use and deployment of Artificial Intelligence-based systems. How we conduct research on algorithmic fairness directly influences our inferences and conclusions regarding algorithmic fairness. To better understand the methodological decisions of studies focused on people’s perceptions of algorithmic fairness, we systematic analysed relevant papers from the CHI and FAccT conferences. We identified 200 relevant papers published between 1993 and 2022 and assessed their study design, participant sample, and geographical location of participants and authors. Our results highlight that studies are predominantly cross-sectional, cover a wide range of participant roles, and that both authors and participants are primarily from the United States. Based on these findings, we reflect on the potential pitfalls and shortcomings in how the community studies algorithmic fairness.