Mobile Human-computer Interaction Research Articles

PACMHCI, VI, MHCI, September 2024 Editorial

Welcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, which brings together contributions from the Mobile Human-Computer Interaction (MHCI) community. This issue showcases innovations in research focused on mobile, wearable, and personal devices. Research in MHCI encompasses both technical innovations and social considerations. Mobile technologies accompany us throughout our daily lives: at home, at work, in traffic, and out in the wild. However, while mobile technologies provide the capacity for constant connectivity, they also pose risks of harm and unintended consequences. There has never been a more pressing time to debate and explore the meaning of digital culture, and to understand how mobile technologies can and should be used to connect us, our data, and mobile applications and services in meaningful ways - true to the conference theme of 'Connecting Cultures'.

A Mandarin Tone Recognition Algorithm Based on Random Forest and Feature Fusion †

In human–computer interaction (HCI) systems for Mandarin learning, tone recognition is of great importance. A brand-new tone recognition method based on random forest (RF) and feature fusion is proposed in this study. Firstly, three fusion feature sets (FFSs) were created by using different fusion methods on sound source features linked to Mandarin syllable tone. Following the construction of the CART decision trees using the three FFSs, modeling and optimization of the corresponding RF tone classifiers were performed. The method was tested and evaluated on the Syllable Corpus of Standard Chinese (SCSC), which is a speaker-independent Mandarin monosyllable corpus. Additionally, the effects were also assessed on small sample sets. The results show that the tone recognition algorithm can achieve high tone recognition accuracy and has good generalization capability and classification ability with unbalanced data. This indicates that the proposed approach is highly efficient and robust and is appropriate for mobile HCI learning systems.

Retracted: Psychological Stress Identification and Evaluation Method Based on Mobile Human-Computer Interaction Equipment.

[This retracts the article DOI: 10.1155/2022/6039789.].

BehavePassDB: Public Database for Mobile Behavioral Biometrics and Benchmark Evaluation

Mobile behavioral biometrics have become a popular topic of research, reaching promising results in terms of authentication, exploiting a multimodal combination of touchscreen and background sensor data. However, there is no way of knowing whether state-of-the-art classifiers in the literature can distinguish between the notion of user and device. In this article, we present a new database, BehavePassDB, structured into separate acquisition sessions and tasks to mimic the most common aspects of mobile Human-Computer Interaction (HCI). BehavePassDB is acquired through a dedicated mobile app installed on the subjects devices, also including the case of different users on the same device for evaluation. We propose a standard experimental protocol and benchmark for the research community to perform a fair comparison of novel approaches with the state of the art11https://github.com/BiDAlab/MobileB2C_BehavePassDB/.. We propose and evaluate a system based on Long-Short Term Memory (LSTM) architecture with triplet loss and modality fusion at score level.

PACMHCI V6, MHCI, September 2022 Editorial

Welcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, the first to bring together contributions from the community on Mobile Human-Computer Interaction (MHCI). The pervasive impact of mobile technologies over the last few years of the global pandemic is clear. They have become the de facto means for connecting children to their virtual classrooms, for remote doctor visits, to keep us 'in touch' with loved ones, for allowing us to keep working, and everything else in between. Yet, as researchers and practitioners, we are humbled by the many gaps that have needed our renewed attention. The call for articles for this first issue focused on contributions on the Mobile HCI theme received 113 valid submissions from across the world. After the first round of reviews, 56 contributions were invited to submit revised versions of their articles. After second round reviews 41 articles were accepted, giving an acceptance rate of 36%. We would like to thank our dedicated team of 35 associate editors and the external reviewers they recruited who have ensured the articles in this issue were rigorously reviewed to form a collection of high-quality articles on the theme of Mobile HCI.

Psychological Stress Identification and Evaluation Method Based on Mobile Human-Computer Interaction Equipment.

Since the 1980s, the research of artificial neural networks in the field of artificial intelligence has become more and more common. It accepts nonlinear parallel processing, has strong learning and flexibility, and can be used for influencing factor analysis. The ideal power values and triggers are obtained in the Hopfield network model using genetic algorithm, which best avoids the drawbacks of the Hopfield network model instillation learning method. Through the BP of mobile human-computer interaction equipment, hereditary, genetic algorithms, and Hi-PLS regression method in the artificial neural network, the psychological pressure of college students is identified, evaluated, and predicted from three dimensions such as learning, life, and personal events. This makes it possible to understand the current physical and mental conditions of the students in a timely manner, guide to relieve anxiety and fear, and reach a safe psychological level. The three test results are less than 1%, which has high research significance and value.

Usability evaluation of mobile devices user interface

Designs of mobile human-computer interaction (HCI) target to create interactive and familiar products that are simple and interested to use. The mobile devices such as smartphones or tablets are becoming more and more popular and may to some extent replace the desktop computer and laptop as the most prominent hardware for HCI. This article presents a technical background for this area, the survey of selected user interface (UI) usability principles of today’s mobile devices design and user’s interaction for evaluation purposes. The experiment conducted through a questionnaire having four elements related to usability, i.e. usefulness, ease of use, ease of learning, and satisfaction. Furthermore, the evaluation process performed using quantitative analysis to understand and investigate the participants’ responses based on the Likert scale. The results suggested that most of the participants show a high level of satisfaction towards mobile devices and its interface. The attained level of satisfaction for each element was recorded as 63% for usefulness, 71% for ease of learning, 78% for ease of use and 70% for satisfaction. The collected results can guide the developers and mobile companies to understand the satisfaction level of mobile users. It will also help them to enhance the design and user interface of mobile applications based on usability and flexibility.

An Approach to Develop Mobile Proxemic Applications

Traditional Human-Computer Interaction (HCI) is being overpowered by the widespread diffusion of smart and mobile devices. Currently, smart environments involve daily day activities covered by a huge variety of applications, which demand new HCI approaches. In this context, proxemic interaction, derived from the proxemic theory, becomes an influential approach to implement new kind of Mobile Human-Computer Interaction (MobileHCI) in smart environments. It is based on five proxemic dimensions: Distance, Identity, Location, Movement, and Orientation (DILMO). However, there is a lack of general and flexible tools and utilities focused on supporting the development of mobile proxemic applications. To respond to this need, we have previously proposed a framework for the design and implementation of proxemic applications for smart environments, whose devices interactions are defined in terms of DILMO dimensions. In this work, we extend this framework by integrating a Domain Specif Language (DSL) to support the designing phase. The framework also provides an API, that allows developers to simplify the process of proxemic information sensing (i.e., detection of DILMO dimensions) with mobile phones and wearable sensors. We perform an exhaustive revision of relevant and recent studies and describe in detail all components of our framework.

How Far Are We From Quantifying Visual Attention in Mobile HCI?

With an ever-increasing number of mobile devices competing for our attention, quantifying when, how often, or for how long users visually attend to their devices has emerged as a core challenge in mobile human-computer interaction. Encouraged by recent advances in automatic eye contact detection using machine learning and device-integrated cameras, we provide a fundamental investigation into the feasibility of quantifying visual attention during everyday mobile interactions. We identify core challenges and sources of errors associated with sensing attention on mobile devices in the wild, including the impact of face and eye visibility, the importance of robust head pose estimation, and the need for accurate gaze estimation. Based on this analysis, we propose future research directions and discuss how eye contact detection represents the foundation for exciting new applications towards next-generation pervasive attentive user interfaces.

Experimental Study: The Effects of Mobile Phone Icons Characteristics on Users’ Age Groups

Mobile phone applications interfaces offer great support to various age group users. There is a lot of Mobile Human Computer Interaction (here after HCI) research work going on, which is found in User Experience – UX issues. Until now little research work is done on Mobile phone icons and its effect on learn ability for various age group users. The aim of this study is to investigate the effects of mobile phone icons characteristics on users’ age groups. In current research work we carry out an exploratory study followed by an in-depth experimental based study in order to find out which icons characteristics effect icon usability for various age group users. From experimental results, it is revealed that elderly have more issues while interpreting mobile phone icons, but specifically icon characteristics-semantic distance, familiarity and complexity- are having potential in order to improve icon user experience. We further discuss how design guidelines can support application designers to design application icons that can be adopted by various age group users and be beneficial for them.

The Intention to Use Mobile Student Portal: A Mobile Human Computer Interaction Study in a University Context in Egypt

Mobile devices are increasingly being used as platforms for interactive services. However, factors affecting adoption of mobile services in some specific contexts, such as student portals, are still underresearched. This paper reports thematic analyses of semistructured interviews conducted with 52 undergraduate users of mobile student portal in a developing country. The results uncover design features affecting the use of mobile student portals, such as (1) Content Sharing, comprising an Integrative Design to share posts from the mobile portal to social media, (2) Personalized Content and Notification, for personalized notification based on student’s academic status, (3) Location-Aware notification, providing location-based updates, (4) User Control, allowing to choose content formats and details level, and (5) Context Switching and Interrupted Behavior, leading to the need for displaying content in small, meaningful chunks to help pick up after interruptions of messages and calls. The findings suggest design recommendations for mobile student portal, in specific.

IMU-to-Segment Assignment and Orientation Alignment for the Lower Body Using Deep Learning.

Human body motion analysis based on wearable inertial measurement units (IMUs) receives a lot of attention from both the research community and the and industrial community. This is due to the significant role in, for instance, mobile health systems, sports and human computer interaction. In sensor based activity recognition, one of the major issues for obtaining reliable results is the sensor placement/assignment on the body. For inertial motion capture (joint kinematics estimation) and analysis, the IMU-to-segment (I2S) assignment and alignment are central issues to obtain biomechanical joint angles. Existing approaches for I2S assignment usually rely on hand crafted features and shallow classification approaches (e.g., support vector machines), with no agreement regarding the most suitable features for the assignment task. Moreover, estimating the complete orientation alignment of an IMU relative to the segment it is attached to using a machine learning approach has not been shown in literature so far. This is likely due to the high amount of training data that have to be recorded to suitably represent possible IMU alignment variations. In this work, we propose online approaches for solving the assignment and alignment tasks for an arbitrary amount of IMUs with respect to a biomechanical lower body model using a deep learning architecture and windows of 128 gyroscope and accelerometer data samples. For this, we combine convolutional neural networks (CNNs) for local filter learning with long-short-term memory (LSTM) recurrent networks as well as generalized recurrent units (GRUs) for learning time dynamic features. The assignment task is casted as a classification problem, while the alignment task is casted as a regression problem. In this framework, we demonstrate the feasibility of augmenting a limited amount of real IMU training data with simulated alignment variations and IMU data for improving the recognition/estimation accuracies. With the proposed approaches and final models we achieved 98.57% average accuracy over all segments for the I2S assignment task (100% when excluding left/right switches) and an average median angle error over all segments and axes of ° for the I2S alignment task.

Reliability and Validity of Low Temporal Resolution Eye Tracking Systems in Cognitive Performance Tasks

Eye tracking experiments are an important contribution to human computer interaction (HCI) research. Eye movements indicate attention, information processing, and cognitive state. Oculomotor activity is usually captured with high temporal resolution eye tracking systems, which are expensive and not affordable for everyone. Moreover, these systems require specific hard- and software. However, affordable and practical systems are needed especially for applied research concerning mobile HCI in everyday life. This study examined the reliability/validity of low temporal resolution devices by comparing data of a table-mounted system with an electrooculogram. Gaze patterns of twenty participants were recorded while performing a visual reaction and a surveillance task. Statistical analyses showed high consistency between both measurement systems for recorded gaze parameters. These results indicate that data from low temporal resolution eye trackers are sufficient to derive performance related oculomotor parameters and that such solutions present a viable alternative for applied HCI research.

Audio Technology and Mobile Human Computer Interaction

Audio-based mobile technology is opening up a range of new interactive possibilities. This paper brings some of those possibilities to light by offering a range of perspectives based in this area. It is not only the technical systems that are developing, but novel approaches to the design and understanding of audio-based mobile systems are evolving to offer new perspectives on interaction and design and support such systems to be applied in areas, such as the humanities.

Adding Expressiveness to Smartwatch Notifications Through Ambient Illumination

The ongoing miniaturization of technology provides the possibility to create more and more powerful devices in smaller form factors. One characteristic of this development is smart wearable devices, such as smartwatches, which open up new possibilities for mobile human-computer interaction. While recent research has revealed that these devices are mainly used to display notifications, the very small screen size can be a hindrance. Consequently, explicit user interaction is, for example, required to browse through notifications to get an overview of them. The authors present an alternative by providing an aggregation and filtering approach to better handle notifications. Furthermore, they investigated several display concepts based on a self-built smartwatch prototype equipped with twelve full-color LEDs to present notifications through ambient illumination. Derived from a user study with twelve participants, the work concludes with guidelines that could be employed when designing notification systems.

A Real-time Smile Elegance Detection System: A Feature-level Fusion and SVM Based Approach

Smile detection in the unconstrained real-world scenario has attracted much attention due to its importance for mobile applications and human computer interaction. However, in many applications, how to determine the attractiveness of a smile face image (i.e., smile elegance) is an interesting task. In this paper, we present a real-time smile elegance detection system based on feature-level fusion and support vector machine (SVM). Specifically, three types of features, including local intensity histogram (LIH), central symmetry local binary pattern (CS-LBP) and Gabor wavelet, are firstly employed to characterize the global and local relationships of the face image. Then, a feature-level fusion method is leveraged to effectively combine these features. Finally, a specific SVM classifier is learnt to classify the smile face image into elegance or inelegance. Since there is no available public smile elegance database, we establish one for the first time. Experimental results on our collected smile elegance database demonstrate the effectiveness and efficiency of the proposed smile elegance detection system.

A synthetic research on the multimedia data encryption based mobile computing security enhancement model and multi-channel mobile human computer interaction framework

With the development of computer network, people could obtain information through the network with stronger impulsion, the dependence of the requirements also becomes higher, this is not only reflected in the increase of information, but to obtain and submit more reflected in real-time and easy to access to information on the pressing needs of the. Therefore, people devoted all aspects from the terminal, network and software platforms to make unremitting efforts. Under this basis, we conduct synthetic research on the data encryption based mobile computing security enhancement model and multi-channel mobile human computer interaction framework in this paper. We firstly introduce the discrete-time Hopfield neural network based data encryption algorithm beyond the analysis on the information flow security and type based model, data security review and attack model under mobile computing environment. We improve learning algorithm of the MD method to avoid the sample mobile and cross interference problems of Hebb rule. Later, we integrate the multi-channel concept to propose the new multi-channel man–machine interaction. In our framework, the 3D interactive technology, speech recognition and synthesis technology, natural language understanding and processing technology, eye tracking technology, posture, input, tactile, force display basic technology are taken into consideration for the synthetic analysis. The result from the experimental simulation proves that our methodology obtains better effectiveness and feasibility from both the angels of data security and interface experience.

MUUX-E, a framework of criteria for evaluating the usability, user experience and educational features of m-learning environments

Higher education students use mobile phones, equipped for Internet access. Mobile technologies can offer effective, satisfying and accessible m-learning experiences. A contribution has been made to knowledge on evaluating m-learning environments and to mobile human-computer interaction (MHCI), with the innovative synthesis of the MUUX-E Framework, which fills a gap in the domain of m-learning. MUUX-E is a single comprehensive, multi-faceted instrument for evaluating m-learning environments, emphasising usability and user experience in mobile educational contexts. It was developed by extensive literature studies on each aspect, and has five categories, 31 criteria and numerous sub-criteria. Using a design-based research paradigm, MUUX-E was applied iteratively to evaluate and enhance successive versions of m-LR, a mobile application created for a Software Engineering module. Participants were students and expert evaluators. MUUX-E served well to identify problems and strengths. The students were more positive than the experts regarding the benefits of m-LR, yet insightfully reported more system problems.

An Evaluation Framework for Context of Use in Mobility

Context information is commonly linked to various physical and user activities embedded in users' everyday lives. Research has mainly focused on sensing and inferring context information, yet the relevancy among these contexts is rarely investigated. In this paper, the authors propose an analytical framework for modeling and evaluating collected context data and the nature of relevancy among these data towards defining context awareness. They validate the proposed framework on a case study using a dataset that incorporates users' activities in various situations and surrounding environment scenarios. The framework provides preliminary guidelines for determining representative measures of user and physical context in the design and evaluation policies of context awareness in mobile human computer interaction.

Rear‐facing mobile camera‐based finger motion command recognition system

A novel rear-facing mobile camera-based finger motion command recognition system is proposed and implemented that can support mobile human–computer interaction (HCI) by using only the index finger of the hand holding the mobile device. This is a hybrid approach that includes (i) maximum morphological gradient combination and an AdaBoost-based robust fingertip detector under dynamic lighting conditions, against complicated backgrounds and (ii) one-handed mobile HCI using finger motion and fingertip pointer tracking.