User Interaction Technology Research Articles

Multimodal emotion recognition: A comprehensive review, trends, and challenges

AbstractAutomatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices.This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Cognitive Computing Technologies > Artificial Intelligence

Augmented reality technology adoption: Case of a mobile application in Turkey

With the increasing use of smart devices, augmented reality (AR) technology has become widespread in mobile devices. As with user interaction technologies, there are factors affecting the use of applications in mobile augmented reality (MAR) applications. In this study, the factors affecting the use of mobile augmented reality in Turkey are investigated. Although AR and MAR are generally investigated during the research period, “Augment”, the application, was used in the survey and interview parts of the research study. The interview consists of three different parts in addition to a quantitative experimental study. More than hundred variables were obtained from articles and interviews which 22 of them were selected. The results showed that the two most important factors that influence usage of MAR applications are security and privacy. These two are followed by ease of learning, visual quality of the application 3D model, and ease of use in importance, respectively. It is recommended that designers and application developers consider these five variables when designing or developing a MAR application.

Образовательный и культурно-просветительский компонент саамских этнических интернет-сообществ.

The article is devoted to pressing issues related to the educational, cultural and enlightening functions of the virtual network communities of the Kola Sami. The results of an empirical study are presented, the purpose of which is a quantitative and qualitative characteristic of the scientific, educational, cultural and educational content of the Sami network resources. The study showed that the educational potential of online communities targeting Sami users is significantly inferior to the capabilities of resources created for a wider audience. Cultural and educational content is most diverse in communities created on the individual initiative of cyber activists. Most Sámi virtual communities are not well developed in terms of user interaction technologies. The factors of interacting destructive processes in online and offline spaces that impede the activities of communities are identified.

Systematic Design and Rapid Development of Motion-Based Touchless Games for Enhancing Students’ Thinking Skills

During the last few years, there has been a growing interest in students getting engaged in digital game-making activities so as to enhance their thinking skills. The findings of studies that have examined the impact of such initiatives are quite positive, especially concerning the promotion of 21st century skills; however, many students seem to face difficulties in getting a deeper understanding of the game development life cycle. Additionally, students often have difficulties in meaningfully reusing and applying the concepts from various subjects, mainly mathematics and physics, into their game-making tasks or in understanding advanced programming commands while creating their games. The present study presents an innovative game-making teaching approach that suggests a series of steps for the systematic design and rapid development of motion-based touchless games, i.e., games that are based on natural user interaction technologies, like the Microsoft Kinect camera. Findings from evaluation studies in two (2) secondary schools indicate that this approach can increase student motivation, strengthen their computational thinking, enhance their understanding of geometric principles and improve their social skills.

Fuzzy logic‐based linguistic summaries of time series: a powerful tool for discovering knowledge on time varying processes and systems under imprecision

We present a novel approach to linguistic data summarization for numeric data, i.e., of the numbers to text type. Linguistic summarization is meant as a process of a comprehensive description of big and complex datasets via short statements in natural language. First, we briefly survey main developments in the traditional, well developed, and powerful approach to linguistic data summarization based on tools and techniques of natural language generation (NLG), notably due to Reiter and his collaborators. We indicate that this approach has a serious limitation on the representation and processing of imprecision that is characteristic for natural languages. We show that a fuzzy logic‐based approach to linguistic data summarization can be a simple yet efficient solution in this respect. We present the linguistic summaries represented by protoforms in the form of linguistically quantified propositions dealt with using tools and techniques of fuzzy logic to grasp an inherent imprecision of natural language. Such linguistic data summaries can provide a human user, whose only natural means of articulation and communication is natural language, with a simple yet effective and efficient means for the representation and manipulation of knowledge about processes and systems. We concentrate on the linguistic summarization of dynamic processes and systems, dealing with data represented as time series. We extend the basic, static data‐oriented concept of a linguistic data summary to the case of time series data, present various possible protoforms of linguistic summaries, and an analysis of their properties and ways of generation. We show two our own real applications of the new tools of linguistic summarization of time series, for the summarization of quotations of an investment (mutual) fund, and of Web server logs, to show the power of the tool. We also mention some other applications known from the literature. We conclude with some remarks on the strength of the linguistic summarization for broadly perceived data mining and knowledge discovery and some possible further research directions. WIREs Data Mining Knowl Discov 2016, 6:37–46. doi: 10.1002/widm.1175This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Computational Intelligence Technologies > Machine Learning

Motor Rehabilitation Using Kinect: A Systematic Review.

Interactive systems are being developed with the intention to help in the engagement of patients on various therapies. Amid the recent technological advances, Kinect™ from Microsoft (Redmond, WA) has helped pave the way on how user interaction technology facilitates and complements many clinical applications. In order to examine the actual status of Kinect developments for rehabilitation, this article presents a systematic review of articles that involve interactive, evaluative, and technical advances related to motor rehabilitation. Systematic research was performed in the IEEE Xplore and PubMed databases using the key word combination "Kinect AND rehabilitation" with the following inclusion criteria: (1) English language, (2) page number >4, (3) Kinect system for assistive interaction or clinical evaluation, or (4) Kinect system for improvement or evaluation of the sensor tracking or movement recognition. Quality assessment was performed by QualSyst standards. In total, 109 articles were found in the database research, from which 31 were included in the review: 13 were focused on the development of assistive systems for rehabilitation, 3 in evaluation, 3 in the applicability category, 7 on validation of Kinect anatomic and clinical evaluation, and 5 on improvement techniques. Quality analysis of all included articles is also presented with their respective QualSyst checklist scores. Research and development possibilities and future works with the Kinect for rehabilitation application are extensive. Methodological improvements when performing studies on this area need to be further investigated.

Enhancing the Effectiveness and Efficiency of Control Room Operators – A Roadmap-based Approach for Selecting Interaction Technologies for Defence and Safety-critical Organisations and Industries

Innovations in user interaction technologies play an increasingly important role in the marketing of products and services. This paper presents a systematic and easy-to-update method for the evaluation of innovations. It discusses criteria for selecting innovations for consumer products and an approach for the definition of selection criteria for defence and safety-critical organisations and industries. Those criteria are illustrated in the example of use cases from ARTEMIS project HoliDes addressing adaptation features in the workplaces of control room operators.

TheWeb‐Extended Mind

AbstractThis article explores the notion of theWeb‐extended mind, which is the idea that the technological and informational elements of theWeb can sometimes serve as part of the mechanistic substrate that realizes human mental states and processes. It is argued that while current forms of theWeb may not be particularly suited to the realization ofWeb‐extended minds, new forms of user interaction technology as well as new approaches to information representation do provide promising new opportunities forWeb‐based forms of cognitive extension. In addition, it is suggested that extended cognitive systems often rely on the emergence of social practices and conventions that shape how a technology is used.Web‐extended minds may thus depend on forms of socio‐technical co‐evolution in which social forces and factors play just as important a role as do the processes of technology design and development.

Methodical aspects of text testing in a driving simulator

A test with 30 test persons was conducted in a driving simulator. The test was a concept exploration and comparison of existing user interaction technologies for text message handling with focus on traffic safety and experience (technology familiarity and learning effects). Focus was put on methodical aspects how to measure and how to analyze the data. Results show difficulties with the eye tracking system (calibration etc.) per se, and also include the subsequent raw data preparation. The physical setup in the car where found important for the test completion.

The Mainstreaming on Ambient Intelligence project

The Mainstreaming on Ambient Intelligence (MonAMI) project, funded under the European Commission’s IST e-Inclusion work programme (FP6 framework), set out to demonstrate that accessible, useful services for older, and disabled people can be delivered using mainstreamed systems and platforms. An open platform, based on an Open Services Gateway initiative (OSGi Alliance™, 2011), framework, was developed to deliver a bouquet of Ambient Assisted Living services. The services were developed with a Design for All (European Commission, 2000) approach, in consultation with potential users in the following areas: • Monitoring: home (environmental and appliance control), health. • Time management. • Safety and security: safety at home, visitor validation, activity detection. • Communication and networking. The platform integrated elements from reliable self-organizing networks, wearable devices, user interaction technology, monitoring capability, and service infrastructures and delivered services on mainstream devices such as broadband internet and third-generation mobile telephones. In addition, to facilitate use and user interaction, the consortium designed an innovative user interface (UI), using Universal Control Hub architecture (Zimmerman, 2009). The services were first tested by older and disabled users in laboratory settings in six feasibility and usability centres across Europe (France, Germany, Slovakia, Spain, Sweden, and the UK). Once the platform, services and UI were found to be feasible, usable, and appropriate to user needs, a living-scale field trial was carried out in three communities (Kosice, Slovakia; Stockholm, Sweden; and Zaragoza, Spain), where 70 users, 45 carers, and 14 care staff employed the services in users’ homes over a three-month period. In addition to testing the scalability of the platform, the trial evaluated the services’ potential of meeting the long-term care needs of older and disabled people and the support needs of caregivers. The commercial viability and economic effects on long-term care financing were also evaluated in order to understand the impact of mainstream implementation of the MonAMI concept, as part of the Ambient Assisted Living Open Association (AALOA, 2010) initiative. The MonAMI project ran for 57 months and came to completion in May, 2011.

Considering the user in the wireless world

The near future promises significant advances in communication capabilities, but one of the keys to success is the capability understanding of the people with regards to its value and usage. In considering the role of the user in the wireless world of the future, the Human Perspective Working Group (WG1) of the Wireless World Research Forum has gathered input and developed positions in four important areas: methods, processes, and best practices for user-centered research and design; reference frameworks for modeling user needs within the context of wireless systems; user scenario creation and analysis; and user interaction technologies. This article provides an overview of WG1's work in these areas that are critical to ensuring that the future wireless world meets and exceeds the expectations of people in the coming decades.

An evaluation of question answering challenge (QAC-1) at the NTCIR workshop 3

The Question Answering Challenge (QAC) was carried out as the first evaluation task on question answering of the NTCIR Workshop 3 [Fukumoto2002] [NTCIR]. Question answering in an open domain is a task for obtaining appropriate answers to given domain independent questions written in natural language from a large corpus. The purpose of the QAC was to develop practical QA systems in an open domain focusing on research of user interaction and information extraction. A further objective was to develop an evaluation method for the question answering system and information resources for evaluation. To evaluate QA technologies, there are several technical aspects to consider for the extraction of answer expressions from knowledge sources. Question type is one aspect of the QA system evaluation. In QAC-1, question types are defined as a noun or noun phrase which indicates names of persons, organizations, and various artifacts and facts, such as money, size, date and so on. Moreover, information related to these can also be considered as answer candidates: for example, names of persons, their affiliations, age and status can be an answer; and for names of organizations, their annual profit, year of establishment and so on. Another aspect to consider is how many answer expressions exist in the knowledge sources. In QAC-1, there may be multiple answers or no answers to questions in general. This aspect makes development of a QA system difficult because, QA system has to check all answer candidates very carefully. User interaction technology requires actual interaction between the computer and person. In actual QA between people, there are typically several interactions which take place in order to confirm the intention of the questions and so on. In QAC-1, we gave one follow-up question for the first question. It will be necessary to resolve an ellipsis in the follow-up question which is frequently occurs in Japanese sentences.