Intelligent Interactive Systems Research Articles

Application of digital entertainment experience based on intelligent interactive system in personalized fitness training system

The application of digital media in fitness environments can improve the entertainment of the fitness process, allowing users to watch entertainment videos while engaging in fitness activities. At the same time, we can add relevant game elements to the fitness process, such as playing tennis through gesture movements, thereby increasing user interest in fitness participation. Therefore, this article has developed a personalized fitness training system using optical sensors and intelligent algorithms, and conducted testing experiments on blood oxygen saturation under two sports states: running and cycling. This article analyzes the intelligent optimization algorithm for optical signals. This algorithm has strong adaptability and intelligence level, and can directly learn and adjust the model based on the characteristics of the data. The automation performance of the intelligent algorithm can effectively reduce labor costs and improve efficiency. Finally, this article provides a basic analysis and testing of the personalized fitness training system. Through the obtained data, personalized guidance is provided to users, helping people make fitness more scientific and intelligent, and improving fitness efficiency.

Medical education and artificial intelligence: Question answering for medical questions based on intelligent interaction

SummaryComputer assisted medical diagnosis technology is widely used in the field of medical assistance to assist doctors in making diagnostic decisions. But as the number of patients increases, the diagnostic pressure on doctors gradually increases, and more efficient computer‐aided medical diagnosis technology is needed to improve the accuracy of doctors' diagnosis. Today's computer‐based medically assisted diagnostic technologies suffer from the inability to fully simulate physical attributes and environmental factors, the large computational resources required for high‐precision models, the need for professional training for user operation, and the limited intuition for innovative design. For improving diagnostic efficiency, this study designs a medical Question answering intelligent interaction system in view of artificial intelligence algorithms. The system is constructed with an active interactive intelligent Q&A model consisting of a medical reasoning module and a medical examination recommendation module. Then, it uses local Bayesian network algorithm as the foundation to establish an intelligent strategy optimization network. And it puts forward the answer selection model of medical Question answering in view of hierarchical interaction for natural language processing tasks in the medical context. The performance test results show that when the diagnostic end threshold of the medical reasoning module is 0.8, the shortest diagnostic path is 3.33. When the diagnostic threshold is 0.85, the maximum length of the diagnostic path is 4.66, and the maximum difference between the diagnostic paths is 1.33, which is basically not affected by the diagnostic end threshold. The local Bayesian network algorithm can reduce the impact of noise features and extract more valuable information. The accuracy of the multilevel interactive answer selection model on the Stanford Natural Language Inference dataset without using external resources reached 89.2%. The ablation test results show that the overall accuracy of the model is 89.64%. The visualization results of the attention weight distribution test between interaction layers show that under different levels of interaction, the attention distribution will undergo significant changes.

Serum albumin hydrogels designed by protein Re-association for self-powered intelligent interactive systems

Advancements in protein structural engineering have made it possible to design protein materials that can mediate between humans and intelligent systems, manifesting a combination of characteristics such as elastic recovery, electrical conductivity, and stability. This work describes the design of a hydrogel using bovine serum albumin (BSA), based on protein-protein interactions and sulfhydryl-alkene click reactions. The multiple α-helices of BSA were intertwined together by the multi-armed olefin derivatives, acting like Lego blocks to form a gel network and locking in bound water and ions, thus imparting mechanical recovery characteristics (demonstrating 80 % recovery after the compressive strain and 638 % after tensile strain), electrical conductivity, resistance to freezing (maintaining mechanical properties after being frozen at -20 °C for 48 h), biocompatibility (the cell viability of mouse pre-osteoblast-like cells in the hydrogels was 97.5 %), antimicrobial properties, and biodegradability to the BSA hydrogels. A multi-channel wireless signal acquisition system based on BSA hydrogel devices designed for bioenergy conversion and harvesting was developed for the real-time detection of human motion along with a human-mechanics interaction system for manipulating robotic gestures with simple human finger movements. This restructuring of the BSA structure represents a significant advancement in the systematic exploration of protein structures and the development of protein materials. These applications serve as a secure bridge between humans and energy storage devices.

I Know This Looks Bad, But I Can Explain: Understanding When AI Should Explain Actions In Human-AI Teams

Explanation of artificial intelligence (AI) decision-making has become an important research area in human–computer interaction (HCI) and computer-supported teamwork research. While plenty of research has investigated AI explanations with an intent to improve AI transparency and human trust in AI, how AI explanations function in teaming environments remains unclear. Given that a major benefit of AI giving explanations is to increase human trust understanding how AI explanations impact human trust is crucial to effective human-AI teamwork. An online experiment was conducted with 156 participants to explore this question by examining how a teammate’s explanations impact the perceived trust of the teammate and the effectiveness of the team and how these impacts vary based on whether the teammate is a human or an AI. This study shows that explanations facilitate trust in AI teammates when explaining why AI disobeyed humans’ orders but hindered trust when explaining why an AI lied to humans. In addition, participants’ personal characteristics (e.g., their gender and the individual’s ethical framework) impacted their perceptions of AI teammates both directly and indirectly in different scenarios. Our study contributes to interactive intelligent systems and HCI by shedding light on how an AI teammate’s actions and corresponding explanations are perceived by humans while identifying factors that impact trust and perceived effectiveness. This work provides an initial understanding of AI explanations in human-AI teams, which can be used for future research to build upon in exploring AI explanation implementation in collaborative environments.

Human-AI Collaboration in Healthcare: A Review of User Perspectives and Challenges

Artificial intelligence (AI) has rapidly become a transformative force across various sectors, including healthcare, virtual support, and security. The convergence of AI and Human-Computer Interaction (HCI) has led to the development of interactive intelligent systems, fostering user engagement. This paper presents a comprehensive review of the intersection of AI and HCI, focusing on user perspectives and challenges in the context of healthcare. Keywords: Artificial intelligence, Human-Computer Interaction, Healthcare, User Perspectives, Challenges

Design of Intelligent Interactive Teaching System for College English Based on Streamlined Bilinear Attention Networks

Abstract Intelligent methods such as deep learning provide an innovative optimization path for English teaching in colleges and universities. In this paper, the attention mechanism is based on traditional CNN. Embedded attention and multilayer convolutional feature fusion are used to complete the design of streamlined bilinear attention networks. The streamlined bilinear attention network is used to obtain the characteristics of students’ classroom behavior in the process of English teaching, combine the characteristics of students’ classroom behavior and computer programming language to design and implement the intelligent interactive teaching system of English in colleges and universities, and test and apply the analysis of the intelligent interactive teaching system of English. The results show that in the query performance test report, even though the number of requests issued in the test is 2,000, the error rate is always kept at 0, the throughput is 39.8, and the response time is kept at 95~1144s, which indicates that this system can meet the performance requirements with a moderate number of users. In the application analysis, the total score of the teaching method of English Intelligent Interactive Teaching System (mean=94.62) is higher than that of the traditional English teaching method (mean=89.65), which indicates that the teaching method of English Intelligent Interactive Teaching System is more capable of improving the interaction and communication ability of the students in English teaching. The system constructed in this study can meet the diverse needs of students and contribute to the quality improvement of English teaching in colleges and universities.

Speech recognition enhancement based on wireless network sensors application in interactive intelligent teaching system

The speech function used in the multi-dimensional speech recognition model based on MTL and RNN is the commonly used MFCC function, and in feature extraction, after various filtering and conversion, deleting part of the speech information is a requirement for 3D audio recognition. Use multidimensional information as much as possible. Based on this, this article not only brings the development of convolutional neural network intelligent teaching, but also proposes a new education model. Heuristic education like flipped classroom is still the content of education research in recent years, and it is important to the current education form in China. It is mainly based on the traditional lecture mode or the classroom form. This article focuses on undergraduate education. It is designed in the classroom based on the video data of the undergraduate interactive activities and the interactive system. It has important research significance and practicality. At the same time, the interactive content of the traditional teaching method is used to perform voice recognition, and the recognition effect of the main body of the interactive system is further improved through the voice information. The spatial perspective is used to divide the lens perspective into seminars and experimental forms, and the pixel change value of each frame provides a specific reference for it to determine the reflection of student interaction activities and the evaluation of student seminars. This article introduces the implementation process of interactive content and interactive activities of the teaching system in the classroom, introduces the speech recognition framework of the MARS Chinese teaching system, uses the model principle, and compares the general speech data set test with the actual ARST model audio. The results show that, MAS model performance is more advantageous.

Monitoring Soft Shape Surface Deformation via Optical Images for the Distinction of Contact State

The deformability of elastomer surfaces offers the opportunity to capture multimodal physical information, which opens up possibilities for applications in intelligent interactive systems and flexible electronics. Accurately identifying the surface deformation of elastomers has been a key issue for understanding the contact state in a soft touch. However, effective monitoring of the deformed surface topography during soft‐touch processes is difficult due to the complexity of interfacial deformation. Herein, the optical detection method based on an ideal diffuse reflection model is proposed to monitor the deformation of the elastomer, enabling the acquisition of a spatially continuous deformation of the surface in a soft touch. This optical system with the parallel incident light can distinguish the elastomers’ surface deformation field by constructing the correlation between the reflected light irradiance and the surface deformation angle. This method allows for the perception of the contact area. Through the deep learning method, the recognition rate can reach more than 84.24% when there are slight differences in the contact of objects with different geometric shapes. The design offers valuable insights for contact state detection tasks and facilitates soft‐touch sensing functions through the high precision of optical signal acquisition.

RETRACTED ARTICLE: Optimized design and application research of smart interactive screen for wireless networks based on federated learning

The rapid development of infinite networks and information technology has promoted the wide deployment and rapid growth of intelligent interactive devices. However, at the same time, touch interaction technology also faces many challenges such as lack of precision. This study combines federated learning with LayerGesture technology to optimize and design a touch interaction system with higher interaction accuracy and applies it to practice. The analysis results show that with the increase in the number of iterations of the federated model, the accuracy of the human–computer recognition interaction and the amount of information contained in it increases, and the accuracy curve reaches stability at about 2800 times and is at the optimal interaction adaptation. At this point, the loss function also decreases gradually, while the loss factor tends to 0, which verifies the stability of the optimized model. According to the participants’ interaction experience and experimental results, the optimized LayerGesture technique of the federated learning model has an average correctness rate of 90.4% and the lowest average selection time, while the average selection time of LayerGesture in the interaction area at the edge of the screen is 2510 ms and the average correctness rate is 93.60%, which is better than the Shift technique. In addition, the subjective survey results indicated that more participants favored the optimized LayerGesture technique. In summary, this paper’s joint learning algorithm contributes to the recognition effectiveness and efficiency of intelligent interactive systems.

Retracted: Design and Construction of an Interactive Intelligent Learning System for English Learners in Higher Education Institutions

Retracted: Design and Construction of an Interactive Intelligent Learning System for English Learners in Higher Education Institutions

FindVehicle and VehicleFinder: a NER dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system

Natural language (NL) based vehicle retrieval is a task aiming to retrieve a vehicle that is most consistent with a given NL query from among all candidate vehicles. Because NL query can be easily obtained, such a task has a promising prospect in building an interactive intelligent traffic system (ITS). Current solutions mainly focus on extracting both text and image features and mapping them to the same latent space to compare the similarity. However, existing methods usually use dependency analysis or semantic role-labelling techniques to find keywords related to vehicle attributes. These techniques may require a lot of pre-processing and post-processing work, and also suffer from extracting the wrong keyword when the NL query is complex. To tackle these problems and simplify, we borrow the idea from named entity recognition (NER) and construct FindVehicle, a NER dataset in the traffic domain. It has 42.3k labelled NL descriptions of vehicle tracks, containing information such as the location, orientation, type and colour of the vehicle. FindVehicle also adopts both overlapping entities and fine-grained entities to meet further requirements. To verify its effectiveness, we propose a baseline NL-based vehicle retrieval model called VehicleFinder. Our experiment shows that by using text encoders pre-trained by FindVehicle, VehicleFinder achieves 87.7% precision and 89.4% recall when retrieving a target vehicle by text command on our homemade dataset based on UA-DETRAC [1]. From loading the command into VehicleFinder to identifying whether the target vehicle is consistent with the command, the time cost is 279.35 ms on one ARM v8.2 CPU and 93.72 ms on one RTX A4000 GPU, which is much faster than the Transformer-based system. The dataset is open-source via the link https://github.com/GuanRunwei/FindVehicle, and the implementation can be found via the link https://github.com/GuanRunwei/VehicleFinder-CTIM.

‘I already forgot half of it’ – Interviewing people with dementia for co-designing an intelligent system

ABSTRACT During the pre-design phase of supportive intelligent systems, interviews with future users, such as people with dementia, aim to explore the problems for which technology may present a solution. However, interviewing someone with dementia in this unstructured design phase is reported to be challenging resulting in systems that are not truly addressing the needs of the people with dementia. Existing guidelines on creating engaging interviews are often based on post-hoc reflections of researchers and participants. In this study we performed line-to-line sequential in-depth analyses on transcripts of 10 interviews conducted to co-design an interactive intelligent gaming system. Both interviewers and interviewees were subsequently asked to reflect on their experiences. Results show that information exchange with people with dementia can be supported by carefully constructed questions, well-designed probes, and an appropriate length of the interview. To create a pleasant atmosphere during the interviews, researchers should design the interview as a fulfilling experience, be prepared to respond respectfully to negative responses and be able to adapt to different signs of consent. Improving the engagement with people with dementia during interviews can enhance the design and development of intelligent systems that cater to their unique challenges and requirements.

Modelling Proper and Improper Sitting Posture of Computer Users Using Machine Vision for a Human–Computer Intelligent Interactive System during COVID-19

Human posture recognition is one of the most challenging tasks due to the variation in human appearance, changes in the background and illumination, additional noise in the frame, and diverse characteristics and amount of data generated. Aside from these, generating a high configuration for recognition of human body parts, occlusion, nearly identical parts of the body, variations of colors due to clothing, and other various factors make this task one of the hardest in computer vision. Therefore, these studies require high-computing devices and machines that could handle the computational load of this task. This study used a small-scale convolutional neural network and a smartphone built-in camera to recognize proper and improper sitting posture in a work-from-home setup. Aside from the recognition of body points, this study also utilized points’ distances and angles to help in recognition. Overall, the study was able to develop two objective datasets capturing the left and right side of the participants with the supervision and guidance of licensed physical therapists. The study shows accuracies of 85.18% and 92.07%, and kappas of 0.691 and 0.838, respectively. The system was developed, implemented, and tested in a work-from-home environment.

Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

Among the primary aims of Artificial Intelligence (AI) is the enhancement of User Experience (UX) by providing deep understanding, profound empathy, tailored assistance, useful recommendations, and natural communication with human interactants while they are achieving their goals through computer use. To this end, AI is used in varying techniques to automate sophisticated functions in UX and thereby changing what UX is apprehended by the users. This is achieved through the development of intelligent interactive systems such as virtual assistants, recommender systems, and intelligent tutoring systems. The changes are well received, as technological achievements but create new challenges of trust, explainability and usability to humans, which in turn need to be amended by further advancements of AI in reciprocity. AI can be utilised to enhance the UX of a system while the quality of the UX can influence the effectiveness of AI. The state of the art in AI for UX is constantly evolving, with a growing focus on designing transparent, explainable, and fair AI systems that prioritise user control and autonomy, protect user data privacy and security, and promote diversity and inclusivity in the design process. Staying up to date with the latest advancements and best practices in this field is crucial. This paper conducts a critical analysis of published academic works and research studies related to AI and UX, exploring their interrelationship and the cause-effect cycle between the two. Ultimately, best practices for achieving a successful interrelationship of AI in UX are identified and listed based on established methods or techniques that have been proven to be effective in previous research reviewed.

Werewolf-XL: A Database for Identifying Spontaneous Affect in Large Competitive Group Interactions

Affective computing and natural human-computer interaction, which would be capable of interpreting and responding intelligently to the social cues of interaction in crowds, are more needed than ever as an individuals affective experience is often related to others in group activities. To develop the next-generation intelligent interactive systems, we require numerous human facial expressions with accurate annotations. However, existing databases usually consider nonspontaneous human behavior (posed or induced), individual or dyadic setting, and a single type of emotion annotation. To address this need, we created the Werewolf-XL database, which contains a total of 890 minutes of spontaneous audio-visual recordings of 129 subjects in a group interaction of nine individuals playing a conversational role-playing game called Werewolf. We provide 131,688 individual utterance-level video clips with internal self-assessment of 18 non-prototypical emotional categories and external assessment of pleasure, arousal, and dominance, including 14,632 speakers' samples and the rest of listeners' samples. Besides, the results of the annotation agreement analysis show fair reliability and validity. Role information and outcomes of the game are also recorded. Furthermore, we provided extensive benchmarks of unimodal and multimodal emotional recognition results. The database is made publicly available.

Computer Users Sitting Posture Classification Using Distinct Feature Points and Small Scale Convolutional Neural Network for Humana Computer Intelligent Interactive System During COVID-19

The sudden change in our workplace practices from face-to-face work to work from home setup due to the pandemic has brought positive and negative impacts on our overall health. In literature, the use of deep learning and specialized cameras in the estimation of the human pose is popular even if there is a need for high computational resources and complex models. For this purpose, this study developed an intelligent and interactive system utilizing a human estimation model with the use of distinct keypoint such as thoracic, thoraco lumbar, and lumbar points in the spine. An objective type of a dataset captured in a work from home environment with the knowledge and guidance of Licensed Physical Therapists to assess proper and improper sitting posture was developed. The study developed and implemented a small-scale convolutional network and low-cost smartphone camera to recognize body key points. Once all the feature points' locations were extracted, additional features such as cosine similarity and point distances were calculated. Next, feature selection and optimization were utilized to classify proper and improper sitting postures. As a result, the study developed (2) datasets and (2) models with an accuracy of 85.18 and 92.07% and kappa of 0.691 and 0.838 respectively.

Research Process of Carbon Dots in Memristors

AbstractThe explosive growth of digital communication promotes advanced memory and computing devices in Big Data and artificial intelligence era. In particular, memristors hold great promise for in‐memory computing and artificial synapses, expected to break through restrictions on hardware computational power and storage capacity caused by the von Neumann bottleneck and declining Moore's Law. The memristance controllability is vital to memristors, in which functional layer materials are key. As novel functional layer materials, carbon dots (CDs) are expected to overtake silicon‐based materials due to their satisfactory modulation effects on memristance and distinguished memristive performances. The combination of CDs and matrix materials may carry out multimode sensation memristors for interactive intelligent systems. This review first gives a brief introduction to memristors and their functional layer materials, especially different CDs and their relations with memristance. Then the modulation effects of CDs on memristance are highlighted, mainly including the local electric field enhancement effect, the electron trapping and detrapping effect, and the photosensitization effect, accompanied by applications of CDs‐based memristors (CDMs). Lastly, challenges and perspectives of CDMs are pointed out. This work is rewarding to understand the role of CDs in memristors, to guide relevant research about CDMs, and to promote implementation for intelligent memory and computing.

Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers

In this paper, we show that the evolution of artificial intelligence (AI) and its increased presence within an interactive system pushes designers to rethink the way in which AI and its users interact and to highlight users’ feelings towards AI. For novice designers, it is crucial to acknowledge that both the user and artificial intelligence possess decision-making capabilities. Such a process may involve mediation between humans and artificial intelligence. This process should also consider the mutual learning that can occur between the two entities over time. Therefore, we explain how to adapt the Human-Centered Design (HCD) process to give centrality to AI as the user, further empowering the interactive system, and to adapt the interaction design to the actual capabilities, limitations, and potentialities of AI. This is to encourage designers to explore the interactions between AI and humans and focus on the potential user experience. We achieve such centrality by extracting and formalizing a new category of AI requirements. We have provocatively named this extension: “Intelligence-Centered”. A design workshop with MsC HCI students was carried out as a case study supporting this change of perspective in design.

Attitudes About Artificially Intelligent Interactive Voice Response Systems Using Amazon Alexa in Cardiovascular Clinics: Insights from the VOICE-COVID-19 Study.

The acceptability of artificially intelligent interactive voice response (AI-IVR) systems in cardiovascular research settings is unclear. As a result, we evaluated peoples' attitudes regarding the Amazon Echo Show 8 device when used for electronic data capture in cardiovascular clinics. Participants were recruited following the Voice-Based Screening for SARS-CoV-2 Exposure in Cardiovascular clinics study. Overall, 215 people enrolled and underwent screening (mean age 46.1; 55% females) in the VOICE-COVID study and 58 people consented to participate in a post-screening survey. Following thematic analysis, four key themes affecting AI-IVR acceptability were identified. These were difficulties with communication (44.8%), limitations with available interaction modalities (41.4%), barriers with the development of therapeutic relationships (25.9%), and concerns with universality and accessibility (8.6%). While there are potential concerns with the use of AI-IVR technologies, these systems appeared to be well accepted in cardiovascular clinics. Increased development of these technologies could significantly improve healthcare access and efficiency.

Covering the Broad Spectrum of Intelligent Systems: From Neuromorphic Engineering to Robotics

Covering the Broad Spectrum of Intelligent Systems: From Neuromorphic Engineering to Robotics