Interactive System Research Articles

Evolution of Industrial Robots from the Perspective of the Metaverse: Integration of Virtual and Physical Realities and Human–Robot Collaboration

During the transition from Industry 4.0 to Industry 5.0, industrial robotics technology faces the need for intelligent and highly integrated development. Metaverse technology creates immersive and interactive virtual environments, allowing technicians to perform simulations and experiments in the virtual world, and overcoming the limitations of traditional industrial operations. This paper explores the application and evolution of metaverse technology in the field of industrial robotics, focusing on the realization of virtual–real integration and human–machine collaboration. It proposes a design framework for a virtual–real interaction system based on the ROS and WEB technologies, supporting robot connectivity, posture display, coordinate axis conversion, and cross-platform multi-robot loading. This paper emphasizes the study of two key technologies for the system: virtual–real model communication and virtual–real model transformation. A general communication mechanism is designed and implemented based on the ROS, using the ROS topic subscription to achieve connection and real-time data communication between physical robots and virtual models, and utilizing URDF model transformation technology for model invocation and display. Compared with traditional simulation software, i.e., KUKA Sim PRO (version 1.1) and RobotStudio (version 6.08), the system improves model loading by 45.58% and 24.72%, and the drive response by 41.50% and 28.75%. This system not only supports virtual simulation and training but also enables the operation of physical industrial robots, provides persistent data storage, and supports action reproduction and offline data analysis and decision making.

Sensor based interactive digital entertainment and gamified training to alleviate basketball player fatigue

In recent years, due to the increasing number of basketball games and training activities, basketball players often face the problem of fatigue and injury. This study aims to develop an interactive digital entertainment and gamification training method to provide basketball players with more efficient training methods and enhance their training fun. Sensors can be installed in various key parts of the athlete’s body, through the use of high-precision sensor equipment, real-time acquisition of athlete action data, the use of machine learning and data mining technology, real-time analysis and modeling of sensor data to extract key information and movement patterns. Using gamified training theory to design interactive digital entertainment system, develop a series of training scenarios and gamified tasks for different technical elements, combine sensor data and motion analysis results, and provide athletes with personalized training plans and feedback mechanisms to help them improve their technical level. Through an interactive digital entertainment system, basketball players can train and compete in a virtual environment. They can complete various training tasks by interacting with the system, which will give real-time evaluation and guidance based on the athlete’s performance to help them correct mistakes, improve technique and improve training results.

Depth perception optimization of mixed reality simulation systems based on multiple‐cue fusion

AbstractMixed reality technology can be applied to simulation training, improve surgical performance, enhance 3D game experience, and so on, attracting extensive attention of researchers. The perception of the user about head‐mounted display MR using HoloLens is critical, especially for precision applications such as virtual hoisting. Designing and adding appropriate depth cues in MR scenes is an effective way to improve users' depth perception. In this study, taking virtual hoisting training system as an example, the depth perception strategy of multi‐cue fusion is proposed to improve the perception effect. Based on the mechanism of human depth perception, five kinds of depth cues are designed. The depth perception effect of adding single cue is studied by perceptual matching experiment. Based on the principle of fuzzy clustering, a multiple‐cue comprehensive depth optimization strategy on viewing distance scale is proposed. Finally, the perceptual matching results demonstrate the effectiveness of the multi‐cue fusion strategy, and the average error is reduced by 20.68% compared with the single‐cue strategy, which can significantly improve the spatial depth perception. This research can provide a reference for improving users' depth perception in interactive MR simulation systems.

Designing for financial inclusion in developing countries: Digital financial service for low-income women in Ghana

Digital financial inclusion initiatives in developing countries have gained salience because of their potential to improve the socioeconomic condition of marginalized groups such as low-income women. However, persistent challenges remain in overcoming the digital divide in developing countries and enhancing access and participation of women in digital financial services. Despite the growing scholarly attention, little is known about how digital technologies might be designed to enable financial inclusion of women in developing countries. Using the technology affordances approach, we extend previous theorizing on inclusive information system, and introduce a relational approach to designing for inclusion. Specifically, we conduct a case study of a digital finance initiative in Ghana involving the design of an interactive voice response system (IVR) for low-income women where systemic barriers to technology adoption and use are pervasive. We show the significance of user feedback, environmental factors, and affordances for more inclusive information system design. We contribute a theoretically grounded framework that takes holistic account of the sociotechnical context of IS design for inclusion.

Interactive Path Editing and Simulation System for Motion Planning and Control of a Collaborative Robot

Robots in hazardous environments demand precise and advanced motion control, making extensive simulations crucial for verifying the safety of motion planning. This paper presents a simulation system that enables interactive path editing, allowing for motion planning in a simulated collaborative robot environment and its real-world application. The system includes a simulation host, a control board, and a robot. Unity 3D on a Windows platform provides the simulation environment, while a virtual Linux environment runs ROS2 for execution. Unity sends edited motion paths to ROS2 using the Unity ROS TCP Connector package. The ROS2 MoveIt framework generates trajectories, which are synchronized back to Unity for simulation and real-world validation. To control the six-axis Indy7 collaborative robot, we used the MIO5272 embedded board as an EtherCAT master. Verified trajectories are sent to the target board, synchronizing the robot with the simulation in position and speed. Data are relayed from the host to the MIO5272 using ROS2 and the Data Distribution Service (DDS) to control the robot via EtherCAT communication. The system enables direct simulation and control of various trajectories for robots in hazardous environments. It represents a major advancement by providing safe and optimized trajectories through efficient motion planning and repeated simulations, offering a clear improvement over traditional time-consuming and error-prone teach pendant methods.

AI-Based Affective Music Generation Systems: A Review of Methods and Challenges

Music is a powerful medium for altering the emotional state of the listener. In recent years, with significant advancements in computing capabilities, artificial intelligence-based (AI-based) approaches have become popular for creating affective music generation (AMG) systems. Entertainment, healthcare, and sensor-integrated interactive system design are a few of the areas in which AI-based affective music generation (AI-AMG) systems may have a significant impact. Given the surge of interest in this topic, this article aims to provide a comprehensive review of controllable AI-AMG systems. The main building blocks of an AI-AMG system are discussed and existing systems are formally categorized based on the core algorithm used for music generation. In addition, this article discusses the main musical features employed to compose affective music, along with the respective AI-based approaches used for tailoring them. Lastly, the main challenges and open questions in this field, as well as their potential solutions, are presented to guide future research. We hope that this review will be useful for readers seeking to understand the state-of-the-art in AI-AMG systems and gain an overview of the methods used for developing them, thereby helping them explore this field in the future.

Multi-factor Authentication Framework for Secured Data Access in Cloud Computing

Aims and Background: Accessing data services on the cloud raises serious concerns about security and privacy. To verify a client's stated identity, the existing authentication approach uses just one authentication factor. However, traditional techniques of one-factor authentication are inefficient since they demand an excessive amount of user information and expose their private data to eavesdropping. Methods: Cloud-based data storage is susceptible to both external and internal threats. Information security relies heavily on authentication, which is a method for verifying user identities before granting them access to protected data. When it comes to protecting data in a cloud computing environment from the most cutting-edge methods of attack, traditional password authentication simply isn't enough. For the purpose of protecting user identities in the cloud, this study presents interactive multi-layer authentication systems. Various access control policies are used when validating users in the cloud. Registration, permissions, and authentication factors for users are all parts of the cloud app's security process. Results: An intrusion detection system is built into the security mechanism to help identify people who could be malevolent. Various techniques are available for user identity verification, including intrusion detection, access control, and multi-factor authentication. The last step is to encrypt the data so that no one else can read it. Conclusion: To guarantee the effectiveness and accuracy of the suggested frameworks and methodology, they are experimentally evaluated. There were few false positives and a high detection rate.

Development of Standalone Extended-Reality-Supported Interactive Industrial Robot Programming System

Extended reality (XR) is one of the most important technologies in developing a new generation of human–machine interfaces (HMIs). In this study, the design and implementation of a standalone interactive XR-supported industrial robot programming system using the Unity game engine is presented. The presented research aims to achieve a cross-platform solution that enables novel tools for robot programming, trajectory validation, and robot programming debugging within an extended reality environment. From a robotics perspective, key design tasks include modeling in the Unity environment based on robot CAD models and control design, which include inverse kinematics solution, trajectory planner development, and motion controller set-up. Furthermore, the integration of real-time vision, touchscreen interaction, and AR/VR headset interaction are involved within the overall system development. A comprehensive approach to integrating Unity with established industrial robot modeling conventions and control strategies is presented. The proposed modeling, control, and programming concepts, procedures, and algorithms are verified using a 6DoF robot with revolute joints. The benefits and challenges of using a standalone XR-supported interactive industrial robot programming system compared to integrated Unity–robotics development frameworks are discussed.

Model-Based Adaptive Control of Bioreactors—A Brief Review

This article summarizes the authors’ experiences in the development and application of the General Dynamical Model Approach related to adaptive linearizing control of biotechnological processes. Special attention has been given to some original, innovative solutions in model-based process control theory: new formalization of biotechnological process kinetics, derivation and tuning of the general software sensor of the full kinetics of biotechnological processes, and a general algorithm for fully adaptive linearizing control with software sensors. These theoretical solutions are the basis of three control strategies—fully adaptive control of the main substrate, partially adaptive control of intermediate metabolite, and recognition and stabilization of the desired physiological state based on the proposed theoretical solutions. Each strategy is illustrated in different case studies. The advantages and limitations of each of them are identified and discussed. The derived algorithms for monitoring and controlling the considered biotechnological processes are realized and included in a software platform named Interactive System for Education in Modelling and Control of Bioprocesses (InSEMCoBio). The InSEMCoBio modules and their main functions are discussed. The effectiveness of the proposed control strategies (achieving maximum productivity) has been proven through a series of simulation investigations of the considered case studies.

Effectiveness of smoking cessation interventions among adults: an overview of systematic reviews

BackgroundThis overview of reviews aims to identify evidence on the benefits (i.e. tobacco use abstinence and reduction in smoking frequency) and harms (i.e. possible adverse events/outcomes) of smoking cessation interventions among adults aged 18 years and older.MethodsWe searched Medline, Embase, PsycINFO, Cochrane Database of Systematic Reviews, Database of s of Reviews of Effects, the CADTH Health Technology Assessment Database and several other websites for grey literature. Searches were conducted on November 12, 2018, updated on September 24, 2020, with publication years 2008 to 2020. Two reviewers independently performed title-abstract and full-text screening considering pre-determined inclusion criteria. Data extraction and quality assessments were initially completed by two reviewers independently (i.e. 73% of included studies (n = 22)) using A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR 2), and the remainder done by one reviewer and verified by another due to resources and feasibility. The application of Grading of Recommendations Assessment, Development and Evaluation (GRADE) was performed by one independent reviewer and verified by another.ResultsA total of 22 Cochrane systematic reviews evaluating the impact of smoking cessation interventions on outcomes such as tobacco use abstinence, reduction in smoking frequency, quality of life and possible adverse events were included. Pharmaceutical (i.e. varenicline, cytisine, nicotine replacement therapy (NRT), bupropion) and behavioural interventions (i.e. physician advice, non-tailored print-based self-help materials, stage-based individual counselling, etc.) showed to have increased smoking cessation; whereas, data for mobile phone-based interventions including text messaging, hypnotherapy, acupuncture, continuous auricular stimulation, laser therapy, electrostimulation, acupressure, St John’s wort, S-adenosyl-L-methionine (SAMe), interactive voice response systems and other combination treatments were unclear. Considering harms related to smoking cessation interventions, small/mild harms (i.e. increased palpitations, chest pain, nausea, insomnia, headache) were observed following NRT, varenicline and cytisine use. There were no data on harms related to behavioural therapies (i.e. individual or group counselling self-help materials, internet interventions), combination therapies or other therapies (i.e. laser therapy, electrostimulation, acupressure, St John’s wort, SAMe).ConclusionResults suggest that pharmacological and behavioural interventions may help the general smoking population quit smoking with observed small/mild harms following NRT or varenicline. Consequently, evidence regarding ideal intervention strategies and the long-term impact of these interventions for preventing smoking was unclear.Systematic review registrationPROSPERO CRD42018099691

Designing preschool children's educational games for enlightenment through decision analysis methods

As an innovative form of digital online education, educational games have potentially significant impacts on preschool children's enlightenment education. Identifying key factors for effective design is crucial in game development. This study identifies key factors for the design of educational games aimed at preschool children's enlightenment through decision analysis methods. The specific research steps include using SET analysis method to identify gaps in development opportunities, employing the Analytic Hierarchy Process to construct an interactive system design model for enlightenment educational games, calculating the weight value of each factor, and pinpointing the key factors for design practice. Based on this methodology, an enlightenment educational game has been developed, and Fuzzy Comprehensive Evaluation method has been utilized to assess the practical implementation. The approach outlined in this study enables designers to swiftly recognize the critical elements in designing educational games for preschool children's enlightenment, facilitating a responsive design process, achieving effective game designs, and ultimately enhancing preschool children's learning interest and outcomes.

Time-varying characteristics analysis of bridge under moving vehicle using a modified time-frequency method with limited sensors

Investigating the bridge dynamic characteristics under operational traffic load is critical for bridge health monitoring. As vehicles traverse the bridge, the bridge frequencies present time-varying characteristics owing to the effect of vehicle-bridge dynamic interaction. The recently introduced variational nonlinear chirp mode decomposition (VNCMD) demonstrates significant benefits in identifying structural instantaneous frequencies (IFs) from the measured responses. However, its practical application is limited by the requirement for artificially setting priori parameters (i.e., upper bound of noise level and number of modal components). In this study, a modified VNCMD (MVNCMD) is proposed to overcome this limitation and is employed to identify the IFs of the bridge under moving vehicle using single sensor data. First, by modifying the optimization function, the proposed MVNCMD adopts a novel algorithm framework that eliminates the necessity for presetting the upper bound of noise level. An autoregressive spectrum-based method is then introduced to preset the number of modal components. The proposed MVNCMD is evaluated using a synthetic non-stationary signal, demonstrating its effectiveness in identifying IFs and overcoming the limitations of the original VNCMD. Furthermore, the numerical simulation involving different parameter analysis and laboratory experiments of the coupled vehicle-bridge system are conducted to validate the effectiveness of the proposed method. It is to be shown that the proposed MVNCMD method is capable of identifying the IFs of the bridge under moving vehicle using single sensor data and outperforms other time-frequency methods in terms of identification accuracy, which provides a robust tool for analyzing the time-varying characteristics of the vehicle-bridge interaction system.

Policy Networks in Ecotourism Governance: Evidence from Local Perspective

This study examined the governance of ecotourism at Dalegan Beach, with a focus on policy networks. The goal is to provide insights concerning actor, role, functions, characteristic, power relations and strategies, as well as factors driving and inhibiting the formation of network. The qualitative method was used to collect data through observations, in-depth interviews with informants, and document studies. The findings showed that the participation and roles of stakeholders, ranging from provincial governments to local communities, had a significant impact on beach tourism management. Despite the establishment of policy networks, challenges such as membership instability and coordination limitations among stakeholders linger. The concept of policy networks as a dynamic interaction system underscores the evolving relationships among various actors in the policy environment. These interactions are influenced by factors such as interests, preferences, and power dynamics among actors, which are used to influence decision-making processes and policy outcomes. The strategies employed by each actor can affect network dynamics and the policy process outcomes. Therefore, changes in the structure or power relations among actors in policy networks can impact the shape and direction of the policies.

A cross-sectional survey of first-aid kit equipment in a family in Sichuan, China

PurposeTo examine residents’ first-aid kit preparation and its influencing factors.DesignCross-sectional survey.MethodsA questionnaire survey was conducted among 449 permanent residents in Sichuan Province using convenience sampling. We examined participants’ demographic characteristics, self-efficacy, health literacy, and personality.FindingsOf the participants, 111 (24.7%) stocked a home first-aid kit. The most frequent supplies were disinfection supplies (91.9%), common medicines (86.5%), and dressing supplies (76.6%). Family per capita monthly income, medical expenses payment method, chronic diseases, general self-efficacy, and health literacy were influencing factors of family first-aid kit preparedness.ConclusionA multilevel and interactive emergency literacy education system should be established to improve residents’ abilities to prevent emergencies.

Study on the Effect of Burial Depth on Selection of Optimal Intensity Measures for Advanced Fragility Analysis of Horseshoe-Shaped Tunnels in Soft Soil

Seismic intensity measures (IMs) can directly affect the seismic risk assessment and the response characteristics of underground structures, especially when considering the key variable of burial depth. This means that the optimal seismic IMs must be selected to match the underground structure under different buried depth conditions. In the field of seismic engineering design, peak ground acceleration (PGA) is widely recognized as the optimal IM, especially in the seismic design code for aboveground structures. However, for the seismic evaluation of underground structures, the applicability and effectiveness still face certain doubts and discussions. In addition, the adverse effects of earthquakes on tunnels in soft soil are particularly prominent. This study aims to determine the optimal IMs applicable to different burial depths for horseshoe-shaped tunnels in soft soil using a nonlinear dynamic time history analysis method, and based on this, establish the seismic fragility curves that can accurately predict the probability of tunnel damage. The nonlinear finite element analysis model for the soil–tunnel interaction system was established. The effects of different burial depths on damage to horseshoe-shaped tunnels in soft soil were systematically studied. By adopting the incremental dynamic analysis (IDA) method and assessing the correlation, efficiency, practicality, and proficiency of the potential IMs, the optimal IMs were determined. The analysis indicates that PGA emerges as the optimal IM for shallow tunnels, whereas peak ground velocity (PGV) stands as the optimal IM for medium-depth tunnels. Furthermore, for deep tunnels, velocity spectral intensity (VSI) emerges as the optimal IM. Finally, the seismic fragility curves for horseshoe-shaped tunnels in soft soil were built. The proposed fragility curves can provide a quantitative tool for evaluating seismic disaster risk, and are of great significance for improving the overall seismic resistance and disaster resilience of society.

The Effect of Electroencephalogram Feedback in Virtual Reality Interactive System on Creativity Performance, Attention Value, and Cognitive Load

This study aimed to systematically investigate the impacts of virtual reality (VR) technology integrated with electroencephalogram (EEG) vibrotactile feedback on undergraduate students’ creativity performance, attention value, and cognitive load. 4P’s creativity model was adopted as the macro strategy, with cognitive load theory and constructivist theory serving as a micro strategy. A 2 × 2 quasi-experimental factor design was utilized. A total of 149 undergraduate students with varying spatial abilities were randomly assigned to an experimental group (using a VR interactive system with EEG vibrotactile feedback) and a control group (without feedback). Covariance analysis (ANCOVA) and analysis of variance (ANOVA) were employed to identify significant differences among the systems and examine interaction effects among variables. The quantitative results provided empirical evidence that the experimental group exhibited significantly higher levels of creativity performance and attention value than the control group. Furthermore, undergraduate students with higher spatial ability demonstrated lower cognitive load, superior creativity performance, and attention value compared to those with lower spatial ability. The study also revealed that the system positively impacts learning in university creativity education and holds promising potential for improving traditional VR interactive training systems. These findings contribute valuable insights for future research on the application of VR interaction technology and EEG technology in cognitive training for creativity.

Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4

Large language models (LLMs) are seen to have tremendous potential in advancing medical diagnosis recently, particularly in dermatological diagnosis, which is a very important task as skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases. Here we present SkinGPT-4, which is an interactive dermatology diagnostic system based on multimodal large language models. We have aligned a pre-trained vision transformer with an LLM named Llama-2-13b-chat by collecting an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors’ notes, and designing a two-step training strategy. We have quantitatively evaluated SkinGPT-4 on 150 real-life cases with board-certified dermatologists. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identify the characteristics and categories of the skin conditions, perform in-depth analysis, and provide interactive treatment recommendations.

Interaction Design in Distributed Software Development: What we know and what we don’t know

Human-Computer Interaction (HCI) is an area focused on human interaction with/through computational systems, among which tangent studies on interaction design stand out. In HCI, interaction design has been consolidated as a practice with the potential to support interactive systems projects. On the other hand, more and more organizations are developing software with geographically distributed teams. However, interaction design in distributed software development (DSD) has yet to be explored, mapped, or structured in the scientific literature. Although there are reports in the scientific literature about interaction design in DSD and some proposed solutions, how interaction design occurs in DSD is still being determined. In this paper, we present a summary of the results of a research that aimed to investigate how interaction design has been practiced in DSD, seeking to understand the practices, challenges, and limitations, as well as to promote and advance the state of knowledge in interaction design in the context of DSD. The research methodology was based on Charles S. Peirce's semiotic methodeutics, bibliographic research method, and mixed methods research to investigate the current state of knowledge and practice on interaction design in DSD. We hope that the results pointed out by this research contribute to the body of knowledge about interaction design at the research frontier between HCI and DSD by (i) providing an overview of research efforts on interaction design in DSD, (ii) providing an overview of the practice of interaction design in DSD, (iii) identify research gaps and discuss future research directions, and (iv) conceiving a set of recommendations for interaction design in DSD.

Analytical solution to 2D dynamic soil-tunnel-building interaction under seismic SH-wave excitation considering the influence of the stiffness of tunnel and building's foundation

This investigation examines the structure-soil-structure interaction (SSSI) problem between a tunnel and nearby aboveground building, considering the impact of the stiffness of tunnel and building's foundation. A two-dimensional (2D) model is utilized, which includes a flexible circular tunnel and a three-story building anchored by a flexible semi-circular foundation embedded in a homogeneous half-space. The response of the system to incident plane SH-wave excitation is solved analytically using the wave function expansion method. The analysis focuses on how the system response is affected by the stiffness of the tunnel and the building's foundation. Tunnel stiffness plays a leading role in tunnel response analysis. The model with a rigid tunnel and rigid foundation can offer a reasonable system response overall, but it fails to accurately represent tunnel deformation and stress distribution. Depending on the stiffness of the building's foundation, the maximum value of the inter-story displacement of the first floor can be overestimated by around 70 %, with even greater overestimations for higher floors. The results help understand observed seismic behaviors and provide insight into the effects of foundation and tunnel stiffness on the tunnel-building interaction system.

Assessing the forecasting power of mean-field approaches for disease spreading using active systems

The forecasting of the temporal evolution of real-world diseases is systematically performed via mean-field, compartmental models, such as the SIR model and its various variants. Here, we investigate the spreading of SIR dynamics over a system of interacting, active agents – whose dynamics generate random contacts among the agents, as well as temporal and spatial correlation and fluctuations – to assess whether mean-field-like SIR models are able to accurately describe the temporal evolution of an epidemics. We find that such models display temporal dynamics that are intrinsically and fundamentally different from the ones obtained in active agent simulations. Specifically, we numerically prove that no effective mean-field SIR model is consistent with the dynamics that emerge in agent-based simulations. Our results call into question the use of such models to forecast the evolution of real-world epidemics, independently of the method used to estimate the basic reproduction number.