Development Of Interactive Systems Research Articles

Recognition and Scoring Physical Exercises via Temporal and Relative Analysis of Skeleton Nodes Extracted from the Kinect Sensor

Human activity recognition is known as the backbone of the development of interactive systems, such as computer games. This process is usually performed by either vision-based or depth sensors. So far, various solutions have been developed for this purpose; however, all the challenges of this process have not been completely resolved. In this paper, a solution based on pattern recognition has been developed for labeling and scoring physical exercises performed in front of the Kinect sensor. Extracting the features from human skeletal joints and then generating relative descriptors among them is the first step of our method. This has led to quantification of the meaningful relationships between different parts of the skeletal joints during exercise performance. In this method, the discriminating descriptors of each exercise motion are used to identify the adaptive kernels of the Constrained Energy Minimization method as a target detector operator. The results indicated an accuracy of 95.9% in the labeling process of physical exercise motions. Scoring the exercise motions was the second step after the labeling process, in which a geometric method was used to interpolate numerical quantities extracted from descriptor vectors to transform into semantic scores. The results demonstrated the scoring process coincided with the scores derived by the sports coach by a 99.5 grade in the R2 index.

Deep transformer-based architecture for the recognition of mathematical equations from real-world math problems

Identifying mathematical equations from real-world math problems presents a unique and challenging task within the field of Natural Language Processing (NLP). It has a wide range of applications in various areas, such as academics, digital content design, and the development of automatic or interactive learning systems. However, the accurate understanding of these equations still needs to be enhanced due to the intrinsic complexity of mathematical symbols and various structural formats. Additionally, the unique syntax, diverse symbols, and complex structure of mathematical equations present significant obstacles that traditional NLP methods and Optical Character Recognition (OCR) systems need help to overcome. In this research, we utilize deep transformer architecture to recognize mathematical equations, and we have utilized our novel dataset with 3433 distinct observations. This dataset, which we have collected to include a diverse range of mathematical equations, is used to predict six (z=x+y, z=x−y, z=xy, z=x/y, y=x! and y=x) basic mathematical equations. We applied different transformer-based architectures, such as BERT, ELECTRA, XLNet, RoBERTa, and DistilBERT, and BERT performs best with 99.80% accuracy. To the best of our knowledge, this is the first NLP work in any language where we recognize the equation from the mathematical text.

Consideration of people’s design preferences for the development of adaptive user interfaces

Abstract Adaptive user interfaces enable the display of user-specific, relevant information in complex interactive systems. The user experience on platforms can be improved by taking the user’s needs (goals, system experience, etc.) and design preferences (in terms of design shapes) into account. In a Germany-wide online survey, n = 1,044 young people (pupils and university students) aged between 14 and 35 were asked about their design shape preferences. The results show that, overall, the shape of the circle appears to be the most attractive for young people (14–35 years) and that gender and age have the greatest influence on design shape preferences. While men and generally older people (19–35 years, university students) prefer basic shapes to more complex shapes, women and generally younger people (14–19 years, pupils) find complex shapes more attractive than basic ones. The identification of preferences with regard to design shapes can provide developers of interactive systems with information for the design of (adaptive) user interfaces.

EFFECTIVENESS OF USING INFORMATION TECHNOLOGY IN LEARNING CASE STUDY OF INTERACTIVE LEARNING SYSTEM IN CLASS A EDUCATIONAL TECHNOLOGY

This study evaluates the effectiveness of information technology in interactive learning among college students. Data was obtained through the questionnaire, showing that information technology in interactive learning is very effective. Most students responded positively to accessibility, flexibility, interactivity, engagement, self-support, and variety of learning materials. Expressly, 100% of students agreed with the questions about these benefits. However, there were differences of opinion on several aspects. Regarding the suitability of learning speed and style, only 72.7 % of students felt that this system was suitable for their needs, while 27.3% thought it needed more. In addition, 90.9 % of students stated the ease of use and effectiveness of interaction, but some were still dissatisfied with these aspects. The conclusion of this study shows that although most students feel positive benefits from using information technology in learning, several areas need improvement, such as adjusting the speed and style of knowledge and the system's ease of use. Recommendations from this study include increasing system flexibility, optimizing interface design, and providing additional support for students. These findings can be a reference for developers of interactive learning systems and educational institutions who want to integrate information technology into the learning process more effectively.

The Development of Interactive Cinematic Film Production Systems, and its Impact on the Design of Motion Pictures

The Development of Interactive Cinematic Film Production Systems, and its Impact on the Design of Motion Pictures

CPU 병렬 처리를 이용한 SPH 유체와 단일보드 컴퓨터의 실시간 상호작용 시스템 개발

CPU 병렬 처리를 이용한 SPH 유체와 단일보드 컴퓨터의 실시간 상호작용 시스템 개발

SECURITY AND PROTECTION OF EDUCATIONAL LMC SYSTEMS

Information technologies play a significant role in the educational process and ensuring quality learning outcomes in the context of distance education. A modern trend is the development of interactive systems, which include elements of audio and video materials, graphics, presentations, internet links, materials from various sources, and in different formats. The addition of interactive content to educational systems increases the risks of information security in modern LMC systems. Emphasis is placed on the need to ensure the protection of software in systems, educational content hosted within them, learning outcomes, and participants' personal data. When building protection, attention should be paid to potential threats such as unauthorized access, flawed software, poor plugin updates, material copying, fraud, and cyber-attacks. Vulnerabilities of the most widespread modern LMC systems are analyzed. Attention is given to the possibilities of using cloud technologies to host system modules, educational materials, or learning outcomes on cloud services. Cloud application guarantees constant access to the learning system, reliable storage of materials, additional protection of confidential and personal information. Therefore, to ensure effective and safe learning considering modern interactive approaches to presenting educational materials, it is important to develop systems and implement technologies that provide constant access to educational content, reliable data protection (including personal data), information integrity, and compliance with confidentiality principles.

Petri Nets for the Iterative Development of Interactive Robotic Systems

We argue for the use of Petri nets as a modeling language for the iterative development process of interactive robotic systems. Petri nets, particularly Timed Colored Petri nets (TCPNs), have the potential to unify various phases of the development process-design, specification, simulation, validation, implementation, and deployment. We additionally discuss future directions for creating a domain-specific variant of TCPNs tailored specifically for HRI systems development.

Audio-Based Emotion Recognition Using Self-Supervised Learning on an Engineered Feature Space.

Emotion recognition models using audio input data can enable the development of interactive systems with applications in mental healthcare, marketing, gaming, and social media analysis. While the field of affective computing using audio data is rich, a major barrier to achieve consistently high-performance models is the paucity of available training labels. Self-supervised learning (SSL) is a family of methods which can learn despite a scarcity of supervised labels by predicting properties of the data itself. To understand the utility of self-supervised learning for audio-based emotion recognition, we have applied self-supervised learning pre-training to the classification of emotions from the CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU- MOSEI)'s acoustic data. Unlike prior papers that have experimented with raw acoustic data, our technique has been applied to encoded acoustic data with 74 parameters of distinctive audio features at discrete timesteps. Our model is first pre-trained to uncover the randomly masked timestamps of the acoustic data. The pre-trained model is then fine-tuned using a small sample of annotated data. The performance of the final model is then evaluated via overall mean absolute error (MAE), mean absolute error (MAE) per emotion, overall four-class accuracy, and four-class accuracy per emotion. These metrics are compared against a baseline deep learning model with an identical backbone architecture. We find that self-supervised learning consistently improves the performance of the model across all metrics, especially when the number of annotated data points in the fine-tuning step is small. Furthermore, we quantify the behaviors of the self-supervised model and its convergence as the amount of annotated data increases. This work characterizes the utility of self-supervised learning for affective computing, demonstrating that self-supervised learning is most useful when the number of training examples is small and that the effect is most pronounced for emotions which are easier to classify such as happy, sad, and angry. This work further demonstrates that self-supervised learning still improves performance when applied to the embedded feature representations rather than the traditional approach of pre-training on the raw input space.

Human-AI Collaboration in Healthcare: A Review of User Perspectives and Challenges

Artificial intelligence (AI) has rapidly become a transformative force across various sectors, including healthcare, virtual support, and security. The convergence of AI and Human-Computer Interaction (HCI) has led to the development of interactive intelligent systems, fostering user engagement. This paper presents a comprehensive review of the intersection of AI and HCI, focusing on user perspectives and challenges in the context of healthcare. Keywords: Artificial intelligence, Human-Computer Interaction, Healthcare, User Perspectives, Challenges

Development and validation of an art-inspired multimodal interactive technology system for a multi-component intervention for older people: a pilot study

IntroductionThe World Health Organization (WHO) acknowledges the presence of a significant body of research on the positive effects of the arts on health, considering a variety of factors including physical well-being, quality of life, and social and community impact. The model that underlies cultural welfare puts the performing arts, visual arts, and cultural heritage at the service of people personal and societal well-being. The potential connections between movements of the body and artistic content have been extensively studied over time, considering movement as a non-verbal language with a universal character.MethodsThis pilot study presents the results of the validation of an innovative multimodal system, the DanzArTe-Emotional Wellbeing Technology, designed to support active and participative experience of older people providing physical and cognitive activation through a full-body physical interaction with a traditional visual work of art of religious subject. DanzArTe supports a replicable treatment protocol for multidimensional frailty, administered through a low cost and scalable technological platform capable of generating real-time visual and auditory feedback (interactive sonification) from the automated analysis of individual as well as joint movement expressive qualities. The study involved 45 participants, 23 of whom participated in the DanzArTe program and 22 who were included in the control group.ResultsThe two groups were similar in terms of age (p = 0.465) and gender (p = 0.683). The results showed that the DanzArTe program had a positive impact on participants' self-perceived psychological health and well-being (Mean Psychological General Well-Being Index—Short T1 = 19.6 ± 4.3 Vs. T2 = 20.8 ± 4.9; p = 0.029). The same trend was not observed in the control group (p = 0.389).DiscussionThe findings suggest that such programs may have a significant impact particularly on the mental and social well-being of older adults and could be a valuable tool for promoting healthy aging and improving quality of life.

IMetaTown: A Metaverse System with Multiple Interactive Functions Based on Virtual Reality.

This work aims to pioneer the development of a real-time interactive and immersive Metaverse Human-Computer Interaction (HCI) system leveraging Virtual Reality (VR). The system incorporates a three-dimensional (3D) face reconstruction method, grounded in weakly supervised learning, to enhance player-player interactions within the Metaverse. The proposed method, two-dimensional (2D) face images, are effectively employed in a 2D Self-Supervised Learning (2DASL) approach, significantly optimizing 3D model learning outcomes and improving the quality of 3D face alignment in HCI systems. The work outlines the functional modules of the system, encompassing user interactions such as hugs and handshakes and communication through voice and text via blockchain. Solutions for managing multiple simultaneous online users are presented. Performance evaluation of the HCI system in a 3D reconstruction scene indicates that the 2DASL face reconstruction method achieves noteworthy results, enhancing the system's interaction capabilities by aiding 3D face modeling through 2D face images. The experimental system achieves a maximum processing speed of 18 frames of image data on a personal computer, meeting real-time processing requirements. User feedback regarding social acceptance, action interaction usability, emotions, and satisfaction with the VR interactive system reveals consistently high scores. The designed VR HCI system exhibits outstanding performance across diverse applications.

Construction of Online Interactive Learning System for University English Driven by Artificial Intelligence

Abstract With the advent of rapid advancements in information technology, online learning has increasingly gained traction, prompting a multitude of colleges and universities to adopt innovative reforms in their educational delivery methods. This research undertakes the development of a college English online interactive learning system empowered by artificial intelligence. It primarily investigates the interactive behaviors of college students in online English learning environments. Initially, the study employs the K-means algorithm to categorize students based on their distinct learning behaviors. Subsequently, it leverages a recommendation algorithm to fine-tune the provision of English teaching resources online. Furthermore, the study integrates an enhanced Apriori algorithm to extract associations between student behaviors and learning outcomes. This analytical framework underpins the empirical evaluation of the system’s efficacy in actual educational settings, aiming to deepen the understanding of students’ online interactive behaviors. The findings reveal that over 95% of students consistently log into the learning platform on scheduled class days. The metrics for course recommendation success, such as the hit rate, the average inverse rank, and the range of normalized discounted cumulative gain, are reported between 18.37-21.94, 1.87-2.08, and 4.44-4.87, respectively. Additionally, system memory utilization remains stable at 57%-66%, corroborating the system’s operational effectiveness. This research contributes valuable insights and benchmarks for the development of robust online interactive learning systems.

Conductive Polymer-Based Interactive Shelving System for Real-Time Inventory Management.

Stockouts constitute a major challenge in the retail industry. Stockouts are caused by errors related to manual stockkeeping and by the misplacement of items on shelves. Such errors account for up to 4% of lost sales. Real-time inventory management systems for misplaced items or missing stock detection in retail stores are limited. Accordingly, a conductive polymer-based interactive shelving system for real-time inventory management is developed. The system comprises an 80 × 48 sensor array fabricated by screen-printing a piezoresistive carbon-based conductive polymer layer onto gold interdigitated electrodes deposited on a flexible substrate. Each sensing pixel has dimensions of 5 mm × 5 mm and a sensing area of 4 mm × 4 mm. The sensor mat can detect the shape and weight features of stockkeeping units (SKUs), which can then be analyzed by a TensorFlow model for SKU identification. The developed system is characterized for functional resistance range, uniformity, repeatability, and durability. The accuracy of SKU identification achieved using shape features only and the accuracy of SKU identification achieved using both shape and weight features is 95% and 99.2%, respectively. The key novelty of the work is the development of a deep learning-embedded interactive smart shelving system for retail inventory management by using the shape and weight features of SKU. Also, the developed system helps to detect the SKU if they are stacked one over the other. Furthermore, multiple sensor mats implemented on various shelves in a retail store can be modularized and integrated for monitoring under the control of a single PC. Accordingly, the proposed retail inventory tracking system can facilitate the development of automated "humanless" shops.

Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems

Evaluation metrics such as precision, recall and normalized discounted cumulative gain have been widely applied in ad hoc retrieval experiments. They have facilitated the assessment of system performance in various topics over the past decade. However, the effectiveness of such metrics in capturing users’ in-situ search experience, especially in complex search tasks that trigger interactive search sessions, is limited. To address this challenge, it is necessary to adaptively adjust the evaluation strategies of search systems to better respond to users’ changing information needs and evaluation criteria. In this work, we adopt a taxonomy of search task states that a user goes through in different scenarios and moments of search sessions, and perform a meta-evaluation of existing metrics to better understand their effectiveness in measuring user satisfaction. We then built models for predicting task states behind queries based on in-session signals. Furthermore, we constructed and meta-evaluated new state-aware evaluation metrics. Our analysis and experimental evaluation are performed on two datasets collected from a field study and a laboratory study, respectively. Results demonstrate that the effectiveness of individual evaluation metrics varies across task states. Meanwhile, task states can be detected from in-session signals. Our new state-aware evaluation metrics could better reflect in-situ user satisfaction than an extensive list of the widely used measures we analyzed in this work in certain states. Findings of our research can inspire the design and meta-evaluation of user-centered adaptive evaluation metrics, and also shed light on the development of state-aware interactive search systems.

Development of a multi scale interactive web-GIS system to monitor farming practices: A case study in Lemnos Island, Greece

The environmental impacts of agricultural activities result from a chain of processes, agricultural practices affect the quantity and quality of natural resources, including biodiversity. Geographic information systems contribute to the systematic recording and monitoring of these agricultural practices in terms of production and to associate these practices with impacts on biodiversity. This paper presents the design of a web GIS system that records farming practices on a web map application (FarmGeoBalance), which is used by the farmers to import, store, revise and visualize farming practices in field level and farm level. The web map application communicates with a GeoDatabase informed with the geographical location of the participating fields where spatial and non-spatial data from the application are stored and standardized. The case study is an extensive arable farming system on Lemnos island, Greece, which covers a surface of 482 km². The system consists of 12 producers with 678 fields, representing 1.7% of the total farmed land of the island. In this paper, we present the web map application architecture and the collected data for the cultivation periods of 2020–2021 and 2021–2022, which include farming practices like fertilization, plant protection, irrigation, soil management, etc., outputs, landscape features of the farms related to biodiversity. For all these, their spatial and non-spatial dimension is presented and they are related to elements of the natural geography of fields and soils. With the use of this system recording and monitoring farming practices can assess the impact on biodiversity at multiple scales. It also highlights the need to include the end users at different phases of the development of the services and the interface. Farmers and their participation are the key component of the effectiveness of this effort: they monitor their everyday and seasonal practices at the level that they are realized in their farms.

Development of a virtual interactive system for Dahua Lou loom based on knowledge ontology-driven technology

The Dahua Lou loom, pivotal to Nanjing Yunjin weaving, constitutes an integral part of global intangible cultural heritage. Its intricate weaving technique remains unmatched by modern machinery, marking it as a vital cultural artefact warranting protection. However, current virtual interactive systems grapple with adequately demonstrating its craftsmanship due to limitations like system iteration, multi-system integration, and data interoperability. To address these challenges, this study proposes a method that synergizes knowledge ontology, virtual reality technology, and data-driven design.Knowledge ontology enables enhanced management, reuse, and wide-ranging dissemination of domain knowledge, improving system interoperability. This methodology is utilized in constructing the loom model, animation demonstration, and in integrating it into the virtual interactive system. This multifaceted application of knowledge ontology significantly bolsters the system’s efficiency and optimizes its development, maintenance, and integration processes.This research provides crucial advancements in domain knowledge modeling, 3D visualization, and virtual reality interactive systems, playing a significant role in preserving cultural heritage. Despite certain limitations, it offers an immersive, intuitive, and enriched design experience. The study concludes with a critique of the developed system, discussions on the encountered challenges, and future research directions include improving and expanding the knowledge ontology based on the opinions and practical experiences of domain experts, enhancing system compatibility, and conducting broader evaluations.

Development of a Human-Robot Interaction System for Industrial Applications

The use of robots is increasing day by day. In this study, it was aimed to develop manufacturing-assistant robot software for small production plants involving non-mass production. The main purpose of this study is to eliminate the difficulty of recruiting an expert robot operator thanks to the developed software and to facilitate the use of robots for non-experts. The developed software consists of three parts: the convolutional neural network (CNN), process selection-trajectory generation, and trajectory regulation modules. Before the operations in these modules are executed, operators record the desired process and the trajectory of the process in the video by hand gestures and index finger. Then recorded video is separated into images. The separated images are classified by the CNN module and the positions of landmarks (joint and fingernail of index finger) were calculated by the same module and using the images. Eight different pre-trained CNN structures were tested in the CNN module, and the best result Xception structure (test loss = 0.0051) was used. The desired process was determined and the trajectory of the process was created with the CNN output data. The connection of the generated trajectory with the object was detected by the trajectory regulation module, and unnecessary trajectory parts were cleaned. Regulated trajectory and desired tasks such as welding or sealing were simulated via an industrial robot in a simulation environment. As a result, an industrial robot could be programmed by non-expert operators for companies whose production line is not standard by using the developed software.

Experience of design and development of a new open access interactive multifunctional database management system for special lexis of biology

Terminology work requires a lot of data processing in the practical research of comparative linguistics, especially when studying linguistic and terminological changes over time. The lack of appropriate wide-usage electronic tools and databases can slow down such research. Thus, an efficient information management solution is necessary to improve the efficiency of terminology and comparative linguistic research. Since 2021, a team of terminologists, translators, researchers, and information system developers worked together for two years, during which the open access interactive multifunctional database management system (IMDS) was designed and developed for data storage and a wide range of statistical and search options, especially for language research purposes and comparative multilingual studies in linguistics and terminology. The information system consists of multiple modules (input, statistics, export, etc.). The data input module was successfully designed and developed and has been used effectively by researchers for entering and collecting special lexis units. The article mainly describes the statistical data that can be obtained from the IMDS, as well as problems encountered during the development of the system. The IMDS data collection has been updated with 50,846 entries of scientific (Latin) organism names, 60,585 entries of local names of different organisms (mainly in Latvian, but also in other languages), 1427 entries of names of organism-caused diseases, 2942 entries of dictionary words and 313 entries of terms with definitions collected based on excerpts from different publications containing common names linked with their scientific names.

Development of a Human-Agent Interaction System including Norm and Emotion in an Evacuation Situation (Student Abstract)

Agent-based modeling and simulation can provide a powerful test environment for crisis management scenarios. Human agent interaction has limitations in representing norms issued by an agent to a human agent that has emotions. In this study, we present an approach to the interaction between a virtual normative agent and a human agent in an evacuation scenario. Through simulation comparisons, it is shown that the method used in this study can more fully simulate the real-life out come of an emergency situation and also improves the au thenticity of the agent interaction.