Real-time Interactive System Research Articles

CPU 병렬 처리를 이용한 SPH 유체와 단일보드 컴퓨터의 실시간 상호작용 시스템 개발

CPU 병렬 처리를 이용한 SPH 유체와 단일보드 컴퓨터의 실시간 상호작용 시스템 개발

EYE-BLINK DETECTION ASSISTIVE SYSTEM FOR PARALYZED PATIENT

Paralysis is defined as the complete loss of muscle function in any part of the body. It occurs when there is a problem with the passage of messages between the muscles and the brain. Some paralyzed people cannot move even a single part of the body other than their eyes. Hence, the main aim of this project is to design a real time interactive system that can assist the paralyzed to control appliances such as lights, fans or by playing pre-recorded audio messages, through a predefined number of eye blinks. Image processing techniques have been implemented in order to detect the eye blinks. In our system, the face tracking is accomplished by using a set of trained Haar cascade classifier, and a template matching technique is employed to track the eye. Initially, the involuntary blinks of the paralyzed person are used to locate the patients eyes by finding the number of connected components in a frame.

A real-time interactive restoration system for intraoral digital videos using segment anything model.

Poor conditions in the intraoral environment often lead to low-quality photos and videos, hindering further clinical diagnosis. To restore these digital records, this study proposes a real-time interactive restoration system using segment anything model. Intraoral digital videos, obtained from the vident-lab dataset through an intraoral camera, serve as the input for interactive restoration system. The initial phase employs an interactive segmentation module leveraging segment anything model. Subsequently, a real-time intraframe restoration module and a video enhancement module were designed. A series of ablation studies were systematically conducted to illustrate the superior design of interactive restoration system. Our quantitative evaluation criteria contain restoration quality, segmentation accuracy, and processing speed. Furthermore, the clinical applicability of the processed videos was evaluated by experts. Extensive experiments demonstrated its performance on segmentation with a mean intersection-over-union of 0.977. On video restoration, it leads to reliable performances with peak signal-to-noise ratio of 37.09 and structural similarity index measure of 0.961, respectively. More visualization results are shown on the https://yogurtsam.github.io/iveproject page. Interactive restoration system demonstrates its potential to serve patients and dentists with reliable and controllable intraoral video restoration.

IMetaTown: A Metaverse System with Multiple Interactive Functions Based on Virtual Reality.

This work aims to pioneer the development of a real-time interactive and immersive Metaverse Human-Computer Interaction (HCI) system leveraging Virtual Reality (VR). The system incorporates a three-dimensional (3D) face reconstruction method, grounded in weakly supervised learning, to enhance player-player interactions within the Metaverse. The proposed method, two-dimensional (2D) face images, are effectively employed in a 2D Self-Supervised Learning (2DASL) approach, significantly optimizing 3D model learning outcomes and improving the quality of 3D face alignment in HCI systems. The work outlines the functional modules of the system, encompassing user interactions such as hugs and handshakes and communication through voice and text via blockchain. Solutions for managing multiple simultaneous online users are presented. Performance evaluation of the HCI system in a 3D reconstruction scene indicates that the 2DASL face reconstruction method achieves noteworthy results, enhancing the system's interaction capabilities by aiding 3D face modeling through 2D face images. The experimental system achieves a maximum processing speed of 18 frames of image data on a personal computer, meeting real-time processing requirements. User feedback regarding social acceptance, action interaction usability, emotions, and satisfaction with the VR interactive system reveals consistently high scores. The designed VR HCI system exhibits outstanding performance across diverse applications.

Evaluation of the Real-time Teleradiology Consultation System in the Emergency Department

ABSTRACT Background and Objectives: Teleradiology involves the transmission of all radiological images to a radiologist in a remote location to ensure an accurate diagnosis. Although various teleradiology systems exist, there is not a fully accepted real-time interactive teleradiology system yet. In this study, we analyzed “the teleradiology consultation system which includes a real-time and instant messaging program (Skype®)” established in an emergency department. Methods: The consultations requested from the teleradiology system implemented in the emergency department (ED) of a Medical Faculty Training and Research Hospital were evaluated between January 2018 and June 2018 retrospectively. Results: A total of 18,284 teleconsultations were included in the study. The mean age of the patients who required teleconsultation was 40.6 ± 24.8 years (range: 0–109 years) and 10,069 (55.1%) were male. And 17,079 (93.4%) computed tomography (CT) was consulted. The majority of CT scans were cranial CT (n = 7113, 41.6%) and upper-lower abdominal CT (n = 5436, 31.8%). And 1205 magnetic resonance imaging (MRIs) (6.6%) were consulted. The majority of MRIs were diffusion MRIs (n = 975, 80.9%). The mean evaluation time of radiological imaging was 66.3 minutes. In 7238 (39.6%) of all consultations, a problem or a demand was reported via interactive communication by radiologists or emergency physicians. They are as follows; the desire to speed up delayed reports (33.0%, n = 2388), problems related to radiological images (32.0%, n = 2316); demand for correction of delay problems from health system technicians (11.6%, n = 839), incomplete reporting (6.6%, n = 477), demand for the acceleration of reports of critical patients (6.4%, n = 463), request for re-evaluation of radiological images (5.2%, n = 376), the demand for additional clinical information of the radiology doctor (2.0%, n = 144), asking radiology doctor the statements in the report or to request for disclosure of findings (1.1%, n = 78), the other reasons (2.0%, n = 144). All of these problems were answered with real-time interviews and solved. Conclusion: The teleradiology system is important in health institutions where there is no 7/24 radiologist. Conclusions: This study showed that the establishment of a teleradiology consultation system integrated with a messaging service in the ED might provide real-time and immediate resolution of the occurring problems.

PlantConnect

Abstract This paper describes PlantConnect, a real-time interactive system that explores human-plant interaction via the human act of breathing, the bioelectrical and photosynthetic activity of plants, and computational intelligence to bring the two together. Part of larger investigations into alternative models for the creation of shared experiences and understanding with the natural world, the work is presented as a concrete implementation of a possible model based upon reciprocal interplay and information flows between human and nonhuman worlds.

IoMT with Deep CNN: AI-Based Intelligent Support System for Pandemic Diseases

The Internet of Medical Things (IoMT) is an extended version of the Internet of Things (IoT). It mainly concentrates on the integration of medical things for servicing needy people who cannot get medical services easily, especially rural area people and aged peoples living alone. The main objective of this work is to design a real time interactive system for providing medical services to the needy who do not have a sufficient medical infrastructure. With the help of this system, people will get medical services at their end with minimal medical infrastructure and less treatment cost. However, the designed system could be upgraded to address the family of SARs viruses, and for experimentation, we have taken COVID-19 as a test case. The proposed system comprises of many modules, such as the user interface, analytics, cloud, etc. The proposed user interface is designed for interactive data collection. At the initial stage, it collects preliminary medical information, such as the pulse oxygen rate and RT-PCR results. With the help of a pulse oximeter, they could get the pulse oxygen level. With the help of swap test kit, they could find COVID-19 positivity. That information is uploaded as preliminary information to the designed proposed system via the designed UI. If the system identifies the COVID positivity, it requests that the person upload X-ray/CT images for ranking the severity of the disease. The system is designed for multi-model data. Hence, it can deal with X-ray, CT images, and textual data (RT-PCR results). Once X-ray/CT images are collected via the designed UI, those images are forwarded to the designed AI module for analytics. The proposed AI system is designed for multi-disease classification. It classifies the patients affected with COVID-19 or pneumonia or any other viral infection. It also measures the intensity level of lung infection for providing suitable treatment to the patients. Numerous deep convolution neural network (DCNN) architectures are available for medical image classification. We used ResNet-50, ResNet-100, ResNet-101, VGG 16, and VGG 19 for better classification. From the experimentation, it observed that ResNet101 and VGG 19 outperform, with an accuracy of 97% for CT images. ResNet101 outperforms with an accuracy of 98% for X-ray images. For obtaining enhanced accuracy, we used a major voting classifier. It combines all the classifiers result and presents the majority voted one. It results in reduced classifier bias. Finally, the proposed system presents an automatic test summary report textually. It can be accessed via user-friendly graphical user interface (GUI). It results in a reduced report generation time and individual bias.

Data pipeline for managing field experiments

As agricultural and environmental research projects become more complex, increasingly with multiple outcomes, the demand for technical support with experiment management and data handling has also increased. Interactive visualisation solutions are user-friendly and provide direct information to facilitate decision making with timely data interpretation. Existing off-the-shelf tools can be expensive and require a specialist to conduct the development of visualisation solutions. We used open-source software to develop a customised near real-time interactive dashboard system to support science experiment decision making. Our customisation allowed for:• Digitalised domain knowledge via open-source solutions to develop decision support systems.• Automated workflow that only executed the necessary components.• Modularised solutions for low maintenance cost and upgrades.

Analysis on the Development of Subway in a Pandemic City

The outbreak of COVID-19 poses a severe challenge to the epidemic prevention and control of urban public transport system. As the backbone of urban public transport, urban rail transit is one of the important links in the epidemic prevention and control of urban public transport system. It faces the challenge of heavy tasks and great pressure. Using the methods of questionnaire survey, field investigation and case analysis, this paper aims to find out how to improve the operation of the subway under the epidemic situation, and finally comes to a conclusion that there is need to further strengthen the control measures to deal with peak passenger flow concentration, and improve the emergency plan. Besides, a traceable and reliable epidemic prevention cooperation system should be created and a real-time dynamic interaction system should be built to open up the information barrier between passengers and operating units. Moreover, it is also important to improve the normalized and refined monitoring and early warning system for high-risk areas, as well as simplifying the display of passengers' personal information and realize automation, intelligence, and integration.

Comparison of Client-Server Solutions in the Development of Massively Multiplayer Online Games on Unity

This paper presents a critique of the traditional approach used to create a multiplayer game in the Unity real-time interactive application development system, especially in the case of a large number of concurrent users. As a hypothesis, an alternative option, which is not common, but which solves many of the problems of the previous approach, is proposed. Two client-server solutions have been compared for developing multiplayer online games in Unity, and the advantages of both approaches have been described for different cases. A game development architecture using a more up-to-date method is proposed: instead of the Mirror library, a standard toolkit for Unity development, microservices written in Golang are used. We present solid proofs of the preference of the alternative approach, the main advantage of which is the support of modern architecture providing high-speed communication between microservices, supported tests on messaging on different platforms.
 The test results confirm the hypothesis put forth, and we can conclude that the Unity bundle with Golang is more effective for multiplayer video games.
 The article also contains basic methods for debugging multi-threaded application in Golang bundled with Unity game development system and suggests a technological method that allows to get a fast way of data transfer between the client and the server.

Let it go! Point of release prediction for virtual throwing

In this paper, we present a novel real-time physical interaction system that allows users to throw a virtual ball without using an intermediary device such as a controller. A dataset of throwing motions was captured from six actors, from which ground truth Point of Release (PoR) frames were calculated. We trained a PoR prediction model using motion features extracted from arm joints, and developed a detection algorithm to predict the PoR of a throwing motion in real-time. Evaluation of the system with pre-recorded throwing motion data results in detection errors of less than 50 ms. Qualitative results from six users of the real-time system in VR indicate that the task of throwing without a controller was very natural, although the system performed better for underarm than overarm throws. Finally, we report the relative importance of different joints and motion features for the PoR prediction task.

Design and implementation of virtual fitting system based on gesture recognition and clothing transfer algorithm

Virtual fitting can bring fast and convenient fitting experience for people. The two core problems of virtual fitting system are human–computer interaction and clothing simulation. Interaction is an important factor in determining the fitting experience. The previous virtual fitting products usually use the mouse and keyboard interaction, and users rarely have a good sense of substitution and interaction. While, the method of using multiple cameras to take user images from different angles and then carry out posture recognition has the defect of low recognition accuracy. In view of clothing simulation and human–computer interaction of virtual fitting system, in order to achieve better customer immersion experience, this paper implemented a real-time interactive virtual fitting system based on Microsoft Kinect motion sensing device, and proposed a gesture determination algorithm based on finger recognition and an image transfer algorithm based on skeleton information matching. Using OpenNI development library and multi-threading technology, we have developed a motion-sensing capture module and a complete real-time virtual fitting system, and the system test results show that it has a good user experience.

HOME AUTOMATION USING EYE BLINK SENSOR FOR PARALYZED PEOPLE

The constant demand to improve daily living standards for paralysis patients serves as a motivation to develop newer technology. The taskonce performed by big traditional computers is now solved with smaller smart devices. Paralysis isdefined as the complete loss of muscle function in any part of the body. It occurs when there is a problem with the passage of messages between the muscles and the brain. The main objective is todesign a real time interactive system that can assistthe paralysis patients to control appliances such aslights, fans etc. The eye blink sensor is able to detect an intentional blink from a normal blink, which is useful for the paralysis patients especially Tetraplegic patients to regulate their home deviceseasily without any help.

Convolutional Neural Network-Based Virtual Reality Real-Time Interactive System Design for Unity3D.

In this paper, the algorithm of the convolutional neural network is used for in-depth research and analysis of real-time interactivity, and a virtual reality real-time interactive system is designed based on Unity3D. Aiming at the problems of high computation and low efficiency of existing high-precision models, based on the lightweight network model, this paper proposes a real-time semantic segmentation method based on an asymmetric codec. The encoder part of the network adopts a newly designed bottleneck residual module based on depth-separable convolution, null convolution, and decomposition convolution to extract local and contextual information without increasing the computational effort. At the same time, channel rearrangement is introduced in the module to facilitate the interaction of information between channels. The approach proposed in this paper can present the architectural system simply and clearly and is more flexible to match various complex business relationships. The system design technology is based on the Spring MVC framework technology, using visualization technology design and implementation. The combination of the Spring MVC framework and Unity3D achieves the separation of the front and back ends of the system, which makes the system stable, real-time, visible, and efficient. Meanwhile, a newly designed global attention guidance module based on the attention mechanism is introduced at the jump connection between codecs to guide the low-level features of encoder structure and high-level features of decoder structure for better integration and accuracy improvement.

Real-time interactive digital healthcare system for post-operative breast cancer patients: study protocol for a randomized controlled trial

BackgroundAfter receiving breast cancer surgery or reconstruction, shoulder dysfunctions including weakness, post-operative pain, shoulder joint instability, and limited range of motion (ROM) often occur. Due to limited ROM, patients may suffer difficulty in activities of daily living, and quality of life may be reduced. The objective of this study is to compare the effects on shoulder ROM of a real-time interactive digital healthcare system and brochure-based home rehabilitation program in post-operative breast cancer patients.MethodsThis study is a prospective, multi-center, assessor-blinded randomized controlled trial. The study aims to recruit 100 breast cancer patients exhibiting limited shoulder ROM after undergoing axillary lymph node dissection or breast reconstruction following mastectomy. Patients will be randomly assigned to two groups for 8 weeks of rehabilitation: a digital healthcare system rehabilitation (intervention) group and a brochure-based rehabilitation (control) group. The primary outcome is the change in ROM of the affected shoulder between baseline and 12 weeks after enrollment. Secondary outcomes include pain in the affected shoulder, as measured using a numerical rating scale, functional outcomes (QuickDASH scores), and quality of life (FACT-B and EQ-5D-5L scores), all of which will be measured on enrollment and 4, 8, and 12 weeks thereafter.DiscussionThis study will compare the effectiveness of a newly developed, augmented reality-based real-time interactive digital healthcare system with that of brochure-based home rehabilitation for improving the shoulder ROM, pain, functional outcomes, and quality of life of post-operative breast cancer patients.Trial registrationClinicalTrials.govNCT04316156. Registered on 20 March 2020.

Practice and Reflection on the Synchronous Online courses of Chinese as a Foreign Language with the Goal of Communication

This article introduces the organization and implementation of online interactive Chinese classes with double teachers, as well as the practice and reflections about the online Chinese classes for the students from Egyptian universities during the COVID-19 pandemic.As a remote real-time interactive teaching system, the double-teacher online class can effectively combine synchronous and asynchronous learning. Compared with the traditional non-interactive online classes and recorded classes, the double-teacher online class is easier for students to accept, and can also improve classroom participation. In addition to emphasizing the interactivity, promoting the interaction and communication between the two teachers, the teachers and students, and among students themselves, this class can also share excellent teachers and teaching resources. In this class, students can communicate with native Chinese speakers and improve their language skills and cross-cultural communication skills consequently. The online Chinese courses for Egyptian universities students are designed with activities focusing on improving listening and speaking skills. On the basis of helping students review and consolidate what they have learned at school, this course added practice for daily communication, and these high-frequency oral communication with native Chinese speakers has become a useful supplement to the students’ Chinese classes at school.The author directly participated in the course design and teaching. Through detailed teaching cases, she explained the organization and implementation of this online Chinese class, and the problems encountered in it and resolutions. Furthermore, the author conducted an evaluation about the online learning of those Egyptian universities students.

Smart Living: An Interactive Control System for Household Appliances

Hand-gesture based control has enormous potential both theoretically and for practical applications due to its convenience and intuitiveness. This study presents a real-time interactive control system for household appliances. The interactive control system allowing wireless control of household appliances using a combination of 11 hand gestures and 2 waving motions is tested on hundreds of samples. It is implemented using a regular personal computer (PC) and existing digital single processing (DSP) platforms. The evaluation results show that the system performs efficiently reaching an accuracy recognition rate of 91% and spending around 30 seconds to complete the control operation for household appliances. The contributions of this work are both academic (1) successful demonstration of the integration of algorithms for solving image detection, processing, and pattern recognition, and practical (2) showing its feasibility and using commonly available hardware and software configurations for practical uses, and finally (3) establishing a mechanism for intuitively interactive control system that facilitates smart living.

In-Line Inspection of Pipeline Defects Detection Using Ring-Type Laser

Pipeline plays a vital role in transporting fluids like oils, water, and petrochemical substances for longer distances. Based on the materials they carry, prolonged usage may cause the initiation of defects in the pipeline. These defects occur due to the formed salt deposits, chemical reaction happens between the inner surface and the transferring substance, prevailing environmental conditions, etc. These defects, if not identified earlier may lead to significant losses to the industry. In this work, an in-line inspection system utilizes the nondestructive way for analyzing the internal defects in the petrochemical pipeline. This system consists of a pipeline inspection robot having two major units namely the visual inspection unit and the power carrier unit. The visual inspection unit makes use of a ring-type laser diode and the camera. The laser diode serves as a light source for capturing good quality images of inspection. This unit is controlled by the Arduino in the power carrier unit which provides the necessary movement throughout the pipe. The inspected images captured by the camera are further processed with the aid of NI vision assistant software. After applying the processing function parameters provided by this software, the defect location can be clearly visualized with high precision. Three sets of defects are introduced in a Polylactide (PLA) pipe based on its position and angle along the circumference of the pipe. Further, this robot system serves as a real-time interactive image synchronization system for acquiring the inspected images. By comparing the actual and calculated defect size, the error percentage obtained was less than 5%.

Application of Machine Learning in Air Hockey Interactive Control System.

In recent years, chip design technology and AI (artificial intelligence) have made significant progress. This forces all of fields to investigate how to increase the competitiveness of products with machine learning technology. In this work, we mainly use deep learning coupled with motor control to realize the real-time interactive system of air hockey, and to verify the feasibility of machine learning in the real-time interactive system. In particular, we use the convolutional neural network YOLO (“you only look once”) to capture the hockey current position. At the same time, the law of reflection and neural networking are applied to predict the end position of the puck Based on the predicted location, the system will control the stepping motor to move the linear slide to realize the real-time interactive air hockey system. Finally, we discuss and verify the accuracy of the prediction of the puck end position and improve the system response time to meet the system requirements.

Spintronic Sensors Based on Magnetic Tunnel Junctions for Wireless Eye Movement Gesture Control.

The tracking of eye gesture movements using wearable technologies can undoubtedly improve quality of life for people with mobility and physical impairments by using spintronic sensors based on the tunnel magnetoresistance (TMR) effect in a human-machine interface. Our design involves integrating three TMR sensors on an eyeglass frame for detecting relative movement between the sensor and tiny magnets embedded in an in-house fabricated contact lens. Using TMR sensors with the sensitivity of 11mV/V/Oe and ten <1 mm3 embedded magnets within a lens, an eye gesture system was implemented with a sampling frequency of up to 28 Hz. Three discrete eye movements were successfully classified when a participant looked up, right or left using a threshold-based classifier. Moreover, our proof-of-concept real-time interaction system was tested on 13 participants, who played a simplified Tetris game using their eye movements. Our results show that all participants were successful in completing the game with an average accuracy of 90.8%.