Touchless Interaction Research Articles

Smart HygineMate Hub: A Smart Vending Machine

Activists and groups concerned with women’s health and hygiene as well as news articles, social media, and discussions have brought attention to the issue of sanitary pads selling machines. The need of convenient access to sanitary pads through vending machines particularly in public locations like schools, colleges, airports, hotels and public restrooms, was made more widely known through the efforts of Women’s Rights Organization, Non-Governmental Organizations (NGOs) and Government Initiatives. As a part of larger discussion surrounding menstrual hygiene and women’s empowerment this topic attracted attention. By automating inventory management, providing touchless interactions, various payment modes and enhancing security, the model provides a streamlined user experience. This technology also yields a valuable data insight, reduced maintenance efforts and facilitates personalized user profiles. Ultimately it contributes in improving menstrual hygiene and accessibility. The lack of scanner which makes it difficult for people in need of sanitary pads in emergencies, the difficulty of promptly monitoring and replenishing the machines, sanitary pads supply are the two of the main issues with the existing sanitary pad vending machines. Additionally consistent resources and work are needed to maintain and keep sanitary pad vending machines operational. Further issue still exists of affordability of the vending machines particularly in the rural areas due to limited resources. These issues mainly attracted to work and research on making a vending machine which will be cost effective, simplified and user-friendly machines with the use of edge cutting technologies.

Touchless Interaction for Smart Glasses in Emergency Medical Services: User Needs and Experiences

The application of smart glasses in healthcare, particularly for providers engaged in hands-on patient care tasks, presents unique design challenges. This study combines participatory design and usability testing to assess the user experiences of touchless interaction methods for smart glasses in the context of Emergency Medical Services (EMS). The participatory design workshops with 16 EMS providers reveal a preference for touchless interaction methods such as voice commands and pinch hand gestures, driven by the need to keep hands free and minimize cross-contamination risk. Despite this preference, the laboratory-based usability testing with 16 EMS providers indicates that both voice commands and hand gestures fall short in task performance compared to the default tangible buttons on smart glasses, primarily due to software limitations and EMS providers’ unfamiliarity with touchless techniques. Our findings reveal specific issues associated with using different interaction methods when operating smart glasses. Building on these insights, we discuss design implications for smart glasses to better align with the dynamic and unique characteristics of fast-paced medical work.

Neuromorphic antennal sensory system

Insect antennae facilitate the nuanced detection of vibrations and deflections, and the non-contact perception of magnetic or chemical stimuli, capabilities not found in mammalian skin. Here, we report a neuromorphic antennal sensory system that emulates the structural, functional, and neuronal characteristics of ant antennae. Our system comprises electronic antennae sensor with three-dimensional flexible structures that detects tactile and magnetic stimuli. The integration of artificial synaptic devices adsorbed with solution-processable MoS2 nanoflakes enables synaptic processing of sensory information. By emulating the architecture of receptor-neuron pathway, our system realizes hardware-level, spatiotemporal perception of tactile contact, surface pattern, and magnetic field (detection limits: 1.3 mN, 50 μm, 9.4 mT). Vibrotactile-perception tasks involving profile and texture classifications were accomplished with high accuracy (> 90%), surpassing human performance in “blind” tactile explorations. Magneto-perception tasks including magnetic navigation and touchless interaction were successfully completed. Our work represents a milestone for neuromorphic sensory systems and biomimetic perceptual intelligence.

PalmSpace: Leveraging the palm for touchless interaction on public touch screen devices

Touchscreen is the primary solution to interact with public devices such as Automated Teller Machines (ATMs). However, the touch modality raises health concerns since users have to touch the screens, and therefore risking the spread of contagious diseases. We design PalmSpace, an alternate input technique leveraging users’ hand palms to interact with public devices. With Palmspace, UI elements are mapped onto the users’ palms and can be accessed by touching various locations directly on the palm. We conduct a series of user studies to evaluate several design options, such as interface layout, item size, preferred item location, and suitable feedback for items. Based on the results, we design PalmSpace and compare its performance with mid-air input. We show that PalmSpace is a potential solution to interact with public devices without using their touchscreen. We conclude with design guidelines for using the palm as an alternative input space for touchscreen devices.

Remote, Touchless Interaction with Medical Images and Telementoring in the Operating Room Using a Kinect-Based Application—A Usability Study

Innovative technologies can improve user usability and satisfaction in computer-based activities in the work environment, including surgeons working in the operating room (OR). A scrubbed surgeon must ask unscrubbed staff to retrieve medical images from a surgical PACS system on a monitor in a hybrid operating room. The study aimed to check users’ usability and satisfaction with the designed and developed Ortho_Kinect_OR application, which enables contactless control of access to medical images during surgery. The application also facilitates access to telemedicine applications such as intraoperative telementoring during orthopedic procedures or tele-education. The application’s usability was tested by assigning standardized tasks to surgeons for PACS and teleconferencing in the operating room. Hand gestures were recognized and translated to function like mouse buttons. Field tests conducted among orthopedic surgeons showed high usability and user satisfaction. PACS access in the operating room did not distract the orthopedic surgeon during orthopedic surgery procedures. All surgeons completed the tests and tasks without any problems. OR field test results showed high agreement among users and were very satisfactory. Junior surgeons and residents pointed out that people with average computer knowledge could use the application. It has been shown that the contactless system designed and built based on the Kinect sensor available on the shelves meets the requirements of the operating room environment and is easy to use. Touchless controller technology provides the opportunity to increase the use of intraoperative imaging previews and improve the safety of surgical patients by improving sterility and reducing unnecessary staff in the operating room. Using the Ortho_Kinect_OR application and the Kinect sensor, it is possible to provide contactless access to videoconference telementoring and PACS in the operating room without an orthopedic surgeon’s unnecessary distraction in the operating room environment.

GESTURE-BASED HUMAN-COMPUTER INTERACTION

This project explores touchless interaction with computers using gesture recognition technology. By combining computer vision and gesture recognition, users can control their computers using hand gestures. The system uses a webcam for input, processes images with OpenCV, tracks hand movements with MediaPipe, and controls mouse and keyboard actions with PyAutoGUI. Users can perform various gestures like clicks, scrolls, and custom Open Paint gesture to control their computer. The project integrates with a paint application, allowing users to create digital art using their hand gestures, enhancing the overall user experience. Users can seamlessly control their computers by executing predefined gestures in the air, eliminating the limitations of conventional input device like mice. KEYWORDS - Gesture-based interaction, Human-Computer Interaction, MediaPipe, OpenCV, Paint Application, PyAutoGUI, Virtual mouse.

Potential of Low-Cost Light Detection and Ranging (LiDAR) Sensors: Case Studies for Enhancing Visitor Experience at a Science Museum

A low-cost light detection and ranging (LiDAR) device has several advantages including being able to perform a wide range of angle measurements, less privacy concerns, and robustness to illumination variance owing to its use of infrared (IR) light. In this study, to enhance the visitor experience at a science museum, three case studies using low-cost LiDAR sensors are presented: (1) an interactive floor projection to learn about the phases of the Moon; (2) an information kiosk with touchless interaction and visitor tracking; and (3) a visitor tracking box with horizontal and vertical scanning. The proposed kiosk system uses a mirror to reflect a portion of the scanning plane of the LiDAR sensor, to allow the capture of touchless interactions, track visitor positions, and count the number of nearby visitors. The visitor tracking box also uses two detection planes reflected by a mirror: the vertical plane is for counting visitors crossing the scanning plane and the horizontal plane is for tracking visitor positions to generate the corresponding heat maps for the visualization of museum hotspots. A series of evaluation experiments were conducted at a science museum, whereby an accuracy of 85% was obtained to estimate the number of visitors, with an accuracy increasing in counting people taller than 140 cm. The interactive floor received a visitor rating of 4.3–4.4 on a scale of 1–5.

Exploring Gaze-assisted and Hand-based Region Selection in Augmented Reality

Region selection is a fundamental task in interactive systems. In 2D user interfaces, users typically use a rectangle selection tool to formulate a region using a mouse or touchpad. Region selection in 3D spaces, especially in Augmented Reality (AR) Head-Mounted Displays (HMDs) is different and challenging because users need to select an intended region via freehand mid-air gestures or eye-based actions that are touchless interactions. In this work, we aim to fill in the gap in the design of region selection techniques in AR HMDs. We first analyzed and discretized the interaction procedure of region selection and explored design possibilities for each step. We then developed four techniques for region selection in AR HMDs, which leveraged users' hand and gaze for unimodal or multimodal interaction. The techniques were evaluated via a user study with a controlled region selection task. The findings led to three design recommendations and two proof-of-concept application examples.

A Dual‐Responsive Artificial Skin for Tactile and Touchless Interfaces (Small 21/2023)

Tactile and Touchless Artificial Skins In article number 2206830, Yifan Wang and co-workers present a dual-responsive artificial skin, which reacts to both pre-contact events and tactile pressure levels. Overlooked valuable information encoded in the proximal inputs from environments can be explored. Versatile human–machine interfaces (HMIs) for tactile and touchless interactions are achieved. Moreover, the proof-of-concept application for materials classification is successfully demonstrated in an entirely touchless mode.

A multimodal user interface for touchless control of robotic ultrasound

PurposePast research contained the investigation and development of robotic ultrasound. In this context, interfaces which allow for interaction with the robotic system are of paramount importance. Few researchers have addressed the issue of developing non-tactile interaction approaches, although they could be beneficial for maintaining sterility during medical procedures. Interaction could be supported by multimodality, which has the potential to enable intuitive and natural interaction. To assess the feasibility of multimodal interaction for non-tactile control of a co-located robotic ultrasound system, a novel human–robot interaction concept was developed.MethodsThe medical use case of needle-based interventions under hybrid computed tomography and ultrasound imaging was analyzed by interviewing four radiologists. From the resulting workflow, interaction tasks were derived which include human–robot interaction. Based on this, characteristics of a multimodal, touchless human–robot interface were elaborated, suitable interaction modalities were identified, and a corresponding interface was developed, which was thereafter evaluated in a user study with eight participants.ResultsThe implemented interface includes voice commands, combined with hand gesture control for discrete control and navigation interaction of the robotic US probe, respectively. The interaction concept was evaluated by the users in the form of a quantitative questionnaire with a average usability. Qualitative analysis of interview results revealed user satisfaction with the implemented interaction methods and potential improvements to the system.ConclusionA multimodal, touchless interaction concept for a robotic US for the use case of needle-based procedures in interventional radiology was developed, incorporating combined voice and hand gesture control. Future steps will include the integration of a solution for the missing haptic feedback and the evaluation of its clinical suitability.

AGILEST approach: Using machine learning agents to facilitate kinesthetic learning in STEM education through real-time touchless hand interaction

There is an increasing interest in creating interactive learning applications using innovative interaction technologies, especially in STEM (Science, technology, engineering, and mathematics) subjects. Recent developments in machine learning have allowed for nearly perfect hand-tracking recognition, introducing a touchless modality for interaction within Augmented Reality (AR) environments. However, the research community has not explored the pedagogical approach of Kinesthetic Learning or “Learning by Doing”, hand tracking, and machine learning agents combined with Augmented Reality technology. Fundamentally, this exploration of touchless interaction technologies has taken on new importance in the new post-COVID world. Meanwhile, machine learning has gained attention for its ability to enhance personalized learning and play a vital new role as a virtual instructor. This paper proposes a novel approach called the AGILEST approach, which uses machine learning Agents to facilitate interactive kinesthetic learning in STEM education through touchless interaction. The first case study for this approach will be an AR learning application for chemistry. This application uses real-time touchless hand interaction for kinesthetic learning and uses a machine learning agent to act as both trainer and assessor of the user. The evaluation of this research has been conducted remotely through a usability study with expert reviewers, which includes 15 young researchers with peer-reviewed work in Human-Computer Interaction & AR and 2 subject experts STEM teachers at the secondary school level. The usability evaluation through NASA Task Load Index (NASA-TLX), Perceived Ease of Use (PUEU), and Perceived Usefulness (PU) with expert reviewers provide positive feedback about this approach for productive learning gain, engagement and interactiveness in learning STEM subjects.

SC59 - The role of the intraoperative touchless interaction with 3D Virtual models performing laparoscopic partial nephrectomy: our preliminary experience with ICON3D

SC59 - The role of the intraoperative touchless interaction with 3D Virtual models performing laparoscopic partial nephrectomy: our preliminary experience with ICON3D

Towards multimodal interaction for needlebased procedures in a virtual radiology suite

Abstract Touchless interaction is popular in the medical domain because it maintains sterility and ensures physicians’ autonomy. Evaluating these technologies, however, proves difficult due to technical and human hurdles. Virtual reality leaves these limitations behind and allows for the exploration of promising concepts by simulating an environment and the interactions that takes place within it.We present a virtual radiology suite in the context of needle-based MR-interventions to evaluate touchless interactions. Hand and foot inputs were implemented on a custom interface and evaluated in a user study (n= 16). Results show that activating the system and manipulating values was faster with foot input. However, multimodal interaction is preferable because it is less demanding.

A Siloxene/Ecoflex Nanocomposite‐Based Triboelectric Nanogenerator with Enhanced Charge Retention by MoS2/LIG for Self‐Powered Touchless Sensor Applications (Adv. Funct. Mater. 27/2022)

Triboelectric Nanogenerators In article number 2113005, Jae Y. Park and co-workers develop a Siloxene/Ecoflex based contactless triboelectric nanogenerator (TENG) to reduce the possible contact situations and improve the robustness and functionality of the TENG. The MoS2/LIG-based charge trapping interlayer exhibits excellent charge trapping property and aids in the realization of the TENG as a self-powered sensor for touchless interactions.

Asymmetric Free-Hand Interaction on a Large Display and Inspirations for Designing Natural User Interfaces

Hand motion sensing-based interaction, abbreviated as ‘free-hand interaction’, provides a natural and intuitive method for touch-less interaction on a large display. But due to inherent usability deficiencies of the unconventional size of the large display and the kinematic limitations of the user’s arm joint movement, a large display-based free-hand interaction is suspected to have different performance across the whole areas of the large display. To verify this, a multi-directional target pointing and selection experiment was designed and conducted based on the ISO 9241-9 evaluation criteria. Results show that (1) free-hand interaction in display areas close to the center of the body had a higher accuracy than that in peripheral-body areas; (2) free-hand interaction was asymmetric at the left side and the right side of the body. More specifically, left-hand interaction in the left-sided display area was more efficient and accurate than in the right-sided display area. For the right-hand interaction, the result was converse; moreover, (3) the dominant hand generated a higher interaction accuracy than the non-dominant hand. Lessons and strategies are discussed for designing user-friendly natural user interfaces in large displays-based interactive applications.

The effects of touchless interaction on usability and sense of presence in a virtual environment

For software applications with a significant level of user involvement, the traditional concept of usability has evolved into the more complex idea of user experience, which also covers emotional, cognitive or physical responses. In virtual reality, user experience also depends on the user perception related to some peculiarities of immersive environments, where also the devices employed for user interaction play a determinant role. This has led to the design of the Presence Questionnaire (PQ) for the evaluation of the effectiveness of virtual environments. This work analyzes the effects of two different interaction modalities on usability and sense of presence: in particular, the Myo armband, a gesture-based device for touchless interaction, is compared with the Vive handheld controller bundled with the HTC Vive headset. A total of 84 subjects were recruited to test the virtual environment and asked them to fill in a questionnaire obtained by combining the Usability Metric for User eXperience (UMUX) questionnaire, the System Usability Scale (SUS) and the presence questionnaire (PQ), which was specifically designed for virtual environments. A comparison between the scores obtained for the two interaction modalities revealed which questionnaire items are significantly influenced by the input interface and deduce some insights about the consequences on human factors.

A Siloxene/Ecoflex Nanocomposite‐Based Triboelectric Nanogenerator with Enhanced Charge Retention by MoS2/LIG for Self‐Powered Touchless Sensor Applications

AbstractThe contactless triboelectric nanogenerator with the capability of touchless interactions has become extremely fascinating for applications in health care, remote security, and augmented reality. Herein, a Siloxene/Ecoflex nanocomposite‐based high‐performance contactless triboelectric nanogenerator (TENG) is newly proposed that induces abundant surface charges owing to the strong electron affinity property of the Siloxene/Ecoflex. Furthermore, charge retention is one of the significant challenges in contactless TENG. For this, molybdenum disulfide (MoS2) incorporated laser‐induced graphene (LIG) is introduced as a charge trapping interlayer, which increases the surface potential by four‐fold, enhances the output performance of the TENG, and exhibits excellent charge trapping property which is supported by the surface potential measurements. In addition, the fabricated TENG exhibits excellent humidity sensing characteristics with a sensitivity of 0.45 V/% whereas the wear and tear of the device is prevented owing to the non‐contact mechanism of TENG. Finally, the TENG is demonstrated as a self‐powered sensor for touchless hand sanitizer applications as well as a wireless controller that transmits triggering signals for controlling the movement of the object in a gaming interface. Considering the ongoing coronavirus pandemic, the proposed contactless TENG is highly promising for touchless interactions and for reducing possible contact situations.

ROLE OF TOUCHLESS ECONOMY TECHNOLOGIES IN TOURISM DEVELOPMENT OF EUROPEAN UNION COUNTRIES IN COVID-19 PERIOD

The article examines the features of touchless economy technologies in tourism sphere in European Union countries in conditions of COVID-19 pandemic. The main definitions of the concept of "touchless economy" are considered. It has been found that a touchless economy is an economic activity that remains possible without close physical interaction between people. The current state of the European tourism market is analyzed and the negative consequences of COVID-19 on its development are assessed. The decrease of tourists number in the European Union countries, as well as the contribution of tourism to the GDP of the world leading countries on tourism market have been studied. It was found that the market for short-term accommodation suffered unevenly in Europe: countries such as Spain (-58.1%) and Italy (-60.2%) suffered more than France (-25.0%) or Germany (-20.6%). Research of 10 largest travel and tourism markets showed that Germany dropped one place from third position in the rankings, following a 46.9 % drop in the sector’s contribution towards GDP. Meanwhile, Italy rose one place from sixth to fifth, despite experiencing a 51% fall in GDP contribution. Notably, France, one of the key destinations globally in terms of international visitor arrivals, rose one position from seventh to sixth, despite its contribution to GDP falling by almost half (48.8%). The customer expectations to increase touchless interaction with enterprises in the period of COVID-19 in European countries are considered. It has been found that even after the pandemic, customers in Europe will still use touchless technologies such as face recognition, contactless interfaces, mobile applications and etc. Research of the spread of digital technologies in tourism across Europe has found that countries of Northern Europe have shown a greater spread of digital technologies than the countries of Eastern and Southern Europe. The main technologies of touchless economy in tourism are singled out and analyzed, such as: mobile applications for online check-in at hotels and airports, online delivery and QR-menus in restaurants, artificial intelligence of digital identity management, etc. The advantages of using the touchless economy technologies in the activities of tourism enterprises in the COVID-19 pandemic conditions are substantiated.

From luxury to necessity: Progress of touchless interaction technology

Touchless Technology is facilitating the move to Zero User Interface(UI) propelled by the COVID-19 pandemic which has accelerated the use of this technology due to hygiene requirements. Zero UI can be defined as a controlled interface that enables user interaction with technology through voice, gestures, hand interaction, eye tracking, and biometrics such as facial recognition and contactless fingerprints. Smart devices, IoT sensors, smart appliances, smart TVs, smart assistants and consumer robotics are predominant examples of devices in which Zero UI is becoming increasingly adopted. These control interfaces include natural interaction modes such as voice or gestures.Touchscreens and shared devices such as kiosks, self-service counters and interactive displays are present in our everyday lives. Each of these interactions however is a concern for consumers in a post-COVID-19 world where hygiene is of utmost importance.The one-stop solution to hygienic interactions includes touchless technology such as voice control, remote mobile screen take over, biometric, and gesture control as Zero User interfaces. With the breakthroughs in image recognition and natural language processing, powered by advanced computer vision and machine learning, “Zero UI” is becoming a new normal. This paper is focusing on the progress of the touchless interaction technology during the COVID-19 pandemic, which actually accelerated development in this concept and moved it from being a luxury to a life necessity.

Touchless InteractionUsing Magnetometer for Mobile Games

This paper proposes an alternative method for conducting touchless interaction. Using a magnetometer sensor, we attempt to map some common gestures into something that can be understood by a magnetometer sensor. To demonstrate the concept, an application and a punching game that utilize this principlewere developed. Using the application, usersareable to perform a common swiping gesture using a magnetometer sensor. When the gamedetects a punch gesture, it will respond accordingly. The application was tested and compared against an existing touchless interaction technology. The test results showed that there are some aspects where the magnetometer solution is better compared to the existing solution. This suggests that touchless interaction using magnetometersmight have a promising future.