Virtual Buttons Research Articles

Intrinsic sense of touch for intuitive physical human-robot interaction.

The sense of touch is a property that allows humans to interact delicately with their physical environment. This article reports on a technological advancement in intuitive human-robot interaction that enables an intrinsic robotic sense of touch without the use of artificial skin or tactile instrumentation. On the basis of high-resolution joint-force-torque sensing in a redundant arrangement, we were able to let the robot sensitively feel the surrounding environment and accurately localize touch trajectories in space and time that were applied on its surface by a human. Through an intertwined combination of manifold learning techniques and artificial neural networks, the robot identified and interpreted those touch trajectories as machine-readable letters, symbols, or numbers. This opens up unexplored opportunities in terms of intuitive and flexible interaction between human and robot. Furthermore, we showed that our concept of so-called virtual buttons can be used to straightforwardly implement a tactile communication link, including switches and slider bars, which are complementary to speech, hardware buttons, and control panels. These interaction elements could be freely placed, moved, and configured in arbitrary locations on the robot structure. The intrinsic sense of touch we proposed in this work can serve as the basis for an advanced category of physical human-robot interaction that has not been possible yet, enabling a shift from conventional modalities toward adaptability, flexibility, and intuitive handling.

Exploring the Affordance Effect of Visual Properties Associated with Virtual Keyboard Buttons on Smartphones

Digital affordances exist on the touchscreen, but the affordance of virtual buttons on smartphones is unknown. Therefore, this study investigated the on-screen affordances by examining whether the visual properties of virtual keyboard buttons could afford possible clicking actions and affect the entry performance. This study employed a three-way within-subject design: 2 (Symbol Position: standard mode vs. corrective mode) × 2 (Button Border: bordered button vs. borderless button) × 26 (Button Position: 26 positions). In the serial tapping task, 30 university students tapped virtual keyboard buttons with four button types. The results confirmed that the symbol-position and button-border variables influenced clicking actions. When employing offset-correcting symbols, the entry accuracy of virtual keyboard buttons increased without the speed-accuracy tradeoff. Additionally, button borders improved the perceived usability of virtual keyboard buttons despite failing to enhance the entry accuracy and speed. However, the button position moderated the effects of the symbol-position and button-border variables.

Validation of a Soft Pneumatic Unit Cell (PUC) in a VR Experience: A Comparison Between Vibrotactile and Soft Pneumatic Haptic Feedback.

Soft pneumatic displays have shown to provide compelling soft haptic feedback. However, they have rarely been tested in Virtual Reality applications, while we are interested in their potential for haptic feedback in the metaverse. Therefore, we designed a fully soft Pneumatic Unit Cell (PUC) and implemented it in a VR button task, in which users could directly use their hands for interaction. Twelve participants were asked to enter six-digit sequences, while being presented with PUC feedback, vibration feedback (VT), or no haptic feedback. Metrics on task performance, kinematics and cognitive load were collected. The results show that both vibration and PUC feedback resulted in participants pressing through the back of buttons less. The kinematic data showed that participants moved more smoothly during PUC feedback compared to vibration feedback. These effects were also reflected in the questionnaire data: participants felt more successful when using either PUCs or VTs, but they perceived the lowest level of stress when using PUCs. Feedback preference ratings also showed that PUC was the most preferred kind of feedback. Concluding, our array of metrics confirm that PUCs are good alternatives for haptic feedback in VR tasks in which electromechanical vibration motors typically excel: creating virtual button clicks.

Substitute Buttons: Exploring Tactile Perception of Physical Buttons for Use as Haptic Proxies

Buttons are everywhere and are one of the most common interaction elements in both physical and digital interfaces. While virtual buttons offer versatility, enhancing them with realistic haptic feedback is challenging. Achieving this requires a comprehensive understanding of the tactile perception of physical buttons and their transferability to virtual counterparts. This research investigates tactile perception concerning button attributes such as shape, size, and roundness and their potential generalization across diverse button types. In our study, participants interacted with each of the 36 buttons in our search space and provided a response to which one they thought they were touching. The findings were used to establish six substitute buttons capable of effectively emulating tactile experiences across various buttons. In a second study, these substitute buttons were validated against virtual buttons in VR. Highlighting the potential use of the substitute buttons as haptic proxies for applications such as encountered-type haptics.

PopTouch: A Submillimeter Thick Dynamically Reconfigured Haptic Interface with Pressable Buttons.

The interactions with touchscreens rely heavily on vision: The virtual buttons and virtual sliders on a touchscreen provide no mechanical sense of the object they seek to represent. This work presents PopTouch: a 500µm thick flexible haptic display that creates pressable physical buttons on demand. PopTouch can be mounted directly on touchscreens or any other smooth surface, flat, or curved. The buttons of PopTouch are independently controlled hydraulically amplified electrostatic zipping taxels (tactile pixels) that generate 1.5mm of out of plane displacement. When pressed by the user, the buttons provide intuitive mechanical feedback thanks to a snap-through characteristic in their force-displacement profile. The snap-through threshold can be as high as 4 N, and is tuned by design and actuation parameters. This work presents two versions of PopTouch: a transparent PopTouch for integration on Touchscreens with built-in touch sensing, such as smartphones and a sensorized PopTouch, with embedded thin-film piezoelectric sensors on each taxel, for integration on substrates without built-in touch sensing, such as a steering wheel. PopTouch adds static and vibrating button-like haptics to any device thanks to its thin profile, flexibility, low power consumption (6mW per button), rapid refresh rate (2Hz), and freely configured array format.

Temporal Detection Threshold of Audio-Tactile Delays With Virtual Button.

Synchronization of audio-tactile stimuli represents a key feature of multisensory interactions. However, information on stimuli synchronization remains scarce, especially with virtual buttons. This work used a click sensation produced with traveling waves and auditory stimulus (a bip-like sound) related to a virtual click for a psychological experiment. Participants accomplish a click gesture and judge if the two stimuli were synchronous or asynchronous. Delay injection was performed on the audio (haptic first) or the click (audio first). In both sessions, one stimulus follows the other with a delay ranging from 0-700 ms. We use weighted and transformed 3-up/1-down staircase procedures to estimate people's sensitivity. We found a threshold of 179 ms and 451 ms for the auditory first and haptic first conditions, respectively. Statistical analysis revealed a significant effect between the two stimuli' order for threshold. Participants' acceptable asynchrony decreased when the delay was on the haptic rather than on the audio. This effect could be due to the natural experience in which the stimuli tend to be first tactile and then sonorous rather than the other way around. Our findings will help designers to create multimodal virtual buttons by managing audio-tactile temporal synchronization.

Effects of Interaction Method, Size, and Distance to Object on Augmented Reality Interfaces

ABSTRACT Augmented reality (AR) technologies are becoming increasingly popular. However, studies regarding the usability of AR interfaces are scarce. This study aims to derive the optimal usability design of the AR interface by analyzing the performance of 12 interface conditions (2 button sizes × 3 distances × 2 interaction types). An experiment was conducted using Microsoft HoloLens, in which participants performed button selection tasks with an array of 3 × 3 virtual buttons. The task completion time and the number of errors were measured along with a subjective satisfaction score. The results showed that a clicker-based interaction was more effective in task completion time, the number of errors and user satisfaction than a hand gesture. Furthermore, large buttons required a significantly shorter task completion time than small buttons. There was no significant difference in the number of errors between the two sizes. A distance of at least 80 cm from the eye to the virtual object is most favorable for a good performance. There was a significant difference in the task completion times between 40 cm and 80 cm and between 40 cm and 120 cm. These results can aid the interface designs for AR applications and devices.

Virtual Button Using IOT

Virtual buttons are a way to control the Internet of things (IoT) bone through a user interface that can be entered anywise. The buttons can be represented as symbols on a web runner or mobile app, and they can be used by the user to perform specific conduct analogous to turning on a light or conforming to the temperature. By using virtual buttons, stoners can interact with IoT bias without having to physically touch the device, making it more accessible. The buttons can also be programmed to perform multiple conducts with a single click, simplifying the user experience and making it easier to control IoT bias. Augment reality increases the associated control of the bias and appliances controlling them currently it's possible to control the appliances ever with the help of smart bias. Numerous styles are used to control the appliances ever but it becomes delicate for the stoner to use the smart bias because of the graphical stoner interface. For making this easier we use stoked reality technology for an easy way to control home appliances connected by the home network. Augmented reality technology can give virtual plates and apply fresh information to specific areas through a camera display.

Информационно-управляющая система процесса пиролиза пропанона

The paper presents a solution to the problem of developing an information and control system for the process of propanone pyrolysis, which is the main source for the production of acetylketene, which is widely used in the synthesis of medicines and food additives, insecticides and fungicides, and paints and varnishes. A mathematical model of the propanone pyrolysis process in tube furnace coils has been developed, which takes into account the heat transfer in the reaction zone of the pyro coil by means of radiation from the walls of red-hot mines. The problem of identifying the kinetic parameters of the pyrolysis model on the basis of experimental data obtained at the facility was posed and solved. The problem of static optimization of the propanone pyrolysis process, which consists in maximizing the selectivity of the process, was posed and solved. The software package is made in C#, which is supported by many SCADA systems. The Hook-Jeeves configuration method was used as an optimizer in combination with the penalty function method. The technical, informational and software support for the information and control system of the process has been developed. To develop an information management system, SCADA software was used - RSView32 systems. It allows you to implement the visualization, accumulation and archiving of technological parameters. The main form of information presentation is the mnemonic diagram of the pyrolysis process, the other forms are called from the mnemonic diagram by pressing the corresponding virtual buttons.

Building English Audio-Visual and Oral Mobile Teaching System Based on Virtual Augmented Reality Technology

Aiming at the problem that the current English textbooks still provide video materials in the form of CD-ROMs, which affects the learning effect, a research method of English audio-visual mobile teaching system based on virtual and augmented reality technology is proposed. The system first builds a recognition map database and stores it in the cloud and names the corresponding video files according to the name of the recognition map, then uses Unity3D to design and render the scene, design the virtual video playback button of the ImageTarget object, and write script code to realize the recognition map database and its corresponding video, access, and finally, generate a user-friendly mobile application. Users only need to point the lens at the book illustration to present the visual effect of superposition of virtual and real and realize the playback of English teaching videos on mobile devices. The results show that compared with the original image grayscale distribution map, the pixels near the 0 value of the pixel distribution map after twice filtering are significantly reduced, which reduces the hole noise of the image, and the original image and the image after twice filtering are not 0. The peak value of the pixel is close, maintaining the detailed characteristics of the image. The application of augmented reality technology to English video teaching enables users to enjoy novel learning methods and an interactive experience combining virtual and real.

Multisensory Pseudo‐Haptics for Rendering Manual Interactions with Virtual Objects

Recent advances in extended reality (XR) technologies make seeing and hearing virtual objects commonplace, yet strategies for synthesizing haptic interactions with virtual objects continue to be limited. Two design principles govern the rendering of believable and intuitive haptic feedback: movement through open space must feel “free” while contact with virtual objects must feel stiff. Herein, a novel multisensory approach that conveys proprioception and effort through illusory visual feedback and refers to the wrist, via a bracelet interface, discrete and continuous interaction forces that would otherwise occur at the hands and fingertips, is presented. Results demonstrate that users reliably discriminate the stiffness of virtual buttons when provided with multisensory pseudohaptic feedback, comprising tactile pseudohaptic feedback (discrete vibrotactile feedback and continuous squeeze cues in a bracelet interface) and visual pseudohaptic illusions of touch interactions. Compared to the use of tactile or visual pseudohaptic feedback alone, multisensory pseudohaptic feedback expands the range of physical stiffnesses that are intuitively associated with the rendered virtual interactions and reduces individual differences in physical‐to‐virtual stiffness mappings. This multisensory approach, which leaves users' hands unencumbered, provides a flexible framework for synthesizing a wide array of touch‐enabled interactions in XR, with great potential for enhancing user experiences.

Tactile motor attention induces sensory attenuation for sounds

Sensory events appear reduced in intensity when we actively produce them. Here, we investigated sensory attenuation in a virtual reality setup that allowed us to manipulate the time of tactile feedback when pressing a virtual button. We asked whether tactile motor attention might shift to the tactile location that makes contact with the button. In experiment one, we found that a tactile impulse was perceived as more intense when button pressing. In a second experiment, participants pushed a button and estimated the intensity of sounds. We found sensory attenuation for sounds only when tactile feedback was provided at the time the movement goal was reached. These data indicate that attentional prioritization for the tactile modality during a goal-directed hand movement might lead to a transient reduction in sensitivity in other modalities, resulting in sensory attenuation for sounds.

사용자 경험 향상을 위한 딥러닝 기반 차량용 AR 매뉴얼

This paper implements an AR manual for a vehicle that can be used even in the vehicle interior space where it is difficult to apply the augmentation method of AR content, which is mainly used, and applies a deep learning model to improve the augmentation matching between real space and virtual objects. Through deep learning, the logo of the steering wheel is recognized regardless of the position, angle, and inclination, and 3D interior space coordinates are generated based on this, and the virtual button is precisely augmented on the actual vehicle parts. Based on the same learning model, the function to recognize the main warning light symbols of the vehicle is also implemented to increase the functionality and usability as an AR manual for vehicles.

The Implementation of NodeMCU ESP8266 for Smart Lamp in the Cilacap State Polytechnic Campus Area

Lamps are household electrical devices. In general, the lights are controlled through switches which require the user to move towards the switch to turn the lights on/off. To save time, lights can be controlled via the internet of things using the NodeMcu ESP8266. The purpose of making a smart lamp using the NodeMCU ESP8266 is to make it easier to turn it on and off with a smartphone without the need to look for a switch. This smart lamp can be controlled via a virtual button on the Blynk application or based on a set clock. The results obtained from the design of this tool are that the smart lamp will turn on at 8.00 AM and will turn off automatically at 3.00 PM, or by pressing the virtual button on Blynk. There is a slight delay when the lights will turn on or off, this is caused by the strength of the internet from the wifi connected to the ESP8266.

Perceived Depth and Roughness of Virtual Buttons With Touchscreens.

With the rapidly increasing penetration of touchscreens in various application sectors, more sophisticated and configurable haptic effects can be rendered on touchscreens (e.g., buttons). In this paper, we presented a design process to instantiate a wide range of vibrotactile stimuli for rendering various virtual buttons on touchscreens. We study the perceived depth and roughness of rendered virtual buttons. There are two stages: the design of the drive signals and the main study. We generated and screened drive signals to render vibrotactile stimuli for virtual buttons through varying envelope shapes, superposition methods, compound waveform composition (CWC) types, durations, and frequencies. The results show that the perceived depth of virtual buttons can be very deep, and the perceived roughness can be very rough around the resonant frequency. Perceived depth and roughness decrease when the frequency increases or decreases from the resonant frequency. A longer duration of vibrotactile stimuli and adding pulse numbers could increase the perceived depth and roughness. Perceived depth and roughness have a similar trend with varying frequencies at a fixed duration.

PLASTIC: Pressure and location-based acoustic sensing touch interface using classification

A method is described for developing a pressure sensor panel that uses a single small electromechanical exciter, delivering a sawtooth wave, and a single sensor attached to a panel or structure. A trained classifier is able to recognize the pressure and location of a finger-sized object pressing on the panel based on the characteristics of the waveform detected at the sensor. The classification results of three experiments are reported: 55 virtual buttons on a panel, the pressure classification sensitivity of a single virtual button, and the location classification resolution of a tightly packed grid of points. All three experiments show excellent accuracy (99.7%, 85.2%, 96.0%, respectively) using a simple trained linear SVM classifier with a 70/30 training/validation data ratio, demonstrating that the PLASTIC method is an effective method for classifying the location and pressure of a touch on a panel or structure.

A Prototype Design of Smart Quranic Garden System using ATmega328 and ESP8266 Microcontrollers

Medicinal plants are one of the leading plants cultivated by farmers in Parongpong District, West Bandung Regency. There are so many benefits of medicinal plants, especially the bidara plant which is the main commodity of a farmer group called the Quran Garden Foundation. The vision of this bidara cultivation is in addition to a health function, as well as an effort to implement the sunnah in Islam. Therefore, the authors initiated the Smart Quranic Garden System prototype which is based on green house system automation technology and Internet of Things (IoT) based security using the ATmega328 and ESP8266 microcontrollers. This research was conducted using experimental methods, namely design and testing. The system built on this prototype includes watering, weather and security. Based on the test results, the pump on the watering system works when the soil moisture sensor detects soil moisture at an ADC value of more than or equal to 700. Whereas in the weather system, the heater will work when DHT11 detects a temperature below 25oC and the blower will work when a temperature is detected above 30oC. . The security system also responds according to the program that is set, namely the relay for the solenoid lock door will open only when the password is correct and the virtual button on Blynk is pressed. It is hoped that the system in this prototype can be implemented so that the growth height of the bidara plant in Parongpong district can be optimized.

Edukasi Pengenalan Huruf Hijaiyah dengan Memanfaatkan Teknologi Augmented Reality

Augmented Reality (AR) technology provides opportunities for science and engineering. AR also has great opportunities in the world of education, which is to provide and display additional information in the form of 3D objects, video, sound, and text on an object. Smartphone software developers have developed Augmented reality technology that was previously developed on PC devices where this technology utilizes the existing camera on a smartphone. With this situation, AR technology has the opportunity to be used in the development of a media for recognizing hijaiyah letters, so that children will be happier learning because of its attractive appearance and teachers or parents can more easily teach lessons to their children. The way to use it is as follows: first, the user puts a registered and printed marker, second, the smartphone camera identifies (tracking) the marker. If the marker is invalid, the user repeats the identification process. If the marker is valid and identified, the marker will display the hijaiyah letter object in three-dimensional form. Third, users can understand the shape and pronunciation of hijaiyah letters by touching the virtual button on the marker

Eyes-Free Tongue Gesture and Tongue Joystick Control of a Five DOF Upper-Limb Exoskeleton for Severely Disabled Individuals.

Spinal cord injury can leave the affected individual severely disabled with a low level of independence and quality of life. Assistive upper-limb exoskeletons are one of the solutions that can enable an individual with tetraplegia (paralysis in both arms and legs) to perform simple activities of daily living by mobilizing the arm. Providing an efficient user interface that can provide full continuous control of such a device—safely and intuitively—with multiple degrees of freedom (DOFs) still remains a challenge. In this study, a control interface for an assistive upper-limb exoskeleton with five DOFs based on an intraoral tongue-computer interface (ITCI) for individuals with tetraplegia was proposed. Furthermore, we evaluated eyes-free use of the ITCI for the first time and compared two tongue-operated control methods, one based on tongue gestures and the other based on dynamic virtual buttons and a joystick-like control. Ten able-bodied participants tongue controlled the exoskeleton for a drinking task with and without visual feedback on a screen in three experimental sessions. As a baseline, the participants performed the drinking task with a standard gamepad. The results showed that it was possible to control the exoskeleton with the tongue even without visual feedback and to perform the drinking task at 65.1% of the speed of the gamepad. In a clinical case study, an individual with tetraplegia further succeeded to fully control the exoskeleton and perform the drinking task only 5.6% slower than the able-bodied group. This study demonstrated the first single-modal control interface that can enable individuals with complete tetraplegia to fully and continuously control a five-DOF upper limb exoskeleton and perform a drinking task after only 2 h of training. The interface was used both with and without visual feedback.

Communication in History: The Key to Understanding

Communication in History: The Key to Understanding