User's Wrist Research Articles

Accurate Raycasting Selection With Rotation Gesture Using a 6-DOF Tracking Device.

A six degrees of freedom (6-DOF) controller is a commonly used input device in three-dimensional user interface (3DUI) applications. However, for the fundamental task of target selection in 3D space, the selection accuracy decreases owing to the Heisenberg effect during the manipulation of the 6-DOF controller. Based on the pointing action of a 6-DOF device, we establish the mathematical model for raycasting and analyze the possibility of using a rotation gesture to select a target. This study proposes a target selection method using the axial rotation of the user's wrist, which reduces the negative impact of a discrete input for triggering the selection on accuracy. The detection model can identify the start time of the user's rotation action. The custom designed control display gain (CD gain) function can maintain the ray's stability during the rotation gesture. This method was verified using a 6-DOF pen and Vive controller in three experiments. The results show that the accuracy of the proposed rotation gesture-based raycasting target selection method is superior to that of the traditional button-pressing method, and it can be integrated into existing tracking systems for further application.

How Positioning Wearable Haptic Interfaces on Limbs Influences Virtual Embodiment.

With increasing use of computer applications and robotic devices in our everyday life, and with the advent of metaverse, there is an urgent need of developing new types of interfaces that facilitate a more intuitive interaction in physical and virtual space. In this work, we investigate the influence of the location of haptic feedback devices on embodiment of virtual hands and user load during an interactive pick-and-place task. To do this, we conducted a user study with a 3x2 repeated measure experiment design: feedback position is varied between the distal phalanx of the index finger and the thumb, the proximal phalanx of the index finger and the thumb, and the wrist. These conditions of feedback are tested with the stimuli applied synchronously to the participant in one case, and with an additional delay of 350 ms in the second case. The results show that the location of the haptic feedback device does not affect embodiment, whereas the delay, i.e., whether the feedback is applied synchronously or asynchronously, affects embodiment. This suggests that for pick-and-place tasks, haptic feedback devices can be placed on the user's wrist without compromising performance making the hands to remain free, allowing unobstructed hand visibility for precise motion tracking, thereby improving accuracy.

An ML-Based Approach to Reconstruct Heart Rate from PPG in Presence of Motion Artifacts.

The heart rate (HR) is a widely used clinical variable that provides important information on a physical user's state. One of the most commonly used methods for ambulatory HR monitoring is photoplethysmography (PPG). The PPG signal retrieved from wearable devices positioned on the user's wrist can be corrupted when the user is performing tasks involving the motion of the arms, wrist, and fingers. In these cases, the obtained HR is altered as well. This problem increases when trying to monitor people with autism spectrum disorder (ASD), who are very reluctant to use foreign bodies, notably hindering the adequate attachment of the device to the user. This work presents a machine learning approach to reconstruct the user's HR signal using an own monitoring wristband especially developed for people with ASD. An experiment is carried out, with users performing different daily life activities in order to build a dataset with the measured signals from the monitoring wristband. From these data, an algorithm is applied to obtain a reliable HR value when these people are performing skill improvement activities where intensive wrist movement may corrupt the PPG.

Synthetic Smartwatch IMU Data Generation from In-the-wild ASL Videos

The scarcity of training data available for IMUs in wearables poses a serious challenge for IMU-based American Sign Language (ASL) recognition. In this paper, we ask the following question: can we "translate" the large number of publicly available, in-the-wild ASL videos to their corresponding IMU data? We answer this question by presenting a video to IMU translation framework (Vi2IMU) that takes as input user videos and estimates the IMU acceleration and gyro from the perspective of user's wrist. Vi2IMU consists of two modules, a wrist orientation estimation module that accounts for wrist rotations by carefully incorporating hand joint positions, and an acceleration and gyro prediction module, that leverages the orientation for transformation while capturing the contributions of hand movements and shape to produce realistic wrist acceleration and gyro data. We evaluate Vi2IMU by translating publicly available ASL videos to their corresponding wrist IMU data and train a gesture recognition model purely using the translated data. Our results show that the model using translated data performs reasonably well compared to the same model trained using measured IMU data.

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values.

The electromyogram (EMG) is a waveform representation of the action potential generated by muscle cells using electrodes. EMG acquired using surface electrodes is called surface EMG (sEMG), and it is the acquisition of muscle action potentials transmitted by volume conduction from the skin. Surface electrodes require disposable conductive gel or adhesive tape to be attached to the skin, which is costly to run, and the tape is hard on the skin when it is removed. Muscle activity can be evaluated by acquiring muscle potentials and analyzing quantitative, temporal, and frequency factors. It is also possible to evaluate muscle fatigue because the frequency of the EMG becomes lower as the muscle becomes fatigued. Research on human activity recognition from EMG signals has been actively conducted and applied to systems that support arm and hand functions. This paper proposes a method for recognizing the muscle activity state of the arm using pulse wave data (PPG: Photoplethysmography) and a method for estimating EMG using pulse wave data. This paper assumes that the PPG sensor is worn on the user's wrist to measure the heart rate. The user also attaches an elastic band to the upper arm, and when the user exerts a force on the arm, the muscles of the upper arm contract. The arteries are then constricted, and the pulse wave measured at the wrist becomes weak. From the change in the pulse wave, the muscle activity of the arm can be recognized and the number of action potentials of the muscle can be estimated. From the evaluation experiment with five subjects, three types of muscle activity were recognized with 80+%, and EMG was estimated with approximately 20% error rate.

Energy-efficient, fully flexible, high-performance tactile sensor based on piezotronic effect: Piezoelectric signal amplified with organic field-effect transistors

Abstract Fully flexible piezoelectric tactile sensors with low power consumption and high sensitivity play an important role in artificial intelligence, advanced manufacturing, and smart wearable devices. Herein, we constructed a high-performance, energy-efficient, and fully flexible piezoelectric tactile sensor based on piezotronic effect through the integration of piezoelectric materials with the mechanical-to-electrical conversion function of β polyvinylidene fluoride (PVDF) nanorod arrays and the signal amplification function of organic field-effect transistor (OFET) devices. PVDF nanorod arrays, which considerably improve the piezoelectric properties of materials, transform the external mechanical force into piezoelectric voltage to drive the OFET. Further, the piezoelectric voltage can be effectively amplified using an OFET to improve the sensitivity of the tactile sensor. Enabled by the unique structure of sensing devices and well-defined active materials, the fabricated tactile sensor exhibited excellent pressure sensitivity, detection limit, and response time of 5.17 kPa⁻1, 175 Pa, and 150 ms, respectively. In addition, using a sophisticated fabrication process, we demonstrated a flexible integrated tactile sensor array with a 3 × 3 cell on a polyethylene terephthalate (PET) substrate to detect the bending angle of the user's wrist on which the fabricated device was mounted. This study uses piezotronic transistors to convert pressure into electrical signals without applying a gate voltage, which can substantially simplify circuit models and sensor distribution; additionally, the fabricated device provides technical support for flexible electronic skin (e-skin).

Mobile-based Human Emotion Recognition based on Speech and Heart rate

Mobile-based human emotion recognition is very challenging subject, most of the approaches suggested and built in this field utilized various contexts that can be derived from the external sensors and the smartphone, but these approaches suffer from different obstacles and challenges. The proposed system integrated human speech signal and heart rate, in one system, to leverage the accuracy of the human emotion recognition. The proposed system is designed to recognize four human emotions; angry, happy, sad and normal. In this system, the smartphone is used to record user speech and send it to a server. The smartwatch, fixed on user wrist, is used to measure user heart rate while the user is speaking and send it, via Bluetooth, to the smartphone which in turn sends it to the server. At the server side, the speech features are extracted from the speech signal to be classified by neural network. To minimize the misclassification of the neural network, the user heart rate measurement is used to direct the extracted speech features to either excited (angry and happy) neural network or to the calm (sad and normal) neural network. In spite of the challenges associated with the system, the system achieved 96.49% for known speakers and 79.05% for unknown speakers

ALeak: Context-Free Side-Channel from Your Smart Watch Leaks Your Typing Privacy

We revisit a crucial privacy problem in this paper — can the sensitive information, like the numeric passwords and personal data, frequently typed by user on mobile devices be inferred through the motion sensors of wearable device on user's wrist, e.g., smart watch or wrist band? Existing works have achieved the initial success under certain context-aware conditions, such as 1) the horizontal keypad plane, 2) the known keyboard size, and/or 3) the last keystroke on a fixed “enter” button. Taking one step further, the key contribution of this paper is to fully demonstrate, more importantly alarm people, the further risks of typing privacy leakage in much more generalized context-free scenarios, which are related to most of us for the daily usage of mobile devices. We validate this feasibility by addressing a series of unsolved challenges and developing a prototype system aLeak . Extensive experiments show the efficacy of aLeak , which achieves promising successful rates in the attack from more than 500 rounds of different users’ typings on various mobile platforms without any context-related information.

Investigations on collaborative remote control of virtual robotic manipulators by using a Kinect v2 sensor

The present work deals with design aspects and testing results of a novel solution regarding the remote control of two virtual robotic manipulators by using hands motion tracking and palms gesture recognition. The spatial position and orientation of the user's left and right hands' wrists are interpreted by a custom built software application and then forwarded as positioning data and gripper status to a V-Rep server running the simulation scene. The goal of the presented setup is for the user to be able to manipulate objects in the scene in a collaborative way, i.e. pick one object with the first robot and give it to the second one for elsewhere depositing. The real-life applications for this approach are to provide a framework for simplified operator training and programming of robots used in pick-and-place or assembly operations, and also in healthcare for assisting patients in hand -coordination and mobility recovery.

SAW

Token-based proximity authentication methods that authenticate users based on physical proximity are effortless, but lack explicit user intentionality, which may result in accidental logins. For example, a user may get logged in when she is near a computer or just passing by, even if she does not intend to use that computer. Lack of user intentionality in proximity-based methods makes them less suitable for multi-user shared computer environments, despite their desired usability benefits over passwords. We present an authentication method for desktops called Seamless Authentication using Wristbands (SAW), which addresses the lack of intentionality limitation of proximity-based methods. SAW uses a low-effort user input step for explicitly conveying user intentionality, while keeping the overall usability of the method better than password-based methods. In SAW, a user wears a wristband that acts as the user's identity token, and to authenticate to a desktop, the user provides a low-effort input by tapping a key on the keyboard multiple times or wiggling the mouse with the wristband hand. This input to the desktop conveys that someone wishes to log in to the desktop, and SAW verifies the user who wishes to log in by confirming the user's proximity and correlating the received keyboard or mouse inputs with the user's wrist movement, as measured by the wristband. In our feasibility user study (n=17), SAW proved quick to authenticate (within two seconds), with a low false-negative rate of 2.5% and worst-case false-positive rate of 1.8%. In our user perception study (n=16), a majority of the participants rated it as more usable than passwords.

ForceHand Glove: A Wearable Force-Feedback Glove With Pneumatic Artificial Muscles (PAMs)

The upper limb of the human body has a higher likeliness of getting affected by muscle fatigue and injury, due to repetitive motion. This letter presents an interactive glove for the wrist of the human body. The assistive model is enabled with pneumatic actuators and stretch sensors, which support the user in performing wrist flexion, extension, pronation, and supination. The purpose of this glove is to provide the subject with a force-feedback feeling, and while a repetitive practice of such training is done, rehabilitation can be achieved. For evaluation, surface electromyographical signals were first used to study the effect of force feedback on associated muscle activation. Average % maximum voluntary isometric contraction of all subjects showed statistically reduced muscle activity in at least one associated muscle while performing all motions, enabled with actuators. Second, the force feedback generated by the glove at the user's wrist has been modeled using experiments with a force transducer. An application was then developed where the user was given the ability to interact with virtual objects and receive feedback through the glove. The evaluation of the feasibility of this application was done by measuring and comparing force profiles while generating an equal force in an ideal and a predicted environment.

Path-Following Guidance Using Phantom Sensation Based Vibrotactile Cues Around the Wrist

Vibrotactile feedback systems for supporting human motion have been widely researched, mainly owing to the low price and reduced size of vibration motors, which allow trainees to actively move while using them. Most existing vibrotactile feedback systems focus on providing information about the human's joint angles, making it necessary to understand multiple simultaneous stimuli to guide the trajectory of a limb. Instead, in the present study, we focus on guiding the motion of the hand by following an endpoint approach and propose a vibrotactile feedback paradigm to convey a direction around the wrist and guide the user's wrist around points in a two-dimensional Cartesian space. In the path-following task, vibrotactile cues are provided as direction information to modify the wrist's path only when the wrist's position deviates significantly from the desired path. The experiment was designed based on the motor learning process to observe the evolution of performance during training. We found that the root mean squared error of participants decreased by 49.3% when provided with vibrotactile feedback. This effect was almost constant throughout training. Additionally, most participants could reproduce the desired path after removing the provided feedback immediately after training. We believe the proposed system can be applied to enhance the process of learning tasks that require hand guidance, such as learning of Japanese calligraphy or upper limb rehabilitation.

Microsoft's Kinect Technology: A Bust That Could Still Become a Boom

Microsoft's Kinect wrist-worn sensor technology is capable of interpreting a user's wrist, palm, and finger motions [1] to control a computer or console. The infrared laser device sends messages to mobile phones and iPads to decode human body movements, resulting in the manipulation of digital files based solely on gestures [2]. This motion-controlled computing has the potential to reduce redundancies and workload blockages in offices.

A Phantom-Sensation Based Paradigm for Continuous Vibrotactile Wrist Guidance in Two-Dimensional Space

Vibrotactile feedback has long been used to improve the human motor learning process through information transfer. However, in most of the approaches described in the literature, only the joint angles of the limb are guided. To guide motions that involve multiple degrees of freedom (DOFs), the user needs to interpret multiple simultaneous vibrotactile cues (at least one cue per joint), which is difficult. In this letter, we guide the position of a user's wrist in space by producing a vibrotactile cue at any place around the wrist by using a vibrotactile illusion known as “Phantom Sensation” (PS). In a user study that involved 10 male participants, the vibrotactile cues produced by PS were reasonably accurately located with an average error of $\boldsymbol{7.9^\circ }$ . To maintain consistency between the vibrotactile cue location and the global frame, we measured the user's wrist rotation and adjusted the position of the vibrotactile cue accordingly. When rotating their hand, the participants recognized the location of the vibrotactile cue with an average error of $\boldsymbol{9.3^\circ }$ . By using the proposed feedback paradigm, we guided users’ wrists in space under two conceptual mappings (“push” and “pull”). This experiment examined the participants’ motion path efficiency (ME), which measures the directness with which the user's wrist moves toward the desired position. The participants reached the desired positions under the vibrotactile feedback alone. The average MEs of the “push” and “pull” mappings were 44.3% and 52.4%, respectively. We will apply the proposed method under the “pull” mapping to guide real life tasks, such as book classification or rehabilitation.

Sensory integration of apparent motion speed and vibration magnitude.

Tactile apparent motion can display directional information in an intuitive way. It can for example be used to give directions to visually impaired individuals, or for waypoint navigation while cycling on busy streets, when vision or audition should not be loaded further. However, although humans can detect very short tactile patterns, discriminating between similar motion speeds has been shown to be difficult. Here we develop and investigate a method where the speed of tactile apparent motion around the user's wrist is coupled with vibration magnitude. This redundant coupling is used to produce tactile patterns from slow&weak to fast&strong. We compared the just noticeable difference (JND) of the coupled and the individual variables. The results show that the perception of the coupled variable can be characterised by JND smaller than JNDs of the individual variables. This allowed us to create short tactile pattens (tactons) for display of direction and speed, which can be distinguished significantly better than tactons based on motion alone. Additionally, most subjects were also able to identify the coupled-variable tactons better than the magnitude-based tactons.

Development of Low Cost Supernumerary Robotic Fingers as an Assistive Device

This paper presents the development of new type of wearable robot namely Supernumerary Robotic Finger (SRF) as an assistive robot for healthy people or people with hemiparesis or hemiplegia. SRF comprises of two manipulators attached in user's wrist. Three flex sensors are utilized to measure the finger bending of the user's finger. The posture of SRF is driven by modified glove sensor. The kinematics of both robotic thumb (RT) and robotic finger (RF) is studied using D-H parameter method and RoboAnalyzer software in order to understand the kinematic behavior of this robot. Each of RT and RF has three degrees of freedom (DOF). The posture of RT and RF is controlled using bending angles of thumb and finger from the user that are read by flex sensor. Based on the experimental results for people with healthy hand, the proposed SRF can assist object manipulation task in grasping, holding, and manipulating an object by using single hand when normally it only can be done by using two hands. From the experimental results on a person with healthy hand, the proposed of SRF can be employed as an assistive device for people with hemiparesis or hemiplegia. This device will enable people with diminished hand function work more independently.

3D 스캐너와 3D 프린터를 활용한 손목보호대 개발

The purpose of this study was to develop a customized wrist brace using 3D scanner and 3D printer. This study included in-depth interviews with people who had wrist pain and ever used wrist braces. The wrist brace has been designed and modeled by the CAD program after 3D scanning the wrist of subjects. Based on the results of the in-depth interviews on wrist brace users and design investigation in the market, a prototype of the wrist brace has been created. The wrist brace does not compress the hand and is easy to put on. In addition, it is adjustable to the wrist of users, allowing them to move his or her wrist without any restrictions. A computer-modeling program produced solid files for the design of the wrist brace after 3D scanning ten subjects' hands. It features a lattice-patterned surface, a velcro adjustment and trimming to smoothen the surface. PLA filaments were used to 3D print the wrist brace. As for the assessment, the wrist brace has been evaluated to be easy to put on and adjust to user's wrist. However, when it comes to appearance and comfortability, it was rated 3.0 or slightly above. This is because 3D printing materials are firm and not flexible compared to other materials such as neoprene or stretchable bands.

SeleCon: Scalable IoT Device Selection and Control Using Hand Gestures.

Although different interaction modalities have been proposed in the field of human-computer interface (HCI), only a few of these techniques could reach the end users because of scalability and usability issues. Given the popularity and the growing number of IoT devices, selecting one out of many devices becomes a hurdle in a typical smarthome environment. Therefore, an easy-to-learn, scalable, and non-intrusive interaction modality has to be explored. In this paper, we propose a pointing approach to interact with devices, as pointing is arguably a natural way for device selection. We introduce SeleCon for device selection and control which uses an ultra-wideband (UWB) equipped smartwatch. To interact with a device in our system, people can point to the device to select it then draw a hand gesture in the air to specify a control action. To this end, SeleCon employs inertial sensors for pointing gesture detection and a UWB transceiver for identifying the selected device from ranging measurements. Furthermore, SeleCon supports an alphabet of gestures that can be used for controlling the selected devices. We performed our experiment in a 9m-by-10m lab space with eight deployed devices. The results demonstrate that SeleCon can achieve 84.5% accuracy for device selection and 97% accuracy for hand gesture recognition. We also show that SeleCon is power efficient to sustain daily use by turning off the UWB transceiver, when a user's wrist is stationary.

ShakeUnlock: Securely Transfer Authentication States Between Mobile Devices

As users start carrying multiple mobile devices, we propose a novel, token based mobile device unlocking approach. Mobile devices are conjointly shaken to transfer the authentication state from an unlocked token device to another device to unlock it. A common use case features a wrist watch as token device, which remains unlocked as long as it is strapped to the user's wrist, and a locked mobile phone, which is unlocked if both devices are shaken conjointly. Shaking can be done single-handedly, requires little user attention (users don't have to look at the device for unlocking it), and does not cause additional cognitive load on users. In case attackers gain control over the locked phone, forging shaking is difficult, which impedes malicious unlocks. We evaluate our approach using acceleration records from our 29 people sized ShakeUnlock database and discuss influence of its constituent parts on the system performance. We further present a performance study using an Android implementation and live data, which shows the true negative rate of observational attacks to be in the range of 0.8—if an attacker manages to gain control over the locked device and shake it in parallel to the device owner shaking the token device.

複数の振動刺激を用いた運動方向呈示による手首の運動誘導

We propose a novel method for motion guidance by creating a motion direction by using multiple vibration stimulations. In order to create the motion direction, we utilize a vibratactile illusion called Phantom Sensation. In this method, moving vibration stimulation on the user's skin, which is generated by controlled the position of Phantom Sensation, creates the motion direction. In this paper, we guide the user's wrist to the desired position using our proposal method. We control the produced direction and transmission time by the relationship between current position and desired position. We recognize that they can arrive at a desired position to move their wrist obeying the produced direction.