User's Fingertip Research Articles

Exploring the impact of the passenger's display on driver workload and driving performance

New production vehicles equipped with multiple screens have various functions and technologies at the user's fingertips that create informative and immersive cockpits not only for the driver but also for the passenger. Despite the growing popularity of the co-pilot display, surprisingly little is known about the impact of its engagement on drivers. Engagement, in the context of this paper, is defined as visual, auditory, and mental involvement but no manual involvement. The present study sought to investigate the effects of co-pilot display engagement on drivers' visual workload, cognitive workload, and driving performance and offered a deeper insight into the design factors and task involvement of the co-pilot display. The study involved a driving simulator testing of 25 participants in five automobiles with a co-pilot display. Our analysis found increased driver visual workload and cognitive workload in the conditions of co-pilot display engagement. Participants exhibited more erratic driving behavior (standard deviation of speed and standard deviation of lane position) while the front passenger was watching a video. Furthermore, five linear mixed models were built to further analyze the effects of task type, screen size, screen layout, and task involvement on dependent measures. Potential applications of this research include refinements to the co-pilot display and workload estimation of co-pilot display engagement.

Computer Vision based Virtual Illustration using OpenCV

Abstract: During the COVID-19 epidemic, it has been extremely difficult to engage kids in the classroom via an online platform. Children needed a classroom that was free of dust as a result. This article describes a novel paint application that recognises hand movements and monitors hand joints using MediaPipe and OpenCV. With the use of hand gestures, this programme enables people to interact with computers in a natural way. The main objective of HCI is to enhance human-computer interaction. This study put out a method for writing in the air with your fingertips without gloves or sensors. Air writing enables us to write characters in open spaces and draw shapes with a specific colour on the fingertip. The colour marker is applied to the user's fingertip and aids the camera in recognising hand motions

HaptoMapping: Visuo-Haptic Augmented Reality by Embedding User-Imperceptible Tactile Display Control Signals in a Projected Image.

This paper proposes HaptoMapping, a projection-based visuo-haptic augmented reality (VHAR) system, that can render visual and haptic content independently and present consistent visuo-haptic sensations on physical surfaces. HaptoMapping controls wearable haptic displays by embedded control signals that are imperceptible to the user in projected images using a pixel-level visible light communication technique. The prototype system is comprised of a high-speed projector and three types of haptic devicesfinger worn, stylus, and arm mounted. The finger-worn and stylus devices present vibrotactile sensations to a users fingertips. The arm-mounted device presents stroking sensations on a users forearm using arrayed actuators with a synchronized hand projection mapping. We identified that the developed systems maximum latency of haptic from visual sensations was 93.4 ms. We conducted user studies on the latency perception of our VHAR system. The results revealed that the developed haptic devices can present haptic sensations without user-perceivable latencies, and the visual-haptic latency tolerance of our VHAR system was 100, 159, 500 ms for the finger-worn, stylus, and arm-mounted devices, respectively. Another user study with the arm-mounted device discovered that the visuo-haptic stroking system maintained both continuity and pleasantness when the spacing between each substrate was relatively sparse, such as 20 mm, and significantly improved both the continuity and pleasantness at 80 and 150 mm/s when compared to the haptic only stroking system. Lastly, we introduced four potential applications in daily scenes. Our system methodology allows for a wide range of VHAR application design without concern for latency and misalignment effects.

A Multicoil Array Transceiver Antenna Design for Touchless Hygienic Artificial Human Interfacing

This article presents an optimized multicoil array transceiver antenna design to localize the position of a human hand for a touchless artificial human interface. The functioning of the system is evolved from combined principles of a two-coil and three-coil magnetic resonance coupling-based wireless power transfer techniques. The proposed system comprises a transceiver antenna platform made of spatially distributed multicoil arrays as transmitter and receiver antennas, and a resonator coil. The resonator coil attached to the fingertip of the user is localized by comparing the voltage gains of the receiver antenna array working as sensors with the decision thresholds. Therefore, the coil arrays of the transceiver platform and the resonator coil are jointly optimized to enhance the localization accuracy by maximizing the separation of receiver antenna readings from the decision thresholds. The fabricated prototype of the optimized antenna system is measured, and performance is analyzed for various movements of the user fingertip. The results reveal the decision threshold limits of the design for localization and prove that the proposed transceiver antenna system is able to track the human hand movement successfully. Unlike other vision and sensor-based interfacing, the proposed transceiver antenna system provides a cost-effective, user-friendly, and hygienic solution.

Smartphones and telemedicine for older people in China: Opportunities and challenges.

The development of 5G technology has brought major advances and diverse challenges in healthcare services for older people, and access to equitable telemedicine is a high priority. Smartphones are most commonly used device to access the internet and bring the convenience of telemedicine to the user's fingertips. In the period where aging and informatization coexist in China, the role of smartphones in their daily lives, long-term care, and future opportunities and challenges needs serious reconsideration. We provide the latest evidence of smartphone use among the older adults of China and expound on issues about fairness in accessing mobile health, the substantial digital divide, and potential drawbacks of problematic smartphone use.

Pseudo‐Hologram with Aerohaptic Feedback for Interactive Volumetric Displays

The ability to display virtual objects midair that can be seen without special headwear is an attractive proposition for the virtual reality systems of the near future. Volumetric display technologies have demonstrated this concept, allowing users to view virtual objects with real dimensions in 3D space while enabling some forms of interaction. Developments have been witnessed in haptic feedback technologies which are used to deliver a sensation of touch in the digital world. As virtual environments find more applications in today's world, haptic feedback devices have potential to integrate with virtual reality systems and contribute to a more immersive and interactive user experience. Herein, an air‐based haptic feedback device, named Aerohaptics, is presented, which delivers mid‐air tactile feedback while the user is manipulating virtual objects within a pseudo‐holographic display. The developed system constitutes a cost‐effective approach with relatively low complexity and various potential applications as no wearable or handheld peripherals are required. The delivered haptic feedback can be accurately directed at specific locations on the user's hand and its intensity can be controlled to suit various interaction scenarios. Through experiments, localized feedback on the user's fingertips as well as varying feedback force according to the user's hand movement is demonstrated.

Sensor based Smart Irrigation System with Monitoring and Controlling using Internet of Things

This paper presents the development of a sensor based smart irrigation system with the capabilities of remote monitoring and controlling of water usage in the agriculture field using Internet of Things (IoT). With the employment of IoT in irrigation system, all agricultural information can be viewed and controlled at the user's fingertips. The system consists of a microcontroller (Node MCU), sensors (soil moisture, DHT11), and irrigation of a water pump with a decision-making system. Sensors are linked to a Wi-Fi module (Node MCU) and are interdependent to provide increased sensitivity to the irrigation system. The data obtained will be uploaded to the cloud (ThingSpeak) and presented in the form of graphs accessible via the website. A web page is used to control the water pump for irrigation purposes. This paper is managed to meet all of its aims to help farmers in terms of time, project cost, labor, water consumption, power consumption, and reliability by implementing the IoT-based smart irrigation system.

Design of Personalized Wearable Haptic Interfaces to Account for Fingertip Size and shape.

The size and shape of fingertips vary significantly across humans, making it challenging to design wearable fingertip interfaces suitable for everyone. Although deemed important, this issue has often been neglected due to the difficulty of customizing devices for each different user. This article presents an innovative approach for automatically adapting the hardware design of a wearable haptic interface for a given user. We consider a three-DoF fingertip cutaneous device, composed of a static body and a mobile platform linked by three articulated legs. The mobile platform is capable of making and breaking contact with the finger pulp and re-angle to replicate contacts with arbitrarily-oriented surfaces. We analyze the performance of this device as a function of its main geometrical dimensions. Then, starting from the user's fingertip characteristics, we define a numerical procedure that best adapts the dimension of the device to: (i) maximize the range of renderable haptic stimuli; (ii) avoid unwanted contacts between the device and the skin; (iii) avoid singular configurations; and (iv) minimize the device encumbrance and weight. Together with the mechanical analysis and evaluation of the adapted design, we present a MATLAB script that calculates the device dimensions customized for a target fingertip as well as an online CAD utility for generating a ready-to-print STL file of the personalized design.

Two-Dimensional Fingertip Force Training With Improved Haptic Sensation via Stochastic Resonance

To learn motor skills using a finger in many haptic training systems, a user places his or her finger in a holder to communicate with a haptic interface. It is known that the force-detection capability at the fingertip is reduced when the user's finger is enclosed in a holder. The learning performance might therefore be impaired in training. Stochastic resonance is known to improve sensitivity at the fingertip. We thus propose a fingertip force learning method using stochastic resonance. We first examine the effect of stochastic resonance on the user haptic performance in a two-dimensional task when the fingertip is within the finger holder. The outcomes indicate that the user fingertip sensitivity increases even when the finger is placed within a holder. We next perform force learning tasks in one- and two-dimensional space. Our proposed method that combines haptic feedback and stochastic resonance is compared with the method in which only haptic feedback is provided to the user in force leaning tasks. Results obtained using the proposed method indicate that the performance outcome of the proposed method is higher than that of the comparison method. This study demonstrates the potential of the proposed method for motor learning tasks.

Multimode Hydraulically Amplified Electrostatic Actuators for Wearable Haptics

The sense of touch is underused in today's virtual reality systems due to lack of wearable, soft, mm-scale transducers to generate dynamic mechanical stimulus on the skin. Extremely thin actuators combining both high force and large displacement are a long-standing challenge in soft actuators. Sub-mm thick flexible hydraulically amplified electrostatic actuators are reported here, capable of both out-of-plane and in-plane motion, providing normal and shear forces to the user's fingertip, hand, or arm. Each actuator consists of a fluid-filled cavity whose shell is made of a metalized polyester boundary and a central elastomer region. When a voltage is applied to the annular electrodes, the fluid is rapidly forced into the stretchable region, forming a raised bump. A 6 mm × 6 mm × 0.8 mm actuator weighs 90 mg, and generates forces of over 300 mN, out-of-plane displacements of 500 µm (over 60% strain), and lateral motion of 760 µm. Response time is below 5 ms, for a specific power of 100 W kg-1 . In user tests, human subjects distinguished normal and different 2-axis shear forces with over 80% accuracy. A flexible 5 × 5 array is demonstrated, integrated in a haptic sleeve.

Electromagnetic Haptic Feedback System for Use With a Graphical Display Using Flat Coils and Sensor Array

We have developed a haptic interaction system which wirelessly generates 3D forces onto a small magnet which can be fixed to a user's fingertip or a handheld stylus. Magnet position sensing and haptic force generation can both be performed through a thin screen, and the actuation and sensing components of the system are sufficiently thin and compact to be easily mounted behind the screen in the same enclosure. In this manner, the system is particularly well suited for use for co-located haptic and graphic display, with 3D force feedback added to an interactive display. The purpose of the work is to describe, analyze, and validate a method and device to generate haptic force feedback on a magnet held near a display screen using thin flat arrays of coils and sensors. The location of the magnet during haptic interaction is obtained by an array of Hall effect sensors. Flat rectangular coils are used to generate Lorentz forces on the interaction magnet and can act in combination to produce 3D forces in any direction, both parallel and normal to the screen surface. The active area of the haptic interaction system prototype is approximately 120 × 120 mm, with effective force generation and position sensing from the screen surface to an elevation of approximately 15 mm. The localization methods do not depend on the size, shape, or magnetization strength of the magnet, so that larger or smaller magnets may be used for greater forces or more ease of manipulation, without modifying the software other than the force actuation model. The design of the system is presented, including modeling and analysis of the sensing and actuation methods. Experimental results are given for field sensing, magnet localization, and haptic interaction with simulated objects. Forces up to 0.3 N are demonstrated to be generated in different specific desired directions while sensing planar magnet position to an accuracy within approximately 2.0 mm.

An IoT-Based Non-Invasive Glucose Level Monitoring System Using Raspberry Pi

Patients diagnosed with diabetes mellitus must monitor their blood glucose levels in order to control the glycaemia. Consequently, they must perform a capillary test at least three times per day and, besides that, a laboratory test once or twice per month. These standard methods pose difficulty for patients since they need to prick their finger in order to determine the glucose concentration, yielding discomfort and distress. In this paper, an Internet of Things (IoT)-based framework for non-invasive blood glucose monitoring is described. The system is based on Raspberry Pi Zero (RPi) energised with a power bank, using a visible laser beam and a Raspberry Pi Camera, all implemented in a glove. Data for the non-invasive monitoring is acquired by the RPi Zero taking a set of pictures of the user fingertip and computing their histograms. Generated data is processed by an artificial neural network (ANN) implemented on a Flask microservice using the Tensorflow libraries. In this paper, all measurements were performed in vivo and the obtained data was validated against laboratory blood tests by means of the mean absolute error (10.37%) and Clarke grid error (90.32% in zone A). Estimated glucose values can be harvested by an end device such as a smartphone for monitoring purposes.

Wearable electrochemical glove-based sensor for rapid and on-site detection of fentanyl

Rapid, on-site detection of fentanyl is of critical importance, as it is an extremely potent synthetic opioid that is prone to abuse. Here we describe a wearable glove-based sensor that can detect fentanyl electrochemically on the fingertips towards decentralized testing for opioids. The glove-based sensor consists of flexible screen-printed carbon electrodes modified with a mixture of multiwalled carbon nanotubes and a room temperature ionic liquid, 4-(3-butyl-1-imidazolio)-1-butanesulfonate. The sensor shows direct oxidation of fentanyl in both liquid and powder forms with a detection limit of 10 μM using square-wave voltammetry. The “Lab-on-a-Glove” sensors, combined with a portable electrochemical analyzer, provide wireless transmission of the measured data to a smartphone or tablet for further analysis. The integrated sampling and sensing methodology on the thumb and index fingers, respectively, enables rapid screening of fentanyl in the presence of a mixture of cutting agents and offers considerable promise for timely point-of-need screening for first responders. Such a glove-based “swipe, scan, sense, and alert” strategy brings chemical analytics directly to the user's fingertips and opens new possibilities for detecting substances of abuse in emergency situations.

CamK: Camera-Based Keystroke Detection and Localization for Small Mobile Devices

Because of the smaller size of mobile devices, text entry with on-screen keyboards becomes inefficient. Therefore, we present CamK, a camera-based text-entry method, which can use a panel (e.g., a piece of paper) with a keyboard layout to input text into small devices. With the built-in camera of the mobile device, CamK captures images during the typing process and utilizes image processing techniques to recognize the typing behavior, i.e., extract the keys, track the user's fingertips, detect, and locate keystrokes. To achieve high accuracy of keystroke localization and low false positive rate of keystroke detection, CamK introduces the initial training and online calibration. To reduce the time latency, CamK optimizes computation-intensive modules by changing image sizes, focusing on target areas, introducing multiple threads, removing the operations of writing or reading images. Finally, we implement CamK on mobile devices running Android. Our experimental results show that CamK can achieve above 95 percent accuracy in keystroke localization, with only a 4.8 percent false positive rate. When compared with on-screen keyboards, CamK can achieve a 1.25X typing speedup for regular text input and 2.5X for random character input. In addition, we introduce word prediction to further improve the input speed for regular text by 13.4 percent.

Usage of Mobile Phones for Library Services by Students of REVA University, Bangalore: A Study

There are evidences of existence of libraries as treasure houses of knowledge in ancient history, earliest reference being “The Great Alexandrian Library”. A continuous transition in the libraries has happened in phased manner over the years to get into the present “Digital Era”. Containers of information have changed stage by stage from parchment to papyrus to paper to digital media. Information & Communication Technology has revolutionized all walks of human life starting from shopping, banking or booking a cinema ticket etc. Technology has become a vital part of common life in different forms. Mobile phones are considered as basic necessity in digital era for many of the chores. The academic world is also an area to take advantages of mobile technology in connecting its stakeholders. For librarians, mobile technology is a boon to extend services quickly to users and connecting them. Library services and sources can be made available on finger tips of users for 24/7 without any geographical jurisdiction. In this regards REVA University central library surveyed 200 students in order to find out the level of smart phones usage among them and what are their expectation.

Effect of Cutaneous Feedback on the Perceived Hardness of a Virtual Object.

We investigate the effect of adding cutaneous cues to kinesthetic feedback on the perception of a virtual object's hardness. A cutaneous haptic interface is designed to deliver hardness information to a user's fingertip along with a force-feedback interface, and the corresponding rendering strategy is implemented. Two sets of experiments are conducted to evaluate the proposed approach for hardness perception using one-finger touch and two-finger grasp. Experimental results indicate that the addition of cutaneous feedback can make the virtual surface feel significantly harder than the nominal stiffness delivered by force-feedback alone. In addition, the perceived hardness is significantly affected by the rate of hardness rendered with a cutaneous interface for the nominal stiffness K = 0.3 and 0.5 N/mm. For two-finger grip, the effect of a virtual object's thickness has a significant effect on the perceived hardness measured in stiffness. When the perceived hardness is converted to Young's modulus, the effect of thickness is insignificant.

A Continuous Identity Authentication Scheme Based on Physiological and Behavioral Characteristics.

Wearable devices have flourished over the past ten years providing great advantages to people and, recently, they have also been used for identity authentication. Most of the authentication methods adopt a one-time authentication manner which cannot provide continuous certification. To address this issue, we present a two-step authentication method based on an own-built fingertip sensor device which can capture motion data (e.g., acceleration and angular velocity) and physiological data (e.g., a photoplethysmography (PPG) signal) simultaneously. When the device is worn on the user’s fingertip, it will automatically recognize whether the wearer is a legitimate user or not. More specifically, multisensor data is collected and analyzed to extract representative and intensive features. Then, human activity recognition is applied as the first step to enhance the practicability of the authentication system. After correctly discriminating the motion state, a one-class machine learning algorithm is applied for identity authentication as the second step. When a user wears the device, the authentication process is carried on automatically at set intervals. Analyses were conducted using data from 40 individuals across various operational scenarios. Extensive experiments were executed to examine the effectiveness of the proposed approach, which achieved an average accuracy rate of 98.5% and an F1-score of 86.67%. Our results suggest that the proposed scheme provides a feasible and practical solution for authentication.

Interactive Image Segmentation Using Semi-transparent Wearable Glasses

Since it is difficult to automatically and precisely extract an object of interest, interactive image segmentation techniques exploit user-provided segmentation seeds. In previous interactive segmentation applications, the segmentation seeds are typically provided by mouse clicks or finger touches. In this paper, the segmentation of an object is studied from the scene that the user sees through semi-transparent wearable glasses. In this application scenario, a front-view camera is used to obtain the segmentation seeds from the user's fingertip position. In particular, two segmentation methodologies called transparent segmentation and semi-transparent segmentation are considered to determine an effective segmentation scheme for the wearable glasses. Extensive user studies are performed to evaluate the user preferences and the segmentation accuracies of the two methodologies.

JustPoint: Identifying Colors with a Natural User Interface.

People with severe visual impairments usually have no way of identifying the colors of objects in their environment. While existing smartphone apps can recognize colors and speak them aloud, they require the user to center the object of interest in the camera's field of view, which is challenging for many users. We developed a smartphone app to address this problem that reads aloud the color of the object pointed to by the user's fingertip, without confusion from background colors. We evaluated the app with nine people who are blind, demonstrating the app's effectiveness and suggesting directions for improvements in the future.

A Three Revolute-Revolute-Spherical Wearable Fingertip Cutaneous Device for Stiffness Rendering.

We present a novel three Revolute-Revolute-Spherical (3RRS) wearable fingertip device for the rendering of stiffness information. It is composed of a static upper body and a mobile end-effector. The upper body is located on the nail side of the finger, supporting three small servo motors, and the mobile end-effector is in contact with the finger pulp. The two parts are connected by three articulated legs, actuated by the motors. The end-effector can move toward the user's fingertip and rotate it to simulate contacts with arbitrarily-oriented surfaces. Moreover, a vibrotactile motor placed below the end-effector conveys vibrations to the fingertip. The proposed device weights 25g for 35 x 50 x 48 mm dimensions. To test the effectiveness of our wearable haptic device and its level of wearability, we carried out two experiments, enrolling 30 human subjects in total. The first experiment tested the capability of our device in differentiating stiffness information, while the second one focused on evaluating its applicability in an immersive virtual reality scenario. Results showed the effectiveness of the proposed wearable solution, with a JND for stiffness of 208.5 17.2N/m. Moreover, all subjects preferred the virtual interaction experience when provided with wearable cutaneous feedback, even if results also showed that subjects found our device still a bit difficult to use.