Gesture Control Research Articles

Piezoelectric sensor based on graphene-doped PVDF nanofibers for sign language translation

The tracking of body motion, such as bending or twisting, plays an important role in modern sign language translation. Here, a subtle flexible self-powered piezoelectric sensor (PES) made of graphene (GR)-doped polyvinylidene fluoride (PVDF) nanofibers is reported. The PES exhibits a high sensitivity to pressing and bending, and there is a stable correlation between bending angle and piezoelectric voltage. The sensitivity can be adjusted by changing the doping concentration of GR. Also, when the PES contacts a source of heat, a pyroelectric signal can be acquired. The positive correlation between temperature and signal can be used to avoid burns. The integrated sensing system based on multiple PESs can accurately recognize the action of each finger in real time, which can be effectively applied in sign language translation. PES-based motion-tracking applications have been effectively used, especially in human–computer interaction, such as gesture control, rehabilitation training, and auxiliary communication.

Face Authenticated Hand Gesture Based Human Computer Interaction for Desktops

Abstract Hand gesture detection and recognition is a cutting-edge technology that is getting progressively applicable in several applications, including the recent trends namely Virtual Reality and Augmented Reality. It is a key part of Human-Computer Interaction which gives an approach to two-way interaction between the computer and the user. Currently, this technology is limited to expensive and highly specialized equipment and gadgets such as Kinect and the Oculus Rift. In this paper, various technologies and methodologies of implementing a gesture detection and recognition system are discussed. The paper also includes the implementation of a face recognition module using the Viola-Jones Algorithm for authentication of the system followed by hand gesture recognition using CNN to perform basic operations on the laptop. Any type of user can use gesture control as an alternative and interesting way to control their laptop. Furthermore, this can be used as a prototype for future implementations in the field of virtual reality as well as augmented reality.

Design of Four Rotor Unmanned Aerial Vehicle Based on Open CV

This design is based on the OpenCV technology of the four rotor UAV, mainly by the control module and image module of two parts. The control module contains a STM32 micro controller, it can generate multiple PWM to control the motor to produce basic movements, while the body is equipped with three axis acceleration of three axis gyroscope MPU-6050 module, electronic compass, barometer and wireless communication module, and use STM32 to read sensor data for gesture calculation and control motor action; The image module is equipped with a ARM Cortex-A53 processor, running the Python-OpenCV operating system in the embedded Linux environment, OpenCV implementation of the UAV aerial features, in addition, the onboard Wifi module can be accessed by SSH remote desktop to obtain detection images, which can be widely used in security monitoring, exploration engineering, home security and other fields.

Design and Realization of Intelligent Wheelchair Based on STM32

In response to the market demand of people who need special care, a smart wheelchair based on STM32 is designed. Through the STM32F103ZET6 development board, it uses the smart car (simulated electric wheelchair) as the carrier with the attitude calculation algorithm, GPS positioning, Lora wireless communication, PWM speed control and distance detection, the gesture control of the movement and steering of the car, automatic obstacle avoidance, GPS Real-time transmission of positioning information, one-key call for help, and video recording of unexpected situations. The gesture recognition rate of the system is stable and the safety is high, and it can meet the requirements of wheelchair intelligence.

Humanoid gesture control ARM with manifold actuation by embedded system

The design and implementation of humanoid gesture control robotic arm with multiple actuations is proposed. Utilization of gesture based design is made here. The design is made such a way that five separate movable fingers, flexion and extension installed wrist with tilt feature. Human palm movements are imitated in robotic arm using the motion glove. To develop the processing of arms with additional movement, to improve the aesthetics both for the safety and to look as less artificial. Already in this field metallic and plastic variants are used. The dimensions are made with the help of human palm dimensions. A human design is not sufficient. It also need the movements that is done by actual human [1]. In this paper we propose the number of movements done by the arm is made to 120 [2].

Gesture controlled quadcopter for defense search operations

This paper is representing an idea on the use of the quadcopter for the sake of the pivotal issues like defense tasks or climate calamities/changes which threaten the world to plunge into hazards. So, we need efficient ways to identify areas of potential hazard risks and monitor them closely in an easier manner. Due to these hazardous and risky challenges it is not always optimal for humans to take up this task rather use a quadcopter for the search and rescue operation to save the lives of the people from risks. The base idea for our drone is to use a DJI NAZA-M Lite Flight controller with GPS antenna, a low latency camera with a 0.33-inch CMOS sensor with a 1.2 mm wide angle (150degree) lens and a transmitter with 2.4 GHz bandwidth makes it suitable at the time of search and rescue. By using an arduino and an MPU6050 sensor, we can map its gyro and accelerometer values to the control sticks of the quadcopter. We have also added a small 5 V speaker along with a couple of other sensors like temperature, rain/humidity sensor, gas sensors etc. The use of these components along with gesture control features can be used at the time of surveillance by defense or at the time of natural calamities for the search and rescue operations making it advantageous to use by any sort of person in these risky situations.

The influence of performing gesture type on interpersonal musical timing, and the role of visual contact and tempo

Bodily gestures play an important role in the communication of expressive intentions between humans. Music ensemble performance, as an outstanding example of nonverbal human communication, offers an exemplary context to study and understand the gestural control and communication of these expressive intentions. An important mechanism in music ensemble performance is the anticipation and control of interpersonal timing. When performing, musicians are involved in a complex system of mutual adaptation which is not completely understood so far. In this study, we investigated the role of performers' gestures in the mediation process of interpersonal timing in a dyad performance. Therefore, we designed an experiment in which we controlled for the use of hand and arm movements in a musical task, in which dyads were asked to synchronously tap out a melody. Next to their comfortable/natural way of tapping, we instructed participants to either perform pronounced expressive hand and arm gestures in between successive taps, or to restrict from any overt body movement. In addition, we looked at effects of visual contact (yes/no) and tempo (slow: 50 beats per minute; fast: 100 beats per minute). The results show that performers' gestures improve interpersonal musical timing, in terms of the consistency and accuracy of onset asynchronies, and of the variability of produced inter-onset intervals. Interestingly, we found that the use of expressive gestures, in regard to comfortable/natural movements, add to these positive timing effects, but only when there is visual contact and at the slow tempo. In addition, we found that the type of gestures employed by musicians may modulate leader-follower dynamics. Together, these findings are explained by human anticipation mechanisms facilitated by gesturing, shedding new light on the principles underlying human communication of expressive intentions, through music.

Integrating gesture control board and image recognition for gesture recognition based on deep learning

ABSTRACT Due to the rise of the Internet of Things, more devices can connect with the Internet. A large amount of data is collected from devices that could be used for different applications. The development of hardware equipment for the Internet of Things not only use for industrial but also for smart homes. The smart home covers broad topics, including remote control of home applications, sensing of humans, temperature-controlling air conditions, and security monitors. When we carry out these topics, the human-machine interface is essential for system applications. A gesture recognition system is applied to many real applications. The reason is that the accuracy rate and real factors are complicated. They commonly use of gesture control service in the market is the sensor board of gesture control. The principle is using the electric field to change and determine the gestures. The limitation requires the close operation, and there is a problem of critical point sensitivity. In this paper, we use the gesture control board to combine with gesture image recognition methods to perform the double authentication gesture recognition. Raspberry Pi is the control center to integrate the intelligent light bulb. HUE makes a gesture recognition system. The results explain that the accuracy rate of the gesture recognition proposed is 90%. Meanwhile, it is higher than the SVM method.

Mid-Air Gesture Control of Multiple Home Devices in Spatial Augmented Reality Prototype

Touchless, mid-air gesture-based interactions with remote devices have been investigated as alternative or complementary to interactions based on remote controls and smartphones. Related studies focus on user elicitation of a gesture vocabulary for one or a few home devices and explore recommendations of respective gesture vocabularies without validating them by empirical testing with interactive prototypes. We have developed an interactive prototype based on spatial Augmented Reality (AR) of seven home devices. Each device responds to touchless gestures (identified from a previous elicitation study) via the MS Kinect sensor. Nineteen users participated in a two-phase test (with and without help provided by a virtual assistant) according to a scenario that required from each user to apply 41 gestural commands (19 unique). We report on main usability indicators: task success, task time, errors (false negative/positives), memorability, perceived usability, and user experience. The main conclusion is that mid-air interaction with multiple home devices is feasible, fairly easy to learn and apply, and enjoyable. The contributions of this paper are (a) validation of a previously elicited gesture set; (b) development of a spatial AR prototype for testing of mid-air gestures, and (c) extensive assessment of gestures and evidence in favor of mid-air interaction in smart environments.

English

English

From perception to action using observed actions to learn gestures

Pervasive computing environments deliver a multitude of possibilities for human–computer interactions. Modern technologies, such as gesture control or speech recognition, allow different devices to be controlled without additional hardware. A drawback of these concepts is that gestures and commands need to be learned. We propose a system that is able to learn actions by observation of the user. To accomplish this, we use a camera and deep learning algorithms in a self-supervised fashion. The user can either train the system directly by showing gestures examples and perform an action, or let the system learn by itself. To evaluate the system, five experiments are carried out. In the first experiment, initial detectors are trained and used to evaluate our training procedure. The following three experiments are used to evaluate the adaption of our system and the applicability to new environments. In the last experiment, the online adaption is evaluated as well as adaption times and intervals are shown.

Aladdin’s magic carpet: Navigation by in-air static hand gesture in autonomous vehicles

ABSTRACT This paper presents a novel and exploratory investigation of how users might control future autonomous vehicles with user-defined in-air static hand gestures. In the era of autonomous vehicles, how to support “driving” without steering control may become a key issue affecting user experience. As the navigation interface of future autonomous cars will be wholly dependent on GPS, and passengers will be unable to make manual control adjustments, verbal or gestural communication will become a primary means to assist them in vehicle control. We thus focus on gesture as an innovative solution for this “final 100 meters” problem in automated navigation. A study (N = 24) conducted in a full chassis simulator shows that hand gesture control is feasible for autonomous vehicle navigation. It further reveals that hand gestures are influenced by task types, local regional norms, and participant culture. In particular, the spatial region where gestures occur can affect execution time, gender can impact user preference and demand, and culture differences and priorities can affect adoption, ease of learning, and user comfort, influencing longer-term use.

Design and Development of Bio-Sensitive Robotic Arm using Gesture Control

In this paper, a model of a robotic arm is designed and manufactured using a 3D printer, which is to be operated via human gesture by using the Accelerometer and Gyroscope Module. This arm is proposed to help in the medical field, in military operations, in hazardous conditions and in industries to maximize human safety. The robot arm is designed such that, it has 5 degrees of freedom controlled by the 3-axis accelerometer mounted on the IC placed on the glove of the user. The module will replicate the movement of the user’s hand to extend, retract, and rotate accordingly to accurately position as required for the application. The user’s finger action is used to manipulate the working of the gripper. Therefore, gesture control in a broad sense is a computerized interface that allows computers to record and interpret those gestures into commands to adhere to actions.

Wireless Gesture Control Wheelchair

Wheelchairs have been used by patients who suffer from various physical disabilities to help them with locomotion and cater their day to day needs with ease. But there are some cases where the movement of a wheelchair is dependent on another individual as is the case with patients who lack the required arm strength and movement to properly push the wheels forward such as quadriplegics, paraplegics, stroke patients, elders etc. Joystick oriented wheelchairs, thought to be a solution to those kinds of patients, can pose different problems as it requires basic shoulder movement. It is not always possible for the aforementioned types of patients. In addition, our solution doesn’t have the positional constraints that a joystick wheelchair might have as it is wireless and can be worn on either hand which allows the patient to sit in their preferred position for minimum discomfort. This project is an attempt to help the disabled move around independently. Thus, in this research work, we present a prototype of a wireless gesture-based wheelchair which can be controlled via hand gestures. The framework consists of a transmitter and a receiver that communicate with each other wirelessly. For wireless transmission, 433Mhz RF Transmitter and Receiver Unit has been used as it transmits data through an antenna at the speed of 1Kbps – 10Kbps and the range can be adjusted as required. The transmitter unit consists of an Arduino LilyPad microcontroller and an accelerometer that has been attached to a hand glove. The accelerometer sensor has been used to register the position of the hand while creating a gesture. This glove is supposed to be worn by the patient allowing them to move their hand conveniently, sending signals to the receiver unit connected to the wheelchair leading to the movement of the wheels in the desired direction. The receiver unit consist of motor drivers that convert the voltage as needed by the wheels. This paper presents an alternative to the commercial wheelchairs as it is cost effective, easy to control and efficient. The working and assembly of the system has been explained in the paper.

Human-computer multimodal interface to internet navigation

Purpose Propose and develop new multimodal interfaces that allow people with motor impairments to control mass use applications in a natural way through gestures and voice. Methods A multimodal interaction interface was developed for using Google Chrome, Gmail and Facebook applications through gestural and verbal commands. The interface activates mouse and keyboard commands from the processing of voice signals and videos of the user’s head movement. The interface does not disable traditional keyboard and mouse functions; moreover, it only requires a webcam and a microphone, which are usually built into portable computers. Results The tests were performed on three groups of people: young adults, older adults and people with motor impairments. The verbal interaction was tested on a total of 189 voice commands with an average performance of 75.7% and a total of 105 dictations. The dictations had an average of 13 words; the system had a performance of 81.1%. Moreover, the gestural interaction activated 126 commands without errors using a drop-down menu; a click was activated 84 times with a success rate of 70.2%. Conclusion The motor impairments group especially valued the option of using Google Chrome, Gmail and Facebook without physically manipulating a mouse and keyboard. This group showed a greater preference for verbal control than for gestural control. An adaptation period is required for the adults group to acquire greater skill in using the interface. The young adults group preferred the types of interactions they are accustomed to due to their familiarity with Information and Communication Technologies (ICT); they considered the interface fun. Implication for rehabilitation Computers and their applications were conceived with unnatural interaction mechanisms, such as the keyboard and mouse, which prevent their use by people with psychophysiological limitations or digital literacy. For this reason, the need arises to design new natural interfaces commanded by gestures and voice. It is necessary to develop low-cost interfaces that can control mass-use applications such as Google Chrome, Facebook and Gmail, which do not require additional hardware using webcams and microphones, which are usually integrated into laptops. The development of these multimodal interfaces improves the quality of life of people with motor impairments, allowing them to have access to Information and Communication Technologies (ICT).

Implementation of Gesture Control Robotic Arm for Automation of Industrial Application

<p>In today’s world, in almost all sectors, most of the work is done by robots or robotic arm having different number of degree of freedoms (DOF’s) as per the requirement. This project deals with the Design and Implementation of a “Wireless Gesture Controlled Robotic Arm with Vision”. The system design is divided into 3 parts namely: Accelerometer Part, Robotic Arm and Platform. It is fundamentally an Accelerometer based framework which controls a Robotic Arm remotely utilizing a, little and minimal effort, 3-pivot (DOF's) accelerometer by means of RF signals. The Robotic Arm is mounted over a versatile stage which is likewise controlled remotely by another accelerometer. One accelerometer is mounted/joined on the human hand, catching its conduct (motions and stances) and hence the mechanical arm moves in like manner and the other accelerometer is mounted on any of the leg of the client/administrator, catching its motions and stances and in this way the stage moves as needs be. In a nutshell, the robotic arm and platform is synchronised with the gestures and postures of the hand and leg of the user / operator, respectively. The different motions performed by robotic arm are: PICK and PLACE / DROP, RAISING and LOWERING the objects. Also, the motions performed by the platform are: FORWARD, BACKWARD, RIGHT and LEFT.</p>

Additive Gaussian noise model and Kalman filter to enhance controllability of gesture-controlled teleoperated soft actuators

Soft actuators are used in bilateral systems as slave-side actuators. To broaden the applications and usage of these soft actuators in teleoperated applications required high accuracy and control. This article focuses on the improvement and control of the soft actuators operated through the Internet using a glove-based gesture control system. A hand-worn sensor glove (a glove with flex sensors and accelerometers) is used to detect hand gestures (specifically bending angle and force generated in a finger while gripping a spherical object) of an operator. The operator’s finger movements are mimicked in a remote location, through the Internet, in a soft actuator. The bending angles of the human finger have converted into the pressure variations inside the actuator using a linear calibration technique. A vertically mounted actuator and a spherical object are used to demonstrate the gripping action. The main problems that occurred during the controlling of this setup are noise and delays. Electronic noise (line noise and circuit noise), mechanical noise (vibration, nonuniformities in actuator fabrication, wear, and tear), elastic effect (energy absorbed during the state transformation), communication delays (delays occurred due to geography, telecommunication infrastructure, and round trip transmission), and other noises (other environmental effects) degraded the performance. This article considered a Brownian motion model, an additive Gaussian noise model, and a Kalman filter to solve these problems. The experimentations are performed in three different locations (to demonstrate the teleoperation) and the recorded improvement in the performance is approximately 17%.

Controlling Anesthesia Hardware With Simple Hand Gestures: Thumbs Up or Thumbs Down?

Modern consumer electronic devices and automobiles are often controlled by interfaces that sense physical gestures and spoken commands. In contrast, patient monitors and anesthesia devices are typically equipped with panel-mounted buttons, dials, and keyboards. The increased use of noncontact gesture-based interfaces in anesthesia may improve patient safety through more intuitive and prompter control of equipment and also through reduced rates of surface contamination. A novel gesture-based controller was designed and retrofitted to a standard GE Solar 8000M patient monitor. This type of technical innovation is rare, due to closely held proprietary input control systems on commercially produced clinical equipment. Nevertheless, we hypothesized that anesthesiologists would find a contactless gesture interface straightforward to use. A gesture-based interface system was developed to control a Solar 8000M patient monitor using a millimeter-wave radar sensor. The system was programmed to detect noncontact "rotate" and "press" gestures to control the patient monitor by implementing a virtual trim knob for interface control. Fifty anesthesiologists tested a prototype interface and evaluated usability by completing a short questionnaire incorporating modified Likert scales. These evaluations were performed in a nonpatient care environment so that respondents were not adversely task loaded during assessment, also allaying any ethical or safety concerns regarding use of this novel interface for patient management. Anesthesia hardware was controlled reliably with 2 distinct gestures above the gesture sensor. The gesture-based interface generally was well received by anesthesiologists (8.09; confidence interval, 8.06-8.12 on a 10-point scale), who preferred the simpler "press" gesture to the "rotate" gesture (8.45; 8.39-8.51 vs 7.73; 7.67-7.79 on a 10-point scale; P = .005). The correlation between the preference scores for the 2 gestures from each anesthesiologist was strong (Pearson r = 0.49; 0.25-0.68; P < .001). Advancing level of training (resident, fellow, attending 1-10 years, attending >10 years) was not correlated with preference scores for either gesture (Spearman ρ = -0.02; -0.30 to 0.26; P = .87 for "press" and Spearman ρ = 0.08; -0.20 to 0.35; P = .58 for "rotate"). The use of gesture sensing for controlling anesthesia equipment was well received by a cohort of anesthesiologists. Even though the simpler "press" gesture was preferred over the "rotate" gesture, the intrarespondent correlation indicates that the preference for gestures as a whole is the stronger effect. No adverse relationship was found between acceptability and anesthesia experience level. Gesture sensing is a promising new area to simplify and improve the interaction between the anesthesiologist and the anesthesia workstation.

Humanoid gesture control arm with manifold actuation using additive manufacturing

The design and implementation of humanoid gesture control robotic arm with multiple actuations is proposed. Utilization of gesture based design is made here. The design is made such a way that five separate movable fingers, flexion and extension installed wrist with tilt feature. Human palm movements are imitated in robotic arm using the motion glove. To develop the processing of arms with additional movement, to improve the aesthetics both for the safety and to look as less artificial. Already in this field metallic and plastic variants are used. The dimensions are made with the help of human palm dimensions. A human design is not sufficient. It also need the movements that is done by actual human [1]. This paper we propose the number of movements done by the arm is made to 120 [2].

A novel design and implementation of wheelchair navigation system using Leap Motion sensor

Purpose In this paper, a novel design for a leap motion wheelchair navigation system is proposed, and the suggested model is implemented on a prototype. The behaviour of the created prototype is closely observed during the different performance tests carried out, and the results are presented throughout this manuscript. Materials and methods In the prototype, a Leap Motion sensor is implemented to acquire navigation data through hand gestures of the users. This navigations system design is specifically implemented to facilitate wheelchair use for amputee users and stroke patients as it does not rely on the movement of the fingers. Through this design, wheelchair movement can be controlled through detection of finger, fist, palm or wrist (for amputees) movement by the leap motion sensor. Bluetooth connection is used as the navigation system’s communication means, removing the need for constant internet connection and providing freedom of movement outside of internet-covered territory. Additionally, two Dynamixel motors are used as movement force, which yield optimal computational time and minimal delay. Results The performance of the designed prototype is tested by considering response time and speed resolution as evaluation metrics. Results suggest that the designed wheelchair will give movement independence to users who cannot use their fingers to control the movement of their wheelchairs, while reducing delay, being independent of internet connection, providing high resolution and minimising detection error. Conclusions The promising results obtained from prototype testing suggest the possibility of real-life application of this wheelchair navigation system, which can greatly assist amputee users and rehabilitation patients. Implications for rehabilitation A novel wheelchair navigations system designed to facilitate amputee users, stroke patients and rehabilitation patients. The proposed system eliminates the reliance on finger movements, is gaze independent, and does not require voice or gesture control, creating much more freedom for users undergoing specific medical conditions or still under rehabilitation or treatment. Results demonstrate very low delay time in wheelchair command to action, allowing improved control for users and reducing the occurrence of control-related accidents. The designed wheelchair navigation system is independent of internet connection, allowing more freedom in range for wheelchair users compared to available cloud based models.