Novint Falcon Research Articles

THE INFLUENCE OF SEX FACTOR ON THE MODELING OF THE HUMAN HAND/ARM INTERACTING WITH A TELEOPERATION SYSTEM

This paper addresses the lack of research on the dynamical differences between male and female hand/arm models during interaction with haptic systems. A linear one-degree-of-freedom experiment is designed using the Novint Falcon robot for this aim. To compare the differences, the study analyzes the pole, stiffness, and damping of a first-order two-parameter spring–damper model. The Mann–Whitney U and Fligner-Killeen tests were used to explore the homogeneity in medians and variances of the groups. The results indicate that there is no significant difference in the median values of poles and stiffnesses between male and female models. However, there is a statistically significant difference in the damping coefficient with a 0.05 significance level and a p-value of 0.0361. Regarding variances, the statistical measures do not indicate any significant differences between each sex. However, the p-value of 0.0669 for the stiffness parameter suggests that the stiffness parameters of the aforementioned groups may come from different distributions.

Development of a 6-DOF robot for haptic interaction with complex virtual environments

In the last decade, the design and implementation of robots have taken a turnaround in areas such as haptics and virtual reality. Unlike the big and heavy industrial robots, haptic ones must be light and suitable to be easily handled by an operator. Moreover, they must have enough actuators to allow a realistic haptic interaction with a complex virtual environment. In this work, we present the two phases Research and Development process of a six-degrees-of-freedom haptic robot. In the first phase, we build on our previous work to design a spherical wrist, improving both the mechanics and electronics of an old one. In the second phase, we design a virtual environment consisting of a ball and beam system with which the operator interacts visually and haptically. The basis of our development is the well-known Novint Falcon parallel robot that acts as the first 3-DOF of the resulting device. The rest is completed with our improved spherical wrist, in which a 6-DOF force sensor that measures the interaction forces between the virtual environment and the operator was mounted. Our ultimate goal is to evaluate the usability of the obtained haptic robot and present it as a viable alternative to current commercial devices.

User study of sensorless adaptive haptic assistance for bilateral teleoperation

This paper presents a sensor-less haptic adaptive assistance scheme for bilateral shared controlled in telerobotics. The paper focuses on the implementation of an adaptive assistance scheme previously developed using force sensors at the master’s side, extending it to the case where this measurement is no longer available. The experimental setup was composed of a virtual slave robot and a real master robot (a commercially-of-the-shelf, low-cost haptic device), namely the Novint Falcon haptic device. Due to the lack of sensors to measure the contact force between the human operator and the haptic control device i.e., only the position of the master robot is measured, a data-driven Unknown Input Observer with disturbance estimation augmentation is proposed, allowing the estimation of the (external) human force under plant uncertainty and external disturbances. The proposed approach is tested via a path tracking (known) with (unknown) obstacle avoidance task, and statistical analysis regarding the scheme’s effectiveness is presented.

Emulating Textures Using Vibrotactile Technology: HaptTech System and its Adaptation to a Commercial Kinesthetic Interface

This article presents the development and testing of a tactile interface prototype, HaptTech, that applies vibrotactile stimulation patterns to the fingertip, as well as an analysis of its performance when coupled to the commercial kinesthetic interface Novint Falcon. Its applicability in digital entertainment expects to improve the level of immersion into virtual reality scenarios. In the first experiment, a group of 75 subjects compared three different vibration stimuli that emulated different textures. They evaluated the stimuli on a 1 to 10 scale, where 1 means ‘non-perceivable differences’ and 10 means ‘entirely differentiable’. The obtained mean values were 9 (textures 1 and 3), 8,29 (textures 1 and 2), and 7,43 (textures 2 and 3), indicating HaptTech’s capability to reproduce differentiable stimuli. In the second experiment, 31 subjects evaluated the coupling between HaptTech and Novint Falcon. They perceived differences when the HaptTech system was activated in the context of a comparison between kinesthetic and vibrotactile plus kinesthetic stimuli. In the third experiment, the subjects evaluated the similarity between the perceived stimuli on the hand and a visualized texture pattern in a virtual environment. The resulting median values were 8, 8, and 9 for textures 1, 2, and 3, respectively, which demonstrate that the subjects perceived a high correspondence for each one.

Position control for haptic device based on discrete-time proportional integral derivative controller

Haptic devices had known as advanced technology with the goal is creating the experiences of touch by applying forces and motions to the operator based on force feedback. Especially in unmanned aerial vehicle (UAV) applications, the position of the end-effector Falcon haptic sets the velocity command for the UAV. And the operator can feel the experience vibration of the vehicle as to the acceleration or collision with other objects through a forces feedback to the haptic device. In some emergency cases, the haptic can report to the user the dangerous situation of the UAV by changing the position of the end-effector which is be obtained by changing the angle of the motor using the inverse kinematic equation. But this solution may not accurate due to the disturbance of the system. Therefore, we proposed a position controller for the haptic based on a discrete-time proportional integral derivative (PID) controller. A Novint Falcon haptic is used to demonstrate our proposal. From hardware parameters, a Jacobian matrix is calculated, which combines with the force output from the PID controller to make the torque for the motors of the haptic. The experiment was shown that the PID has high accuracy and a small error position.

A gaming system with haptic feedback to improve upper extremity function: A prospective case series

BACKGROUND: Video games can be used to motivate repetitive movements in paediatric rehabilitation. Most upper limb videogaming therapies do not however include haptic feedback which can limit their impact. OBJECTIVE: To explore the effectiveness of interactive computer play with haptic feedback for improving arm function in children with cerebral palsy (CP). METHODS: Eleven children with hemiplegic CP attended 12 therapist-guided sessions in which they used a gaming station composed of the Novint Falcon, custom-built handles, physical supports for the child’s arm, games, and an application to manage and calibrate therapeutic settings. Outcome measures included Quality of Upper Extremity Skills Test (QUEST) and Canadian Occupational Performance Measure (COPM). The study protocol is registered on clinicaltrials.gov (NCT04298411). RESULTS: Participants completed a mean of 3858 wrist extensions and 6665 elbow/shoulder movements during the therapist-guided sessions. Clinically important improvements were observed on the dissociated and grasp dimensions on the QUEST and the performance and satisfaction scales of the COPM (all p< 0.05). CONCLUSION: This study suggests that computer play with haptic feedback could be a useful and playful option to help improve the hand/arm capacities of children with CP and warrants further study. The opportunities and challenges of using low-cost, mainstream gaming software and hardware for therapeutic applications are discussed.

Bi-manual Haptic-based Periodontal Simulation with Finger Support and Vibrotactile Feedback

The rise of virtual reality and haptic technologies has created exciting new applications in medical training and education. In a dental simulation, haptic technology can create the illusion of substances (teeth, gingiva, bone, etc.) by providing interaction forces within a simulated virtual world of the mouth. In this article, a haptic periodontal training simulation system, named Haptodont, is developed and evaluated for simulating periodontal probing. Thirty-two faculty members from New York University College of Dentistry were recruited and divided into three groups to evaluate three fundamental functionalities: Group 1 evaluated bi-manual 3 Degrees of Freedome (DoF) haptic interaction, Group 2 evaluated bi-manual 3 DoF haptic interaction with a finger support mechanism, and Group 3 evaluated bi-manual 3 DoF haptic interaction with finger support mechanism and vibrotactile feedback. The probe and mirror interactions were simulated with the Geomagic Touch haptic device whereas the finger support was implemented using the Novint Falcon device. The three groups conducted two probing tasks: healthy gingiva scenario with no pockets (2- to 3-mm depth) and periodontitis scenario with deep pockets (4- to 8-mm depth). Results demonstrated that experts performed comparably to clinical settings in terms of probing depth error (within 0.3 to 0.6 mm) and probing forces (less than 0.5 N). Furthermore, the finger support mechanism significantly improved the probing accuracy for periodontitis condition in the lingual region. The argument that probing the lingual region is more difficult than the buccal region is supported by quantitative evidence (significantly higher probing depth error and probing force). Further research is planned to improve the usability of the finger support, integrate the Haptodont system into the pre-clinical curriculum, and evaluate the Haptodont system with dental students as a learning tool.

ParaVR: a virtual reality training simulator for paramedic skills maintenance

Background: Virtual reality (VR) technology is emerging as a powerful tool in medical training and has potential benefits for paramedic education. Aim: The aim of this paper is to report the development of ParaVR, which uses VR to maintain paramedics' skills. Methods: Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust (WAST) developed ParaVR in four stages: identifying requirements and specifications; alpha version development; beta version development; and management—development of software, further funding and commercialisation. Results: Needle cricothyrotomy and needle thoracostomy emerged as candidates for the prototype ParaVR. The Oculus Rift head-mounted display was combined with Novint Falcon haptic device and a virtual environment crafted using 3D modelling software, which was ported to the Oculus Go virtual reality headset and the Google Cardboard VR platform. Conclusion: VR is an emerging educational tool with the potential to enhance paramedic skills development and maintenance. The ParaVR programme is the first step in the authors' development, testing and scaling up of this technology.

Hybrid force/position control of master and slave robotic systems

ABSTRACT For certain robotic applications, the manipulator should be tele-operated to reach a specified position or track a desired trajectory, with desired compliance, in a hazardous working environment. Hence, how to construct a two-way position/force-transmitted master/slave system for tele-manipulating end-effector motion by an operator is a relevant research topic. Here, a novel low-cost system structure has been constructed with a Novint Falcon tactile device as the master control and a 5 DOF robot arm the slave end. Appropriate position and contact force transformation between both mechanisms were established and transmitted through an RS232 interface. An operator can manipulate the Novint Falcon tactile device to deliver the position and force output commands to the slave robot arm controller for monitoring the position and force responses of the end-effector. A torque/force sensor was installed at the robotic wrist to measure the contact force, and feedback to the master end as hand feeling for two-way impedance motion harmonization. The experimental results show that the operator can tele-operate the robotic manipulator to follow the commanded trajectory with appropriate contact force and get proportional force feedback to the operator’s hand, via the Novint Falcon device.

Serious Game Platform with Haptic Feedback and EMG Monitoring for Upper Limb Rehabilitation and Smoothness Quantification on Spinal Cord Injury Patients

Cervical Spinal Cord injury (SCI) is a neurological disease that produces, as a consequence, impairments of the upper limb function. This paper illustrates a virtual reality platform based on three serious games for upper limb rehabilitation with electromyography monitoring, providing force feedback to the patient. In the rehabilitation process proposed, haptic feedback was provided to the patients to strength the arm muscles by means of the Novint Falcon device. This end-effector device was used to manipulate the serious games. During the therapy performance, the system recorded electromyography signals from the patient’s arm muscles, which may be used to monitor muscle contraction. The work presented a virtual reality system developed for spinal cord-injured patients. Each virtual reality environment could be modified in strength and duration according to the patients’ needs and was implemented for recording quantitative data about the motor performance. The platform was validated as a proof of concept in cervical spinal cord-injured patients. Results showed that this rehabilitation platform could be used for obtaining objective information in relation to motor control characteristics.

Shared Autonomy of a Flexible Manipulator in Constrained Endoluminal Surgical Tasks

Automation in surgery is becoming more and more active in recent years. From the editorial of Science Robotics in 2017, there are six levels of autonomy and the existing tele-operation lies in level 0 (no autonomy). Although full autonomy would be fictional in the current moment, task/shared autonomy would be highly achievable. In endoluminal surgeries, surgeons rely on manual operation of flexible instruments for tissue manipulation and dissection. The techniques are difficult to learn, which makes them a good candidate for automation. In this letter, we introduce our work on shared autonomy of a flexible manipulator in simulated constrained endoluminal tasks, e.g., tissue dissection task in transanal total mesorectal excision and endoscopic submucosal dissection operations. During the procedure, the operator defines the dissection trajectory with a Novint Falcon device by marking via points. The dissection trajectory is dynamically planned based on the feedback from stereo endoscope system with consideration of the deformation of material. Furthermore, trajectory of the flexible manipulator is controlled by an adaptive controller with consideration of the possible collision between instruments and surrounding environment. This framework is validated by a mock up test with a three-degree-of-freedom flexible manipulator.

Perturbed Point-to-Point Reaching Tasks in a 3D Environment Using a Portable Haptic Device

In this paper, we propose a new protocol, integrating Virtual Reality with the Novint Falcon, to evaluate motion performance during perturbed 3D reaching tasks. The protocol consists of six 3D point-to-point reaching tasks, performed using Falcon with six opposing force fields. Twenty subjects were enrolled in the study. During each task, subjects reached 80 targets and the protocol was repeated over three different days. The trajectories of the end-effector were recorded to calculate: duration of movement, length ratio, lateral deviation, aiming angle, speed metric, and normalized jerk. The coefficient of variation was calculated to study the intra-subject variability and the intra-class correlation coefficient to assess the reliability of the indices. Two-way repeated measurement ANOVA tests were performed for all indices in order to ascertain the effects of force and direction on the trajectories. Duration of movement, length ratio and speed metric have proven to be the most repeatable and reliable indices. Considering the force fields, subjects were able to optimize the trajectory in terms of duration and accuracy but not in terms of smoothness. Considering the directions, the best motor performance occurred when the trajectories were performed in the upper quadrant compared to those performed in the lower quadrant.

An experimental oscillation damping impedance control for the Novint Falcon haptic device based on the phase trajectory length function concept

This paper aims at designing an oscillationless open-loop impedance controller for a well-known low-cost haptic device, namely the Novint Falcon haptic device. In this regard, the oscillation number index standing for the ratio of the traversed path in the corresponding phase portrait systems for a certain state to its radial value is employed to evaluate the oscillatory behavior of the system. Based on the identified parameters of the dynamic equation, a precise model of the intended robot is developed in the SimMechanics simulator. According to the obtained results in the simulation, one can infer that there is a certain impedance value in which the value of the oscillation number index has its minimum value. The obtained experimental results reveal that the oscillation number index based controller possesses lower oscillation in comparison to the Falcon’s official software development kit. In fact, by tuning the system’s impedance will result in a better performance for the haptic purposes.

Carpal Tunnel Syndrome Rehabilitation Through Force Feedback

The present paper proposes a carpal tunnel syndrome oriented system by means of the use of haptic devices with force feedback. The system, based on entertainment, handles different applications, as well as movements in a 3D graphic environment, which is devised with daily tasks that allow to develop skills and abilities to reduce the patient´s affection. The system is designed with Unity3D along with Novint Falcon haptic device. In which, the patient interacts with the developed applications while receives force feedback, at the same time, the patient performs physiotherapeutic exercises that attack the affection in a proper manner to improve patient´s health. Experimental results manifest system´s validity, which generates necessary and efficient exercises for the process of carpal tunnel rehabilitation, besides, it deploys a human-machine interaction oriented to the development of physic therapies.

Adaptive Control of a Robot-Assisted Tele-Surgery in Interaction With Hybrid Tissues

In a haptic teleoperation system, which interacts with unknown and hybrid environments, it is important to achieve stability and transparency. In medical usages, the utilization of knowledge on the tissues behavior in a controller design can improve the performance of the surgery in a robot-assisted telesurgery. Simultaneous interaction with hard and soft tissues makes it difficult to achieve stability and transparency. To deal with this difficulty, two controller schemes are designed. At first, a nonlinear mathematical model (inspired by the Hunt-Crossley (HC) model), which has the properties of soft and hard tissues, is combined with the slave side dynamic. In the second approach, the reaction force applied by hybrid tissues during the transition between tissues of different properties is modeled as an unknown force acting on the slave side. In a four-channel (4-CH) architecture, nonlinear adaptive controllers are designed without any knowledge about the parameters of the master, the slave robot, and the environment. For both control schemes, Lyapunov candidate functions provide a way to ensure the stability and transparency in the presence of uncertainties. The testbed comprises two Novint Falcon robots functioning as master and slave robots. Moreover, the experiments are performed on various objects, including a soft cube, a hard cube, and a phantom tissue. This paper rigorously evaluates the performances of the proposed methods, comparing them with each other and other previous schemes. Experimental and numerical results demonstrate the effectiveness of the proposed control schemes.

Occlusion-Free Visual Servoing for the Shared Autonomy Teleoperation of Dual-Arm Robots

Visual servoing in telerobotics provides information to the operator about the remote location and assists in the task execution to reduce stress on the user. Occlusions in this scenario can, on the one hand, lead to the visual servoing failure, and, on the other hand, degrade the user experience and navigation performance due to the obstructed vision of elements of interest. In this letter, we consider a teleoperation system composed of two robot arms where one is remotely operated, while the other is autonomous and equipped with an eye-in-hand camera sensor. We propose a reactive unified convex optimization based controller that allows the execution of occlusion-free tasks by autonomously adjusting the camera robot, so as to keep the teleoperated one in the field of view. The occlusion avoidance is formulated inside the optimization as a constraint in the image space, it is formally derived from a collision avoidance analogy and made robust against noisy measurements and dynamic environment. A state machine is used to switch the control policy whenever an occlusion might occur. We validate our approach with experiments on a 14 d.o.f. dual-arm ABB YuMi robot equipped with an red, green, blue (RGB) camera and teleoperated by a 3 d.o.f. Novint Falcon device.

Experimental dynamic identification and model feed-forward control of Novint Falcon haptic device

This paper addresses the dynamic identification and model feed-forward control implementation of a Novint Falcon haptic device. Firstly, inverse and forward kinematic problem of the device is obtained. Subsequently, equations describing the robot’s dynamics is derived using Newton–Euler approach which includes friction of active joints. The unknown dynamic parameters including mass and moment of inertia of constituting parts along with dry and viscous friction coefficients is identified practically by a white-box identification approach using LS technique. Furthermore, the torque digital scale coefficient which Novint Falcon’s firmware uses is identified. To the end of obtaining some of regressors a novel numerical-based differentiation approach is presented. Having an experimentally verified dynamic model in hand, the model is used in a model feed-forward impedance controller in order to omit the effect of natural dynamics of the robot, including active joints frictions from displayed impedance. Experimental tests reveal that the implemented force sensor-less controller performs 26% more accurate in terms of output force comparing device’s official controller.

T-S Fuzzy-Based Optimal Control for Minimally Invasive Robotic Surgery with Input Saturation

A minimally invasive surgery robot is difficult to control when actuator saturation exists. In this paper, a Takagi-Sugeno fuzzy model-based controller is designed for a minimally invasive surgery robot with actuator saturation, which is difficult to control. The contractively invariant ellipsoid theorem is applied for the actuator saturation. The proposed scheme can be derived using the H-infinity control theorem and parallel distributed compensation. The result is rebuilt in the form of linear matrix inequalities for easier calculation by computer. Meanwhile, the uniformly ultimately bounded stable and the prescribed H-infinity control performance can be guaranteed. The proposed scheme is simulated in a Novint Falcon haptic device system.

Low-end haptic devices for knee bone drilling in a serious game

PurposeThe purpose of this paper is to examine the application of low-end, low-fidelity (gaming/consumer-level) haptic devices for medical-based, surgical skills development (surgical bone-based drilling in particular) with serious games and virtual simulations as an affordable training solution with the potential of complementing current and traditional training methods.Design/methodology/approachThe authors present the adaptation of two low-end haptic devices (Novint Falcon and Geomagic 3D Touch) to simulate a surgical drill drilling through bone for a serious game developed for total knee arthroplasty training. The implementation was possible through the analysis of the bone drilling mechanics. The authors provide a quantitative comparison of both haptic devices with respect to forces, movements, and development.FindingsAlthough further testing is required, the initial results demonstrate that the low-end, consumer-level haptic devices can be incorporated into virtual environments/serious games to allow for the simulation of surgical drilling. The authors also believe that the results will generalize and allow these devices to be used to simulate a variety of technical-based medical procedures.Originality/valueIn contrast to previous work where the focus is placed on cost-prohibitive haptic devices, this approach considers affordable consumer-level solutions that can be easily incorporated into a variety of serious games and virtual simulations. This holds promise that haptic-based virtual simulation and serious games become more widespread, ultimately ensuring that medical trainees are better prepared before exposure to live patients.

Force-Feedback Integration with NASA’s Next Generation Air Transportation System Cockpit Situation Display

NASA’s Next Generation Air Transportation System Volumetric Cockpit Situation Display is an advanced software tool for real-time flight management from the cockpit. In its current development, users interact with visual elements on the display to retrieve operational information and manipulate objects using a standard computer mouse. In the present study, force feedback is integrated in the Cockpit Situation Display framework and its effectiveness on the performance of two tasks is investigated: object selection and route manipulation. For the object-selection task, an attractive force feedback was used to guide the user to the target. For the route-manipulation task, guidance was generated as variable force feedback, corresponding to the changes to the flight path, relative to obstacles. The effectiveness of the proposed force-feedback models was evaluated in user studies using a Novint Falcon haptic device. It is found that force feedback improves the movement time in the object-selection tasks for diagonal movements. For the route-manipulation tasks, force feedback results in a longer movement time as compared to the mouse; however, force feedback increases the movement precision. Further investigation on the interaction effect of feedback levels and task complexity is needed to evaluate the overall effectiveness of force feedback for use with the Cockpit Situation Display.