Teleoperation Applications Research Articles

A flexible multimodal tactile display array for virtual shape and texture

Multimodal tactile display systems that are capable of delivering versatile tactile information effectively are of great interest for virtual reality and tele-operation applications. In this paper, we present a new flexible multimodal tactile display system that delivers shape and texture information simultaneously using a single actuator. The proposed tactile display is actuated by a combination of electromagnetic and pneumatic forces and thus exhibits high tactile force to deliver shape information and large vibration amplitude to deliver texture information. In addition, the proposed tactile display device was implemented on a flexible PDMS structure to allow for tight attachment on any curved body to transfer reliable tactile information. By applying a current pulse of 300 mA for 5 ms, we readily controlled the state of the tactile actuator and measured the actuation displacement of 900 μm and the generated force of 20.1 gf. The average vibration amplitude of 30 μm was measured over a wide dynamic range (25---650 Hz) with an applied current of 74 mA. Lastly, by delivering various tactile information to users, we verified the functionalities of the implemented 4 ? 1 multimodal tactile display array. User study results showed that the fabricated multimodal tactile display system successfully delivered both shape information with an accuracy of 73 % and texture information with an accuracy of 90 %. This work demonstrates the potential of our multimodal tactile display system for uses in various applications such as tele-operation, tactile communication, and visual presentation for visually impaired.

Position Control Mathematical Modelling and Operational Evaluation of Tele-Operated Electro-Hydraulic Actuator (T-EHA)

This study deals with a master-slave system for a tele-operated electro-hydraulic actuator (T-EHA) that focuses on a hydraulic system to remotely control a mini excavator. Tele-operation using such system is useful for tele-operation support of heavy construction and road restoration, typically in post-disaster areas. This paper presents the current development of position control electro-hydraulic actuator for such remote tele-operation application. A 2.4 GHz radio-controlled transmitter and receiver unit, which is also known as the master, has been utilized as the remote controller for an electro-hydraulic actuator. The electro-hydraulic actuator, which serves as the slave has been fabricated by using a tie-rod cylinder, and coupled with a 24 VDC electro-hydraulic valve. Position control mathematical modelling and operational evaluation have been studied with regard to the tele-operated electro-hydraulic actuator.

Service-oriented approach to fault tolerance in CPSs

Cyber-physical systems (CPSs) are open and interconnected embedded systems that control or interact with physical processes. Failures in CPSs can lead to loss of production time, damage to the equipment and environment, or loss of life, meaning that dependability and resilience are key properties for their design. However, existing fault tolerance and safety approaches are inadequate for complex, networked and dynamic CPSs. Service-orientation, on the other hand, is generally considered to be a robust architectural style, but there is a limited amount of research on fault tolerance of service-oriented architecture (SOA), especially on distributed real-time systems. We propose an approach that utilizes the loosely coupled nature of services to implement fault tolerance using a middleware-based real-time SOA (RTSOA) for CPSs. The approach, based on the concepts of fault isolation and recovery at the service level, is empirically evaluated using a demanding bilateral teleoperation (remote handling) application. The empirical evaluation demonstrates that RTSOA supports real-time fault detection and recovery, use of services as a unit of fault isolation, and it provides capability to implement fault tolerance patterns flexibly and without significant overhead.

A 3-PRS parallel manipulator for ankle rehabilitation: towards a low-cost robotic rehabilitation

SUMMARYThis paper presents the design, kinematics, dynamics and control of a low-cost parallel rehabilitation robot developed at the Universitat Politècnica de Valencia. Several position and force controllers have been tested to ensure accurate tracking performances. An orthopedic boot, equipped with a force sensor, has been placed over the platform of the parallel robot to perform exercises for injured ankles. Passive, active-assistive and active-resistive exercises have been implemented to train dorsi/plantar flexion, inversion and eversion ankle movements. In order to implement the controllers, the component-based middleware Orocos has been used with the advantage over other solutions that the whole scheme control can be implemented modularly. These modules are independent and can be configured and reconfigured in both configuration and runtime. This means that no specific knowledge is needed by medical staff, for example, to carry out rehabilitation exercises using this low-cost parallel robot. The integration between Orocos and ROS, with a CAD model displaying the actual position of the rehabilitation robot in real time, makes it possible to develop a teleoperation application. In addition, a teleoperated rehabilitation exercise can be performed by a specialist using a Wiimote (or any other Bluetooth device).

Bilateral shared autonomous systems with passive and nonpassive input forces under time varying delay

In this paper, we address stability and tracking control problem of bilateral shared autonomous systems in the presence of passive and nonpassive input interaction forces. The design comprises delayed position and position–velocity signals with the known and unknown structures of the master and slave manipulator dynamics. Using novel Lyapunov–Krasovskii functional, stability and tracking conditions of the coupled master–slave shared autonomous systems are developed under symmetrical and unsymmetrical time varying data transmission delays. This condition allows the designer to estimate the control design parameters to ensure position, velocity and synchronizing errors of the master and slave manipulators. Finally, evaluation results are presented to demonstrate the validity of the proposed design for real-time teleoperation applications.

DEVELOPMENT OF A SIX-DIMENSIONAL SENSOR FOR MINIMALLY INVASIVE ROBOTIC SURGERY

In minimally invasive robotic surgery (MIRS), the force/torque which occurred between instruments and organs cannot be accessed by surgeon. This paper presents development of a six-dimensional sensor based on double-hole parallel crossing beam, which can be integrated into instruments of MIRS. The size of sensor is 9.8 mm (diameter) × 6 mm (height). The structure of the sensor can acquire the force signals directly. The decoupling mechanism of the sensor was analyzed. The result of the finite element analysis (FEA) showed that the maximum coupling error was 3.8%. The machining error of the sensor was also investigated, and it was feasible for numerical control (NC) machine tools to manufacture the components of the sensor. The experimental calibration and soft tissue experiment indicated that the developed sensor can measure the force/torque loaded on the instrument and can be used to obtain the force feedback in the application of teleoperation surgical robot.

A method for passivity analysis of multilateral haptic systems

This paper presents a novel criterion to study the stability of multilateral teleoperation systems based on passivity. Such systems (modeled as n-port networks) have recently found interesting applications in cooperative haptic teleoperation and haptic-assisted training. The criterion provides researchers with an analytical, closed-form, necessary, and sufficient condition useful for both analysis and design of multilateral haptic teleoperation systems. The paper shows that when the proposed conditions reduce to the well-known Raisbeck’s passivity criterion for 2-port networks. The proposed conditions are used to study the passivity (and consequently the stability) of a dual-user haptic system for control of a single teleoperated robot. Simulations and experiments are performed to further test the validity of the proposed criterion.

Redundantly actuated 3-RRR spherical parallel manipulator used as a haptic device: improving dexterity and eliminating singularity

SUMMARYThis paper discusses the study of a spherical parallel manipulator (SPM) used as a haptic device for tele-operation applications. The SPM presents poor behavior in singular configurations. Redundancy is used to eliminate the parallel singularity without major changes in the mechanical structure. Comparisons in terms of kinematic and dynamic behavior between the non-redundant and redundant SPM are presented. The results prove the advantage of introducing redundancy in the actuator and sensor to improve the behavior of the SPM. A new control strategy for the redundant SPM is also proposed. The control strategy has been successfully tested and validated on a SimMechanics model.

An Optimization Approach to Teleoperation of the Thumb of a Humanoid Robot Hand: Kinematic Mapping and Calibration

The complex kinematic structure of a human thumb makes it difficult to capture and control the thumb motions. A further complication is that mapping the fingertip position alone leads to inadequate grasping postures for current robotic hands, many of which are equipped with tactile sensors on the volar side of the fingers. This paper aimed to use a data glove as the input device to teleoperate the thumb of a humanoid robotic hand. An experiment protocol was developed with only minimum hardware involved to compensate for the differences in kinematic structures between a robotic hand and a human hand. A nonlinear constrained-optimization formulation was proposed to map and calibrate the motion of a human thumb to that of a robotic thumb by minimizing the maximum errors (minimax algorithms) of fingertip position while subject to the constraint of the normals of the surfaces of the thumb and the index fingertips within a friction cone. The proposed approach could be extended to other teleoperation applications, where the master and slave devices differ in kinematic structure.

Teleoperation of ABB industrial robots

Purpose – The purpose of this paper is to analyse teleoperation of an ABB industrial robot with an ABB IRC5 controller. A method to improve motion smoothness and decrease latency using the existing ABB IRC5 robot controller without access to any low-level interface is proposed. Design/methodology/approach – The proposed control algorithm includes a high-level proportional-integral-derivative controller (PID) controller used to dynamically generate reference velocities for different travel ranges of the tool centre point (TCP) of the robot. Communication with the ABB IRC5 controller was performed utilising the ABB PC software development kit. The multitasking feature of the IRC5 controller was used to enhance the communication frequency between the controller and the remote application. Trajectory tracking experiments of a pre-defined three-dimensional trajectory were carried out and the benefits of the proposed algorithm were demonstrated. The robot was intentionally installed on a wobbly table and its vibrations were recorded using a six-degrees-of-freedom force/torque sensor fitted to the tool mounting interface of the robot. The robot vibrations were used as a measure of the smoothness of the tracking movements. Findings – A communication rate of up to 250 Hz between the computer and the controller was established using C# .Net. Experimental results demonstrating the robot TCP, tracking errors and robot vibrations for different control approaches were provided and analysed. It was demonstrated that the proposed approach results in the smoothest motion with tracking errors of < 0.2 mm. Research limitations/implications – The proposed approach may be employed to produce smooth motion for a remotely operated ABB industrial robot with the existing ABB IRC5 controller. However, to achieve high-bandwidth path following, the inherent latency of the controller must be overcome, for example by utilising a low-level interface. It is particularly useful for applications including a large number of short manipulation segments, which is typical in teleoperation applications. Social implications – Using the proposed technique, off-the-shelf industrial robots can be used for research and industrial applications where remote control is required. Originality/value – Although low-level control interface for industrial robots seems to be the ideal long-term solution for teleoperation applications, the proposed remote control technique allows out-of-the-box ABB industrial robots with IRC5 controllers to achieve high efficiency and manipulation smoothness without requirements of any low-level programming interface.

Nonlinear control for teleoperation systems with time varying delay

In this paper, we introduce delay-dependent control strategies for bilateral teleoperation systems in the presence of passive and constant input forces under time varying delay. We first design teleoperation systems where the local and remote sites are coupled by position signals of the master and slave manipulator. The design also combined undelayed position and velocity signals with nonlinear adaptive control terms to deal with the parametric uncertainties associated with the dynamical model of the master and slave manipulator. Then, we develop teleoperators by delaying position and velocity signals of the master and slave manipulator. Using Lyapunov–Krasovskii function, delay-dependent stability and tracking conditions for both teleoperators are developed in the presence of symmetrical and unsymmetrical time varying delays. The stability conditions are established in the presence of passive and constant human and environment interaction forces with the master and slave manipulators. Finally, simulation results are presented to demonstrate the validity of the theoretical development of the proposed designs for real-time teleoperation applications.

Teleoperation Systems With Symmetric and Unsymmetric Time Varying Communication Delay

This paper addresses the stability and tracking control problem for passive and nonpassive input-based teleoperation systems under time varying telecommunication delays. The design comprises delayed position signals with the local velocity and known structure of the master and slave manipulator dynamics. Using Lyapunov-Krasovskii-like functional, we derive the stability condition for the closed-loop teleoperators under symmetrical and unsymmetrical time varying delays. These conditions ensure the control stability and tracking of the closed-loop teleoperation systems in the presence of passive and nonpassive input forces. Finally, various simulation results are presented to confirm the validity of the proposed designs for real-time teleoperation application.

Haptic Hand Exoskeleton for Precision Grasp Simulation

This paper outlines the design and the development of a novel robotic hand exoskeleton (HE) conceived for haptic interaction in the context of virtual reality (VR) and teleoperation (TO) applications. The device allows exerting controlled forces on fingertips of the index and thumb of the operator. The new exoskeleton features several design solutions adopted with the aim of optimizing force accuracy and resolution. The use of remote centers of motion mechanisms allows achieving a compact and lightweight design. An improved stiffness of the transmission and reduced requirements for the electromechanical actuators are obtained thanks to a novel principle for integrating speed reduction into torque transmission systems. A custom designed force sensor and integrated electronics are employed to further improve performances. The electromechanical design of the device and the experimental characterization are presented.

Transparency Improved Sliding-mode Control Design for Bilateral Teleoperation Systems by Using Virtual Manipulator Concept

This paper proposes a new methodology for designing sliding mode bilateral controllers based virtual manipulator concept that aims to reach the most transparency considering parameter uncertainties and disturbances. This method shows a modified scheme of the sliding mode control that leads the alleviation of the tradeoff between position and force tracking, which is crucial in certain teleoperation applications. An analysis of stability and transparency of the closed-loop teleoperation system is carried out. In addition, a new fuzzy logic scheme is also proposed to eliminate the chattering phenomena caused by sign function in the sliding mode controller. Simulation results show the superior performance of the proposed fuzzy sliding mode scheme, with respect to previous method, particularly in term of significantly ameliorating transparency in the presence of unknown parameters and disturbances.

Sliding-mode bilateral teleoperation control design for master–slave pneumatic servo systems

This paper presents a novel bilateral control design scheme for pneumatic master–slave teleoperation systems that are actuated by low-cost solenoid valves. The motivation for using pneumatic actuators in lieu of electrical actuators is that the former has higher force to mass ratio than the latter and is inert to magnetic fields, which is crucial in certain teleoperation applications such as MRI-guided, robot-assisted surgery. A sliding mode approach, called the three-mode control scheme, is incorporated into a two-channel bilateral teleoperation architecture, which can implement a position–position, force–force, or force–position scheme. An analysis of stability and transparency of the closed-loop teleoperation system is carried out. The proposed control design performance is experimentally verified on a single-degree-of-freedom pneumatic teleoperation system actuated by on/off valves. Experimental results show high accuracies in terms of position and force tracking under free-space motion and hard-contact motion in the teleoperation system. Another purpose of this paper is to demonstrate the possibility to improve the valve lifetime by increasing the number of control levels. To do this, a new control design, called the five-mode control scheme, is developed and compared with the three-mode scheme in time domain as well as in frequency domain.

Application of Teleoperation for Rehabilitation Training Robot

The paper introduces the structure of telerehabilitation robot system based on network. The detailed design and implementation of the established experimental system is presented, which included the design and the implementation of hardware and software and the settings of the program work and the experiments. The experimental results of the telerehabilitation robot system are shown, which demonstrated the feasibility of the application of teleoperation for rehabilitation robot.

Issues of identifying, estimating and using delay times in telecontrol systems based on TCP/IP networks

Today's performant equipment and high speed connections make network control systems a reality. For a series of teleoperation applications like the ones from network control systems domain it is essential to estimate the time delay between the acquisition moment of primary data that needs to be sent and the moment it is available for the receiver's use. This paper presents a methodology for estimating the time delay from experimental measurements, while also taking into account the information loss during transmission. A series of unwanted exceptions that may occur in signal transmission are identified and solution methods are proposed. Based on the above study, three real scenarios were designed for three TCP/IP network types. These scenarios were implemented through simulations for a telecontrol system that uses adaptive PD compensators for the time-varying delays. Finally, the improvement in the system's performances with the PD compensators were validated using real life scenarios.

Integrating Teams of Mobile Robots in Wireless Ad-Hoc Networks

Modern applications of mobile robot teams or robot teleoperation often demand wireless any-to-any communication in combination with highly dynamical network topologies to accomplish more and more complex tasks. These communication systems should also not rely on any pre-existing infrastructure, have low costs, and should allow for a seamless integration into existing communication networks. The well tested technology IEEE 802.11 wireless LAN offers all these possibilities. Especially the capability of realizing ad-hoc networks offers a high potential for WLAN to be used in the area of networked robots. Mesh networks and mobile ad-hoc networks are able to provide mechanisms for realizing stable communication links for networked robots. This work gives a brief overview on mesh networks and ad-hoc networks and further shows the behavior of different ad-hoc routing protocols (AODV, OLSR, DSR, BATMAN) in typical scenarios for remote operation of mobile robots. Also the differences between the application scenarios of mobile robot teams and in pure telecommunication environments are explained. The presented results are not only based on simulations. The ad-hoc routing protocols are compared in test scenarios with real mobile robot hardware experiments and the analysis is done according to well established methods and valid metrics well known from literature. Hints for the setup and parameter tuning of selected ad-hoc routing protocols are given to enable a better teleoperation of mobile robots and to improve the communication inside teams of mobile robots and humans. Finally a first approach is presented how to increase the system performance also on the application layer.

A Review of exoskeleton-type systems and their key technologies

The exoskeleton-type system is a brand new type of man—machine intelligent system. It fully combines human intelligence and machine power so that machine intelligence and human operator's power are both enhanced. Therefore, it achieves a high-level performance that neither could separately. This paper describes the basic exoskeleton concepts from biological system to man—machine intelligent systems. It is followed by an overview of the development history of exoskeleton-type systems and their two main applications in teleoperation and human power augmentation. Besides the key technologies in exoskeleton-type systems, the research is presented from several viewpoints of the biomechanical design, system structure modelling, cooperation and function allocation, control strategy, and safety evaluation.

Bioinspired Robotic Dual-Camera System for High-Resolution Vision

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Due to the limited resolution of both cameras and displays, acuity of artificial vision systems is currently well below the human eye. Visual acuity, in cameras as well as in animal eyes, can be increased by making smaller receptors or bigger eyes. In some applications, the size of the camera is constrained, so alternative solutions must be sought. This paper presents a robotic dual-camera vision system whose design is inspired by the visual system of jumping spiders (<emphasis emphasistype="italic">Salticidae</emphasis> family). The system is composed of a telephoto camera whose field of view (FOV) can be moved within the larger FOV of a wide-angle camera and allows to form a high-resolution image, i.e., an image with the FOV of the wide-angle camera, yet having the same resolution as the telephoto camera. We describe the design of the robotic system, the direct and inverse kinematics, and the image processing algorithms that allow to build the high-resolution image. Images from experiments are presented, together with a discussion on sources of errors and possible solutions. The system is particularly useful for fixed-camera monitoring or teleoperation applications, such as remote surveillance and minimally invasive surgery. The system achieves seven times higher resolution than typical commercial endoscopes. </para>