Semi-autonomous Control Research Articles

The Design and Realization of Real-Time Video Transmission System Using in Tracked Robot

In order to solve the poor quality of video transmission system on the platform for tracked robot motion, we have designed a video transmission system of tracked robot motion platform adopting a streaming media SDK of DirectShow and the real-time transmission protocol of RTP, and we also analyzed and improved the communication quality of the system. The experimental results show that the new design has greatly advantage in reducing the packet loss ratio and time delay phenomenon, and enhance the quality of the video transmission. The conclusion of the research has a great significance in promoting the accuracy and timeliness in the semi-autonomous control of tracked robot under complicated environment.

Stereovision and augmented reality for closed-loop control of grasping in hand prostheses

Objective. Technologically advanced assistive devices are nowadays available to restore grasping, but effective and effortless control integrating both feed-forward (commands) and feedback (sensory information) is still missing. The goal of this work was to develop a user friendly interface for the semi-automatic and closed-loop control of grasping and to test its feasibility. Approach. We developed a controller based on stereovision to automatically select grasp type and size and augmented reality (AR) to provide artificial proprioceptive feedback. The system was experimentally tested in healthy subjects using a dexterous hand prosthesis to grasp a set of daily objects. The subjects wore AR glasses with an integrated stereo-camera pair, and triggered the system via a simple myoelectric interface. Main results. The results demonstrated that the subjects got easily acquainted with the semi-autonomous control. The stereovision grasp decoder successfully estimated the grasp type and size in realistic, cluttered environments. When allowed (forced) to correct the automatic system decisions, the subjects successfully utilized the AR feedback and achieved close to ideal system performance. Significance. The new method implements a high level, low effort control of complex functions in addition to the low level closed-loop control. The latter is achieved by providing rich visual feedback, which is integrated into the real life environment. The proposed system is an effective interface applicable with small alterations for many advanced prosthetic and orthotic/therapeutic rehabilitation devices.

A learning-based semi-autonomous controller for robotic exploration of unknown disaster scenes while searching for victims.

Semi-autonomous control schemes can address the limitations of both teleoperation and fully autonomous robotic control of rescue robots in disaster environments by allowing a human operator to cooperate and share such tasks with a rescue robot as navigation, exploration, and victim identification. In this paper, we present a unique hierarchical reinforcement learning-based semi-autonomous control architecture for rescue robots operating in cluttered and unknown urban search and rescue (USAR) environments. The aim of the controller is to enable a rescue robot to continuously learn from its own experiences in an environment in order to improve its overall performance in exploration of unknown disaster scenes. A direction-based exploration technique is integrated in the controller to expand the search area of the robot via the classification of regions and the rubble piles within these regions. Both simulations and physical experiments in USAR-like environments verify the robustness of the proposed HRL-based semi-autonomous controller to unknown cluttered scenes with different sizes and varying types of configurations.

A margin-based approach to vehicle threat assessment

In this paper we propose a novel approach to the threat assessment problem for Advanced Driver Assistance System (ADAS) and autonomous navigation decision making support. This threat assessment is based on estimation of the control margin afforded to a vehicle and is performed in a multi–threat framework. Given sensor information available about the surrounding environment, an algorithm first identifies corridors of travel through which the vehicle can safely navigate. The second stage then assesses the threat posed to the vehicle in each identified corridor via a metric associated with available control margin. For this purpose, the corridors are approximated by sets of trajectories generated from a lattice sampled in the vehicle's input space. The level of threat can then serve to influence autonomous navigation as an input to a decision–making layer. It also potentially allows a semi–autonomous control system to honour driver intent while ensuring safe and robust navigation in hazardous events. The benefit of such an approach is compared to common threat metrics in canonical scenarios. The method is also applied to the multi–lane road environment of highway navigation by post processing human driving data gathered from a simulator.

Remote Control System for Mobile Robot with Semi-autonomous Control Based on Degree of Collision Danger

Remote Control System for Mobile Robot with Semi-autonomous Control Based on Degree of Collision Danger

Task performance evaluation of asymmetric semiautonomous teleoperation of mobile twin-arm robotic manipulators.

A series of human factors experiments involving maneuvering and grasping tasks are carried out to evaluate the effectiveness of a novel asymmetric semiautonomous teleoperation (AST) control design framework for teleoperation of mobile twin-arm robotic manipulators. Simplified configurations are examined first to explore control strategies for different aspects of such teleoperation tasks. These include teleoperation of a nonholonomic mobile base, telemanipulation of a dual-arm robot, and dual-arm/dual-operator teleoperation task scenarios. In two sets of experiments with a planar nonholonomic mobile base, teleoperation via a 3DOF planar haptic interface with position mapping and force reflection of the nonholonomic constraint decreases task-completion-time (TCT) and reduces unwanted collisions. In dual-arm and dual-operator teleoperation maneuverability experiments, the assignment of decoupled and nonconflicting control frames reduces TCT and unwanted contacts. The use of so-called "soft" constraints via passive semiautonomous control reduces TCT and unwanted block drops in telegrasping experiments with a twin-arm manipulator. A final comprehensive experiment encompassing elements of the simplified configurations demonstrates the effectiveness of AST control framework in dual-operator teleoperation of a twin-arm mobile manipulator.

Using Haptic Feedback in Human Robotic Swarms Interaction

Robotic swarms display emergent behaviors that are robust to failure of individual robots, although they can not necessarily accomplish complex tasks with these behaviors. The research objective is to make use of their robust behaviors to accomplish complex tasks in many types of environments. For now, it is difficult to affect swarm “goals”, and therefore difficult them to direct to perform complex tasks. The extant literature on Human Swarm Interaction (HSI) focuses on demonstrating the usefulness of human operator inputs for swarms to accomplish complex tasks. The human typically gets visual feedback of the state of the swarm and influences the robots through a computer interface. This paper presents a user study investigating the effectiveness of haptic feedback in improving HSI. We use methods developed in studies using haptics in multi-robot systems (where the communication and structure is very rigid) and potential field algorithms developed for fully-autonomous swarms to determine the benefits of haptic feedback from the semi-autonomous control algorithm. In some environments, haptic feedback proved beneficial whereas in other environments haptic feedback did not improve performance over visual feedback alone. However, presence of haptic feedback did not degrade the performance under any of the experimental conditions. This supports our working hypothesis that haptic feedback is potentially useful in HSI.

Robotic Urban Search and Rescue: A Survey from the Control Perspective

Robotic urban search and rescue (USAR) is a challenging yet promising research area which has significant application potentials as has been seen during the rescue and recovery operations of recent disaster events. To date, the majority of rescue robots used in the field are teleoperated. In order to minimize a robot operator's workload in time-critical disaster scenes, recent efforts have been made to equip these robots with some level of autonomy. This paper provides a detailed overview of developments in the exciting and challenging area of robotic control for USAR environments. In particular, we discuss the efforts that have been made in the literature towards: 1) developing low-level controllers for rescue robot autonomy in traversing uneven terrain and stairs, and perception-based simultaneous localization and mapping (SLAM) algorithms for developing 3D maps of USAR scenes, 2) task sharing of multiple tasks between operator and robot via semi-autonomous control, and 3) high-level control schemes that have been designed for multi-robot rescue teams.

Control of semi-autonomous teleoperation system with time delays

In teleoperation systems operating in complex environments, due to the cognitive limitations of the human operator and lack of complete information about the remote environment, safety and performance of such systems can potentially be comprised. In order to ensure the safety and enhance the efficiency of complex teleoperation systems operating in cluttered environments, in this paper we investigate a semi-autonomous control framework for bilateral teleoperation. The semi-autonomous teleoperation system is composed of heterogeneous master and slave robots, where the slave robot is assumed to be a redundant manipulator. Considering robots with different configurations, and in the presence of dynamic uncertainties and asymmetric communication delays, we first develop a control algorithm to ensure position and velocity tracking in the task space. Additionally in the absence of dynamic uncertainty, and in the presence of human operator and environmental forces, all signals of the proposed teleoperation system are proven to be ultimately bounded. The redundancy of the slave robot is then utilized for achieving autonomous sub-task control, such as singularity avoidance, joint limits, and collision avoidance. The control algorithms for the proposed semi-autonomous teleoperation system are validated through numerical simulations on a non-redundant master and a redundant slave robot.

2A2-T08 Semi-autonomous Steering Control for Cycling Wheelchair(Welfare Robotics and Mechatronics (3))

Aim at safety and maneuverability, this paper proposes a semi-autonomous control of cycling wheelchair by steering power assist. In this study, a dynamic window approach is utilized to be the theoretical basis of navigation, steering power assist by servo motor is utilized to realize the actual maneuver. The proposed control scheme is implemented oil a mobile robot, its effectiveness is assessed by experiment.

Development of Smart Wheelchair System for a User with Severe Motor Impairment

Users with severe motor impairment may find it difficult to operate a wheelchair when they are in tight space (e.g., passing doorway) or when avoiding obstacles since they cannot command the wheelchair by means of a conventional joystick. Here we propose a framework that can assist users to overcome such circumstances using a hierarchical semi-autonomous control strategy. Initially multimodal user inputs based on momentary switch and yaw head angle are analyzed to decide a maneuvering mode and assign the direction of travel. Then, environmental information is perceived using the combination of a laser range finder and the Kinect sensor for determining safety map around wheelchair's vicinity. Eventually, the user's inputs are provided to the navigation planner along with the safety map to moderate motion that is collision free and the best for user preference. Experimental results demonstrate the feasibility of the proposed approach.

Visual Marker System for Autonomous Object Handling by Assistive Robotic Arm

The environment structuring concept has been used to design and develop a visual marker system for automating a robotic arm that assists those with upperlimb problems to lead independent lives. Confirmation experiments demonstrate the feasibility of our system in meeting different practical performance requirements. Combining semiautonomous control, simple visual feedback, and our system enabled the assistive robotic arm to handle objects autonomously – something that had not been previously conventionally possible.

Enabling Safety and Mobility through Connectivity

Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) networks within the Intelligent Transportation System (ITS) lead to safety and mobility improvements in vehicle road traffic. This paper presents case studies that support the realization of the ITS architecture as an evolutionary process, beginning with driver information systems for enhancing feedback to the users, semi-autonomous control systems for improved vehicle system management, and fully autonomous control for improving vehicle cooperation and management. The paper will also demonstrate how the automotive, telecom, and data and service providers are working together to develop new ITS technologies.

Robot-Assisted Risky Intervention, Search, Rescue and Environmental Surveillance

Technology has become the solution to many long-standing problems, and while current technologies may be effective, it is far from fully addressing the hug, complex, difficult and challenging tasks associated with disaster missions and risky intervention. The challenge is in finding creative, reliable and applicable technical solutions in such highly constrained and uncertain environment. In addition, it is necessary to overcome constrains on resources by developing innovative, cost effective and practical technology. Robotics can play important intelligent and technological roles that support first response equipment in harsh and dangerous environments while replacing rescue personnel from entering unreachable or unsafe places. Robotics solutions that are well adapted to local conditions of unstructured and unknown environment can greatly improve safety and security of personnel as well as work efficiency, productivity and flexibility. Solving and fulfilling the needs of such tasks presents challenges in robotic mechanical structure and mobility, sensors and sensor fusion, autonomous and semi autonomous control, planning and navigation, and machine intelligence. This paper categorizes the source of disasters and associated missions, and highlights the needs for suitable and reliable technology and technical and functional requirements of robotic systems to fulfill task objectives. In addition, it shows that robotic technologies can be used for disasters prevention or early warning, intervention and recovery efforts during disasters with all possible kinds of relevant missions while ensuring quality of service and safety of human beings. Some of these missions may include: demining, search and rescue, surveillance, reconnaissance and risk assessment, evacuation assistance, intrusion/victim detection and assessment, etc.

Control of a Semi-Autonomous UGV using Lossy Low-Bandwidth Communication

This paper describes various mechanisms for supervisory remote operation of an UGV. Contrary to common Wi-Fi (IEEE 802.11) approaches our approach enables communication via a lossy low-bandwidth radio link. The mechanisms include the massive reduction of environmental sensor data for transmission to the operator. The vehicle itself is capable of semi-autonomous control. Doing so, the operator is removed from the high frequent vehicle control loop and maintains situational awareness.

An optimal-control-based framework for trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in hazard avoidance scenarios

This paper formulates the vehicle navigation task as a constrained optimal control problem with constraints bounding a traversable region of the environment. A model predictive controller iteratively plans an optimal vehicle trajectory through the constrained corridor and uses this trajectory to establish the minimum threat posed to the vehicle given its current state and driver inputs. Based on this threat assessment, the level of controller intervention required to prevent departure from the traversable corridor is calculated and driver/controller inputs are scaled accordingly. Simulated and experimental results are presented to demonstrate multiple threat metrics and configurable intervention laws.

Control under limited information: Special issue (part II)

Control under limited information: Special issue (part II)

Situation Awareness and Team Communication in Robot Control

Operator performance in robotics tasks is impacted by the situation awareness (SA) that individuals or teams are able to acquire and maintain on the robot, the associated environments, and the tasks to be completed. SA can be influenced by team processes when multiple humans must work together for coordinating control of multiple robotic systems. In this study, we investigated the relationship between SA and team verbal communications in a collaborative robotics tasks involving robots under semi-autonomous control. Results indicate that the content, format, and amount of communication can significantly impact operator SA and task performance.

Micro-assembly Robot System

A micro-assembly robot system with multi-manipulator cooperation is developed.The system includes two master micromanipulators with 4 DOFs and one slave micromanipulator with 3 DOFs and adopts double optical path orthogonal micro-vision as the main means to obtain the poses and environmental information of robot and micro-objects.To adapt to the operation requirements of micro-objects of different shapes and materials,the vacuum micro-gripper and piezoelectric ceramic micro-gripper are designed respectively,and their control methods are given on the basis of analyzing the working principle of micro-grippers.Problems such as micro-image acquisition and visual servo are studied,including wiener inverse filter restoration of blurred image,auto focus of microscope based on image analysis,and micro-vision servo control based on the image features of distance and angle.A semi-autonomous control scheme with human-robot interaction is utilized in the micro assembly operation.Assembly results demonstrate that the system works reliably and can accomplish micro assembly task that has complicated technological requirements.Presently the smallest assembly accuracy of micro-parts can attain 30 μm.

Semiautonomous control of multiple networked Lagrangian systems

AbstractIn this paper we study the problem of coordinating the positions of a group of Lagrangian systems with communication constraints exhibited through delays and limited data rates. We show that in a bidirectional connected communication graph structure, it is possible to design PD‐type control laws that render the overall nonnetworked system input‐to‐state stable. Then, we exploit the robustness property of these control laws along with a small‐gain condition on the allowable delays to infer the stability of the overall networked system. We illustrate how the obtained results can be specialized to handle the problem of position–position control for three networked 2DOF manipulators as well as teleoperation of robotic formations. Copyright © 2008 John Wiley & Sons, Ltd.