Autonomous Robotic Assembly Research Articles

On Efficient and Flexible Autonomous Robotic Insertion Assembly in the Presence of Uncertainty

On Efficient and Flexible Autonomous Robotic Insertion Assembly in the Presence of Uncertainty

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Assembly positioning by visual servoing (VS) is a basis for autonomous robotic assembly. In practice, VS control suffers potential stability and convergence problems due to image and physical constraints, e.g., field of view constraints, image local minima, obstacle collisions, and occlusion. Therefore, this article proposes a novel deep reinforcement learning-based hybrid visual servoing (DRL-HVS) controller for motion planning of VS tasks. DRL-HVS controller takes current observed image features and camera pose as inputs, and the core parameters of hybrid VS are dynamically optimized using a deep deterministic policy gradient (DDPG) algorithm to obtain an optimal motion scheme, considering image/physical constraints and robot motion performance. In addition, an adaptive exploration strategy is proposed to further improve the training efficiency by adaptively tuning the exploration noise parameters. In this way, the offline pretrained DRL-HVS controller in the virtual environment, where the DDPG actor–critic network is continuously optimized, can be quickly deployed to a real robot system for real-time control. Experiments based on an eye-in-hand VS system are conducted with a calibrated HIKVISION RGB camera mounted on the end-effector of a GSK-RB03A1 six degree-of-freedom (6-DoF) robot. Basic VS task experiments show that the proposed controller achieves better performance than the existing methods: the servoing time is 24% smaller than that of the five-dimensional VS method, a 100% success rate with the perturbed ranges of the initial position within 25 mm for translation and 20° for rotation, and a 48% efficiency improvement. Moreover, a planetary gear component assembly process case study, where the robot aims to automatically put the gears on the gear shafts, is conducted to demonstrate the applicability of the proposed method in practice.

An Autonomous Task Assignment Paradigm for Autonomous Robotic In-Space Assembly.

The development of autonomous robotic systems is a key component in the expansion of space exploration and the development of infrastructures for in-space applications. An important capability for these robotic systems is the ability to maintain and repair structures in the absence of human input by autonomously generating valid task sequences and task to robot allocations. To this end, a novel stochastic problem formulation paired with a mixed integer programming assembly schedule generator has been developed to articulate the elements, constraints, and state of an assembly project and solve for an optimal assembly schedule. The developed formulations were tested with a set of hardware experiments that included generating an optimal schedule for an assembly and rescheduling during an assembly to plan a repair. This formulation and validation work provides a path forward for future research in the development of an autonomous system capable of building and maintaining in-space infrastructures.

Towards Autonomous Robotic Assembly: Using Combined Visual and Tactile Sensing for Adaptive Task Execution

Robotic assembly tasks are typically implemented in static settings in which parts are kept at fixed locations by making use of part holders. Very few works deal with the problem of moving parts in industrial assembly applications. However, having autonomous robots that are able to execute assembly tasks in dynamic environments could lead to more flexible facilities with reduced implementation efforts for individual products. In this paper, we present a general approach towards autonomous robotic assembly that combines visual and intrinsic tactile sensing to continuously track parts within a single Bayesian framework. Based on this, it is possible to implement object-centric assembly skills that are guided by the estimated poses of the parts, including cases where occlusions block the vision system. In particular, we investigate the application of this approach for peg-in-hole assembly. A tilt-and-align strategy is implemented using a Cartesian impedance controller, and combined with an adaptive path executor. Experimental results with multiple part combinations are provided and analyzed in detail.

High-Speed Autonomous Robotic Assembly Using In-Hand Manipulation and Re-Grasping

This paper presents an autonomous robotic assembly system for Soma cube blocks, which, after observing the individual blocks and their assembled shape, quickly plans and executes the assembly motion sequence that picks up each block and incrementally build the target shape. A multi stage planner is used to find the suitable assembly solutions, assembly sequences and grip sequences considering various constraints, and re-grasping is used when the block target pose is not directly realizable or the block pose is ambiguous. The suggested system is implemented for a commercial UR5e robotic arm and a novel two degrees of freedom (DOF) gripper capable of in-hand manipulation, which further speeds up the manipulation speed. It was experimentally validated through a public competitive demonstration, where the suggested system completed all assembly tasks reliably with outstanding performance.

Dynamics and control for in-space assembly robots with large translational and rotational maneuvers

There are two major characteristics of in-space structure assembly missions. First, in order to improve the transport efficiency of assembly robot and reduce the number of structural modules, the size of each structural module is usually designed as large as possible. Second, it is necessary for autonomous assembly robot to quickly transport such large-sized flexible structural modules to the desired locations for integration. However, the coupling effect between the deformation of structural module and large-scale maneuver of robot is inevitably involved during transportation. In this paper, a dynamics modeling method for in-space assembly (ISA) robot with large translational and rotational maneuvers was proposed. The descriptions of dual quaternions were expanded, and a general expression for the dynamics model of rigid-flex system was derived to describe motion of the assembly robot. Innovation of this method is that it considered the complex coupling effect among deformation of structural modules, translational and rotational maneuvers of the assembly robot during transportation in ISA missions. On this basis, the proportional-derivative (PD) controllers were adopted as the in-space structural module assembly method. Finally, taking the ISA of antenna as an example, successful application of the proposed dynamics model and controller to the assembly mission was verified.

A Machine Learning Framework for Real-Time Identification of Successful Snap-Fit Assemblies

Snap-fit assemblies are widely used in the manufacturing of several product types, allowing part joining, while the parts remain unprocessed. The locking mechanism of a snap-fit is usually done within the object structure, not allowing visual identification of the successful process completion. Humans consider the forces developed between the two parts or the snapping sound, as an indication of success. This is difficult to realize in robotic assembly, and the process success is usually identified at a product quality control stage. The aim of this article is to migrate the human ability to identify a successful snap assembly to autonomous robotic assembly, via a machine learning framework, enabled by human-robot collaboration for rich data collection and labeling. The proposed framework allows learning while minimizing complexity, cost, and time. A generic feature set is proposed, which can produce good identification results in different snap assembly types. A feature transformation is also introduced that is fundamental for the real-time operation of the proposed framework and the identification of successful snap-assemblies. Three different objects are used to experimentally validate the approach using a KUKA LWR4+ robotic arm, resulting in high classification and real-time identification accuracy. Finally, a comparison with a model-based method is conducted.

BIM-assisted object recognition for the on-site autonomous robotic assembly of discrete structures

Robots-operating autonomous assembly applications in an unstructured environment require precise methods to locate the building components on site. However, the current available object detection systems are not well-optimised for construction applications, due to the tedious setups incorporated for referencing an object to a system and inability to cope with the elements imperfections. In this paper, we propose a flexible object pose estimation framework to enable robots to autonomously handle building components on-site with an error tolerance to build a specific design target without the need to sort or label them. We implemented an object recognition approach that uses the virtual representation model of all the objects found in a BIM model to autonomously search for the best-matched objects in a scene. The design layout is used to guide the robot to grasp and manipulate the found elements to build the desired structure. We verify our proposed framework by testing it in an automatic discrete wall assembly workflow. Although the precision is not as expected, we analyse the possible reasons that might cause this imprecision, which paves the path for future improvements.

Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites

Unlike robotics in the manufacturing industry, on-site construction robotics has to consider and address two unique challenges: 1) the rugged, evolving, and unstructured environment of typical work sites; and 2) the reversed spatial relationship between the product and the manipulator, i.e., the manipulator has to travel to and localize itself at the work face, rather than a partially complete product arriving at an anchored manipulator. The presented research designed and implemented algorithms that address these challenges and enable autonomous robotic assembly of freeform modular structures on construction sites. Building on the authors' previous work in computer-vision-based pose estimation, the designed algorithms enable a mobile robotic manipulator to: 1) autonomously identify and grasp prismatic building components (e.g., bricks, blocks) that are typically non-unique and arbitrarily stored on-site; and 2) assemble these components into pre-designed modular structures. The algorithms use a single camera and a visual marker-based metrology to rapidly establish local reference frames and to detect staged building components. Based on the design of the structure being assembled, the algorithms automatically determine the assembly sequence. Furthermore, if a 3D camera is mounted on the manipulator, 3D point clouds can be readily captured and registered into a same reference frame through our marker-based metrology and the manipulator's internal encoders, either after construction to facilitate as-built Building Information Model (BIM) generation, or during construction to document details of the progress. Implemented using a 7-axis KUKA KR100 robotic manipulator, the presented robotic system has successfully assembled various structures and created as-built 3D point cloud models autonomously, demonstrating the designed algorithms' effectiveness in autonomous on-site construction robotics applications.

An autonomous mobile manipulator for assembly tasks

The fundamental difference between autonomous robotic assembly and traditional hard automation, currently utilized in large-scale manufacturing production, lies in the specific approaches used in locating, acquiring, manipulating, aligning, and assembling parts. An autonomous robotic assembly manipulator offers high flexibility and high capability to deal with the inherent system uncertainties, unknowns, and exceptions. This paper presents an autonomous mobile manipulator that effectively overcomes inherent system uncertainties and exceptions by utilizing control strategies that employ coordinated control, combine visual and force servoing, and incorporate sophisticated reactive task control. The mobile manipulation system has been demonstrated experimentally to achieve high reliability for a "peg-in-hole" type of insertion assembly task that is commonly encountered in automotive wiring harness assembly.

Assembly and task planning using Petri nets: A survey

The automatic synthesis of a plan to perform a robotized assembly task from a high-level description of the product to be assembled is a difficult issue which involves assembly planning and task planning. This paper summarizes the problems that an autonomous robotic assembly system must tackle both regarding assembly planning and task planning and reviews the approaches that use Petri nets as a formalism to develop the corresponding planners. The need for integration of both assembly and task planning is made evident, and approaches covering that are described.

Autonomous Robotic Assembly Using Collaborative Behavior Based Agents

This work studies the design of a robotic assembly system as multiagent system. Any multidevice system or any system whose performance is naturally decomposable can be interpreted as the corporation of agents. Such scheme comprises the ability of creating the collaborative technical system which can provide the achieving of the social intelligence. The social behavior is the highest form of intelligence that can provide the solving of very complex problems, autonomous creation of new procedures and efficient adaptation to new tasks. The presented multiagent model is based on the processing units, which include the recognition networks, problem solving algorithms and learning engines. It integrates: perception, recognition, learning and communication capabilities. The reinforcement learning method is used here to evaluate robot behavior and to induce new, or improve the existing, knowledge. The acquired action (task) plan is stored as experience which can be used in solving similar future problems. To provide the recognition of problem similarities, the Adaptive Fuzzy Shadowed (AFS) neural network is applied.

Dynamic task mapping in a real-time controller for distributed cooperating robot systems

An intelligent control architecture for autonomous robot systems must be modular, scaleable, extendible, and adaptable to the available capacity of computer control systems. To support these properties, the concept of an adaptive control structure, ACS, is presented in this paper. Experiments with a small autonomous assembly robot show the successful use of ACS.

Dynamic Task Mapping for Real-Time Controller of Distributed Cooperative Robot Systems

Intelligent control architecture of autonomous robot systems must be modular, scaleable, extendible, and adaptable to the available capacity of computer control systems. For the support of these properties, the concept of adaptive control structure, ACS, is presented in this paper. Experiments with small autonomous assembly robot show the successful use of ACS.

Assembly of Blocks by Autonomous Assembly Robot with Intelligence (ARI)

Summary Robots in the next decade should be able to easily understand the job to be performed, automatically plan action procedures and successfully execute the plan. This paper describes the robot ARI [Assembly Robot with Intelligence) which achieved these objectives for simple assembling operations. ARI has two eyes, a neck, two arms and a trunk -like upper half of a human body. The robot is controlled by a engineering work-station and several micro-computers. By simply observing a sample assembly of toy blocks, ARI can autonomously construct a replica. The robot first determines the 3-dimensional structure of the sample assembly and then plans a procedure to construct a replica. ARI next carries out the procedure with the aid of vision and force sensors.

The development of an autonomous assembly robot

This paper presents various projects which are being developed at the Institute for Informatics at the Robotics Research Group. Several of these projects will be integrated to support the development of an autonomous assembly robot. With its two arms under sensor guidance, the robot will be capable of assembling a standard benchmark (workpieces). To achieve mobility, the robot will be equipped with a self-propelled platform. Several vision systems are employed in order to support the navigation of the robot and the assembly task. An AI-based system is being developed for assembly planning, implicit programming, route planning (obstacle avoidance), assembly monitoring, and error recovery.