General-purpose Robot Research Articles

VR map construction for orchard robot teleoperation based on dual-source positioning and sparse point cloud segmentation

The unstructured nature of orchard environments presents significant challenges for autonomous navigation of orchard robots. Teleoperation, combined with virtual reality (VR), has emerged as a promising solution to overcome the limitations of on-board autonomous navigation capabilities in general-purpose robots. However, constructing accurate and semantically meaningful VR maps for orchard environments remains a challenge. This study proposes a novel VR map construction framework for orchard robot teleoperation visualization. It comprises two key components: dual-source combined positioning and semantic segmentation based on sparse point clouds. First, a novel sliding window-based data fusion approach is proposed to address the positioning challenges in semi-obstructed orchard environments. This approach combines GNSS data and laser matching to achieve robust and accurate positioning for orchard robots. Second, the framework capitalizes on the installation characteristics of mobile robot radars to achieve real-time classification of sparse point clouds. This allows for the removal of unstable elements, such as leaves, which would hinder effective teleoperation. By integrating these components, a VR map suitable for VR teleoperation of orchard robots is achieved. To validate the effectiveness of the proposed method, a VR teleoperation platform was constructed. The experimental results demonstrate that the method successfully fulfills the fundamental requirements for map visualization during VR remote operation of orchard robots. Furthermore, this study provides a valuable reference for the application of digital twins in agricultural robotics.

Development of an automatic medium exchange system using a compact general-purpose robot equipped with image recognition functionality．

Development of an automatic medium exchange system using a compact general-purpose robot equipped with image recognition functionality．

Design of 4-DOF hybrid parallel robots with an integrated three-fingered robot end effector

Grippers are essential components of robotic automation, allowing general purpose robots to perform specialized tasks. This paper aims to design a novel family of 4-DOF hybrid parallel robots, which has an integrated three-fingered robot end effector. As the proposed hybrid parallel robot features a parallel structure, it holds the potential to offer advantages synonymous with parallel mechanisms. The three-fingered robot end effector can perform stable grasping of objects with different sizes and shapes. First, the condition for stable grasping is derived and the synthesis method is proposed. Second, a family of novel 4-DOF hybrid parallel robots is designed through integrating a newly invented three-fingered robot end effector. Finally, a comparative study including the workspace analysis, singularity analysis and performance evaluation of an example hybrid parallel robot with two different topologies is carried out. Then the hybrid parallel robot with a better performance can be selected. The results of performance analysis demonstrate the correctness and rationality of the synthesis method and the high application potential of the obtained hybrid parallel robots.

Robotic Perception-Motion Synergy for Novel Rope Wrapping Tasks

This paper introduces a novel and general method to address the problem of using a general-purpose robot manipulator with a parallel gripper to wrap a deformable linear object (DLO), called a rope, around a rigid object, called a rod, autonomously. Such a robotic wrapping task has broad potential applications in automotive, electromechanical industries construction manufacturing, etc., but has hardly been studied. Our method does not require prior knowledge of the physical and geometrical properties of the objects but enables the robot to use real-time RGB-D perception to determine the wrapping state and feedback control to achieve high-quality results. As such, it provides the robot manipulator with the general capabilities to handle wrapping tasks of different rods or ropes. We tested our method on 6 combinations of 3 different ropes and 2 rods. The result shows that the wrapping quality improved and converged within 5 wraps for all test cases.

Perception for General-purpose Robot Manipulation

Perception for General-purpose Robot Manipulation

Probabilistic framework for reliable optimal design of gearboxes in general-purpose industrial robots considering random use conditions

AbstractA vertically articulated robot with 6-degrees of freedom (DoF), called a general-purpose robot, can perform a myriad of different tasks within a workspace. This paper newly presents a probabilistic framework for the reliable optimal design of gearboxes used in general-purpose industrial robots, which considers random use conditions. To account for random use conditions, the start and end positions of a single motion profile are described as the random variable, which is statistically modeled as a uniform distribution based on the assumption that we have no information about the robot use pattern. Then, each sample of the random variable is converted to the corresponding motion profile by using an on-line trajectory planner. Monte Carlo simulation is implemented for the uncertainty propagation analysis, due to the heuristic feature of the on-line trajectory planner. In the design optimization formulation, the peak torque constraint and maximum bending moment constraint are described in a probabilistic manner. The system-level lifetime is calculated by combining component scale factors. The effectiveness of the proposed framework is demonstrated by examining a case study of a gearbox size problem for a 6-DoF serial industrial robot. The benefits of this study are as follows: Firstly, the proposed framework can evaluate the performance considering random use conditions. Secondly, torque reliability and bending moment reliability are newly proposed to ensure the dynamic performance of an industrial robot. Thirdly, the system-level lifetime by combining component scale factors gives more user-oriented and intuitive measure in an industrial robot design.

Digital pipette: open hardware for liquid transfer in self-driving laboratories

We propose an economical 3D-printed pipette, which aims to overcome the limitations of two-finger robot grippers. It enables general-purpose robot arms to achieve high precision in liquid transfer tasks that is comparable to commercial devices.

CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

General-purpose robots coexisting with humans in their environment must learn to relate human language to their perceptions and actions to be useful in a range of daily tasks. Moreover, they need to acquire a diverse repertoire of general-purpose skills that allow composing long-horizon tasks by following unconstrained language instructions. In this letter, we present Composing Actions from Language and Vision (CALVIN) ( <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</u> omposing <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> ctions from <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> anguage and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Vi</u> sio <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</u> ), an open-source simulated benchmark to learn long-horizon language-conditioned tasks. Our aim is to make it possible to develop agents that can solve many robotic manipulation tasks over a long horizon, from onboard sensors, and specified only via human language. CALVIN tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets and supports flexible specification of sensor suites. We evaluate the agents in zero-shot to novel language instructions and to novel environments. We show that a baseline model based on multi-context imitation learning performs poorly on CALVIN, suggesting that there is significant room for developing innovative agents that learn to relate human language to their world models with this benchmark.

Modularized robot machining system with scalability, portability and parallelism Modular design for machine tool to proove concept and drilling experiment using a general purpose robot

Modularized robot machining system with scalability, portability and parallelism Modular design for machine tool to proove concept and drilling experiment using a general purpose robot

Specification Patterns for Robotic Missions

Mobile and general-purpose robots increasingly support everyday life, requiring dependable robotics control software. Creating such software mainly amounts to implementing complex behaviors known as missions. Recognizing this need, a large number of domain-specific specification languages has been proposed. These, in addition to traditional logical languages, allow the use of formally specified missions for synthesis, verification, simulation or guiding implementation. For instance, the logical language LTL is commonly used by experts to specify missions as an input for planners, which synthesize a robot's required behavior. Unfortunately, domain-specific languages are usually tied to specific robot models, while logical languages such as LTL are difficult to use by non-experts. We present a catalog of 22 mission specification patterns for mobile robots, together with tooling for instantiating, composing, and compiling the patterns to create mission specifications. The patterns provide solutions for recurrent specification problems; each pattern details the usage intent, known uses, relationships to other patterns, and—most importantly—a template mission specification in temporal logic. Our tooling produces specifications expressed in the temporal logics LTL and CTL to be used by planners, simulators or model checkers. The patterns originate from 245 mission requirements extracted from the robotics literature, and they are evaluated upon a total of 441 real-world mission requirements and 1251 mission specifications. Five of these reflect scenarios defined with two well-known industrial partners developing human-size robots. We further validate our patterns’ correctness with simulators and two different types of real robots.

Review of the techniques used in motor‐cognitive human‐robot skill transfer

A conventional robot programming method extensively limits the reusability of skills in the developmental aspect. Engineers programme a robot in a targeted manner for the realisation of predefined skills. The low reusability of general-purpose robot skills is mainly reflected in inability in novel and complex scenarios. Skill transfer aims to transfer human skills to general-purpose manipulators or mobile robots to replicate human-like behaviours. Skill transfer methods that are commonly used at present, such as learning from demonstrated (LfD) or imitation learning, endow the robot with the expert's low-level motor and high-level decision-making ability, so that skills can be reproduced and generalised according to perceived context. The improvement of robot cognition usually relates to an improvement in the autonomous high-level decision-making ability. Based on the idea of establishing a generic or specialised robot skill library, robots are expected to autonomously reason about the needs for using skills and plan compound movements according to sensory input. In recent years, in this area, many successful studies have demonstrated their effectiveness. Herein, a detailed review is provided on the transferring techniques of skills, applications, advancements, and limitations, especially in the LfD. Future research directions are also suggested.

Gaze-Based Dual Resolution Deep Imitation Learning for High-Precision Dexterous Robot Manipulation

A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency.

Origami Folding by Multifingered Hands with Motion Primitives.

Origami, a traditional Japanese art, is an example of superior handwork produced by human hands. Achieving such extreme dexterity is one of the goals of robotic technology. In the work described in this paper, we developed a new general-purpose robot system with sufficient capabilities for performing Origami. We decomposed the complex folding motions into simple primitives and generated the overall motion as a combination of these primitives. Also, to measure the paper deformation in real-time, we built an estimator using a physical simulator and a depth camera. As a result, our experimental system achieved consecutive valley folds and a squash fold.

Robotic grasping with obstacle avoidance using octrees

This paper considers the problems of the integration of independent manipulator control systems. Areas of control of the manipulator are: recognition of objects and obstacles, identification of objects to be grasped, determination of reliable positions by the grasping device, planning of movement of the manipulator to certain positions with avoidance of obstacles, and recognition of slipping or determination of reliable grasping. This issue is a current problem primarily in industry, general-purpose robots, and experimental robots. This paper considers current publications that address these issues. Existing algorithms and approaches have been found in the management of both parts of the robot manipulator and solutions that combine several areas, or the integration of several existing approaches. There is a brief review of current literature and publications on the above algorithms and approaches. The advantages and disadvantages of the considered methods and approaches are determined. There are solutions that cover either some areas or only one of them, which does not meet the requirements of the problem. Using existing approaches, integration points of existing implementations are identified to get the best results. In the process, a system was developed that analyzes the environment, finds obstacles, objects for interaction, poses for grasping, plans the movement of the manipulator to a specific position, and ensures reliable grasping of the object. The next step was to test the system, test the performance, and adjust the parameters for the best results. The resulting system was developed by the research team of RT-Lions, Technik University, Reutlingen. The hardware research robot includes an Intel Realsense camera, a Sawyer Arm manipulator from Rethink Robotics, and an internally grabbing device.

Emotional model for a multi-robot system with emergent behavior

&lt;p&gt;This article describes an emotional model for a general-purpose robot operating in a multi-robot system with emergent behavior. The model considers four basic emotions: anger, rejection, sadness and joy, plus a neutral emotional state, which affect the behavior of the robot, both individually and collectively. The emotional state of each robot in the system is constructed through the conjunction of a series of factors related to their individual and collective actions, which are: safety, load, acting and interaction, which serve as input to an emotional process that results in an index of satisfaction of the robot that establishes the emotional state in which it is in a certain moment. The emotional state of a robot influences its interactions with the other robots and with the environment, that is, it determines its emergent behavior in the system. This paper presents the design of this model, and establishes some considerations for its implementation.&lt;/p&gt;

Automatic Image and Video Caption Generation With Deep Learning: A Concise Review and Algorithmic Overlap

Methodologies that utilize Deep Learning offer great potential for applications that automatically attempt to generate captions or descriptions about images and video frames. Image and video captioning are considered to be intellectually challenging problems in imaging science. The application domains include automatic caption (or description) generation for images and videos for people who suffer from various degrees of visual impairment; the automatic creation of metadata for images and videos (indexing) for use by search engines; general-purpose robot vision systems; and many others. Each of these application domains can positively and significantly impact many other task-specific applications. This article is not meant to be a comprehensive review of image captioning; rather, it is a concise review of both image captioning and video captioning methodologies based on deep learning. This study treats both image and video captioning by emphasizing the algorithmic overlap between the two.

Review of Deep Learning Methods in Robotic Grasp Detection

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods have driven robotics researchers to explore the use of deep learning methods in task-generalised robotic applications. This paper reviews the current state-of-the-art in regards to the application of deep learning methods to generalised robotic grasping and discusses how each element of the deep learning approach has improved the overall performance of robotic grasp detection. Several of the most promising approaches are evaluated and the most suitable for real-time grasp detection is identified as the one-shot detection method. The availability of suitable volumes of appropriate training data is identified as a major obstacle for effective utilisation of the deep learning approaches, and the use of transfer learning techniques is proposed as a potential mechanism to address this. Finally, current trends in the field and future potential research directions are discussed.

BWIBots: A platform for bridging the gap between AI and human–robot interaction research

Recent progress in both AI and robotics have enabled the development of general purpose robot platforms that are capable of executing a wide variety of complex, temporally extended service tasks in open environments. This article introduces a novel, custom-designed multi-robot platform for research on AI, robotics, and especially human–robot interaction for service robots. Called BWIBots, the robots were designed as a part of the Building-Wide Intelligence (BWI) project at the University of Texas at Austin. The article begins with a description of, and justification for, the hardware and software design decisions underlying the BWIBots, with the aim of informing the design of such platforms in the future. It then proceeds to present an overview of various research contributions that have enabled the BWIBots to better (a) execute action sequences to complete user requests, (b) efficiently ask questions to resolve user requests, (c) understand human commands given in natural language, and (d) understand human intention from afar. The article concludes with a look forward towards future research opportunities and applications enabled by the BWIBot platform.

Selecting industrial robots for milling applications using AHP

Industrial robots are usually used for pick-and-place applications, which require only point-to-point motion control. However, the recent developments, both in robot technology and in Computer Automated Machining (CAM) software, allow the use of these equipment in applications which require continuous path control, such as multi-axis milling processes. However, the producers do not offer robots specifically developed for this kind of application, thus the user has to choose the most appropriate robot for this goal from a wide range of general purpose robot types. This research work proposes a method based upon Analytic Hierarchy Process (AHP) for selecting the industrial robot for milling applications. Several attributes of the robots are proposed as selection criteria, and a simulation study of the milling capabilities of the robotic structure is also taken into consideration.

Architecture for in-space robotic assembly of a modular space telescope

An architecture and conceptual design for a robotically assembled, modular space telescope (RAMST) that enables extremely large space telescopes to be conceived is presented. The distinguishing features of the RAMST architecture compared with prior concepts include the use of a modular deployable structure, a general-purpose robot, and advanced metrology, with the option of formation flying. To demonstrate the feasibility of the robotic assembly concept, we present a reference design using the RAMST architecture for a formation flying 100-m telescope that is assembled in Earth orbit and operated at the Sun–Earth Lagrange Point 2.