Future Robotic Systems Research Articles

Optimising robotic operation speed with edge computing via 5G network: Insights from selective harvesting robots

AbstractSelective harvesting by autonomous robots will be a critical enabling technology for future farming. Increases in inflation and shortages of skilled labor are driving factors that can help encourage user acceptability of robotic harvesting. For example, robotic strawberry harvesting requires real‐time high‐precision fruit localization, three‐dimensional (3D) mapping, and path planning for 3D cluster manipulation. Whilst industry and academia have developed multiple strawberry harvesting robots, none have yet achieved human–cost parity. Achieving this goal requires increased picking speed (perception, control, and movement), accuracy, and the development of low‐cost robotic system designs. We propose the edge‐server over 5G for Selective Harvesting (E5SH) system, which is an integration of high bandwidth and low latency Fifth‐Generation (5G) mobile network into a crop harvesting robotic platform, which we view as an enabler for future robotic harvesting systems. We also consider processing scale and speed in conjunction with system environmental and energy costs. A system architecture is presented and evaluated with support from quantitative results from a series of experiments that compare the performance of the system in response to different architecture choices, including image segmentation models, network infrastructure (5G vs. Wireless Fidelity), and messaging protocols, such as Message Queuing Telemetry Transport and Transport Control Protocol Robot Operating System. Our results demonstrate that the E5SH system delivers step‐change peak processing performance speedup of above 18‐fold than a standalone embedded computing Nvidia Jetson Xavier NX system.

Scan Time Reduction by Fewer Projections - an Approach for Circular and Spherical Trajectories

For most computed tomography scans a circular or helical trajectory is used to acquire the necessary radiographic projections. However, these standard trajectories are not necessarily the best choice since the shape of the inspected object is not considered which leads to a suboptimal sampling of the part. In contrast, workpiece-specific and more efficient scan strategies hold the potential for time and cost savings. Such adaptions might be of particular interest for future robotic CT systems that allow greater flexibility in the positioning of X-ray tube, object, and detector than conventional setups or multi-source applications with predefined positions by the setup. We propose a scan planning algorithm to adapt the acquisition trajectory considering the shape of the investigated part. For this task, we present an algorithm that is able to identify the projections from a set of projections that contribute most to the reconstruction of the object. As input information, only the localization of the area of interest and projections from the feasible range of motion are necessary. Also, a reasonable amount of projections to pick from needs to be provided, which can originate from an earlier scan or computer simulations that consider the range of motion of the available CT setup. The optimization method has been implemented for circular and spherical trajectories and can be extended towards arbitrary acquisition paths. Since for modern CT systems the duration for the scan itself takes significantly longer than the reconstruction, this approach appears to be well-suited to significantly reduce the throughput time if several identical parts need to be inspected. Additionally, new applications such as inline CT are conceivable and certain artifacts can be suppressed if spherical trajectories are used instead of circular ones. For the subsequent reconstruction itself no previous knowledge of the object is included, i.e. the reconstruction workflow does not need to be modified. We demonstrate the capabilities of the optimization method for two industrial relevant examples and show that a scan time reduction of more than 50 % is feasible for some cases by reducing the number of acquired projections. Furthermore, for all investigated projection numbers and parts the optimized trajectory yields an identical to clearly improved image quality in the reconstructed volume than the reference trajectories.

Implementation of singularity-free inverse kinematics for humanoid robotic arm using Bayesian optimized deep neural network

This study presents the efficacy of deep learning techniques in controlling the arm of a humanoid robot without resorting to inverse kinematic analysis. Emphasizing real-time applicability, a straightforward deep neural network (DNN) structure is developed and optimized using hyperparameter Bayesian optimization. The Keras Bayesian Optimization Tuner with Gaussian process fine-tunes the DNN architecture. Utilizing a simulation environment and generating around 10 billion datasets, the optimized DNN is evaluated on both simulated and real robots. Impressively, the trained DNN exhibits a notable ability to predict robot control within a 25 ms timeframe, achieving a total Mean Absolute Error (MAE) of 0.02 for the xyz-axes. These results underscore the potential of deep learning-based approaches in humanoid robot arm control, eliminating the need for inverse kinematic equations and providing valuable insights for future robotic system development.

Periscope: A Robotic Camera System to Support Remote Physical Collaboration

We investigate how robotic camera systems can offer new capabilities to computer-supported cooperative work through the design, development, and evaluation of a prototype system called Periscope. With Periscope, a local worker completes manipulation tasks with guidance from a remote helper who observes the workspace through a camera mounted on a semi-autonomous robotic arm that is co-located with the worker. Our key insight is that the helper, the worker, and the robot should all share responsibility of the camera view-an approach we call shared camera control. Using this approach, we present a set of modes that distribute the control of the camera between the human collaborators and the autonomous robot depending on task needs. We demonstrate the system's utility and the promise of shared camera control through a preliminary study where 12 dyads collaboratively worked on assembly tasks. Finally, we discuss design and research implications of our work for future robotic camera systems that facilitate remote collaboration.

Design of Novel Modified Double-Ended Forward Converter for Stepper Motor Drive

This paper presents the design and analysis of a modified double-ended forward converter (DEFC) for stepper motor-based robotic applications. The proposed converter topology provides galvanic isolation between the input and output while also higher efficiency with a smooth operative system, making it suitable for use in robotic systems that require both power and control signals to be transmitted. The paper also discusses the control strategy for the converter, which uses Proportional Integral (PI) to regulate the output voltage and current. The control strategy is implemented using a microcontroller-based system, which provides precise control of the output parameters. The converter is tested using a stepper motor-based load, and the results demonstrate the effectiveness of the proposed topology and control strategy. In addition to the experimental results, the paper also presents a detailed analysis of the converter’s performance. The analysis includes the input voltage and current, capacitor voltage, MOSFET parameters, output voltage and current, and calculation of efficiency. The analysis results show that the proposed converter topology and control strategy offer high efficiency comparing to existing converting approaches. Overall, the proposed double-ended forward converter offers a suitable solution for stepper motor-based robotic applications, providing efficient and reliable power and control signals. The results demonstrate the effectiveness of the proposed converter topology and control strategy, making it a promising option for use in future robotic systems.

The case study of new approach to robot programming and layout design by supporting virtual and augmented reality

In currently designed robotic systems, great emphasis is focused to optimal choice of the appropriate position (smart place) of a particular type of industrial robot. Depending on the chosen application, whether it is a handling, technological or assembly operation, it is very important to choose a suitable type of robot and end effector. Many problems in actually deployed production systems arise from the inappropriately chosen kinematic structure of an industrial robot, where either the robot cannot achieve the required production cycle, or its size or load capacity is incorrectly selected for a given type of the operation, or in some cases the robot does not achieve all required robot targets of the future robotic workplace. In the currently available methods of designing future robotic systems, the pre-production phase of designing and creating various robotic simulations scenarios a very important role in order to verify the robot’s reachability to the required positions, as well as eliminate possible collisions states with different peripherals, etc.. This article aims to point out to the new possibilities of deploying modern technologies in the form of the use of virtual and augmented reality in the chosen case study. The authors aim to show and verify in the VR and AR environment the suitability of the selected type of industrial robot for the selected operation, to simulate and validate the proposed robotic trajectories as well as the final robotic Pick and Place operation for the selected case study.

Self-powered bionic antenna based on triboelectric nanogenerator for micro-robotic tactile sensing

Antennae are the main sensory organs of insects which allow them to gather information from environment through direct contact and active sensing. In this study, a self-powered bionic antenna (SBA) was developed to emulate the structure and function of insect antenna for robotic tactile sensing. The sensing component of SBA based on the triboelectric nanogenerator (TENG) technology enables the conversion of mechanical stimuli into voltage signal without requiring external power source, and is capable of identifying different contact materials. To imitate the active sensing performance of insect antenna, a two-stage actuator was designed and integrated into the SBA, enabling active sensing in both the horizontal and vertical planes. The ultra-sensitive, lightweight and drivable design of SBA renders it highly compatible with micro soft robots featured with low actuation force and compact size, facilitating both passive and active real-time sensing and making it a promising solution for future robotic tactile systems.

Enabling automated facility maintenance from articulated robot Collision-Free designs

Key challenges found in some facilities maintenance (FM) operations include the dependency on placing sophisticated workers in undesirable working conditions that could make them susceptible to accidents. Robot-assisted in-situ facilities maintenance represents new opportunities by assisting human operators when exposed to harsh environments. However, given the complexities and potential varieties of advanced robotic machinery available for use in facility maintenance workplaces in the future, there is still a legitimate risk of collisions between future robotic systems and the facility structure during the robot-assisted processes. This paper proposes and tests a simulation-based collision-free design method for future facilities that may be benefited from a robot-assisted FM process. At the most fundamental level, we consider interactions between the articulated manipulator (robot) and the surrounding facility structure. Inverse Kinematics (IK) and game engines are used to simulate the possible collisions between a given robot and the parameterized facility layout. To account for the uncertainty about the kinematic specifications of the future robotic machinery, a Monte Carlo method is used to model unique priority circumstances for each joint on the articulated manipulator. The simulation result, presented as the robot work zone envelope, is then used to estimate the collision probability given a certain design parameter, and the corresponding optimal design that balances the initial construction cost and the FM costs. The method is assessed for identifying a 3D spatial collision probability cloud within reach of a 7 DOF articulated manipulator that is marginally positioned within a pressure valve piping facility. These insights can provide future facility designers, managers, and equipment operators with a quick and flexible 3D spatial collision probability indicator to better design the facility and proximity limitations of any articulated manipulator positioned throughout a confined space for future robot-assisted FM processes.

Characterization of the Workspace and Limits of Operation of Laser Treatments for Vascular Lesions of the Lower Limbs.

The increase of the aging population brings numerous challenges to health and aesthetic segments. Here, the use of laser therapy for dermatology is expected to increase since it allows for non-invasive and infection-free treatments. However, existing laser devices require doctors’ manually handling and visually inspecting the skin. As such, the treatment outcome is dependent on the user’s expertise, which frequently results in ineffective treatments and side effects. This study aims to determine the workspace and limits of operation of laser treatments for vascular lesions of the lower limbs. The results of this study can be used to develop a robotic-guided technology to help address the aforementioned problems. Specifically, workspace and limits of operation were studied in eight vascular laser treatments. For it, an electromagnetic tracking system was used to collect the real-time positioning of the laser during the treatments. The computed average workspace length, height, and width were 0.84 ± 0.15, 0.41 ± 0.06, and 0.78 ± 0.16 m, respectively. This corresponds to an average volume of treatment of 0.277 ± 0.093 m3. The average treatment time was 23.2 ± 10.2 min, with an average laser orientation of 40.6 ± 5.6 degrees. Additionally, the average velocities of 0.124 ± 0.103 m/s and 31.5 + 25.4 deg/s were measured. This knowledge characterizes the vascular laser treatment workspace and limits of operation, which may ease the understanding for future robotic system development.

Human–Robot Interaction: A Review and Analysis on Variable Admittance Control, Safety, and Perspectives

Human–robot interaction (HRI) is a broad research topic, which is defined as understanding, designing, developing, and evaluating the robotic system to be used with or by humans. This paper presents a survey on the control, safety, and perspectives for HRI systems. The first part of this paper reviews the variable admittance (VA) control for human–robot co-manipulation tasks, where the virtual damping, inertia, or both are adjusted. An overview of the published research for the VA control approaches, their methods, the accomplished collaborative co-manipulation tasks and applications, and the criteria for evaluating them are presented and compared. Then, the performance of various VA controllers is compared and investigated. In the second part, the safety of HRI systems is discussed. The various methods for detection of human–robot collisions (model-based and data-based) are investigated and compared. Furthermore, the criteria, the main aspects, and the requirements for the determination of the collision and their thresholds are discussed. The performance measure and the effectiveness of each method are analyzed and compared. The third and final part of the paper discusses the perspectives, necessity, influences, and expectations of the HRI for future robotic systems.

Remote vascular interventional surgery robotics: a literature review.

Vascular interventional doctors are exposed to radiation hazards during surgery and endure high work intensity. Remote vascular interventional surgery robotics is a hot research field, in which researchers aim to not only protect the health of interventional doctors, but to also improve surgical accuracy and efficiency. However, the current vascular interventional robots have numerous shortcomings, such as poor haptic feedback, few compatible surgeries and instruments, and cumbersome maintenance and operational procedures. Nevertheless, vascular interventional surgery combined with robotics provides more cutting-edge directions, such as Internet remote surgery combined with 5G network technology and the application of artificial intelligence in surgical procedures. To summarize the developmental status and key technical points of intravascular interventional surgical robotics research, we performed a systematic literature search to retrieve original articles related to remote vascular interventional surgery robotics published up to December 2020. This review, which includes 113 articles published in English, introduces the mechanical and structural characteristics of various aspects of vascular interventional surgical robotics, discusses the current key features of vascular interventional surgical robotics in force sensing, haptic feedback, and control methods, and summarizes current frontiers in autonomous surgery, long-distance robotic telesurgery, and magnetic resonance imaging (MRI)-compatible structures. On the basis of summarizing the current research status of remote vascular interventional surgery robotics, we aim to propose a variety of prospects for future robotic systems.

Origami-inspired folding assembly of dielectric elastomers for programmable soft robots

Origami has become an optimal methodological choice for creating complex three-dimensional (3D) structures and soft robots. The simple and low-cost origami-inspired folding assembly provides a new method for developing 3D soft robots, which is ideal for future intelligent robotic systems. Here, we present a series of materials, structural designs, and fabrication methods for developing independent, electrically controlled origami 3D soft robots for walking and soft manipulators. The 3D soft robots are based on soft actuators, which are multilayer structures with a dielectric elastomer (DE) film as the deformation layer and a laser-cut PET film as the supporting flexible frame. The triangular and rectangular design of the soft actuators allows them to be easily assembled into crawling soft robots and pyramidal- and square-shaped 3D structures. The crawling robot exhibits very stable crawling behaviors and can carry loads while walking. Inspired by origami folding, the pyramidal and square-shaped 3D soft robots exhibit programmable out-of-plane deformations and easy switching between two-dimensional (2D) and 3D structures. The electrically controllable origami deformation allows the 3D soft robots to be used as soft manipulators for grasping and precisely locking 3D objects. This work proves that origami-inspired fold-based assembly of DE actuators is a good reference for the development of soft actuators and future intelligent multifunctional soft robots.

3D‐Printed Pneumatically Controlled Soft Suction Cups for Gripping Fragile, Small, and Rough Objects

A 3D‐printed pneumatically actuated soft suction gripper with an elastomer film is proposed. Suction in such gripper is actively controlled by applying a negative pressure behind the film. The elastomeric gripper body is 3D‐printed, making it easy to customize and integrate into future robotic gripping systems. The gripper can pick a wide variety of objects, such as delicate fruits, small parts, and parts with uneven loads, with high pull‐off forces (over 7.4 N with ∅ 20 mm/55 kPa). The achieved pull‐off forces are significantly higher than the previously reported suction cup grippers with films and more comparable with commercial vacuum grippers. The pull‐off forces show no significant differences with surfaces of varying roughness (up to root‐mean‐square roughness of 5.66 μm) and the gripper is able to pick and release target objects repeatedly. The gripper is also compared with a commercial vacuum gripper with comparable dimensions. It outperforms the commercial gripper in the case of fragile objects, objects smaller than the gripper diameter, and objects with uneven loads. It can apply high pull‐off forces while having controllable release, and is suitable for gripping a wide variety of real‐world objects, including heavy, rough, small, thin, and fragile ones.

Autonomous pick-and-place using the dVRK

PurposeRobotic-assisted partial nephrectomy (RAPN) is a tissue-preserving approach to treating renal cancer, where ultrasound (US) imaging is used for intra-operative identification of tumour margins and localisation of blood vessels. With the da Vinci Surgical System (Sunnyvale, CA), the US probe is inserted through an auxiliary access port, grasped by the robotic tool and moved over the surface of the kidney. Images from US probe are displayed separately to the surgical site video within the surgical console leaving the surgeon to interpret and co-registers information which is challenging and complicates the procedural workflow.MethodsWe introduce a novel software architecture to support a hardware soft robotic rail designed to automate intra-operative US acquisition. As a preliminary step towards complete task automation, we automatically grasp the rail and position it on the tissue surface so that the surgeon is then able to manipulate manually the US probe along it.ResultsA preliminary clinical study, involving five surgeons, was carried out to evaluate the potential performance of the system. Results indicate that the proposed semi-autonomous approach reduced the time needed to complete a US scan compared to manual tele-operation.ConclusionProcedural automation can be an important workflow enhancement functionality in future robotic surgery systems. We have shown a preliminary study on semi-autonomous US imaging, and this could support more efficient data acquisition.

Activity, Plan, and Goal Recognition: A Review.

Recognizing the actions, plans, and goals of a person in an unconstrained environment is a key feature that future robotic systems will need in order to achieve a natural human-machine interaction. Indeed, we humans are constantly understanding and predicting the actions and goals of others, which allows us to interact in intuitive and safe ways. While action and plan recognition are tasks that humans perform naturally and with little effort, they are still an unresolved problem from the point of view of artificial intelligence. The immense variety of possible actions and plans that may be encountered in an unconstrained environment makes current approaches be far from human-like performance. In addition, while very different types of algorithms have been proposed to tackle the problem of activity, plan, and goal (intention) recognition, these tend to focus in only one part of the problem (e.g., action recognition), and techniques that address the problem as a whole have been not so thoroughly explored. This review is meant to provide a general view of the problem of activity, plan, and goal recognition as a whole. It presents a description of the problem, both from the human perspective and from the computational perspective, and proposes a classification of the main types of approaches that have been proposed to address it (logic-based, classical machine learning, deep learning, and brain-inspired), together with a description and comparison of the classes. This general view of the problem can help on the identification of research gaps, and may also provide inspiration for the development of new approaches that address the problem in a unified way.

Electrostatic bellow muscle actuators and energy harvesters that stack up.

Future robotic systems will be pervasive technologies operating autonomously in unknown spaces that are shared with humans. Such complex interactions make it compulsory for them to be lightweight, soft, and efficient in a way to guarantee safety, robustness, and long-term operation. Such a set of qualities can be achieved using soft multipurpose systems that combine, integrate, and commute between conventional electromechanical and fluidic drives, as well as harvest energy during inactive actuation phases for increased energy efficiency. Here, we present an electrostatic actuator made of thin films and liquid dielectrics combined with rigid polymeric stiffening elements to form a circular electrostatic bellow muscle (EBM) unit capable of out-of-plane contraction. These units are easy to manufacture and can be arranged in arrays and stacks, which can be used as a contractile artificial muscle, as a pump for fluid-driven soft robots, or as an energy harvester. As an artificial muscle, EBMs of 20 to 40 millimeters in diameter can exert forces of up to 6 newtons, lift loads over a hundred times their own weight, and reach contractions of over 40% with strain rates over 1200% per second, with a bandwidth over 10 hertz. As a pump driver, these EBMs produce flow rates of up to 0.63 liters per minute and maximum pressure head of 6 kilopascals, whereas as generator, they reach a conversion efficiency close to 20%. The compact shape, low cost, simple assembling procedure, high reliability, and large contractions make the EBM a promising technology for high-performance robotic systems.

New approach of designing robotics production systems using immersive virtual reality environment

Production requirements are growing and companies needs to adapt to the latest trends with state-of-art technologies. Virtual reality (VR) technology promises big changes in Industry sector, even this technology is still evolving. This immersive technology allows to visualization and interaction of complex engineering data in virtual environment, provide various inspection of 3D objects and allows to communicate changes on objects or process between production teams and customers. This paper discus about potential of VR systems in Industry and purpose of usage to improve development process and the use of this technology in the design of future robotic systems. The goal is to compare causal designing methods of robotic work station and method with using VR environment and their possibilities. Development supported by VR systems shortens preproduction phase and brings to user immersive experience with designing product or process without creating physical prototype or real production systems.

Intelligent Liquid Integrated Functional Entity: A Basic Way to Innovate Future Advanced Biomimetic Soft Robotics

The liquid is an essential component of natural creatures with profound capacities in assisting homeostatic regulation of physiological functions. However, its participation in more diverse biological behaviors, such as intelligence, motion, energy, etc., is seriously neglected and is undisclosed thus far. Herein, it is considered whether liquid systems can offer pivotal hints in the development of intelligent machines. Starting from this core point, a basic conjecture is proposed that specifically equips a robotic system with liquid that brings about revolutionary design of intelligent systems and plays an elementary role in molding future advanced soft robots. Through a comparative analogy between artificially made machines and natural animals, the basic category of the so‐termed Intelligent Liquid Integrated Functional Entity (I‐LIFE) is drafted and thus enabled. Then, functions of such unconventional robotic liquid are expounded from five aspects, including motion, energy supply, material performance tuning, sensing, and intelligence, respectively. Typical application examples are also illustrated. Furthermore, a group of intelligent liquids are recommended as candidates for developing future robotic systems. The fundamental scientific and technological challenges lying behind these recommendations are outlined. Finally, prospects of I‐LIFE are pointed out, which stimulates the increasing design of a future generation of innovative intelligent robotic systems.

A Robust Visual System for Small Target Motion Detection Against Cluttered Moving Backgrounds.

Monitoring small objects against cluttered moving backgrounds is a huge challenge to future robotic vision systems. As a source of inspiration, insects are quite apt at searching for mates and tracking prey, which always appear as small dim speckles in the visual field. The exquisite sensitivity of insects for small target motion, as revealed recently, is coming from a class of specific neurons called small target motion detectors (STMDs). Although a few STMD-based models have been proposed, these existing models only use motion information for small target detection and cannot discriminate small targets from small-target-like background features (named fake features). To address this problem, this paper proposes a novel visual system model (STMD+) for small target motion detection, which is composed of four subsystems-ommatidia, motion pathway, contrast pathway, and mushroom body. Compared with the existing STMD-based models, the additional contrast pathway extracts directional contrast from luminance signals to eliminate false positive background motion. The directional contrast and the extracted motion information by the motion pathway are integrated into the mushroom body for small target discrimination. Extensive experiments showed the significant and consistent improvements of the proposed visual system model over the existing STMD-based models against fake features.

A Novel Single Camera Robotic Approach for Three-Dimensional Digital Image Correlation with Targetless Extrinsic Calibration and Expanded View Angles

Three-Dimensional Digital Image Correlation (3D-DIC) is a technique used widely in both academic and industrial settings to measure object morphology, velocity, and full-field strain. However, conventional methods require the use of two synchronized cameras and a lengthy calibration process of taking thirty or more images of specialized target, after which the cameras must be kept fixed. This paper proposes a new approach for obtaining 3D-DIC measurements using only a single camera moved between different positions by a robotic arm, with the extrinsic calibration parameters obtained by manipulation of the internal position and orientation information known to the robot, obviating the need for a calibration target in order to calculate the extrinsic matrix for various positions (after the intrinsic matrix has been established). The specifications of a prototype are presented, and the method is quantitatively and qualitatively compared to the traditional 3D-DIC approach to determine relative accuracy and reliability in shape measurements. By correcting for errors in the reported orientation of the robotic arm, it was possible to achieve the same accuracy as the traditional 3D-DIC approach for stereo angles up to 30o, beyond which the loss of image overlap results in linear increasing errors that are up to an order of magnitude greater. The method is also further extended to allow for images from more than two positions to be combined in a single analysis, which enables an expanded view of up to 360° using only a single camera and without the additional calibrations required by current multi-camera expanded view systems. The integration of this new 3D-DIC approach into future robotic systems will facilitate new capabilities for performing vital tasks, such as material analysis, infrastructure assessment, and human-robot interactions.