3D Manipulation Research Articles

3D manipulation of small multicellular organisms in soft microtubes

Small multicellular organisms, such as C. elegans and zebrafish larvae, are essential models in biological research, but their 3D manipulation poses challenges due to their small size. Manual positioning is labor-intensive and imprecise. To address this, we developed a soft PDMS microtube that can be gently stretched and released, immobilizing organisms without anesthetization and damage. This tube enables easy rotation and adjustment of orientation, facilitating comprehensive imaging. Functionality testing showed effective immobilization as well as precise imaging and microinjection. Zebrafish larvae were successfully injected with fluorescein in the hindbrain without anesthesia. This technique offers a simple, efficient, and non-damaging method for 3D manipulation and imaging of small multicellular organisms and provides a versatile tool for biological research practices.

A Force Control Method Integrating Human Skills for Complex Surface Finishing

Force control is one of the core modules for surface finishing such as grinding, polishing and sanding. However, the current force control methods based on human skills lack in-depth analysis of data patterns or are only applicable to flat surfaces. In addition, surface finishing is mainly performed by hand, resulting in low processing efficiency and poor product consistency. Therefore, this paper proposes a force control method that incorporates human skills to achieve relatively accurate force skill transfer and complex surface finishing. Firstly, human skills consisting of the force skill and the motion skill are learned. The force skill is used to generate the desired force. Then, a series of discrete poses are obtained based on human demonstration and combined with the motion skill to generate the desired trajectory. Finally, a computed-torque impedance control method is proposed to achieve relatively accurate force skill transfer and complex surface finishing by incorporating the desired trajectory and the desired force. The experiments are conducted on a platform composed of a 7-DOF collaborative robot manipulator from Franka Emika and a complex violin surface. The results demonstrate that the proposed force control method can achieve relatively accurate force skill transfer and improve the surface quality of the workpiece.

Acoustography by Beam Engineering and Acoustic Control Node: BEACON.

Acoustic manipulation has emerged as a valuable tool for precision controls and dynamic programming of cells and particles. However, conventional acoustic manipulation approaches lack the finesse necessary to form intricate, configurable, continuous, and 3D patterning of particles. Here, this study reports acoustography by Beam Engineering and Acoustic Control Node (BEACON), which delivers intricate, configurable patterns by guiding particles along custom paths with independent phase modulation. Leveraging analytical methods of orbital angular momentum beam via iterative Wirtinger hologram algorithm, this study accomplish acoustography by facilitating orbital angular momentum traps, enabling continuous 2D and 3D acoustic manipulation of microparticles in any desired geometry, with phase modulation independent of intensity. Utilizing on-chip acoustography, the BEACON platform markedly increases the space-bandwidth product to 31000 while attaining an enhanced resolution with a pixel size of ≈25µm, surpassing the typical resolution of over 200µm in previous holographic particle manipulation methods. The capabilities of BEACON are demonstrated in creating intricate triple helical tracing structures using microdroplets (20µm in diameter) and those carrying DNA to validate the effectiveness of the acoustography and phase control methods. This study offers new particle manipulation opportunities, paving the way for next-generation biomedical systems and the future of contact-free precision manufacturing.

Tunable Acoustic Tweezer System for Precise Three-Dimensional Particle Manipulation

This study introduces a tunable acoustic tweezer system designed for precise three-dimensional particle trapping and manipulation. The system utilizes a dual-liquid-layer acoustic lens, which enables the dynamic control of the focal length through the adjustable curvature of a latex membrane. This tunability is essential for generating the acoustic forces necessary for effective manipulation of particles, particularly along the direction of acoustic wave propagation (z-axis). Experiments conducted with spherical particles as small as 1.5 mm in diameter demonstrated the system’s capability for stable trapping and manipulation. Performance was rigorously evaluated through both z-axis and 3D manipulation tests. In the z-axis experiments, the system achieved a manipulation range of 33.4–53.4 mm, with a root-mean-square error and standard deviation of 0.044 ± 0.045 mm, which highlights its precision. Further, the 3D manipulation experiments showed that particles could be accurately guided along complex paths, including multilayer rectangular and helical trajectories, with minimal deviation. A visual feedback-based particle navigation system significantly enhanced positional accuracy, reducing errors relative to open-loop control. These results confirm that the tunable acoustic tweezer system is a robust tool for applications requiring precise control of particles with diameter of 1.5 mm in three-dimensional environments. Considering its ability to dynamically adjust the focal point and maintain stable trapping, this system is well suited for tasks demanding high precision, such as targeted particle delivery and other applications involving advanced material manipulation.

In-field performance evaluation of robotic arm developed for harvesting cotton bolls

Robotic harvesting of cotton bolls combines the advantages of manual picking and mechanical harvesting, including selective picking and reduced labor dependency. Thus, we designed, developed, and tested a cotton-picking robotic arm equipped with a depth camera, a 4-DoF manipulator, and a vacuum-type end-effector. Convolutional neural networks were implemented for cotton boll recognition, while an artificial neural network-assisted particle swarm optimization-based model was utilized for solving inverse kinematics. Linear actuators, controlled by a microcontroller and relays, positioned the end-effector for cotton boll harvesting. Laboratory testing resulted in picking 44–49 cotton bolls per hour, taking approximately 73.0 s per boll. During field testing, the robotic arm picked 40–42 bolls per hour with an average time of 86.0 s per boll. Picking efficiency ranged from 76.19% to 85.71% in the lab and 66.32% to 72.04% in the field. The complex structure of cotton plants sometimes hindered the end-effector’s path, resulting in lower picking efficiency during field testing. As observed in present study, manual pickers achieved a significantly higher picking rate, harvesting 1.381 kg of cotton per hour, nearly 4.7 times more than the robotic arm. Manual pickers achieved 98.04% picking efficiency, significantly higher than the robotic arm’s 68.25%. This significant difference in efficiency is due to the agility, adaptability, and decision-making abilities of human pickers. The developed robotic arm demonstrates competitive efficiency in the domain of harvesting robots, contributing to advancements in robotic agriculture. Future research and development efforts in this field hold the potential to enhance the efficiency and productivity of robotic cotton harvesters.

Integrated Intelligent Control of Redundant Degrees-of-Freedom Manipulators via the Fusion of Deep Reinforcement Learning and Forward Kinematics Models

Redundant degree-of-freedom (DOF) manipulators offer increased flexibility and are better suited for obstacle avoidance, yet precise control of these systems remains a significant challenge. This paper addresses the issues of slow training convergence and suboptimal stability that plague current deep reinforcement learning (DRL)-based control strategies for redundant DOF manipulators. We propose a novel DRL-based intelligent control strategy, FK-DRL, which integrates the manipulator’s forward kinematics (FK) model into the control framework. Initially, we conceptualize the control task as a Markov decision process (MDP) and construct the FK model for the manipulator. Subsequently, we expound on the integration principles and training procedures for amalgamating the FK model with existing DRL algorithms. Our experimental analysis, applied to 7-DOF and 4-DOF manipulators in simulated and real-world environments, evaluates the FK-DRL strategy’s performance. The results indicate that compared to classical DRL algorithms, the FK-DDPG, FK-TD3, and FK-SAC algorithms improved the success rates of intelligent control tasks for the 7-DOF manipulator by 21%, 87%, and 64%, respectively, and the training convergence speeds increased by 21%, 18%, and 68%, respectively. These outcomes validate the proposed algorithm’s effectiveness and advantages in redundant manipulator control using DRL and FK models.

Pick and Place Control of a 3-DOF Robot Manipulator Based on Image and Pattern Recognition

Board games like chess serve as an excellent testbed for human–robot interactions, where advancements can lead to broader human–robot cooperation systems. This paper presents a chess-playing robotic system to demonstrate controlled pick and place operations using a 3-DoF manipulator with image and speech recognition. The system identifies chessboard square coordinates through image processing and centroid detection before mapping them onto the physical board. User voice input is processed and transcribed into a string from which the system extracts the current and destination locations of a chess piece with a word error rate of 8.64%. Using an inverse-kinematics algorithm, the system calculates the joint angles needed to position the end effector at the desired coordinates actuating the robot. The developed system was evaluated experimentally on the 3-DoF manipulator with a voice command used to direct the robot movement in grasping a chess piece. Consideration was made involving both the own pieces as well as capturing the opponent’s pieces and moving the captured piece outside the board workspace.

Guaranteed real-time cooperative collision avoidance for n-DOF manipulators

Abstract This paper presents a decentralized, cooperative, real-time avoidance control strategy for robotic manipulators. The proposed avoidance control law builds on the concepts of artificial potential field functions and provides tighter bounds on the minimum safe distance when compared to traditional potential-based controllers. Moreover, the proposed avoidance control law is given in analytical, continuous closed form, avoiding the use of optimization techniques and discrete algorithms, and is rigorously proven to guarantee collision avoidance at all times. Examples of planar and 3D manipulators with cylindrical links under the proposed avoidance control are given and compared with the traditional approach of modeling links and obstacles with multiple spheres. The results show that the proposed avoidance control law can achieve, in general, faster convergence, smaller tracking errors, and lower control torques than the traditional approach. Furthermore, we provide extensions of the avoidance control to robotic manipulators with bounded control torques.

Learning complex nonlinear dynamics of a 3D translational parallel manipulator using neural network

This study investigates modeling the dynamics of a 3D translational parallel manipulator with closed chains using feedforward neural networks (FFNNs). The dataset exceeds 50,000 samples, incorporating experimental data collected from a robot prototype using MATLAB® real-time workshop and the National InstrumentsTM DAQ toolbox, as well as CAD simulation data from MSC ADAMS software. While achieving satisfactory mean squared error (MSE), some predictions did not fully capture the manipulator's dynamics, with small overfitting observed. A Deep Neural Network (DNN) was tested but faced overfitting and high computational costs, despite being trained on a subset of the dataset. This highlighted the limitations of DNNs for modeling such complicated parallel robots with closed chains and parallelograms. FFNNs were preferred for their simplicity and lower overfitting risk. L2 regularization and k-fold validation were applied to improve performance. Transfer learning (TL) was also employed, fine-tuning a new network with weights from pre-trained FFNNs using a smaller, unseen dataset. This approach significantly reduced MSE and completely eliminated overfitting, demonstrating the effectiveness of TL in refining model performance for forward and inverse dynamics. These findings suggest that FFNNs, combined with TL, L2 regularization, and k-fold validation, offer a robust method for accurately modeling complex robotic dynamics, enhancing control and optimization strategies for complicated robotic systems. Training for all networks was conducted within the MATLAB® environment.

Inverse kinematic solution and singularity avoidance using a deep deterministic policy gradient approach

<p>The robotic arm emerges as a subject of paramount significance within the industrial landscape, particularly in addressing the complexities of its kinematics. A significant research challenge lies in resolving the inverse kinematics of multiple degree of freedom (M-DOF) robotic arms. The inverse kinematics of M-DOF robotic arms pose a challenging problem to resolve, thus it involves consideration of singularities which affect the arm robot movement. This study aims a novel approach utilizing deep reinforcement learning (DRL) to tackle the inverse kinematic problem of the 6-DOF PUMA manipulator as a representative case within the M-DOF manipulator. The research employs Jacobian matrix for the kinematics system that can solve the singularity, and deep deterministic policy gradient (DDPG) as the kinematics solver. This chosen technique offers enhancing speed and ensuring stability. The results of inverse kinematic solution using DDPG were experimentally validated on a 6-DOF PUMA arm robot. The DDPG successfully solves inverse kinematic solution and avoids the singularity with 1,000 episodes and yielding a commendable total reward of 1,018.</p>

Analysis of Polynomial Interpolation Characteristics Based on 6DOF Manipulator Trajectory Planning

Aiming at the problems of choosing and using polynomials of different times and mixed polynomials in the process of manipulator trajectory planning, such as difficulty, wide range of light and low accuracy, in this paper, the application of different polynomials in manipulator trajectory planning is systematically analyzed by using MATLAB Robot Toolbox simulation platform and Polynomial interpolation algorithm. Using a 6-dof robotic arm Puma560 as a simulation object, the effects of three-, five-, and seven-Polynomial interpolation on joint position and pose in joint space trajectory planning were analyzed, a preliminary understanding of the application of different polynomials, and secondly, the more complex application of the fifth and seventh degree polynomials in Descartes space for the trajectory planning of the manipulator, the influence of trajectory planning with different degree polynomials on the position and pose of manipulator joint is clarified. The results show that the higher the number of polynomials, the smoother the transition of the joint at the critical path points, but the greater the maximum angular velocity and angular acceleration of the joint, the Polynomial interpolation of joint space and Descartes space trajectory planning are 14% and 23% lower than the average of absolute maximum angular velocity and absolute maximum angular acceleration of the Polynomial interpolation, respectively. Based on the analysis of the research results, this paper presents the applicable conditions and applications of Polynomial interpolation method in trajectory planning, and further summarizes the application conditions and methods of mixed polynomials.

Motion Planning for Dynamic Three-Dimensional Manipulation for Unknown Flexible Linear Object

Generally, deformable objects have large and nonlinear deformations. Because of these characteristics, recognition and estimation of their movement are difficult. Many studies have been conducted aimed at manipulating deformable objects at will. However, they have been focused on situations wherein a rope’s properties are already known from prior experiments. In our previous work, we proposed a motion planning algorithm to manipulate unknown ropes using a robot arm. Our approach considered three steps: motion generation, manipulation, and parameter estimation. By repeating these three steps, a parameterized flexible linear object model that can express the actual rope movements was estimated, and manipulation was realized. However, our previous work was limited to 2D space manipulation. In this paper, we extend our previously proposed method to address casting manipulation in a 3D space. Casting manipulation involves targeting the flexible linear object tips at the desired object. While our previous studies focused solely on two-dimensional manipulation, this work examines the applicability of the same approach in 3D space. Moreover, 3D manipulation using an unknown flexible linear object has never been reported for dynamic manipulation with flexible linear objects. In this work, we show that our proposed method can be used for 3D manipulation.

Path planning of 6-DOF free-floating space robotic manipulators using reinforcement learning

This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Design and implementation of power socket soldering robot arm system

Abstract In order to solve the problem of high cost and low efficiency in manual welding of power socket welding components, a soldering robot arm system was developed. The gripper, 5-DOF manipulator, clamp, support, and control system were designed. The coordinate system of the robot arm was constructed, and the working space of the robot arm was analyzed by using the MATLAB robot toolbox. It was concluded that the x0 coordinate range of the welding working area was [175mm, 215mm], the y0 coordinate range was [55mm, -55mm], and the z0 coordinate was 90mm. At the same time, the motion trajectory of the robot arm was determined. Using Raspberry PI as a controller, the control system is developed by programming Python language. The prototype is made, and a welding test is carried out. The test shows that the system has completed the welding operation, and the welding effect is good.

Selective Grasping for Complex-Shaped Parts Using Topological Skeleton Extraction

To enhance the autonomy and flexibility of robotic systems, a crucial role is played by the capacity to perceive and grasp objects. More in detail, robot manipulators must detect the presence of the objects within their workspace, identify the grasping point, and compute a trajectory for approaching the objects with a pose of the end-effector suitable for performing the task. These can be challenging tasks in the presence of complex geometries, where multiple grasping-point candidates can be detected. In this paper, we present a novel approach for dealing with complex-shaped automotive parts consisting of a deep-learning-based method for topological skeleton extraction and an active grasping pose selection mechanism. In particular, we use a modified version of the well-known Lightweight OpenPose algorithm to estimate the topological skeleton of real-world automotive parts. The estimated skeleton is used to select the best grasping pose for the object at hand. Our approach is designed to be more computationally efficient with respect to other existing grasping pose detection methods. Quantitative experiments conducted with a 7 DoF manipulator on different real-world automotive components demonstrate the effectiveness of the proposed approach with a success rate of 87.04%.

A speed coordination control method based on D-S evidence synthesis theory

An adaptive speed coordination control method based on Dempster–Shafer (D-S) evidence synthesis theory is proposed to achieve the speed coordination of the slave manipulator under the condition of a large transmission delay in space teleoperation. First, the D-S evidence synthesis theory is applied to transform the speed coordination rule method. The model for predicting the manipulator’s future state is given to gain confidence in each state. Subsequently, performance comparison experiments of D-S evidence synthesis control theory, cascade control, fuzzy control, and adaptive fuzzy control are completed on the 3-degree-of-freedom (3-DOF) manipulator simulation platform. Finally, according to the experimental results, the accuracy of D-S evidence synthesis theory is 7.49% better than cascade control, 16.84% better than fuzzy control, and 28.45% better than adaptive fuzzy control. The adaptability of D-S evidence synthesis theory is generally superior to cascade control, slightly inferior to fuzzy control and inferior to adaptive fuzzy control.

A novel proposed approach for optimal design of a 2-Dof manipulator arm

The manipulator arm design is generally related to a search for a compromise between conflicting requirements. Finding this compromise requires the resolution of a bi-level optimization problem (the respect of some specific constraints involves solving other optimization sub-problems). In this paper, we propose a novel approach for the optimal dimensioning of a two-degrees-of-freedom manipulator arm, where the objective function is defined by the minimization of the structure’s overall mass. The constraints that are taken into consideration can be divided into three categories: geometrical (bounds on design variables, admissible limit of the static deflection at the endpoint of the effector), kinematic (bounds on joint velocities and the accelerations of the actuators), and dynamic (admissible limit of the dynamic deflection at the endpoint of the effector). To achieve the penalizing dynamic configurations in terms of actuator torques and dynamic deflection, a quasi-static deformation model and a recurrent dynamic model were used. The static and dynamic deflection constraints are treated as optimization sub-problems, where their outcomes are integrated, at each iteration, into the global optimization process. The validation of the developed quasi-static deformation model as well as the proposed optimal dimensioning approach is conducted by comparing results with Ansys software simulations. Notably, The proposed approach demonstrates a significantly faster computation time compared to the Ansys software (06 s for the proposed approach vs 21 h for Ansys). Even though Ansys software performs optimal dimensioning for an imposed trajectory.

Topology-preserved distorted space path planning

Purpose Fast obstacle avoidance path planning is a challenging task for multijoint robots navigating through cluttered workspaces. This paper aims to address this issue by proposing an improved path-planning method based on the distorted space (DS) method, specifically designed for high-dimensional complex environments. Design/methodology/approach The proposed method, termed topology-preserved distorted space (TP-DS) method, mitigates the limitations of the original DS method by preserving space topology through elastic deformation. By applying distinct spring constants, the TP-DS autonomously shrinks obstacles to microscopic areas within the configuration space, maintaining consistent topology. This enhancement extends the application scope of the DS method to handle complex environments effectively. Findings Comparative analysis demonstrates that the proposed TP-DS method outperforms traditional methods regarding planning efficiency. Successful obstacle avoidance tasks in the cluttered workspace validate its applicability on a physical 6-DOF manipulator, highlighting its potential for industrial implementations. Originality/value The novel TP-DS method generates a topology-preserved collision-free space by leveraging elastic deformation and shows significant capability and efficiency in planning obstacle-avoidance paths in complex application scenarios.

Individual and collective manipulation of multifunctional bimodal droplets in three dimensions.

Spatiotemporally controllable droplet manipulation is vital across numerous applications, particularly in miniature droplet robots known for their exceptional deformability. Despite notable advancements, current droplet control methods are predominantly limited to two-dimensional (2D) deformation and motion of an individual droplet, with minimal exploration of 3D manipulation and collective droplet behaviors. Here, we introduce a bimodal actuation strategy, merging magnetic and optical fields, for remote and programmable 3D guidance of individual ferrofluidic droplets and droplet collectives. The magnetic field induces a magnetic dipole force, prompting the formation of droplet collectives. Simultaneously, the optical field triggers isothermal changes in interfacial tension through Marangoni flows, enhancing buoyancy and facilitating 3D movements of individual and collective droplets. Moreover, these droplets can function autonomously as soft robots, capable of transporting objects. Alternatively, when combined with a hydrogel shell, they assemble into jellyfish-like robots, driven by sunlight. These findings present an efficient strategy for droplet manipulation, broadening the capabilities of droplet-based robotics.

Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics.

In this paper, a practical discrete-time control method with adaptive image feature prediction for the image-based visual servoing (IBVS) scheme is presented. In the discrete-time IBVS inner-loop/outer-loop control architecture, the time delay caused by image capture and computation is noticed. Considering the dynamic characteristics of a 6-DOF manipulator velocity input system, we propose a linear dynamic model to describe the motion of a robot end effector. Furthermore, for better estimation of image features and smoothing of the robot's velocity input, we propose an adaptive image feature prediction method that employs past image feature data and real robot velocity data to adopt the prediction parameters. The experimental results on a 6-DOF robotic arm demonstrate that the proposed method can ensure system stability and accelerate system convergence.