3D Manipulation Research Articles

FUZZY FRACTIONAL ORDER SLIDING MODE CONTROL FOR OPTIMAL ENERGY CONSUMPTION-TRANSIENT RESPONSE TRADE-OFF IN ROBOTIC SYSTEMS

This work presents a new fuzzy logic parameter tuning-based fractional order sliding mode control (F-FOSMC) strategy for the trajectory tracking of robotic systems. The primary objective is to achieve an optimal control compromise seeking to minimize energy consumption for required motions, all while increasing both accuracy and rapidity of the transient response, addressing therefore a critical challenge in the field of robotics. Our approach integrates a sophisticated fuzzy logic fractional order mechanism to dynamically fine-tune the noninteger derivation parameter of the FOSMC, offering adaptability to varying operational conditions. This intelligent fuzzy logic controller tuning optimizes the FOSMC performance, demonstrates a remarkable capacity to handle uncertainties and disturbances intrinsic to robotic applications and allows to optimize the trade-off between the energy consumption and transient response efficiency throughout the robot’s motion. The effectiveness of the proposed study is thoroughly examined via simulations conducted on a 3 DOF manipulator robotic system. Results showcase that F-FOSMC significantly reduces control energy consumption while preserving rapid and efficient transient response characteristics.

Investigating the Potential of Haptic Props for 3D Object Manipulation in Handheld AR.

The manipulation of virtual 3D objects is essential for a variety of handheldAR scenarios. However, the mapping of commonly supported 2D touch gestures to manipulations in 3D space is not trivial. As an alternative, our work explores the use of haptic props that facilitate direct manipulation of virtual 3D objects with 6degrees of freedom. In an experiment, we instructed 20 participants to solve 2D and 3D docking tasks in AR, to compare traditional 2D touch gestures with prop-based interactions using three prop shapes (cube, rhombicuboctahedron, sphere). Our findings highlight benefits of haptic props for 3D manipulation tasks with respect to task performance, user experience, preference, and workload. For 2D tasks, the benefits of haptic props are less pronounced. Finally, while we found no significant impact of prop shape on task performance, this appears to be subject to personal preference.

CompositingVis: Exploring Interactions for Creating Composite Visualizations in Immersive Environments.

Composite visualization represents a widely embraced design that combines multiple visual representations to create an integrated view. However, the traditional approach of creating composite visualizations in immersive environments typically occurs asynchronously outside of the immersive space and is carried out by experienced experts. In this work, we aim to empower users to participate in the creation of composite visualization within immersive environments through embodied interactions. This could provide a flexible and fluid experience with immersive visualization and has the potential to facilitate understanding of the relationship between visualization views. We begin with developing a design space of embodied interactions to create various types of composite visualizations with the consideration of data relationships. Drawing inspiration from people's natural experience of manipulating physical objects, we design interactions based on the combination of 3D manipulations in immersive environments. Building upon the design space, we present a series of case studies showcasing the interaction to create different kinds of composite visualizations in virtual reality. Subsequently, we conduct a user study to evaluate the usability of the derived interaction techniques and user experience of creating composite visualizations through embodied interactions. We find that empowering users to participate in composite visualizations through embodied interactions enables them to flexibly leverage different visualization views for understanding and communicating the relationships between different views, which underscores the potential of several future application scenarios.

FBi-RRT: a path planning algorithm for manipulators with heuristic node expansion

AbstractThis paper is concerned with the problem of collision-free path planning for manipulators in multi-obstacle scenarios. Aiming at overcoming the deficiencies of existing algorithms in excessive time consumption and poor expansion quality, a path planning algorithm named Fast Bi-directional Rapidly-exploring Random Tree (FBi-RRT) with novel heuristic node expansion is proposed, which includes a selective-expansion strategy and a vertical-exploration strategy. The selective-expansion strategy is designed to guide the selection of the nearest-neighbor node to avoid the repeated expansion failure, thereby shortening the overall planning time. Also, the vertical-exploration strategy is developed to regulate the expansion direction of the collision nodes to escape from the obstacle space with less blindness, thus improving the expansion quality and further reducing time cost. Compared with previous planning algorithms, FBi-RRT can generate a feasible path for manipulators in a drastically shorter time. To validate the effectiveness of the proposed heuristic node expansion, FBi-RRT is conducted on a 6-DOF manipulator and tested in five scenarios. The experimental results demonstrate that FBi-RRT outperforms the existing methods in time consumption and expansion quality.

Adaptive fuzzy sliding mode control of the manipulator based on an improved super-twisting algorithm

The high-precision trajectory tracking and chatter-free performance of the manipulators are of great significance to the improvement of the automation level. Therefore, in view of the problems of chattering and tracking performance degradation of multi-joint manipulators caused by uncertain factors such as external disturbances and modeling errors, an adaptive fuzzy sliding mode control of the manipulator based on an improved super-twisting algorithm is proposed. First, fuzzy control is chosen to approximate the uncertainty of the manipulator model, which reduces the influence of unknown disturbances on the system, and then the adaptive law is utilized to automatically modify and improve the fuzzy rules to enhance the approximation capability; Second, an improved super-twisting algorithm is used on this basis to put the discontinuous term into the integral term to avoid chattering and eliminate the approximation error; Finally, a comparative simulation experiment is carried out with the 3-DOF manipulator as the research object. The experimental results show that the method in this paper improves the convergence speed by up to 69.74% and the tracking error accuracy by up to 85.11% over the fuzzy control based on adaptive reaching law with no chattering and little difference in control inputs, thus verifying the feasibility and superiority of the method.

Accurate Wide-Range Scaling Manipulation of the Model with a 6-DOF Tracking Device

In virtual reality applications, translation, orientation, and scaling are the three basic properties of 3D model manipulation. The 6-DOF tracking device can be used to directly manipulate the former two properties. However, the scaling property cannot be directly mapped with a hand-held tracking device because it is physically impossible. In this study, we first compare various schemes for mapping the translation and orientation of a handheld tracking device to the 3D model’s scaling property. Next, we propose a scaling manipulation framework that combines the advantages of different schemes. This enables accurate and natural wide-range scaling with the orthogonal combination design of the translation and orientation data of the tracking device. The efficiency and accuracy of the proposed framework were examined in a user study, which verified that it improved the scaling error of wide-range scaling by over 55% without a significant decrease in time performance. Finally, two scaling scenarios for the application of this framework were presented.

Tensor train for global optimization problems in robotics

The convergence of many numerical optimization techniques is highly dependent on the initial guess given to the solver. To address this issue, we propose a novel approach that utilizes tensor methods to initialize existing optimization solvers near global optima. Our method does not require access to a database of good solutions. We first transform the cost function, which depends on both task parameters and optimization variables, into a probability density function. Unlike existing approaches, the joint probability distribution of the task parameters and optimization variables is approximated using the Tensor Train model, which enables efficient conditioning and sampling. We treat the task parameters as random variables, and for a given task, we generate samples for decision variables from the conditional distribution to initialize the optimization solver. Our method can produce multiple solutions (when they exist) faster than existing methods. We first evaluate the approach on benchmark functions for numerical optimization that are hard to solve using gradient-based optimization solvers with a naive initialization. The results show that the proposed method can generate samples close to global optima and from multiple modes. We then demonstrate the generality and relevance of our framework to robotics by applying it to inverse kinematics with obstacles and motion planning problems with a 7-DoF manipulator.

Hybrid impedance and admittance control for optimal robot–environment interaction

AbstractCompliant interaction between robots and the environment is crucial for completing contact-rich tasks. However, obtaining and implementing optimal interaction behavior in complex unknown environments remains a challenge. This article develops a hybrid impedance and admittance control (HIAC) scheme for robots subjected to the second-order unknown environment. To obtain the second-order target impedance model that represents the optimal interaction behavior without the accurate environment dynamics and acceleration feedback, an impedance adaptation method with virtual inertia is proposed. Since impedance control and admittance control have complementary structures and result in unsatisfactory performance in a wide range of environmental stiffness due to their fixed causality, a hybrid system framework suitable for the second-order environment is proposed to generate a series of intermediate controllers which interpolate between the responses of impedance and admittance controls by using a switching controller and adjusting its switching duty cycle. In addition, the optimal intermediate controller is selected using a mapping of the optimal duty cycle to provide the optimal implementation performance for the target impedance model. The proposed HIAC scheme can achieve the desired interaction and impedance implementation performance while ensuring system stability. Simulation and experimental studies are performed to verify the effectiveness of our scheme with a 2-DOF manipulator and a 7-DOF Franka EMIKA panda robot, respectively.

STUDYING THE APPLICATION OF CAD/CAM/CAE SYSTEMS IN THE DESIGN OF COMPONENTS FOR PERSONAL BALLISTIC PROTECTION EQUIPMENT

The article presents a study of the possibilities for applying the virtual set of tools for 3D modeling, manipulation, and testing of components that are part of individual ballistic protection equipment.

3D Motion Manipulation for Micro- and Nanomachines: Progress and Future Directions.

In the past decade, micro- and nanomachines (MNMs) have made outstanding achievements in the fields of targeted drug delivery, tumor therapy, microsurgery, biological detection, and environmental monitoring and remediation. Researchers have made significant efforts to accelerate the rapid development of MNMs capable of moving through fluids by means of different energy sources (chemical reactions, ultrasound, light, electricity, magnetism, heat, or their combinations). However, the motion of MNMs is primarily investigated in confined two-dimensional (2D) horizontal setups. Furthermore, three-dimensional (3D) motion control remains challenging, especially for vertical movement and control, significantly limiting its potential applications in cargo transportation, environmental remediation, and biotherapy. Hence, an urgent need is to develop MNMs that can overcome self-gravity and controllably move in 3D spaces. This review delves into the latest progress made in MNMs with 3D motion capabilities under different manipulation approaches, discusses the underlying motion mechanisms, explores potential design concepts inspired by nature for controllable 3D motion in MNMs, and presents the available 3D observation and tracking systems.

Dynamic analysis and sliding mode control method of 5-DOF manipulator

Background The five degree of freedom (5-DOF) manipulator greatly improves the machining efficiency and accuracy because of its high flexibility. They see wide application in various automation fields. The dynamic analysis and modeling of manipulators is of great significance to improve the working accuracy of a manipulator. Methods For the robot task of sheet metal bending, a 5-DOF manipulator based on sliding mode control strategy is designed in this paper. Firstly, the dynamics of the 5-DOF manipulator is analyzed and the dynamic equation is established. Secondly, based on the principle of sliding mode control, a proportional integral (PI) sliding mode control method for 5-DOF manipulator based on nominal model is proposed. PI sliding mode control method has the advantages of solving system deviation problem, eliminating steady-state error, strong adaptability, good robustness, sliding mode design and easy implementation. Finally, the sliding mode control simulation experiment of 5-DOF manipulator is carried out to verify its stability. Results The 5-DOF manipulator with PI sliding mode control has a good control effect by overcoming the influence of modeling error due to its strong robustness, and effectively realizes good control stability. Conclusions The experimental results show that the 5-DOF manipulator has good response speed and stability. The results also suggest that the manipulator can be widely used in complex scenarios such as medical surgery or industrial production line with high safety requirements.

Handling Four DOF Robot to Move Objects Based on Color and Weight using Fuzzy Logic Control

Manipulators are increasingly used in industry to improve efficiency in jobs that require precision, long duration, and repetitive work. This research was conducted on a laboratory scale to control manipulators on a pick-and-place system in the product storage and packing area. The object of this research is a four-degree-of-freedom (4-DOF) manipulator controlled using a fuzzy logic system. The hardware used is a conveyor machine to model the product delivery process, Dobot Magician as a 4-DOF manipulator, HX711 load cell serves as a weight sensor, TCS-3200 serves as a color sensor, and Arduino Mega 2560 as a controller. The software used is Dobot Studio as the main program to control the movement of the robot and Matlab to develop the Fuzzy Logic Control (FLC) function, which is embedded in the Arduino. Fuzzy logic control processes weight variables and color variables read by sensors as information data to control the movement of the manipulator. The results showed that the manipulator was able to pick up and place objects according to the path-planning coordinates. The testing data states that the precision and accuracy of the average coordinates of product pick and place against the path planning has an error deviation of 1.8%.

Eagle-inspired manipulator with adaptive grasping and collapsible mechanism and modular DOF for UAV operations

This paper presents an eagle-inspired manipulator designed to enhance the efficiency of medium and large multi-rotor UAVs during contact operations such as picking, obstacle cleaning, and grasping. Drawing inspiration from the skeleton structure and motion law of an eagle’s leg, we propose a prototype design of a large load, wide range of movement, multiple DOF, and collapsible manipulator. The main structure of the manipulator consists of a planetary reducer and a coupled parallel linkage mechanism, which enables it to have greater payload capacity while also increasing its range of motion. The grasping mechanism is based on the motion law of an eagle’s claw, which effectively increases the contact area with targets. To further enhance the DOF of the grasping mechanism, we have designed an ankle mechanism that utilizes a pair of double helical telescopic mechanisms to constrain the DOF of the universal joint, connecting the grasping mechanism to the main structure of the manipulator. Regarding the comparison with existing techniques, our eagle-inspired manipulator has several advantages over previous designs. On one hand, the grasp planning strategy, which is inspired by the motion law of an eagle’s claw, can achieve more reliable and accurate grasping than existing techniques. On the other hand, the manipulator’s collapsible design facilitates easier transportation and storage while maintaining its wide range of motion. What’s more, the modular DOF design allows for greater flexibility in adapting to different payloads and operating environments when compared to existing technologies. We outline the development process of this manipulator, including kinematic modeling and control. The ability of the manipulator system is tested, and flight test experiments are conducted on a UAV carrying the manipulator to demonstrate its practicality and effectiveness.

Empirically evaluating virtual reality’s effect on reservoir engineering tasks

To help determine in what ways virtual reality (VR) technologies may benefit reservoir engineering workflows, we conducted a usability study on a prototype VR tool for performing reservoir model analysis tasks. By leveraging the strengths of VR technologies, this tool’s aim is to help advance reservoir analysis workflows beyond conventional methods by improving how one understands, analyzes, and interacts with reservoir model visualizations. To evaluate our tool’s VR approach to this, the study presented herein was conducted with reservoir engineering experts who used the VR tool to perform three common reservoir model analysis tasks: the spatial filtering of model cells using movable planes, the cross-comparison of multiple models, and well path planning. Our study found that accomplishing these tasks with the VR tool was generally regarded as easier, quicker, more effective, and more intuitive than traditional model analysis software while maintaining a feeling of low task workload on average. Overall, participants provided positive feedback regarding their experience with using VR to perform reservoir engineering work tasks, and in general, it was found to improve multi-model cross-analysis and rough object manipulation in 3D. This indicates the potential for VR to be better than conventional means for some work tasks and participants also expressed they could see it best utilized as an addition to current software in their reservoir model analysis workflows. There were, however, some concerns voiced when considering the full adoption of VR into their work that would be best first addressed before this took place.

Multi-functional single cell manipulation method based on a multichannel micropipette

Cell manipulation techniques that provide versatility in handling various cell types and performing diverse manipulations are essential for advancing biological research. This study presents a novel and adaptable approach utilizing a multifunctional micropipette for precise single-cell operations, including three-dimensional (3D) manipulation, trapping, lysis, and aspiration. The multifunctional micropipette comprises two liquid metal-based channels with high electrical conductivity, along with a negative pressure-based aspiration channel. By utilizing the liquid metal electrodes, the system enables the application of electrical stimulation, inducing both dielectrophoretic force and electroporation on cells. Through the implementation of dielectrophoretic force and integration with a 3D-movable stage, single cells can be patterned, manipulated in 3D, accurately trapped, and separated with a minimal adjacent distance of 10 µm. By utilizing the electroporation phenomenon, rapid lysis of suspension cells can be achieved within 50 ms, while adherent cells can be lysed within 200 ms, utilizing a square-wave electrical signal with a frequency of 1 kHz and an amplitude of 40 Vpp. Furthermore, the study explores the impact of electrode size on the lysis of adherent cell groups. Additionally, the utilization of negative pressure for lysate aspiration is also demonstrated. In summary, the reported technique is versatile for single cell manipulation.

Interactive 3D Annotation of Objects in Moving Videos from Sparse Multi-view Frames

Segmenting and determining the 3D bounding boxes of objects of interest in RGB videos is an important task for a variety of applications such as augmented reality, navigation, and robotics. Supervised machine learning techniques are commonly used for this, but they need training datasets: sets of images with associated 3D bounding boxes manually defined by human annotators using a labelling tool. However, precisely placing 3D bounding boxes can be difficult using conventional 3D manipulation tools on a 2D interface. To alleviate that burden, we propose a novel technique with which 3D bounding boxes can be created by simply drawing 2D bounding rectangles on multiple frames of a video sequence showing the object from different angles. The method uses reconstructed dense 3D point clouds from the video and computes tightly fitting 3D bounding boxes of desired objects selected by back-projecting the 2D rectangles. We show concrete application scenarios of our interface, including training dataset creation and editing 3D spaces and videos. An evaluation comparing our technique with a conventional 3D annotation tool shows that our method results in higher accuracy. We also confirm that the bounding boxes created with our interface have a lower variance, likely yielding more consistent labels and datasets.

An inverse kinematics solution with trajectory scaling for redundant manipulators

Kinematic redundancy offers robots many useful capabilities, such as an ability to fulfill several tasks simultaneously. However, the inverse kinematics (IK) problem for redundant manipulators is not trivial. Often, pseudoinverse-based methods are used, allowing for the projection of the desired joint space velocity vector on the null space of the manipulator’s Jacobian. Unfortunately, it is not straightforward to include in the solution the joint position, velocity and acceleration bounds. On the other hand, the IK problem can be formulated as an optimization problem. A quadratic programming (QP) formulation is especially attractive for an online implementation, since it is computationally much lighter than nonlinear optimization methods. In a QP IK formulation, the joint constraints can be easily included. However, if the desired end effector trajectory becomes too demanding, then the joint space solution might not exist. To overcome this obstacle, we propose a QP IK formulation with trajectory scaling. The method is successfully tested on a model of KUKA LWR4+ 7-DOF manipulator; the joint constraints are satisfied and the motion is slowed down only when it is necessary.

Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation

AbstractNeural Radiance Fields (NeRF) have significantly advanced the generation of highly realistic and expressive 3D scenes. However, the task of editing NeRF, particularly in terms of geometry modification, poses a significant challenge. This issue has obstructed NeRF's wider adoption across various applications. To tackle the problem of efficiently editing neural implicit fields, we introduce Neural Impostor, a hybrid representation incorporating an explicit tetrahedral mesh alongside a multigrid implicit field designated for each tetrahedron within the explicit mesh. Our framework bridges the explicit shape manipulation and the geometric editing of implicit fields by utilizing multigrid barycentric coordinate encoding, thus offering a pragmatic solution to deform, composite, and generate neural implicit fields while maintaining a complex volumetric appearance. Furthermore, we propose a comprehensive pipeline for editing neural implicit fields based on a set of explicit geometric editing operations. We show the robustness and adaptability of our system through diverse examples and experiments, including the editing of both synthetic objects and real captured data. Finally, we demonstrate the authoring process of a hybrid synthetic‐captured object utilizing a variety of editing operations, underlining the transformative potential of Neural Impostor in the field of 3D content creation and manipulation.

Adaptive Model Predictive Control for Underwater Manipulators Using Gaussian Process Regression

In this paper, the precise control of the underwater manipulator has studied under the conditions of uncertain underwater dynamics and time-varying external interference. An improved adaptive model predictive control (MPC) method is proposed for a multiple-degrees-of-freedom (DOF) underwater manipulator. In this method, the Gaussian process regression (GPR) algorithm has been embedded into the precise trajectory tracking control of the underwater manipulator. The GPR algorithm has been used to predict the water resistance, additional mass, buoyancy and external interference in real time, and the control law has been calculated by the terminal constraint MPC to realize the adaptive internal and external interference compensation. In addition, a more accurate dynamic model of the underwater 6-DOF manipulator is established by combining Lagrange equation with Morrison formula. Finally, the effectiveness of the adaptive MPC using GPR method is verified by a series of comparative simulations.

Emergency ejection characteristics of space manipulator multi-body system

AbstractSpace manipulators are typically installed on spacecraft using an emergency separation device (ESD). In the event of a malfunction, the ESD ejects the manipulator from the spacecraft. However, due to the relative rotation of the manipulator’s joints during the ejection, the equivalent ejection mass varies depending on different attitudes. This paper focuses on studying manipulators equipped with separation slide rails and analyzes their ejection characteristics under different attitudes to determine the optimal manipulator attitude for ejection. Initially, the ejection dynamics model of the space manipulator is established using the Lagrangian method, based on the kinetic energy equation, kinematics equation, and the boundary condition between the manipulator and ESD. Afterward, the space dynamics model is transformed into the dynamic model of plane ejection state by recursion formula. From this model, the equivalent ejection mass and ejection velocity are obtained, and the joint angular variation during ejection is acquired by considering joint friction torque. Using the law of conservation of angular momentum, the ejection angular velocity is then calculated. Finally, this study selected a 7-DOF space manipulator as an example and adjusted the damping parameter B of the joint for more precise calculations by choosing the attitude with a relatively larger joint angular variation. The modified model was then tested for its applicability to other attitudes. After determining the value of B, the correctness of the algorithm was validated by MATLAB calculation, ADAMS simulation, and real object ejection test.