Robotic Control Research Articles

Security and Privacy Issues in Deep Reinforcement Learning: Threats and Countermeasures

Deep Reinforcement Learning (DRL) is an essential subfield of Artificial Intelligence (AI), where agents interact with environments to learn policies for solving complex tasks. In recent years, DRL has achieved remarkable breakthroughs in various tasks, including video games, robotic control, quantitative trading, and autonomous driving. Despite its accomplishments, security and privacy-related issues still prevent us from deploying trustworthy DRL applications. For example, by manipulating the environment, an attacker can influence an agent’s actions, misleading it to behave abnormally. Additionally, an attacker can infer private training data and environmental information by maliciously interacting with DRL models, causing a privacy breach. In this survey, we systematically investigate the recent progress of security and privacy issues in the context of DRL. First, we present a holistic review of security-related attacks within DRL systems from the perspectives of single-agent and multi-agent systems and review privacy-related attacks. Second, we review and classify defense methods used to address security-related challenges, including robust learning, anomaly detection, and game theory approaches. Third, we review and classify privacy-preserving technologies, including encryption, differential privacy, and policy confusion. We conclude the survey by discussing open issues and possible directions for future research in this field.

Advancing Robotic Gripper Control with the Integration of Flexible Printed Pressure Sensors

A pressure sensor array fabricated on a flexible substrate using the printing method can be easily mounted in various locations by combining it with a wireless communication circuit. This is particularly advantageous for applications where wiring is difficult, such as the tip of a robot gripper. Additionally, printed devices are highly efficient in material utilization, facilitating large areas and low‐cost production. In recent years, there has been growing interest in the harmonious interaction between robots and humans, and the intelligentization of robots is a worthwhile research area. Herein, a method for realizing intelligent gripping force control and object recognition robots using a printed flexible pressure sensor array in combination with a wireless communication circuit is proposed. By mounting the sensor array on a robot gripper and performing arithmetic processing on software, these capabilities are achieved.

Paraffin-Enabled Superlattice Customization for a Photostimulated Gradient-Responsive Artificial Reflex Arc.

The development of photostimulated-motion artificial reflex arcs - a neural circuit inspired by light-driven motion reflexes - holds significant promises for advancements in robotic perception, navigation, and motion control. However, the fabrication of such systems, especially those that accommodate multiple actions and exhibit gradient responses, remains challenging. Here, a gradient-responsive photostimulated-motion artificial reflex arc is developed by integrating a programmable and tunable photoreceptor based on folded MoS2 at different twist angles. The twisted folded bilayer MoS2 used as photoreceptors can be customized via the transfer technique using patternable paraffin, where the twist angle and fold-line could be controlled. The photoluminescence (PL) intensity is 3.7 times higher at a twist angle of 29° compared to that at 0°, showing a monotonically decreasing indirect bandgap. Through tunable interlayer carrier transport, photoreceptors fabricated using folded bilayer MoS2 at different twist angles demonstrate gradient response time, enabling the photostimulated-motion artificial reflex arc for multiaction responses. They are transformed to digital command flow and studied via machine learning to control the gestures of a robotic hand, showing a prototype of photostimulated gradient-responsive artificial reflex arcs for the first time. This work provides a unique idea for developing intelligent soft robots and next-generation human-computer interfaces.

A Method for Unseen Object Six Degrees of Freedom Pose Estimation Based on Segment Anything Model and Hybrid Distance Optimization

Six degrees of freedom pose estimation technology constitutes the cornerstone for precise robotic control and similar tasks. Addressing the limitations of current 6-DoF pose estimation methods in handling object occlusions and unknown objects, we have developed a novel two-stage 6-DoF pose estimation method that integrates RGB-D data with CAD models. Initially, targeting high-quality zero-shot object instance segmentation tasks, we innovated the CAE-SAM model based on the SAM framework. In addressing the SAM model’s boundary blur, mask voids, and over-segmentation issues, this paper introduces innovative strategies such as local spatial-feature-enhancement modules, global context markers, and a bounding box generator. Subsequently, we proposed a registration method optimized through a hybrid distance metric to diminish the dependency of point cloud registration algorithms on sensitive hyperparameters. Experimental results on the HQSeg-44K dataset substantiate the notable improvements in instance segmentation accuracy and robustness rendered by the CAE-SAM model. Moreover, the efficacy of this two-stage method is further corroborated using a 6-DoF pose dataset of workpieces constructed with CloudCompare and RealSense. For unseen targets, the ADD metric achieved 2.973 mm, and the ADD-S metric reached 1.472 mm. This paper significantly enhances pose estimation performance and streamlines the algorithm’s deployment and maintenance procedures.

Control of the robotic arm system with an SSVEP-based BCI

Recent studies on brain–computer interfaces (BCIs) implemented in robotic systems have shown that the system’s effectiveness in assisting individuals with movement disorders to enhance their human–computer interaction skills. However, achieving precise and rapid online completion of tasks remains a challenge for manipulators with multiple degrees of freedom (DOFs). In this paper, we explore a time-sharing control strategy for studying motion control of a robotic arm based on steady-state visual evoked potentials. The signals are generated by the joint frequency-phase modulation method, analyzed with the filter-bank canonical correlation analysis algorithm, and identified to control the six-DOF robotic arm for task execution. The shared control strategy not only reduces user’s cognitive fatigue but also enhances system in practical environments. The use of high-frequency stimuli significantly improves user comfort, and hybrid coding increases the universality of the BCI system. Additionally, by setting multiple locations and actions randomly, the robotic arm can adaptively program the optimal path. The online results showed that BCI instructions of the proposed system could be accurately chosen from six options within 6.45 s. Subjects used an average of 12 commands for the robotic arm to achieve the proposed task with an average accuracy of 98.21%. These findings validate the feasibility and effectiveness of applying the system to robotic control. The control strategy proposed in this study exhibits versatility in controlling robots to perform various complex tasks across different domains.

Bio-Inspired Space Robotic Control Compared to Alternatives.

Controlling robots in space with necessarily low material and structural stiffness is quite challenging at least in part due to the resulting very low structural resonant frequencies or natural vibration. The frequencies are sometimes so low that the very act of controlling the robot with medium or high bandwidth controllers leads to excitation of resonant vibrations in the robot appendages. Biomimetics or biomimicry emulates models, systems, and elements of nature for solving such complex problems. Recent seminal publications have re-introduced the viability of optimal command shaping, and one recent instantiation mimics baseball pitching to propose control of highly flexible space robots. The readership will find a perhaps dizzying array of thirteen decently performing alternatives in the literature but could be left bereft selecting a method(s) deemed to be best suited for a particular application. Bio-inspired control of space robotics is presented in a quite substantial (perhaps not comprehensive) comparison, and the conclusions of this study indicate the three top performing methods based on minimizing control effort (i.e., fuel) usage, tracking error mean, and tracking error deviation, where 96%, 119%, and 80% performance improvement, respectively, are achieved.

Real-Time Performance Benchmarking of RISC-V Architecture: Implementation and Verification on an EtherCAT-Based Robotic Control System

RISC-V offers a modular technical approach combined with an open, royalty-free instruction set architecture (ISA). However, despite its advantages as a fundamental building block for many embedded systems, the escalating complexity and functional demands of real-time applications have made adhering to response time deadlines challenging. For real-time applications of RISC-V, real-time performance analysis is required for various ISAs. In this paper, we analyze the real-time performance of RISC-V through two real-time approaches based on processor architectures. For real-time operating system (RTOS) applications, we adopted FreeRTOS and evaluated its performance on HiFive1 Rev B (RISC-V) and STM3240G-EVAL (ARM M). For real-time Linux, we utilized Linux with the Preempt-RT patch and tested its performance on VisionFive 2 (RISC-V), MIO5272 (x86-64), and Raspberry Pi 4 B (ARM A). Through these experiments, we examined the response times on the real-time mechanisms of each operating system. Additionally, in the Preempt-RT experiments, scheduling latencies were evaluated by means of the cyclictest. These are very important parameters for implementing real-time applications comprised of multi-tasking. Finally, in order to show the real-time capabilities of RISC-V practically, we implemented motion control of a six-axis collaborative robot, which was performed on the VisionFive 2. This implementation provided a comparative result of RISC-V’s performance against the x86-64 architecture. Ultimately, the results indicated that the real-time performance of RISC-V for real-time applications was feasible. A noticeable achievement of this research is its first implementation of an EtherCAT master on RISC-V designed for real-time applications. The successful implementation of the EtherCAT master on RISC-V shows real-time capabilities for a wide range of real-time applications.

Utilizing Inverse Kinematics for Precise Guidance in Planning 6-DoF Robot End-Effector Movements

The solution of the kinematic inverse determines a substantial part of the robotic arm's control accuracy. Researchers frequently employ standard problem-solving techniques such as numerical, algebraic, iterative, and geometric methods. Although geometric like trigonometrical method has been widely studied, and their application is strongly dependent on the shape and dimensions of the robot. The complexity of the steps makes this approach difficult for researchers. In order to give a clearance and easiness, the step-by-step features of inverse kinematics are described in this research. The study begins with forward kinematics and refers to the DH-parameter in Homogeneous Matrix Transformations calculation. The existence of specific elements applied to mathematical derivation constituted the basis of forward kinematic discussions. And based on geometrical analysis, the inverse kinematic is then derived. Furthermore, simulations are performed to demonstrate the actual implementation of IK and the solution is then used to initiate the path planning process.

An design of robotic arm control algorithm for explosive disposal robots

Automatic explosive disposal robot have great potential to replace manual Explosive ordnance disposal(EOD) robots. In order to face the practical application scenarios of complex transformations, it is of great practical significance to study a fully automatic and autonomous EOD robot that can transfer dangerous goods to a safe area. In this paper, a mechanical arm algorithm based on the inverse kinematics of robot is presented. The relative position of the bomb is input by the electromagnetic induction device to guide the mechanical arm to the corresponding position. After obtaining the bomb, the deviation angle of the mechanical arm is input in real time by the gyroscope to achieve the relative stability of the object transportation. The experimental results indicate that the explosive disposal robot has certain practical significance.

NADOL: Neuromorphic Architecture for Spike-Driven Online Learning by Dendrites.

Biologically plausible learning with neuronal dendrites is a promising perspective to improve the spike-driven learning capability by introducing dendritic processing as an additional hyperparameter. Neuromorphic computing is an effective and essential solution towards spike-based machine intelligence and neural learning systems. However, on-line learning capability for neuromorphic models is still an open challenge. In this study a novel neuromorphic architecture with dendritic on-line learning (NADOL) is presented, which is a novel efficient methodology for brain-inspired intelligence on embedded hardware. With the feature of distributed processing using spiking neural network, NADOL can cut down the power consumption and enhance the learning efficiency and convergence speed. A detailed analysis for NADOL is presented, which demonstrates the effects of different conditions on learning capabilities, including neuron number in hidden layer, dendritic segregation parameters, feedback connection, and connection sparseness with various levels of amplification. Piecewise linear approximation approach is used to cut down the computational resource cost. The experimental results demonstrate a remarkable learning capability that surpasses other solutions, with NADOL exhibiting superior performance over the GPU platform in dendritic learning. This study's applicability extends across diverse domains, including the Internet of Things, robotic control, and brain-machine interfaces. Moreover, it signifies a pivotal step in bridging the gap between artificial intelligence and neuroscience through the introduction of an innovative neuromorphic paradigm.

Inverse-Reinforcement-Learning-Based Robotic Ultrasound Active Compliance Control in Uncertain Environments

Robotic ultrasound systems (RUS) have gained increasing attention because they can automate repetitive procedures and relieve operators' workloads. However, the complexity and uncertainty of the human surface pose a challenge for stable scanning control. This paper proposes a general active compliance control strategy based on inverse reinforcement learning (IRL) to perform adaptable scanning for uncertain and unstructured environments. We analyze the manual scanning process pattern and propose a velocity-and-force-related control strategy to achieve variable force control and handle unpredictable deformation. Then, a hybrid policy optimization framework is proposed to improve transferability. In this framework, a reinforcement learning (RL) policy with a predefined reward is built to establish the relationship between contact force and posture. Furthermore, the policy is re-optimized using IRL and generated demonstrations for IRL training. The policy is trained on simple standard phantoms and further evaluated for stability and transferability in unseen and complex environments. Quantitative results show that the difference between the proposed method and the 3-D reconstructed model in terms of posture is (2.3±1.3°, 1.9±1.2°) in continuous scans. Overall, our method provides a solution for improving the usability of robotic ultrasound systems in real-world environments.

A lightweight convolutional neural network for automated crack inspection

Convolutional neural networks (CNN) have been increasingly embraced in building inspection and maintenance to realize automated crack inspection. However, CNNs are still less frequently utilized in the construction industry because most of them are designed with complex architectures that need to be executed by high-powered and expensive GPUs. To improve its practical applicability, this study aims to design a CNN model with a lightweight architecture for crack inspection. The lightweight design is expected to be feasible on a variety of electronic devices, such as mobile phones. To do so, layer pruning and parameter reduction methods were employed to design model depth and weight, respectively. Specifically, the CNN depth was first determined by training and testing 24 experimental groups that contain sequentially increased convolutional and pooling blocks. Secondly, 4 × 4 convolutional filters with 2 strides were inserted between adjacent convolutional layers to reduce model weight and memory. As a result, only 6.82 million weights and 24 M of memory were remined, compared to 138 million weights and 421 M of memory in the widely utilized model VGG-16. During an on-site validation, the proposed lightweight CNN was successfully executed on a mobile robot powered by a CPU-driven robotic control board. Both cracked and non-cracked surfaces could be detected automatically, remotely, and in real-time. Outstanding performance was also demonstrated, with F1-scores in the training and testing datasets of 96.8% and 92.4%, respectively. Overall, the designed lightweight CNN model is validated as more practicable for on-site inspection devices, which facilitate the wide spread of computer vision-based technologies in the construction industry. Meanwhile, the findings provide a potential solution for balancing the weight and accuracy of CNNs.

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

In this paper, we propose a novel Reinforcement Learning (RL) algorithm for robotic motion control, that is, a constrained Deep Deterministic Policy Gradient (DDPG) deviation learning strategy to assist biped robots in walking safely and accurately. The previous research on this topic highlighted the limitations in the controller’s ability to accurately track foot placement on discrete terrains and the lack of consideration for safety concerns. In this study, we address these challenges by focusing on ensuring the overall system’s safety. To begin with, we tackle the inverse kinematics problem by introducing constraints to the damping least squares method. This enhancement not only addresses singularity issues but also guarantees safe ranges for joint angles, thus ensuring the stability and reliability of the system. Based on this, we propose the adoption of the constrained DDPG method to correct controller deviations. In constrained DDPG, we incorporate a constraint layer into the Actor network, incorporating joint deviations as state inputs. By conducting offline training within the range of safe angles, it serves as a deviation corrector. Lastly, we validate the effectiveness of our proposed approach by conducting dynamic simulations using the CRANE biped robot. Through comprehensive assessments, including singularity analysis, constraint effectiveness evaluation, and walking experiments on discrete terrains, we demonstrate the superiority and practicality of our approach in enhancing walking performance while ensuring safety. Overall, our research contributes to the advancement of biped robot locomotion by addressing gait optimisation from multiple perspectives, including singularity handling, safety constraints, and deviation learning.

Collocation methods for second and higher order systems

It is often unnoticed that the predominant way to use collocation methods is fundamentally flawed when applied to optimal control in robotics. Such methods assume that the system dynamics is given by a first order ODE, whereas robots are often governed by a second or higher order ODE involving configuration variables and their time derivatives. To apply a collocation method, therefore, the usual practice is to resort to the well known procedure of casting an Mth order ODE into M first order ones. This manipulation, which in the continuous domain is perfectly valid, leads to inconsistencies when the problem is discretized. Since the configuration variables and their time derivatives are approximated with polynomials of the same degree, their differential dependencies cannot be fulfilled, and the actual dynamics is not satisfied, not even at the collocation points. This paper draws attention to this problem, and develops improved versions of the trapezoidal and Hermite–Simpson collocation methods that do not present these inconsistencies. In many cases, the new methods reduce the dynamics transcription error in one order of magnitude, or even more, without noticeably increasing the cost of computing the solutions.

Supporting AI Planning and Collaboration for Robotic Applications Using TAEMS

This paper describes a general approach to integrating higher-level reasoning mechanisms including planning and scheduling methods with lower-level robotic control processes. We adopt a domain-independent task representation language TAEMS to describe the knowledge of tasks, resources, and their interrelationships. This TAEMS representation language serves as the input of the reasoning functions, which generate a schedule of executable methods to be executed by the robot in the physical world. In the execution process of this goal-directed plan, the robot also needs to attend to basic functions. The potential interactions between the plan and these basic functions would lead to interesting challenges that will be discussed. An integrated development platform with a simulator that supports real-world physics is also presented.

A Semi-Autonomous Hierarchical Control Framework for Prosthetic Hands Inspired by Dual Streams of Human.

The routine use of prosthetic hands significantly enhances amputees' daily lives, yet it often introduces cognitive load and reduces reaction speed. To address this issue, we introduce a wearable semi-autonomous hierarchical control framework tailored for amputees. Drawing inspiration from the visual processing stream in humans, a fully autonomous bionic controller is integrated into the prosthetic hand control system to offload cognitive burden, complemented by a Human-in-the-Loop (HIL) control method. In the ventral-stream phase, the controller integrates multi-modal information from the user's hand-eye coordination and biological instincts to analyze the user's movement intention and manipulate primitive switches in the variable domain of view. Transitioning to the dorsal-stream phase, precise force control is attained through the HIL control strategy, combining feedback from the prosthetic hand's sensors and the user's electromyographic (EMG) signals. The effectiveness of the proposed interface is demonstrated by the experimental results. Our approach presents a more effective method of interaction between a robotic control system and the human.

Application of Fuzzy Logic for Robotic Arm Joint Control of an Explosive Ordnance Disposal Unit

Precise luffing of payload in mobile cranes is a crucial part of material handling and safety in industrial operations. Explosive ordnance disposal uses a bomb disposal robot to safely disable explosive devices at a safe distance. This robot is a mobile jib crane type with a gripper as an end effector instead of the typical suspension cable with a hook. The common challenge of this crane type is the arm/jib movement sensitivity with respect to tip-over stability of the crane body. This is directly influenced by the payload and constrained by the degree of freedom. This paper presents a control strategy of the joint for luffing such that the back wheels remain in contact with the surface ground and avoid bucking at any given instance of weight change in the payload. Fuzzy logic was applied to control the motor torque and luffing angle of the arm in response to the load and gripper opening size to maintain the tip-over stability margin at the highest value possible. The response curves at different configurations of the two input signals were also analyzed based on the rules set to determine its precision with respect to the expected response curve.

On the Stability of a 3DOF Vibrating System Close to Resonances

Abstract Purpose In the current work, the motion of a three degrees-of-freedom (DOF) dynamical system as a vibrating model is examined. The proposed system is of high importance in vibration engineering applications, such as the analysis of the control of flexible arm robotics, flexible arm vibrational motion as a dynamic system, pump compressors, transportation devices, rotor dynamics, shipboard cranes, and human or walking analysis robotics. Methods Lagrange's equations (LE) are used to derive the equations of motion of the controlling system. The analytic solutions (AS) are obtained utilizing the multiple-scales method (MSM) up to the third order. Results The framework for removing secular terms provides the requirements for the solvability of this problem. Various resonance scenarios are categorized and the modulation equations (ME) are constructed. To graphically demonstrate the beneficial impacts of the distinct parameters of the problem, the time histories (TH) of the approximate solutions as well as the resonance curves (RC) are depicted. The Runge-Kutta algorithm (RKA) is employed to obtain the numerical solutions (NS) of the regulating system. Conclusion A comparison of the AS and NS reveals the accuracy of the perturbation approach. The stability/instability zones are studied using Routh-Hurwitz criteria (RHC), and then they are examined using a steady-state situation. Basically, the used perturbation method is considered a traditional method that is applied to solve a new dynamical system. Then, the achieved results are considered new because they weren’t obtained previously, which indicates the novelty of this work.

Effectiveness of Robotic Intervention on Improving Social Development and Participation of Children with Autism Spectrum Disorder - A Randomised Controlled Trial.

Evidence-based robotic intervention programmes for children with autism spectrum disorder (ASD) have been limited. As yet, there is insufficient evidence to inform therapists, teachers, and service providers on effectiveness of robotic intervention to enhance social development and participation of children with ASD in a real context. This study used a randomised controlled trial to test the efficacy of robotic intervention programmes in enhancing the social development and participation of children with ASD. 60 children with ASD were included. The participants were randomly assigned to the following groups: (1) robotic intervention programme (n = 20), (2) human-instructed programme (n = 20), and (3) control group (n = 20). Both the performance-based behavioural change in social communication and parent-reported change in social responsiveness were evaluated. The participants in the robotic intervention group demonstrated statistically significant changes in both the performance-based assessment and parent-reported change in social participation. Significant differences were found in the communication and reciprocal social interactions scores between the experimental group and the control and comparison groups in the performance-based assessment (p < 0.01). The effectiveness of robotic intervention programme to enhance the social communication and participation was confirmed. Future studies may also consider adding a maintenance phase to document how the effects of the intervention carry over to the participants over a longer period. (Clinical trial number: NCT04879303; Date of registration: 10 May 2021).

What weeding robots need to know about ecology

In weed control the aims of securing crop productivity and protecting biodiversity are often difficult to reconcile. Currently, the development of autonomous in-field intervention technology, such as field robots, is creating new potential for minimizing trade-offs between these two aims. To exploit this potential, weed management strategies need to adapt. However, it is currently unclear which kind of input information (e.g. weed cover, number of weeds, weed species identity) is required for such a targeted approach, and which impacts the robotic application has on the trade-off between crop yield and biodiversity. Here, we used a dataset from organically farmed fields to assess several weed management strategies, simulating robot-supported weed control. Specifically, we used within-field heterogeneity of several weed and crop productivity variables to model effects of different kinds of input information for a hypothetical, spatially selective robotic weed control system. The results showed that, at a defined yield loss, gamma diversity (number of weed species on the entire investigated area) is maintainable to a large degree, even without information on weed or crop heterogeneity within the field being used to decide where to weed. However, to maintain alpha diversity (average number of weed species per plot), more spatially explicit input information is required, such as on the number of species per plot, weed quantity (weed cover per species), and weed competitiveness. Consequently, a weeding robot would have to be technically capable of distinguishing between individual weed species, measuring weed cover, processing captured information in real time and removing weeds at per-plant level. Further, it could be shown that the success of such a complex weed management strategy is independent of the degree of spatial heterogeneity of crop yield and of the present level of weed species richness.