Robotic Control Research Articles

Autonomous ecologies of construction: Collaborative modular robotic material eco-systems with deep multi-agent reinforcement learning

To address the unprecedented challenges of construction pressurized by the global climate crisis, housing shortage, and growing shortage of skilled labor, this research presents a radical shift in the construction lifecycle of buildings, from linear processes that produce static continuous buildings to interrelational processes linking adaptative eco-systems of collaborative robots and reconfigurable building parts. Inspired by natural builders, the interdisciplinary field of collective robotic construction (CRC) offers the potential for scalable, adaptive, and resilient construction with simple robots. We establish a design framework for autonomous collaborative robotic construction (ACRC) through modular robotic material eco-systems (MRMES) trained with deep multi-agent reinforcement learning (DMARL). This involves the integration of three core aspects: (1) modular robotic material eco-systems (2) cyber-physical simulation and control with bidirectional feedback (3) adaptive intelligence through deep multi-agent reinforcement learning. The framework is implemented through three comparable case studies for collaborative modular robotic assembly of reconfigurable building parts.

Fixed-Time Control with an Improved Sparrow Search Algorithm for Robotic Arm Performance Optimization

This paper presents an innovative approach that integrates a fixed-time control (FTC) algorithm with an improved sparrow search algorithm (ISSA) to enhance the trajectory tracking accuracy of a two-degree-of-freedom (two-DOF) robotic arm. The FTC algorithm, which incorporates barrier Lyapunov function (BLF) and adaptive neural network strategies, ensures rapid convergence, effective vibration suppression, and the robust handling of system uncertainties and input saturation. The ISSA, inspired by the foraging behavior of sparrows, improves search efficiency through dynamic weight adjustments and chaotic mapping, balancing global and local search capabilities. By optimizing control parameters, ISSA minimizes tracking errors. Simulation results demonstrate that the combined FTC and ISSA approach significantly reduces tracking errors and improves response speed compared to the use of FTC alone, underscoring its potential for achieving high-precision control in robotic arms and offering a promising direction for precise robotic control applications.

Motion controller for multi-joint robotic arm with deep cascade gated Bayesian broad learning system

Intelligent controllers based on the broad learning system can simplify the process of model parameter adjustment, finding wide applications in the motion control of multi-joint robotic arms. However, motion controllers for multi-joint robotic arms based on broad learning system exhibit insufficient precision and overlook the impact of joint motion commonalities on controller design. Therefore, this paper proposes a novel motion control strategy for a multi-joint robotic arm based on a deep cascade feature-enhancement gated Bayesian broad learning system. Firstly, the motion controller of the deep cascade feature-enhancement Bayesian broad learning system is constructed to enhance the robotic arm motion control precision. Secondly, an incremental node generation module with an attention-gated mechanism is constructed to capture the unique motion characteristics of the target joints, which is further combined with model generalization to simplify the motion control process of the multi-joint robotic arm. Finally, controller convergence is enhanced by combining it with the Lyapunov theory to constrain the learning parameters. Simulations and physical experiments are designed to verify the feasibility and superiority of the proposed motion control strategy. The results demonstrated that the strategy improved the accuracy of robotic arm motion control, with the root mean square error in position tracking reduced to 0.0019 rad. This represents a 93.39% reduction in error compared to existing techniques.

Multi-attribute dynamic attenuation learning improved spiking actor network

Deep reinforcement learning (DRL) has shown promising results in solving robotic control and decision tasks, which can learn the high-dimensional state and action information well. Despite their successes, conventional neural-based DRL models are criticized for low energy efficiency, making them laborious to be widely applied in low-power electronics. With more biologically plausible plasticity principles, spiking neural networks (SNNs) are now considered an energy-efficient and robust alternative. The most existing dynamics and learning paradigms for spiking neurons with a common Leaky Integrate-and-Fire (LIF) neuron model often result in relatively low efficiency and poor robustness. To address these limitations, we propose a multi-attribute dynamic attenuation learning improved spiking actor network (MADA-SAN) for reinforcement learning to achieve effective decision-making. The resistance, membrane voltage and membrane current of spiking neurons are updated from a fixed value into dynamic attenuation. By enhancing the temporal relation dependencies in neurons, this model can learn the spatio-temporal relevance of complex continuous information well. Extensive experimental results show MADA-SAN performs better than its counterpart deep actor network on six continuous control tasks from OpenAI gym. Besides, we further validated the proposed MADA-LIF can achieve comparable performance with other state-of-the-art algorithms on MNIST and DVS-gesture recognition tasks.

Research on Bionic Robot Motion Control Based on Reinforcement Learning

This research explores the application of reinforcement learning (RL) to enhance the motion control of bio-inspired robots across various environments. Focusing on underwater, terrestrial, and aerial robotic models, the study integrates model-free Q-learning, Deep Q Network (DQN), State-Action-Reward-State-Action (SARSA), and Double Deep Q Network (DDQN) algorithms. These methods are employed to achieve adaptive and efficient movement strategies without relying on pre-defined environmental models. The methodologies range from real-time data capture using live camera feeds for terrestrial robots to simulating aquatic and flight dynamics in controlled environments. Experimental results confirm the efficacy of these RL methods, demonstrating significant improvements in the robots' ability to adapt to dynamic and unknown environments, optimize movement efficiency, and navigate complex scenarios. The findings suggest a promising direction for future robotic applications, emphasizing the need for further research on optimizing these algorithms and expanding their real-world applicability. This study highlights the potential of RL to revolutionize robotic motion control, making robots more versatile and capable in varied and unpredictable settings.

Initial experience with a 5G cellular network and robotic arm-based remote echocardiographic system in an all-comers setting

Abstract Background Echocardiography is the test-of-choice for the initial evaluation of many cardiac diseases and requires expertise that is often limited in rural area. Robotic arm-assisted remote echocardigram has been attampted for teleconsultation but was limited to heart failure patients(ref), primarily due to the network delay in telecommunication and the subsequent incoordination in robotic arm control. Purpose We aim to assess the feasibility and diagnostic accuracy of a 5G cellular network and robotic arm-based remote echocardiographic system in an all-comers setting. Methods 51 pts were enrolled from the outpatient cardiology clinic of our institute. All pts underwent standard comprehensive echocardiography on a 5G cellular network and robotic arm-based remote echocardiographic system (the robotic part was deployed at our hospital and the control part was located at another campus 20 kilometers away) (Figure 1)and a conventional echocardiographic platform (at our hospital) successively. The order in which patients were examined on the remote and conventional instruments was randomly determined. There was no interval between the two examinations, and examinations of the same patient were performed by experienced but different cardiologists, who were blinded to each other’s diagnosis. The doctor who used the remote system was also randomly allocated and had not been previously specifically trained. The exams were real-time and diagnoses were made right after the exams. Results Of the 51 pts, the image quality was sufficient for diagnosis in 50 patients (48% female, 40±18 years). 1 pt was excluded because some key views could not be obtained using the remote system (98% of exams successful). 17 pts had positive findings, including 10 with a primary diagnosis of valvulopathy (1 Barlow's syndrome, 1 bicuspid aortic valve and 8 less-than-moderate regurgitation), 2 cardiac surgery follow-ups (1 case of aortic valve replacement and septal myectomy, and 1 case of mitral valve replacement and tricuspid annuloplasty), and 2 hypertrophic cardiomyopathy (including 1 case of obstruction at papillary muscle level), 2 with abnormal left ventricular wall motion(including 1 case of apical mural thrombus) (Figure 2), and 1 with congenital heart disease (secumdum atrial septal defect). The main diagnosis was consistent in 98% of cases (papillary muscle level obstruction was missed in one case), using conventional echocardiographic findings as the standard. Time for image acquision using remote echocardiography was significantly longer than conventional one (24 minutes 36 seconds ± 4 minutes 52 seconds vs. 16 minutes 15 seconds ± 1 minute 54 seconds, p<0.0001, paired t-test). Conclusions Comprehensive echocardiographic exam with a 5G cellular network and robotic arm-based remote system is feasible with relatively good diagnostic accuracy.Figure 1Figure 2

Hyper CLS-Data-Based Robotic Interface and Its Application to Intelligent Peg-in-Hole Task Robot Incorporating a CNN Model for Defect Detection

Various types of numerical control (NC) machine tools can be standardly operated and controlled based on NC data that can be easily generated using widespread CAD/CAM systems. On the other hand, the operation environments of industrial robots still depend on conventional teaching and playback systems provided by the makers, so it seems that they have not been standardized and unified like NC machine tools yet. Additionally, robotic functional extensions, e.g., the easy implementation of a machine learning model, such as a convolutional neural network (CNN), a visual feedback controller, cooperative control for multiple robots, and so on, has not been sufficiently realized yet. In this paper, a hyper cutter location source (HCLS)-data-based robotic interface is proposed to cope with the issues. Due to the HCLS-data-based robot interface, the robotic control sequence can be visually and unifiedly described as NC codes. In addition, a VGG19-based CNN model for defect detection, whose classification accuracy is over 99% and average time for forward calculation is 70 ms, can be systematically incorporated into a robotic control application that handles multiple robots. The effectiveness and validity of the proposed system are demonstrated through a cooperative pick and place task using three small-sized industrial robot MG400s and a peg-in-hole task while checking undesirable defects in workpieces with a CNN model without using any programmable logic controller (PLC). The specifications of the PC used for the experiments are CPU: Intel(R) Core(TM) i9-10850K CPU 3.60 GHz, GPU: NVIDIA GeForce RTX 3090, Main memory: 64 GB.

Vision Systems for a UR5 Cobot on a Quality Control Robotic Station

This paper addresses the development of a vision system for the UR5 cobot and the corresponding operating algorithm of the robotic quality control station. The hardware–software architecture of the developed robotic station consisting of a UR5 cobot equipped with a web camera and a stationary industrial camera with a lighting system is presented. Image processing and analysis algorithms are described, the method of control and communication between the station components is discussed, and operating scenarios are presented as a single robotic station and as a part of the robotic line. Based on the results which were obtained, the level of measurement noise, accuracy, and repeatability of the developed vision system were estimated. A novel complete vision system architecture with hardware–software modules for the UR5 cobot quality control station is shown and discussed. The software part based on Python 3.12 language, OpenCV 4.7.0.68 libraries, and the PolyScope 1.8 environment incorporates modules for system calibration, image acquisition, preprocessing and image analysis (for objects’ location and geometric measurements) and for robotic cell communication and control. The hardware part of the UR5 cobot vision system is based on a PC with two independent and distinct cameras: one permanently affixed and the other mounted to the cobot’s flange. This innovative setup, combined with software architecture, broadens the scope of existing robotic quality control applications.

An efficient infinity norm minimization algorithm for under-determined inverse problems

The problem of solving under-determined systems of linear equations with minimum peak magnitude (ℓ∞ norm) has numerous applications in signal processing. These include Peak-to-Average Power Ratio (PAPR) reduction in MIMO-OFDM systems, vector quantization, approximate nearest neighbor search, optimal control in robotics, and power grid optimization. Several methods have been proposed to address this problem, but they often face limitations in computational speed or representation accuracy. Some methods also impose constraints on the frame matrix, such as restrictions on the type of its entries or its aspect ratio. In this paper, we present the Fast Iterative Peak Shrinkage Algorithm (FIPSA), which iterates over feasible solutions to consistently reduce peak magnitude and provably converge to near-optimal solutions. Our experimental results, conducted across various frame matrix types and aspect ratios, demonstrate that FIPSA consistently achieves near-minimal ℓ∞ norm values. In addition, it operates 1.3 to 7.3 times faster than previous methods, while maintaining an average representation error of 10−15. Notably, these advancements are achieved without imposing any constraints on the frame matrix.

Advancing Distributed Distributional Deterministic Policy Gradients for Autonomous Robotic Control Across Diverse Environments

Advancing Distributed Distributional Deterministic Policy Gradients for Autonomous Robotic Control Across Diverse Environments

Mixed‐Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

ABSTRACTMany optimal control problems require the simultaneous output of discrete and continuous control variables. These problems are typically formulated as mixed‐integer optimal control (MIOC) problems, which are challenging to solve due to the complexity of the solution space. Numerical methods such as branch‐and‐bound are computationally expensive and undesirable for real‐time control. This article proposes a novel hybrid‐action reinforcement learning (HARL) algorithm, twin delayed deep deterministic actor‐Q (TD3AQ), for MIOC problems. TD3AQ leverages actor‐critic and Q‐learning methods to manage discrete and continuous action spaces simultaneously. The proposed algorithm is evaluated on a plug‐in hybrid electric vehicle (PHEV) energy management problem, where real‐time control of the discrete variables, clutch engagement/disengagement and gear shift, and continuous variable, engine torque, is essential to maximize fuel economy while satisfying driving constraints. Simulation results show that TD3AQ achieves near‐optimal control, with only a 4.69% difference from dynamic programming (DP), and outperforms baseline reinforcement learning algorithms for hybrid action spaces. The sub‐millisecond execution time indicates potential applicability in other time‐critical scenarios, such as autonomous driving or robotic control.

Editorial for Special Issue of Motion Planning and Advanced Control for Robotics

Advancements in robotics are increasingly essential in addressing complex challenges associated with motion planning and control, particularly in environments that require physical interaction with various elements [...]

Bionic Multi-Legged Robots with Flexible Bodies: Design, Motion, and Control.

Bionic multi-legged robots with flexible bodies embody human ingenuity in imitating, learning, and exploring the natural world. In contrast to rigid-body robots, these robots with flexible bodies exhibit superior locomotive capabilities. The flexible body of the robot not only boosts the moving speed and walking stability but also enhances adaptability across complex terrains. This article focuses on the innovative design of flexible bodies. Firstly, the structural designs, including artificial spines and single/multi-axis articulation mechanisms, are outlined systematically. Secondly, the enhancement of robotic motion by flexible bodies is reviewed, examining the impact that body degrees of freedom, stiffness, and coordinated control between the body and limbs have on robotic motion. Thirdly, existing robotic control methods, organized by control architectures, are comprehensively overviewed in this article. Finally, the application prospects of bionic multi-legged robots with flexible bodies are offered, and the challenges that may arise in their future development are listed. This article aims to serve as a reference for bionic robot research.

Non-invasive brain-machine interface control with artificial intelligence copilots.

Motor brain-machine interfaces (BMIs) decode neural signals to help people with paralysis move and communicate. Even with important advances in the last two decades, BMIs face key obstacles to clinical viability. Invasive BMIs achieve proficient cursor and robotic arm control but require neurosurgery, posing significant risk to patients. Non-invasive BMIs do not have neurosurgical risk, but achieve lower performance, sometimes being prohibitively frustrating to use and preventing widespread adoption. We take a step toward breaking this performance-risk tradeoff by building performant non-invasive BMIs. The critical limitation that bounds decoder performance in non-invasive BMIs is their poor neural signal-to-noise ratio. To overcome this, we contribute (1) a novel EEG decoding approach and (2) artificial intelligence (AI) copilots that infer task goals and aid action completion. We demonstrate that with this "AI-BMI," in tandem with a new adaptive decoding approach using a convolutional neural network (CNN) and ReFIT-like Kalman filter (KF), healthy users and a paralyzed participant can autonomously and proficiently control computer cursors and robotic arms. Using an AI copilot improves goal acquisition speed by up to 4.3 × in the standard center-out 8 cursor control task and enables users to control a robotic arm to perform the sequential pick-and-place task, moving 4 randomly placed blocks to 4 randomly chosen locations. As AI copilots improve, this approach may result in clinically viable non-invasive AI-BMIs.

Rapid-Learning Collaborative Pushing and Grasping via Deep Reinforcement Learning and Image Masking

When multiple objects are positioned close together or stacked, pre-grasp operations such as pushing objects can be used to create space for the grasp, thereby improving the grasping success rate. This study develops a model based on a deep Q-learning network architecture and introduces a fully convolutional network to accurately identify pixels in the workspace image that correspond to target locations for exploration. In addition, this study incorporates image masking to limit the exploration area of the robotic arm, ensuring that the agent consistently explores regions containing objects. This approach effectively addresses the sparse reward problem and improves the convergence rate of the model. Experimental results from both simulated and real-world environments show that the proposed method accelerates the learning of effective grasping strategies. When image masking is applied, the success rate in the grasping task reaches 80% after 600 iterations. The time required to reach 80% success rate is 25% shorter when image masking is used compared to when it is not used. The main finding of this study is the direct integration of image masking technique with a deep reinforcement learning (DRL) algorithm, which offers significant advancement in robotic arm control. Furthermore, this study shows that image masking technique can substantially reduce training time and improve the object grasping success rate. This innovation enables the robotic arm to better adapt to scenarios that conventional DRL methods cannot handle, thereby improving training efficiency and performance in complex and dynamic industrial applications.

EddyBot: A multichannel FPGA-based Eddy Current Testing mobile robot

We introduce EddyBot, a novel integration of a multi-channel FPGA-based Eddy Current Testing (ECT) scanner within a mobile robot designed for autonomous non-destructive testing (NDT). EddyBot distinguishes itself by leveraging advanced signal processing, real-time robotics control, and efficient data synchronization to significantly enhance inspection accuracy and efficiency. Our contributions are (i) the deployment of a Zynq-7020 FPGA for fast data acquisition and processing, (ii) alongside the Robot Operating System (ROS) for precise and real-time control with Kalman filtering (KF) sensor fusion for pose estimation. (iii) We further optimize defect detection and localization through a novel data synchronization approach, correlating the robot’s pose with ECT signals to a mean synchronization delay of only 11.77 ms. This allows for precise localization of defects within 0.18 cm, improving the overall reliability. The robot’s performance was evaluated on aluminum and ferromagnetic specimens, highlighting the system’s capability to detect defects with high accuracy.

Design and Construction of 5-Dof EMG Based Robotic Arm System

Electromyography (EMG) is an alternate method of obtaining muscle signal outputs. As a result, the current development in designing my electric plans has piqued the attention of researchers in this subject. This is because Standard controllers lack essential components, limiting the use of limbs to operate equipment, namely an arm controlled by robotics. EMG signals are subject to noise, including crosstalk, motion artifacts, ambient noise, and intrinsic noise. Electromyography preparation requires careful selection of muscle groups, electrode placement, and environment quality, all of which impact the signal output. The goal of this study is to create an EMG-based robotic arm control system that can be used to help the elderly, individuals with impairments, and those who operate in dangerous places. Initially, a literature review on current human-robot interaction approaches and analysis of the kinematics of a 5 DOF robotic arm has been conducted. Then, utilizing Electromyogram (EMG) data obtained from the muscles of the elbow, away for controlling a 5 DOF robotic arm is provided. Two accelerometers are utilized to record the human arm's gesture and posture, this information is then communicated to the robotic arm as an input. In the experiments, wrist rotation, left, right, up, and down movements were used to establish controlled motion of a 5 DOF robotic arm. The project's goal has been met with the successful development of an EMG-controlled robotic arm. The robotic arm may yet be improved by including Implementing a wireless sensor network with multiple channels.

Integrated Deep Learning Architectures for Perception, Control, and Decision-making in Robotics: a Framework for Sensing, Cognition, and Transparent Decision-making

The increasing complexity of robotic applications demands innovative approaches for addressing problems that lack analytical solutions, with deep learning (DL) emerging as a key tool for enabling robots to learn and adapt in dynamic environments. This survey reviews existing DL techniques in robotics, categorizing the major challenges and exploring successful solutions that leverage DL for perception, control, and decision-making. We discuss the use of modular versus end-to-end DL architectures, providing guidelines for selecting appropriate model structures and training strategies. The review also examines advancements in neuro-robotics systems (NRS), where the convergence of neuroscience and robotics is driving the development of robots with embodied intelligence, enabling more natural human-robot interactions. Recent progress in neural mechanisms for perception, cognition, learning, and control is highlighted, offering insights into creating future neuro-robots. Furthermore, we address the challenge of uncertainty estimation in neural networks, crucial for reliable robotic decision-making, and evaluate existing frameworks such as Bayesian belief networks and Monte Carlo sampling that improve uncertainty modeling without requiring architectural changes. Lastly, we explore efforts to enhance transparency in robotic systems through integrated reasoning and learning methods, focusing on architectures that combine logical reasoning with deep learning to provide explainable decision-making. This survey aims to offer a structured overview of current research and guide future developments in neuro-robotics.

Herbicide‐resistant weed management with robots: A weed ecological–economic model

AbstractThe heavy reliance on herbicides for weed control has led to an increase in resistant weeds in the United States. Robotic weed control is emerging as an alternative technology for removing weeds mechanically using artificial intelligence. We develop an integrated weed ecological and economic dynamic (I‐WEED) model to examine the biophysical and economic drivers of adopting robotic weed management and simulate the optimal timing and intensity of robotic adoption within and across growing seasons. We specify a cohort‐based weed growth model that relates yield damages to effective weed density and treats the susceptibility of weeds to herbicides as a renewable resource that can be regenerated by using mechanical weeding robots, due to a fitness cost that makes resistant weeds less prolific. Compared to myopic weed management which ignores resistance development, forward‐looking management leads to earlier adoption of robots and treating robots as complements instead of substitutes to herbicides. This weed management results in adopting fewer robots, deploying robots on a smaller portion of the land, higher profitability, and lower yield loss in the long run, relative to myopic management. Counterintuitively, myopic management leads to a lower resistance level through its higher robot adoption intensity. We also find that a lower level of initial weed seed resistance and/or a higher fitness cost result in a higher level of resistance because they create incentives for farmers to delay the adoption of robotic weed control. Our analysis shows the importance of jointly considering the interactions between weed ecology and economics in analyzing the incentives and effects of robotic weed management on weed resistance.

Bridging vision and touch: advancing robotic interaction prediction with self-supervised multimodal learning.

Predicting the consequences of the agent's actions on its environment is a pivotal challenge in robotic learning, which plays a key role in developing higher cognitive skills for intelligent robots. While current methods have predominantly relied on vision and motion data to generate the predicted videos, more comprehensive sensory perception is required for complex physical interactions such as contact-rich manipulation or highly dynamic tasks. In this work, we investigate the interdependence between vision and tactile sensation in the scenario of dynamic robotic interaction. A multi-modal fusion mechanism is introduced to the action-conditioned video prediction model to forecast future scenes, which enriches the single-modality prototype with a compressed latent representation of multiple sensory inputs. Additionally, to accomplish the interactive setting, we built a robotic interaction system that is equipped with both web cameras and vision-based tactile sensors to collect the dataset of vision-tactile sequences and the corresponding robot action data. Finally, through a series of qualitative and quantitative comparative study of different prediction architecture and tasks, we present insightful analysis of the cross-modality influence between vision, tactile and action, revealing the asymmetrical impact that exists between the sensations when contributing to interpreting the environment information. This opens possibilities for more adaptive and efficient robotic control in complex environments, with implications for dexterous manipulation and human-robot interaction.