Robot Control Applications Research Articles

Performance investigation of MVMD-MSI algorithm in frequency recognition for SSVEP-based brain-computer interface and its application in robotic arm control.

This study focuses on improving the performance of steady-state visual evoked potential (SSVEP) in brain-computer interfaces (BCIs) for robotic control systems. The challenge lies in effectively reducing the impact of artifacts on raw data to enhance the performance both in quality and reliability. The proposed MVMD-MSI algorithm combines the advantages of multivariate variational mode decomposition (MVMD) and multivariate synchronization index (MSI). Compared to widely used algorithms, the novelty of this method is its capability of decomposing nonlinear and non-stationary EEG signals into intrinsic mode functions (IMF) across different frequency bands with the best center frequency and bandwidth. Therefore, SSVEP decoding performance can be improved by this method, and the effectiveness of MVMD-MSI is evaluated by the robot with 6 degrees-of-freedom. Offline experiments were conducted to optimize the algorithm's parameters, resulting in significant improvements. Additionally, the algorithm showed good performance even with fewer channels and shorter data lengths. In online experiments, the algorithm achieved an average accuracy of 98.31% at 1.8s, confirming its feasibility and effectiveness for real-time SSVEP BCI-based robotic arm applications. The MVMD-MSI algorithm, as proposed, represents a significant advancement in SSVEP analysis for robotic control systems. It enhances decoding performance and shows promise for practical application in this field.

Meso Hybridized Silk Fibroin Watchband for Wearable Biopotential Sensing and AI Gesture Signaling.

Human biopotential signals, such as electrocardiography, are closely linked to health and chronic conditions. Electromyography, corresponds to muscle actions and is pertinent to human-machine interactions. Here, we present a type of smart and flexible watchband that includes a mini flexible electrode array based on Mo-Au filament mesh, combined with mesoscopic hybridized silk fibroin films. As the layer in contact with the skin, waterborne polyurethane and SF create a highly flexible and permeable meso-hybridized SF/WPU layer, ensuring skin-friendliness and comfortable wearing. The flexible FM electrodes are created by integrating Mo-Au FM into 2D-interconnected networks. Molybdenum filaments provide high rigidity and are coated with Aurum to enhance conductivity. The use of Mo-Au FMs in warp-knitted patterns results in high SNR (43.22 dB), high sensitivity (44.43 mV/kg), and significant motion noise reduction due to the pattern's elastic deformability and skin-gripping properties. Leveraging these unique technologies, these smart watchbands excel in prolonged sensing operation, grasping force detection, and gesture recognition. Through smart raining via deep learning, we achieved an unparalleled recognition rate (96% across 20 volunteers of different genders) among other EMG sensing devices. These results have significant implications for human-machine interaction, including applications in underwater robot control, drone operation, and autonomous vehicle control.

Efficient Robot Manipulation via Reinforcement Learning with Dynamic Movement Primitives-Based Policy

Reinforcement learning (RL) that autonomously explores optimal control policies has become a crucial direction for developing intelligent robots while Dynamic Movement Primitives (DMPs) serve as a powerful tool for efficiently expressing robot trajectories. This article explores an efficient integration of RL and DMP to enhance the learning efficiency and control performance of reinforcement learning in robot manipulation tasks by focusing on the forms of control actions and their smoothness. A novel approach, DDPG-DMP, is proposed to address the efficiency and feasibility issues in the current RL approaches that employ DMP to generate control actions. The proposed method naturally integrates a DMP-based policy into the actor–critic framework of the traditional RL approach Deep Deterministic Policy Gradient (DDPG) and derives the corresponding update formulas to learn the networks that properly decide the parameters of DMPs. A novel inverse controller is further introduced to adaptively learn the translation from observed states into various robot control signals through DMPs, eliminating the requirement for human prior knowledge. Evaluated on five robot arm control benchmark tasks, DDPG-DMP demonstrates significant advantages in control performance, learning efficiency, and smoothness of robot actions compared to related baselines, highlighting its potential in complex robot control applications.

Fixed-Time Control with an Improved Sparrow Search Algorithm for Robotic Arm Performance Optimization

This paper presents an innovative approach that integrates a fixed-time control (FTC) algorithm with an improved sparrow search algorithm (ISSA) to enhance the trajectory tracking accuracy of a two-degree-of-freedom (two-DOF) robotic arm. The FTC algorithm, which incorporates barrier Lyapunov function (BLF) and adaptive neural network strategies, ensures rapid convergence, effective vibration suppression, and the robust handling of system uncertainties and input saturation. The ISSA, inspired by the foraging behavior of sparrows, improves search efficiency through dynamic weight adjustments and chaotic mapping, balancing global and local search capabilities. By optimizing control parameters, ISSA minimizes tracking errors. Simulation results demonstrate that the combined FTC and ISSA approach significantly reduces tracking errors and improves response speed compared to the use of FTC alone, underscoring its potential for achieving high-precision control in robotic arms and offering a promising direction for precise robotic control applications.

Hyper CLS-Data-Based Robotic Interface and Its Application to Intelligent Peg-in-Hole Task Robot Incorporating a CNN Model for Defect Detection

Various types of numerical control (NC) machine tools can be standardly operated and controlled based on NC data that can be easily generated using widespread CAD/CAM systems. On the other hand, the operation environments of industrial robots still depend on conventional teaching and playback systems provided by the makers, so it seems that they have not been standardized and unified like NC machine tools yet. Additionally, robotic functional extensions, e.g., the easy implementation of a machine learning model, such as a convolutional neural network (CNN), a visual feedback controller, cooperative control for multiple robots, and so on, has not been sufficiently realized yet. In this paper, a hyper cutter location source (HCLS)-data-based robotic interface is proposed to cope with the issues. Due to the HCLS-data-based robot interface, the robotic control sequence can be visually and unifiedly described as NC codes. In addition, a VGG19-based CNN model for defect detection, whose classification accuracy is over 99% and average time for forward calculation is 70 ms, can be systematically incorporated into a robotic control application that handles multiple robots. The effectiveness and validity of the proposed system are demonstrated through a cooperative pick and place task using three small-sized industrial robot MG400s and a peg-in-hole task while checking undesirable defects in workpieces with a CNN model without using any programmable logic controller (PLC). The specifications of the PC used for the experiments are CPU: Intel(R) Core(TM) i9-10850K CPU 3.60 GHz, GPU: NVIDIA GeForce RTX 3090, Main memory: 64 GB.

Vision Systems for a UR5 Cobot on a Quality Control Robotic Station

This paper addresses the development of a vision system for the UR5 cobot and the corresponding operating algorithm of the robotic quality control station. The hardware–software architecture of the developed robotic station consisting of a UR5 cobot equipped with a web camera and a stationary industrial camera with a lighting system is presented. Image processing and analysis algorithms are described, the method of control and communication between the station components is discussed, and operating scenarios are presented as a single robotic station and as a part of the robotic line. Based on the results which were obtained, the level of measurement noise, accuracy, and repeatability of the developed vision system were estimated. A novel complete vision system architecture with hardware–software modules for the UR5 cobot quality control station is shown and discussed. The software part based on Python 3.12 language, OpenCV 4.7.0.68 libraries, and the PolyScope 1.8 environment incorporates modules for system calibration, image acquisition, preprocessing and image analysis (for objects’ location and geometric measurements) and for robotic cell communication and control. The hardware part of the UR5 cobot vision system is based on a PC with two independent and distinct cameras: one permanently affixed and the other mounted to the cobot’s flange. This innovative setup, combined with software architecture, broadens the scope of existing robotic quality control applications.

Large measurement range curvature sensor based on all-fiber Mach-Zehnder interferometer using triple-ring-core few-mode fiber.

This study experimentally demonstrates a large measurement range curvature sensor based on a Mach-Zehnder interferometer (MZI) in a triple-ring-core fiber (TRCF). The sensor is fabricated by fusion splicing a segment of TRCF between two pieces of single-mode fiber (SMF), forming the SMF-TRCF-SMF sandwich structure. Since the TRCF can support the propagation of a few guided modes, the fundamental mode interferes with high-order modes in the sensing part to produce a periodic interference spectrum. Bending causes the effective refractive index of the mode to change, which shifts the interference spectrum. The bending measurement is achieved by monitoring the wavelength variation of interference dips. The proposed sensors can realize curvature measurements up to 100 m-1, with a bending sensitivity of -225.9 pm/m-1. It is believed that the proposed MZI sensor may have potential applications in robot hand gesture recognition and control.

Mechanoreception of pneumatic soft robotic finger without tactile sensor based on dual-position feature

Purpose Mechanoreception is crucial for robotic planning and control applications, and for robotic fingers, mechanoreception is generally obtained through tactile sensors. As a new type of robotic finger, the soft finger also requires mechanoreception, like contact force and object stiffness. Unlike rigid fingers, soft fingers have elastic structures, meaning there is a connection between force and deformation of the soft fingers. It allows soft fingers to achieve mechanoreception without using tactile sensors. This study aims to provide a mechanoreception sensing scheme of the soft finger without any tactile sensors. Design/methodology/approach This research uses bending sensors to measure the actual bending state under force and calculates the virtual bending state under assumed no-load conditions using pressure sensors and statics model. The difference between the virtual and actual finger states is the finger deformation under load, and its product with the finger stiffness can be used to calculate the contact force. There are distinctions between the virtual and actual finger state change rates in the pressing process. The difference caused by the stiffness of different objects is different, which can be used to identify the object stiffness. Findings Contact force perception can achieve a detection accuracy of 0.117 N root mean square error within the range of 0–6 N contact force. The contact object stiffness perception has a detection average deviation of about 15%, and the detection standard deviation is 10% for low-stiffness objects and 20% for high-stiffness objects. It performs better at detecting the stiffness of low-stiffness objects, which is consistent with the sensory ability of human fingers. Originality/value This paper proposes a universal mechanoreception method for soft fingers that only uses indispensable bending and pressure sensors without tactile sensors. It helps to reduce the hardware complexity of soft robots. Meanwhile, the soft finger no longer needs to deploy the tactile sensor at the fingertip, which can benefit the optimization design of the fingertip structure without considering the complex sensor installation. On the other hand, this approach is no longer confined to adding components needed. It can fully use the soft robot body’s physical elasticity to convert sensor signals. Essentially, It treats the soft actuators as soft sensors.

GAN-BodyPose: Real-time 3D human body pose data key point detection and quality assessment assisted by generative adversarial network

With the rapid advancement of deep learning and computer vision, these technologies are becoming increasingly vital in areas like virtual reality, medical diagnosis, and sports training. Existing methods for real-time 3D human body pose keypoint detection and quality assessment face significant challenges such as insufficient detection accuracy, low computational efficiency, and high data quality requirements. To address these challenges, we propose an innovative solution, GAN-BodyPose. This approach integrates 3D convolutional neural networks, self-attention mechanisms, and generative adversarial networks to deliver efficient and accurate detection and assessment in real time. The GAN-BodyPose framework combines 3D-CNN and self-attention for effective feature extraction and keypoint detection, enhanced further by generative adversarial networks for superior data quality and accuracy. Our extensive evaluations using a large-scale 3D human body pose dataset demonstrated that GAN-BodyPose outperforms traditional methods, showing improvements in processing speed (15% faster), accuracy in terms of Mean Per Joint Position Error (reduced by approximately 2.2%), and an Area Under the Curve (AUC) score increased by approximately 9.5% compared to HR-Net and other datasets. Additionally, it achieves lower Floating-Point Operations (FLOPs) by about 9.3%, indicating more efficient computational performance. These advancements underline the potential of our approach to significantly enhance user experiences in virtual reality, motion capture, and other real-time applications. The successful application of GAN-BodyPose promises greater efficiency and precision in fields ranging from game development to medical diagnostics, and robust support for human-computer interaction and gesture recognition. This research represents a substantial contribution to deep learning applications in robot control, decision-making, and broadens the research foundation in these domains.

AI-Driven solid-state device to enable natural control of upper-extremity robotic exoskeletons

The paper introduces the ‘Smart Handle’, a compact, lightweight, fully solid-state control device based on machine learning (ML) models, enabling intuitive manipulation across all three degrees-of-freedom (DOF) of translation and orientation. This handle-type control device utilizes low-cost, easily accessible sensors and materials. It addresses the need for natural control mechanisms in upper-extremity exoskeletons used in rehabilitation and occupational settings, offering position and orientation trajectories crucial for kinematic models. Additionally, individuals with motor difficulties using mobility chairs could benefit from its ability to learn and interpret diverse inputs as control commands. Through empirical study, it was demonstrated that the device adeptly learns and outputs continuous translation and rotation information from training sets focused on single DOF motions. Personalized calibration enables consistent movement classification with over 95% accuracy. Successful proof-of-concept experiments were conducted in exoskeleton and wheeled robot control applications, showcasing the Smart Handle's adaptability and reliability in diverse settings. This research underscores the device's potential across multiple domains, providing a foundation for enhanced intuitive control mechanisms in various technological applications.

Multi-Axis Force/Torque Sensor Technologies: Design Principles and Robotic Force Control Applications: A Review

Multi-Axis Force/Torque Sensor Technologies: Design Principles and Robotic Force Control Applications: A Review

On-Manifold Model Predictive Control for Trajectory Tracking on Robotic Systems

Model predictive control (MPC) has the ability to incorporate state and input constraints and extendability to systems with multiple inputs and outputs, making it a popular technique for robot control and other real-world applications. However, robotic systems usually evolve on manifolds, which are often over-parameterized or minimally parameterized (but with singularity) in a MPC. How to naturally incorporate the system manifold constraints without over-parameterization or singularity is a fundamental problem when deploying MPC on robotic systems. In this paper, we propose a unified on-manifold MPC framework to address the mentioned issue. This framework first formulates the MPC based on a canonical representation of on-manifold systems. Then, the on-manifold MPC formulation is solved by linearizing the system at each point along the trajectory under tracking. There are two main advantages of the proposed scheme. The first is that the linearized system leads to an equivalent error system represented by a set of minimal parameters without any singularities. Secondly, the process of system modeling, error-system derivation, linearization and control has the manifold constraints completely decoupled from the system descriptions, enabling to develop a symbolic MPC framework that naturally encapsulates the manifold constraints. In this symbolic framework, one need only to supply system-specific descriptions without dealing with the manifold constraints. To validate the generality of the proposed framework, we implement it on two different robotic platforms, a quadrotor unmanned aerial vehicle (UAV) evolving on a Lie group, and an unmanned ground vehicle (UGV) moving on a curved surface with non-Lie group structure. To validate the non-singularity and tracking performance of the framework, we test it in tracking aggressive UAV trajectories. Experimental results show that with a single, global on-manifold MPC controller, the quadrotor tracks highly aggressive trajectories with large actuator efforts and attitude variation (360 degrees) in both roll and pitch directions, without controller switching.

AI-Assisted Disease Monitoring Using Stretchable Polymer-Based Sensors.

Flexible sensors have attracted significant attention for medical applications. Herein, an AI-assisted stretchable polymer-based (AISP) sensor has been developed based on the Beer-Lambert law for disease monitoring and telenursing. Benefiting from the use of superior polymer materials, the AISP sensor features a high tensile strain of up to 100%, durability of >10,000 tests, excellent waterproofness, and no effect of temperature (1.6-60.9 °C). Such advantages support the capability that the AISP can be flexibly pasted on the skin surface as a wearable device for real-time monitoring of multiple physiological parameters. An AISP sensor-based swallowing recognition technique has been proposed with a high accuracy of up to 88.89%. Likewise, it has been expanded to a remote nursing assistance system to meet critical patients' physiological requirements and daily care. The hands-free communication experiment and robot control applications have also been successfully conducted based on the constructed system. Such merits demonstrate its potential as a medical toolkit and indicate promise for intelligent healthcare.

Modeling variability in self-adapting robotic systems

Autonomous robots operating in everyday environments, such as hospitals, private houses, and public roads, are context-aware self-adaptive systems, i.e. they exploit knowledge about their resources and the environment to trigger runtime adaptation, so that they exhibit a behavior adequate to the current context. For these systems, context-aware self-adaptation requires to design the robot control application as a dynamically reconfigurable software architecture and to specify the adaptation logic for reconfiguring its variable aspects (e.g. the modules that implement various obstacle detection algorithms or control different distance sensors) according to specific criteria (e.g. enhancing robustness against variable illumination conditions). Despite self-adaptation is an intrinsic capability of autonomous robots, ad-hoc approaches are used in practice to design reconfigurable robot architectures. In order to enhance system maintainability, the control logic and the adaptation logic should be loosely coupled. For this purpose, the adaptation logic should be defined against an explicit representation of software variability in the robot control architecture. In this paper we propose a modeling approach, which consists in explicitly representing robot software variability with the MARTE::ARM-Variability metamodel, which has been designed as an extension of the UML MARTE profile. We evaluate the applicability of the proposed approach by exemplifying the software architecture design of a robot navigation framework and by analyzing the support provided by the ROS infrastructure for runtime reconfiguration of its variable aspects.

Potential field-based dual heuristic programming for path-following and obstacle avoidance of wheeled mobile robots

AbstractPath-following control of wheeled mobile robots has been a crucial research topic in robotic control theory and applications. In path-following control with obstacles, the path-following control and collision avoidance goals might be conflicting, making it challenging to obtain near-optimal solutions for path-following control and obstacle avoidance with low tracking error and input energy consumption. To address this problem, we propose a potential field-based dual heuristic programming (P-DHP) algorithm with an actor–critic (AC) structure for path-following control of mobile robots with obstacle avoidance. In the proposed P-DHP, the path-following control and collision avoidance problems are decoupled into two ones to resolve the control conflict. Firstly, a neural network-based AC is constructed to approximate the near-optimal path-following control policy in a no-obstacle environment. Then, with the trained path-following control policy fixed, a potential field-based control policy structure is constructed by another AC network to generate opposite control forces as the robot moves toward the obstacle, which can guarantee the robot’s control safety and reduce the tracking error and input energy consumption in obstacle avoidance. The simulated and experimental results show that P-DHP can realize near-optimal path-following control with the satisfaction of safety constraints and outperforms state-of-the-art approaches in control performance.

A predefined-time and anti-noise varying-parameter ZNN model for solving time-varying complex Stein equations

A predefined-time and anti-noise varying-parameter zeroing neural network (PTAN-VPZNN) is designed to resolve time-varying complex Stein equations in this paper. Differing from the existing ZNNs, the merits of the proposed PTAN-VPZNN include: 1) a varying parameter that improves ZNN model’s convergence speed, which is more compatible with characteristics of the actual hardware parameter; 2) a noise-tolerant activation function which enables the PTAN-VPZNN model to solve Stein equations under noisy environments. Thence, the PTAN-VPZNN model has better convergence performance and noise immunity ability. Moreover, the predefined-time convergence of the PTAN-VPZNN is presented and the robustness of the PTAN-VPZNN is analyzed under constant noise, through rigorous theoretical derivations. Numerical studies demonstrate that the performance of the PTAN-VPZNN is better than the existing ZNNs including a linear ZNN (LZNN), a nonlinear ZNN (NLZNN), a finite-time convergent ZNN (FTCZNN) and a predefined-time convergent ZNN (PTCZNN), when solving Stein equations with or without noise involved. Finally, the PTAN-VPZNN is applied to a mobile manipulator for completing a path-tracking task, showing its potential application in robot control.

Hand Gesture Recognition Using Densely Connected Deep Residual Network and Channel Attention Module for Mobile Robot Control

Navigating a mobile robot in an indoor or outdoor environment is an interesting research area for human-robot collaboration (HRC). In such a scenario, hand gesture offers some unique abilities for HRC to provide non-verbal communication between the user and robot. This paper proposes a novel real-time hand gesture recognition (HGR) technique for mobile robot control application. A compact convolutional neural network (CNN) based HGR system denoted as densely connected residual channel attention module (DRCAM) is proposed to recognize the vision-based hand gestures effectively. Since fingers are the most vital sign for HGR; an attention mechanism using multi-scale representation is proposed, which emphasizes finger information more effectively. The proposed CNN model employs the cascading structure of residual blocks with a multi-scale channel attention module to learn low to high-level information of hand gestures. In addition, the cascading structures are connected through dense connectivity, which strengthens the feature propagation and facilitates feature reuse. Experiments are conducted on an ingenuously developed dataset and a publicly available American sign language finger-spelling (ASL-FS) benchmarked dataset to evaluate the performance of the proposed technique. The experimental result illustrates that the proposed DRCAM network outperforms the state-of-the-art methods in terms of mean accuracy using the leave-one-subject-out cross-validation test. Finally, the training model is used to develop a software-based user interface system for the control of a mobile robot in a real-time environment.

HOW DOES ARTIFICIAL INTELLIGENCE HELP ASTRONOMY? A REVIEW

Artificial intelligence (AI) is a discipline of computing that focuses mostly on transferring human intelligence and mental processes into machines that can assist humans in many ways. Machine learning (ML) is the approach of choice in AI for creating useful software for computer vision, speech recognition, natural language processing, robot control, and other applications. Some of the most common analyses of large, complicated and multidimensional data sets in astronomy can be performed by using ML methods. It can be used for automating observatory scheduling to increase the effective utilization and scientific return from telescopes. It is also used for image recognition, classification of galaxies and planet recognition. This paper offers an in-depth review of the evolution of artificial intelligence and the use of AI and ML in the field of astronomy, especially for data analysis, image recognition, astronomical scheduling, classification of galaxies and planet recognition. It adds to the existing literature on use of artificial intelligence for astronomical applications and is a useful resource for students and researchers.

An MVMD-CCA Recognition Algorithm in SSVEP-Based BCI and Its Application in Robot Control.

This article proposes a novel recognition algorithm for the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) system. By combining the advantages of multivariate variational mode decomposition (MVMD) and canonical correlation analysis (CCA), an MVMD-CCA algorithm is investigated to improve the detection ability of SSVEP electroencephalogram (EEG) signals. In comparison with the classical filter bank canonical correlation analysis (FBCCA), the nonlinear and non-stationary EEG signals are decomposed into a fixed number of sub-bands by MVMD, which can enhance the effect of SSVEP-related sub-bands. The experimental results show that MVMD-CCA can effectively reduce the influence of noise and EEG artifacts and improve the performance of SSVEP-based BCI. The offline experiments show that the average accuracies of MVMD-CCA in the training dataset and testing dataset are improved by 3.08% and 1.67%, respectively. In the SSVEP-based online robotic manipulator grasping experiment, the recognition accuracies of the four subjects are 92.5%, 93.33%, 90.83%, and 91.67%, respectively.

A Generalized Index for Fault-Tolerant Control in Operational Space Under Free-Swinging Actuation Failure

Actuation failure and fault-tolerant control under the actuation failure scenario have drawn more attention in accordance with the recent increasing demand for reliable robot control applications such for long-term and remote operation. The emergence of control torque loss, i.e., the free-swinging failure, is particularly challenging when the robot performs dynamic operational space tasks due to complexities stemming from redundancies in the kinematic structure as well as a dynamical disturbance in the under-actuated multi-body system. To reinforce robustness and accuracy of task-space control under the failure condition, this letter proposes a performance index, named generalized failure-susceptibility (GFS), which is formulated to render thorough dynamic and kinematic effects caused by the un-actuated joints. The GFS index is then exploited with the hierarchical task controller, where self-motion is controlled to minimize the index in real-time. Several experiments with a seven-degrees-of-freedom torque-controlled robot verify that the proposed control strategy with the GFS index effectively improves fault tolerance against anticipating actuation failure.