The more the merrier: running multiple neuromorphic components on-chip for robotic control

This paper investigates spiking neural networks (SNN) for novel robotic controllers with the aim of improving accuracy in trajectory tracking. By emulating the operation of the human brain through the incorporation of temporal coding mechanisms, SNN offer greater adaptability and efficiency in information processing, providing significant advantages in the representation of temporal information in robotic arm control compared to conventional neural networks. Exploring specific implementations of SNN in robot control, this study analyzes neuron models and learning mechanisms inherent to SNN. Based on the principles of the Neural Engineering Framework (NEF), a novel spiking PID controller is designed and simulated for a 3-DoF robotic arm using Nengo and MATLAB R2022b. The controller demonstrated good accuracy and efficiency in following designated trajectories, showing minimal deviations, overshoots, or oscillations. A thorough quantitative assessment, utilizing performance metrics like root mean square error (RMSE) and the integral of the absolute value of the time-weighted error (ITAE), provides additional validation for the efficacy of the SNN-based controller. Competitive performance was observed, surpassing a fuzzy controller by 5% in terms of the ITAE index and a conventional PID controller by 6% in the ITAE index and 30% in RMSE performance. This work highlights the utility of NEF and SNN in developing effective robotic controllers, laying the groundwork for future research focused on SNN adaptability in dynamic environments and advanced robotic applications.

In response to the frequent safety accidents of industrial robots, this paper designs and implements a safety detection system for robot control. It can perform real-time security detection of robot operations on industrial production lines to improve the security and reliability of robot control systems. This paper designs and implements a robot control system based Snort-BASE for real-time online detection of DoS attacks. The system uses a six-degree-of-freedom robotic arm as an example, uses Snort to record the network communication data of the robot arm control system in real time, and filters the network traffic through self-defined rules, and then uses the BASE analysis platform to achieve security analysis of the network traffic. The solution verifies the effectiveness of online real-time detection of attacks and visualisation of attack records by designing simulated robotic arm and real robotic arm attack experiments respectively, thus achieving the security of network communication of the robot remote control system.

To meet the needs of small and medium-sized processing plants by establishing a software and hardware platform for a robotic arm control system with a hierarchical structure and designing a robot robotic arm motion control system that is adaptable to multiple environments and cost-effective. In this paper, the industrial robot robotic arm motion control system is designed based on the consistent fusion tracking algorithm under the decentralized Internet. While using the VisualC++ development environment for writing human-machine interface programs using the MFC framework, the device controls the robotic arm body to complete predetermined movements or operational tasks based on command information, sensing information, and the robot controller. The AMS1086CD-3.3 low-voltage differential linear regulator chip used in this design effectively implements the circuit design from 4.8V to 3.4V and successfully meets the 3.4V voltage supply required by the controller. The industrial robot robotic arm control system adopts an adaptive backstepping algorithm controller, and the error of the tracking system at 0 seconds in the simulation results is always kept within the range of [-1~1] that meets the requirements, which effectively reduces the control error of the robotic arm position. Therefore, the robotic arm control studied in this paper is composed of a complete control system in hardware and software, and the designed adaptive backstepping algorithm controller achieves the demand for good control performance of the robotic arm proper.

Low latency and low energy consumption are the indispensable characteristics of Edge Computing applications. With the fusion of Edge Computing and Artificial Intelligence (AI) into Edge Intelligence, this need is more than ever. Of late, Spiking Neural Networks have shown a promise for low latency and low power AI when deployed on a neuromorphic hardware e.g., Intel's Loihi. In this paper, we present a Spiking Reservoir Computing model, based on the Legendre Memory Units which processes temporal data on Loihi hardware. Such a model is greatly suitable for the battery-powered AI enabled edge devices which call for a prompt processing of the temporal sensor-signals with high energy efficiency. We experiment our model with the ECG5000 dataset on the Loihi boards to show its efficacy.

Humanoid robots have garnered significant attention for their adaptability to dynamic tasks. Despite advancements in vision-based sensing and deep learning, tactile sensing -- crucial for safe human-robot interaction and dexterous manipulation -- remains underutilized. This thesis addresses the challenges of integrating touch, a fundamental human sensory modality, into robotic systems, focusing on bio-inspired approaches and neuromorphic computation.<br/><br/>The human tactile system, characterized by mechanoreceptors, neural pathways, and proprioceptive integration, provides a blueprint for designing tactile sensors. Drawing on these principles, recent developments in tactile sensing technologies are reviewed, from low-resolution designs to high-resolution sensors enabled by advanced manufacturing. This work highlights the advantages of event-driven processing for efficient tactile encoding, which mimics biological systems by transmitting data only upon input changes. Using these methods, a novel tactile Braille dataset was created, featuring the iCub fingertip sliding over 3D-printed Braille letters. Encoding strategies inspired by Fast Adaptive (FA) mechanoreceptors reduced data by up to 122 times. Neuromorphic hardware (Intel Loihi) demonstrated significant power savings during letter classification, achieving efficiency gains over traditional GPU-based methods, though at a cost to accuracy.<br/><br/>To further advance encoding methods, the WiN-GUI was developed, allowing researchers to visualize and optimize spike-pattern encoding in real-time. This tool integrates temporal, spatial, and spatio-temporal datasets to evaluate neuron behavior and classification performance, with applications extending beyond tactile sensing. Complementing this, human haptic object exploration was analyzed to uncover kinematic synergies and Exploratory Procedures (EPs). Using a multimodal dataset of blindfolded participants solving tactile discrimination tasks, this study identified recurring movement patterns and dimensionality reduction strategies that could inform robotic hand control.<br/><br/>By bridging insights from biology, neuromorphic engineering, and robotics, this work offers scalable solutions for touch-sensitive robots designed for dynamic and human-centric environments, paving the way for safer and more capable systems.

With the development of robotics, robotic arms are becoming more and more widely used in industrial production, service industries and daily life. However, the traditional robotic arm control method has limitations in the face of complex environments and tasks, and how to achieve efficient control of the robotic arm has become one of the current research hotspots. Based on deep reinforcement learning, this study proposes a robotic arm control framework based on the strategy gradient method. Initially, we designed an end-to-end neural network model to learn the strategy of the robotic arm to select the optimal action in different states. The neural network model utilizes a convolutional neural network to efficiently process the visual input and continuous action space of the robotic arm. Subsequently, we use a deep deterministic strategy gradient algorithm to train the model, and continuously update the neural network parameters by interacting with the environment, so that the robotic arm can gradually improve its performance when performing tasks. Through experimental validation in a simulated environment and on a real robotic arm platform, our method has achieved significant performance improvements across multiple tasks.

Industry 4.0 and digital factories leverage key technologies such as virtual reality (VR) to design, simulate, optimize, and interact with physical production systems remotely or collaboratively. Recent advancements in industrial and aerospace applications demonstrate that the integration of VR with versatile robotic control techniques can unlock new opportunities. VR offers an "extended arm" to physical environments by providing a user-friendly human-machine interface, enhancing sustainability, testability, and more in robotic control systems. This research focuses on a widely used and continuously evolving robotic arm, designed to function in an integrated system operating in both real and virtual environments. A 3-degree-of-freedom (3DOF) robotic arm was initially developed and later upgraded to a 5DOF system. The robotic arm integrated with Virtual Reality (VR) interfaces based on the Robot Operating System (ROS) and Unity3D platforms. In the real-world scenario, inverse kinematics applied for the robot's movement mechanism, while gradient descent algorithms used for the virtual model. Forward kinematics using homogeneous transformation matrices employed to verify the accuracy of the robotic arm's movements. VR headsets and controllers used to manage the virtual robotic arm's control. The goal of this study is to develop integrated systems that enable control of real-world operations via remote observation, addressing challenges like prototyping costs and safety risks. By offering an innovative approach to robotic arm control through VR integration, this work contributes to the advancement of Industry 4.0 technologies. The findings suggest potential applications across various industries, opening up new possibilities for remote operation and training in complex manufacturing environments. DOI: 10.61416/ceai.v28i1.9592

Self-body awareness refers to the recognition of one's body as one's own and consists of two senses: “sense of body ownership” and “sense of agency.” In telexistence/telepresence robot operation, time delays in the robot's motion degrade self-body awareness of the robot body. We investigated how self-body recognition can be affected in a telexistence robot operation in a VR space when the robot is presented with a real robot arm that simulates a real robot with a delay and a virtual robot arm, or several virtual robot arms, with a delay less than that of the real robot. These experimental conditions include a ‘Predictive Display,’ which is well known as a time delay countermeasure. The results suggest that virtual robot arms presented consecutively with less delay than a real robot arm do not induce a sense of body ownership to the real robot arm, but they enhance the sense of agency to the real robot arm, and that sense of agency is stronger when the task requires precision.

ABSTRACTThe rise of digital technology and Artificial Intelligence (AI) has led to the increased use of smart robots in various sectors. However, security and trust are significant concerns about deploying robots in critical infrastructures. Therefore, a secure and reliable robotic command control system is essential for successful robot integration. None of the prevailing systems focused on attack prediction during cloud‐based robot control and data processing. Hence, this paper proposes a secure model called RCA‐assisted attack detection and robotic command verification using LADA‐C‐RNN and S‐Fuzzy. The robot controller is initially registered using the user ID and password in the cloud application. During login, the SCTDA is used to verify the robot controller's authority. Then, the robot controller's task is subjected to the attack detection phase. In the attack detection phase, the dataset is initially gathered and preprocessed. Thereafter, the temporal pattern analysis is done, followed by feature extraction. Subsequently, the optimal features are selected via GMJFOA. Then, the selected features are inputted to the LADA‐C‐RNN, which performs attack detection. Next, the normal data is fed into the traffic prioritization. Then, the prioritized tasks are inputted to the robot command data verification, thus increasing the security level. Finally, the proposed approach had minimum latency with 98.42% accuracy.

A mobile robot was developed to directly apply chemical to individual weed, plant specific direct chemical application. The robot has a mobile platform equipped with a laptop, a robotic arm with the direct application end effector and a machine vision. The laptop computer and the arm were connected to the robots microcontroller via a serial interface, and the machine vision was controlled by the computer via IEEE 1394 interface: the machine vision detects weeds and the arm with the end effector eliminates weeds. A C++ program was written to control the robot and its components: the program included an image processing algorithm for the machine vision to detect weeds under variable outdoor illumination, and a robotic arm motion control algorithm to eliminate detected weeds. Detected weeds were eliminated by the end effector via cutting a stem and sweeping chemical on its surface. As a result, the developed robot showed the fully automated processes of plant specific direct chemical application: detecting and eliminating weeds, examining the application results, and executing the re-application. While the robot was testing in the field, the robot was able to detect the plants despite outdoor illumination changes. The robot controlled 63.6 % of the weeds in the field. Developed robot may enlighten the potential to reduce the chemical use, to improve chemical efficiency and to reduce environmental contamination from agrochemical.

An adaptive control scheme is proposed for the end-effector trajectory tracking control of free-floating space robots. In order to cope with the nonlinear parameterization problem of the dynamic model of the free-floating space robot system, the system is modeled as an extended robot which is composed of a pseudo-arm representing the base motions and a real robot arm. An on-line estimation of the unknown parameters along with a computed-torque controller is used to track the desired trajectory. The proposed control scheme does not require measurement of the accelerations of the base and the real robot arm. A two-link planar space robot system is simulated to illustrate the validity and effectiveness of the proposed control scheme.

: This paper presents an approach to robot arm control based on exploiting the dynamical properties of an adaptive oscillator circuit coupled to the joints of an arm. The approach is implemented on a real robot arm, and swings pendulums at their natural frequencies, turns cranks and manipulates slinky toys. These actions are all achieved using the same architecture, without any modeling of the arm or its environment. The simple nature of the oscillators, and the lack of modeling results in a robust and very simple system.

The use of robot arms in various industrial settings has changed the way tasks are completed. However, safety concerns for both humans and robots in these collaborative environments remain a critical challenge. Traditional approaches to visualising safety zones, including physical barriers and warning signs, may not always be effective in dynamic environments or where multiple robots and humans are working simultaneously. Mixed reality technologies offer dynamic and intuitive visualisations of safety zones in real time, with the potential to overcome these limitations. In this study, we compare the effectiveness of safety zone visualisations in virtual and real robot arm environments using the Microsoft HoloLens 2. We tested our system with a collaborative pick-and-place application that mimics a real manufacturing scenario in an industrial robot cell. We investigated the impact of safety zone shape, size, and appearance in this application. Visualisations that used virtual cage bars were found to be the most preferred safety zone configuration for a real robot arm. However, the results for this aspect were mixed for a virtual robot arm experiment. These results raise the question of whether or not safety visualisations can initially be tested in a virtual scenario and the results transferred to a real robot arm scenario, which has implications for the testing of trust and safety in human–robot collaboration environments.

Spiking Neural Networks (SNNs) is a promising paradigm for efficient event-driven processing of spatio-temporally sparse data streams. Spiking Neural Networks (SNNs) have inspired the design of and can take advantage of the emerging class of neuromorphic processors like Intel Loihi. These novel hardware architectures expose a variety of constraints that affect firmware, compiler, and algorithm development alike. To enable rapid and flexible development of SNN algorithms on Loihi, we developed NxTF: a programming interface derived from Keras and compiler optimized for mapping deep convolutional SNNs to the multi-core Intel Loihi architecture. We evaluate NxTF on Deep Neural Networks (DNNs) trained directly on spikes as well as models converted from traditional DNNs, processing both sparse event-based and dense frame-based datasets. Further, we assess the effectiveness of the compiler to distribute models across a large number of cores and to compress models by exploiting Loihi’s weight-sharing features. Finally, we evaluate model accuracy, energy, and time-to-solution compared to other architectures. The compiler achieves near-optimal resource utilization of 80% across 16 Loihi chips for a 28-layer, 4M parameter MobileNet model with input size 128×128. In addition, we report the lowest error rate of 8.52% for the CIFAR-10 dataset on neuromorphic hardware, using an off-the-shelf MobileNet.

In recent, control approaches for the human-like behavior in the field of service robotics have been attracting considerable attention, since most humans or animals perform various tasks uncomplicatedly. Hence simple control methods based on gaining a physical insight into human reaching movement in redundancy of DOFs have been proposed. In comparison with the conventional approaches, the proposed method tries to control directly robotic systems in task-space with the control signal composed of linear superposition of three terms 1) joint-damping, 2) virtual spring, and 3) virtual damper in task-space. In particular, our work contains a muscle tension effect of a human under the gravity. This give birth to energy efficient natural motions avoiding problems on repeatability of the motion and ill-posedness problems emerged in most of redundant DOF systems. Thus, this paper exhibits expendability of the position control into the orientation control and compliant behavior. It is verified with a real robotic arm that satisfies human-like movements and motion repeatability under kinematic redundancy of joints.