Actual Robot Research Articles

Robot visual navigation estimation and target localization based on neural network

Abstract The high computational cost, complex external environment, and limited computing resources of embedded system are some major problems in traditional autonomous robot navigation methods. To overcome these problems, a mobile robot path planning navigation system based on panoramic vision was proposed. This method first describes the structure and functions of the navigation system. It explains how to use the environment to explore and map in order to create a panoramic vision sensor. Finally, it elaborates on the breadth-first search based on regression neural network (RNN) method, the Voronoi skeleton diagram method, the algorithm principle, and how to navigate by the planning path implementation of practical strategies. The simulation results illustrate that the breadth-first search method and the Voronoi skeleton graph method based on panoramic view have a high speed. The accessibility of RNN planning algorithm can effectively solve the difficult problems such as high computing overhead, complex navigation environment, and limited computing resources. In the actual robot navigation experiment, the difference in real-time performance and optimality performance that exists between the two algorithms is reflected in the length and duration of the course taken by the robot. When applied to a variety of site environments, the breadth-first search method requires between 23.2 and 45.3% more time to calculate the planned path than the Voronoi skeleton graph method, despite the fact that the planned path length is between 20.7 and 35.9% shorter using the breadth-first search method. It serves as a guide for choosing the appropriate algorithm to implement in practical applications.

Parameter Adjustment of EROS Humanoid Robot Soccer using a Motion Visualization

Humanoid robot is a robot whose overall appearance is formed based on the human body and can interact with equipment and the environment created by humans. The robot's balance becomes fundamental in carrying out various tasks in designing humanoid robots. To deal with this, the adjustment of the humanoid robot movement is crucial in this work, research related to the virtual visualization of robots. Virtual robot visualization can be done by creating a simulator that contains dynamic parameters, including the physics of the robot. With the simulation containing dynamic parameters, the humanoid robot movement can be tried many times until the robot movement is robust. Applying the URDF (Unified Robot Description Format) model to the Gazebo simulator, which is supported by the ROS (Robot Operating System) framework, can make a simulator with dynamic parameters mimicking a real environment. In order to make a robust robot motion, feedback is needed in position and torque to find out the difference between simulation and reality. On the other hand, simulations can be done without cost or risk and, most importantly, mimic the actual robot soccer environment.

Towards Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and Generative Models

To enable robots to achieve high level objectives, engineers typically write scripts that apply existing specialized skills, such as navigation, object detection and manipulation to achieve these goals. Writing good scripts is challenging since they must intelligently balance the inherent stochasticity of a physical robot's actions and sensors, and the limited information it has. In principle, AI planning can be used to address this challenge and generate good behavior policies automatically. But this requires passing three hurdles. First, the AI must understand each skill's impact on the world. Second, we must bridge the gap between the more abstract level at which we understand what a skill does and the low-level state variables used within its code. Third, much integration effort is required to tie together all components. We describe an approach for integrating robot skills into a working autonomous robot controller that schedules its skills to achieve a specified task and carries four key advantages. 1) Our Generative Skill Documentation Language (GSDL) makes code documentation simpler, compact, and more expressive using ideas from probabilistic programming languages. 2) An expressive abstraction mapping (AM) bridges the gap between low-level robot code and the abstract AI planning model. 3) Any properly documented skill can be used by the controller without any additional programming effort, providing a Plug'n Play experience. 4) A POMDP solver schedules skill execution while properly balancing partial observability, stochastic behavior, and noisy sensing.

A Manta Ray Robot with Soft Material Based Flapping Wing

Recent research on robotic fish mainly focused on the bionic structure design and realizing the movement with smart materials. Although many robotic fish have been proposed, most of these works were oriented toward shallow water environments and are mostly built with purely rigid structures, limiting the mobility and practical usability of robotic fish. Inspired by the stability of the real manta ray, a manta ray robot design is proposed with soft material made flapping wing based on an open-source ROV (Remotely Operated Vehicle). The flapping wing structure with three different materials mimics the wide pectoral fins of real manta rays, which have bones, muscles, and skin. Furthermore, its modular design makes it easy to install and disassemble. The kinematic and hydrodynamic analysis of the manta ray robot are simulated in this paper. The actual manta ray robot is fabricated and several sets of test are performed in the pool. The robot can swim forward continually and stably with a simple rolling and pitching pattern.

Opportunities for Transforming Fundamental Physics Researchinto an Innovative Product

Currently, humanity, our entire civilization needs long–term forecasts of emergencies and systems of individualized control of the actions of people and robots (AI systems) in the event of an emergency in order to radically reduce human and material losses. This was shown by the events of this year – catastrophic in scale fires, floods, earthquakes, apparently associated with global space phenomena, and man–made disasters, including epidemics associated with the pace of economic activity – the concentration of the world's population in megacities, globalization of the economy, population mobility, etc. The article shows that only the introduction of a mass service of individualized subscriber rescue management (IUSA) of an infocommunication network with sensors developed as a result of fundamental physical studies, and its full–fledged industrial operation can be accelerated by creating the Association of Researchers of Applied Physics Problems (ARAPP).

Multi-Objective Multi-Learner Robot Trajectory Prediction Method for IoT Mobile Robot Systems

Robot trajectory prediction is an essential part of building digital twin systems and ensuring the high-performance navigation of IoT mobile robots. In the study, a novel two-stage multi-objective multi-learner model is proposed for robot trajectory prediction. Five machine learning models are adopted as base learners, including autoregressive moving average, multi-layer perceptron, Elman neural network, deep echo state network, and long short-term memory. A non-dominated sorting genetic algorithm III is applied to automatically combine these base learners, generating an accurate and robust ensemble model. The proposed model is tested on several actual robot trajectory datasets and evaluated by several metrics. Moreover, different existing optimization algorithms are also applied to compare with the proposed model. The results demonstrate that the proposed model can achieve satisfactory accuracy and robustness for different datasets. It is suitable for the accurate prediction of robot trajectory.

Visual-Inertial Odometry With Online Calibration of Velocity-Control Based Kinematic Motion Models

Visual-inertial odometry (VIO) is an important technology for autonomous robots with power and payload constraints. In this paper, we propose a novel approach for VIO with stereo cameras which integrates and calibrates the velocity-control based kinematic motion model of wheeled mobile robots online. Including such a motion model can help to improve the accuracy of VIO. Compared to several previous approaches proposed to integrate wheel odometer measurements for this purpose, our method does not require wheel encoders and can be applied when the robot motion can be modeled with velocity-control based kinematic motion model. We use radial basis function (RBF) kernels to compensate for the time delay and deviations between control commands and actual robot motion. The motion model is calibrated online by the VIO system and can be used as a forward model for motion control and planning. We evaluate our approach with data obtained in variously sized indoor environments, demonstrate improvements over a pure VIO method, and evaluate the prediction accuracy of the online calibrated model.

Making humanoid robots teaching assistants by using natural language processing (NLP) cloud-based services

Humanoid robots have a substantial potential to serve as teaching and social assistants. However, the expectations of the children from robots to interact like humans are huge. This study presents a general model for understanding the natural language in human-robot interaction by applying Generative Pre-trained Transformer (GPT) language models as a service in the Internet of Things. Thus, the physical presence of the robot can help in fine-tuning the GPT model by prompts derived from the environmental context and subsequent robot actions for embodiment understanding of the GPT outputs. The model uses web or cloud services for Natural Language Processing (NLP) to produce and play human-like text, question answering or text generation. Verbal questions are processed either via a local speech recognition software or via a Speech-to-Text (STT) cloud service. The converted question into machine-readable code is sent to one of the GPT language models with zero- or few-shot learning prompts. GPT-J model has been tested and deployed either in the web or cloud with options for varying the parameters for controlling the haphazardness of the generated text. The robot produces human-like text by using Text-to-Speech (TTS) cloud services that convert the response into audio format rendered on the robot to be played. Useful requirements how the model to be used in order to be feasible were determined based on the evaluation of the outputs given from the different NLP and GPT-J web or cloud-services. We designed and implemented the model in order to be used by a humanoid NAO-type robot in the speech language therapy practice, however it can be used for other open-source and programmable robots and in different contexts.

Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

A virtual fixture (VF) is a constraint built into software that assists a human operator in moving a remote tool along a preferred path via an augmented guidance force to improve teleoperation performance. However, teleoperation generally applies to unknown or dynamic environments, which are challenging for VF use. Most researchers have assumed that VFs are pre-defined or generated automatically; however, these processes are complicated and unreliable in unknown environments where teleoperation is in high demand. Recently, a few researchers have addressed this issue by introducing a user-interactive method of generating VFs in unknown environments. However, these methods are limited to generating a single type of primitive for a single robot tool. Moreover, the accuracy of the VF generated by these methods depends on the accuracy of the human input. Thus, applications of these methods are limited. To overcome those limitations, this work introduces a novel interactive VF generation method that includes a new method of representing VFs as a composition of components. A feature-based user interface allows the human operator to intuitively specify the VF components. The new VF representation accommodates a variety of robot tools and actions. Using the feature-based interface, the process of VF generation is more intuitive and accurate. In this study, the proposed method is evaluated with human subjects in three teleoperation experiments: peg-in-hole, pipe-sawing, and pipe-welding. The experimental results show that the VFs generated by the proposed approach result in a higher manipulation quality while demonstrating the lowest total workload in all experiments. The peg-in-hole task teleoperation was the safest in terms of failure proportion and exerted force of the robot tool. In the pipe-sawing task, the positioning of the robot tool was the most accurate. In the pipe-welding task, the quality of weld was the best in terms of measured tool-trajectory smoothness and visual weld observation.

Loop Closure Detection Based on Differentiable Manifold

Loop closure detection is an important part of SLAM (simultaneous location and mapping), which can effectively reduce the cumulative error of the system after long period of exploration. The existing loop closure detection methods are mainly to evenly distribute the accumulated error in the robot trajectory, but the motion error of the actual robot is also related to its motion speed and rotation angle, while the corrected motion trajectory of the robot is difficult to match the real trajectory. Based on the analysis of the mechanism of robot motion error, this paper proposes a novel loop closure detection method based on differentiable manifold, which mainly includes real-time pose based on manifold tangent space and smooth motion trajectory model of robot based on differential geometry. Firstly, we introduce the Frenet framework structure and establish the corresponding manifold tangent space theory for the keyframe pose nodes. The real-time problem of robot motion is equivalent to the problem of finding the optimal angle tangent vector. Secondly, the motion speed between keyframes is used to determine the characteristics of the robot motion trajectory. We calculate the curvature and torsion of the curve composed of several nodes based on the manifold tangent space and then combine the curve interpolation and fitting of the keyframe nodes to achieve the approximation of the robot motion trajectory, and the smooth curve of the robot trajectory is obtained. Finally, the experiment verifies that the method in this paper can effectively ensure the continuity and smoothness of the robot’s trajectory, thereby reducing the cumulative error of the system and improving the accuracy of loop closure detection.

A Multimodal Information Fusion Model for Robot Action Recognition with Time Series

The current robotics field, led by a new generation of information technology, is moving into a new stage of human-machine collaborative operation. Unlike traditional robots that need to use isolation rails to maintain a certain safety distance from people, the new generation of human-machine collaboration systems can work side by side with humans without spatial obstruction, giving full play to the expertise of people and machines through an intelligent assignment of operational tasks and improving work patterns to achieve increased efficiency. The robot’s efficient and accurate recognition of human movements has become a key factor in measuring robot performance. Usually, the data for action recognition is video data, and video data is time-series data. Time series describe the response results of a certain system at different times. Therefore, the study of time series can be used to recognize the structural characteristics of the system and reveal its operation law. As a result, this paper proposes a time series-based action recognition model with multimodal information fusion and applies it to a robot to realize friendly human-robot interaction. Multifeatures can characterize data information comprehensively, and in this study, the spatial flow and motion flow features of the dataset are extracted separately, and each feature is input into a bidirectional long and short-term memory network (BiLSTM). A confidence fusion method was used to obtain the final action recognition results. Experiment results on the publicly available datasets NTU-RGB + D and MSR Action 3D show that the method proposed in this paper can improve action recognition accuracy.

Online learning control for path-aware global optimization with nonlinear mobile robots

Consider a robot with nonlinear dynamics that must quickly find a global optimum of an objective function defined over its operating area, e.g., a chemical concentration, physical measurement, quantity of material etc. The function is initially unknown and must be learned online from samples acquired in a single trajectory. Applying classical optimization methods in this scenario would be highly suboptimal, since they would place the next sample arbitrarily far, without taking into account robot motion constraints, and would not revise the path based on new information accumulated along it. To address these limitations, a novel algorithm called Path-Aware Optimistic Optimization (OOPA) is proposed. The decision of which robot action to apply is formulated as an optimal control problem in which the rewards are refinements of the upper bound on the objective, weighted by bound and objective values to focus the search around optima. OOPA is evaluated in extensive simulations where it is compared to path-unaware optimization baselines, and in a real experiment in which a ROBOTIS TurtleBot3 successfully searches for the lowest grayscale location on a 2D surface.

Improved JPS Path Optimization for Mobile Robots Based on Angle-Propagation Theta* Algorithm

The Jump Point Search (JPS) algorithm ignores the possibility of any-angle walking, so the paths found by the JPS algorithm under the discrete grid map still have a gap with the real paths. To address the above problems, this paper improves the path optimization strategy of the JPS algorithm by combining the viewable angle of the Angle-Propagation Theta* (AP Theta*) algorithm, and it proposes the AP-JPS algorithm based on an any-angle pathfinding strategy. First, based on the JPS algorithm, this paper proposes a vision triangle judgment method to optimize the generated path by selecting the successor search point. Secondly, the idea of the node viewable angle in the AP Theta* algorithm is introduced to modify the line of sight (LOS) reachability detection between two nodes. Finally, the paths are optimized using a seventh-order polynomial based on minimum snap, so that the AP-JPS algorithm generates paths that better match the actual robot motion. The feasibility and effectiveness of this method are proved by simulation experiments and comparison with other algorithms. The results show that the path planning algorithm in this paper obtains paths with good smoothness in environments with different obstacle densities and different map sizes. In the algorithm comparison experiments, it can be seen that the AP-JPS algorithm reduces the path by 1.61–4.68% and the total turning angle of the path by 58.71–84.67% compared with the JPS algorithm. The AP-JPS algorithm reduces the computing time by 98.59–99.22% compared with the AP-Theta* algorithm.

Division-merge based inverse kinematics for multi-DOFs humanoid robots in unstructured environments

A Two-step Division-Merge (TDM) based inverse kinematics (IK) mechanism is proposed for robots with multiple degrees of freedom (multi-DOFs) in unstructured environments. For any robot with n-DOFs (n>6), a division point is selected first to divide it into a lower part subsystem from its base to the division point, together with an upper part one from the point to its end-effector. Once a collision-free configuration is found for the upper part subsystem to match with the target, the lower part subsystem is then planned to merge with the upper part subsystem at the division point. Specifically, the division point is selected such that the lower part subsystem has 6-DOFs, while the upper part subsystem has (n-6)-DOFs. Furthermore, a Reverse & Random Rotation (RRR) algorithm is also proposed to solve the IK of the upper part subsystem in unstructured environments, while the IK of the lower part subsystem is solved analytically. The proposed TDM based mechanism is verified via experiments using a lab-customized humanoid apple harvesting robot with 11- and 13-DOFs, respectively. Under the various scenarios with different target and obstacle configurations, errors in position coordinates and orientation angles between the target pose and those of the end-effector obtained with forward kinematics (FK) are calculated. Results show that the errors in position and orientation are less than 0.005 mm and 0.007°, respectively, and the average computational time is less than 2 s for the IK solution of the entire robot. Such results indicate that the proposed mechanism is reliable and capable of reducing the complexity of IK for multi-DOFs humanoid robots. With the proposed algorithm, the apple harvesting success rate of a lab-customized 11-DOFs robot reaches 89.2%, which can be further increased up to 93.3% with error compensation, while the average time starting from the robot action to the end-effector reaching the target pose of the apple is around 12.3 s.

Multilegged Underwater Running With Articulated Legs

Drawing inspiration from the locomotion modalities of animals, legged robots demonstrated the potential to traverse irregular and unstructured environments. Successful approaches exploited single-leg templates, like the spring-loaded inverted pendulum (SLIP), as a reference for the control of multilegged machines. Nevertheless, the anchoring between the low-order model and the actual multilegged structure is still an open challenge. This article proposes a novel strategy to derive actuation inputs for a multilegged robot by expressing the control requirements in terms of jump height and forward speed (derived from the limit cycle). We found that these requirements could be associated with a specific maximum force, successively split on an arbitrary number of legs and their relative actuation sets. The proposed approach has been validated in multibody simulation and real-world experiments by employing the underwater hexapod robot SILVER2. Results show that locomotion performances of the low-order model are reflected by the simulated and actual robot, showing that the articulated-USLIP (a-USLIP) model can faithfully explain the multilegged behavior under the imposed control inputs once hydrodynamic parameters have been tuned. More importantly, the proposed controller can be translated to the terrestrial case with minimal modifications and extended with additional layers to obtain more complex behaviors.

Toward Active Physical Human–Robot Interaction: Quantifying the Human State During Interactions

Unanticipated physical actions from the robot on humans [active physical human–robot interaction (pHRI)] may be inevitable with the deployment of robots in human-populated environments. However, it is still unclear how humans would perceive such actions and how the robot should execute them in a physically and psychologically safe manner. The objective of this article is to explore the possibility of quantifying the humans’ physical and mental state during an active physical interaction with a robot, by means of a laboratory experiment. We hypothesize that the active robot actions could cause measurable alterations in users’ data, which could be related to their perceptions and personalities. In the experiment, the user plays a visual game using the robot, which has a hidden task that results in active physical actions on the user. We collect data from physical and physiological sensors, and the perceptions and personalities via questionnaires and a semi-structured interview. Statistical analysis and clustering of the data collected from a total of 35 participants showed the relationships between participants’ physical and physiological data and their age, gender, perception, and personalities. Further developments based on these exploratory outcomes can be used to implement an active pHRI controller that can account for both the physical and the mental state of users.

VISUAL ODOMETRY OF A MOBILE PALETTE ROBOT USING GROUND PLANE IMAGE FROM A FISHEYE CAMERA

Abstract. In this paper, we present a method of mobile robot’s visual odometry using the visual feature tracking in the ground plane image generated from the fisheye image. In order to extract the feature information on the ground, we use a fisheye camera that has a larger FOV than a general pinhole camera, so that we can capture more information on the ground plane. However, due to the large distortion, it is difficult to extract the visual features in the fisheye image. The distortion can be eliminated, but various problems arise, such as a decrease in the resolution of an image or losing the wide angle of the fisheye camera. We propose the EUCM-Cubemap projection model to convert the fisheye image into the cubemap image without losing the FOV of the fisheye image. And we create the Ground Plane Image, a virtual image that vertically looks at the ground from a cube map image. So, the ground plane image is generated so that it is captured from a virtual camera perpendicular to the ground. In the ground plane image, the motion vector obtained by feature tracking between previous and current frames is proportional to the actual robot’s motion in the 2D ground plane. Thus, if we know the actual scale of the motion vector, we can estimate the mobile robot’s velocity and steering angle on the virtual wheel generated by the ground plane image. The scale of the vector can be estimated using the position and focal length of the camera. Using these parameters, we estimate the mobile robot’s pose by applying the bicycle kinematic model. Experimental results show that the proposed method can replace other conventional odometry methods for mobile robots. And, in the future, it is expected to be used in a variety of fields such as visual-based control or path planning.

Human but not robotic gaze facilitates action prediction

SummaryDo people ascribe intentions to humanoid robots as they would to humans or non-human-like animated objects? In six experiments, we compared people’s ability to extract non-mentalistic (i.e., where an agent is looking) and mentalistic (i.e., what an agent is looking at; what an agent is going to do) information from gaze and directional cues performed by humans, human-like robots, and a non-human-like object. People were faster to infer the mental content of human agents compared to robotic agents. Furthermore, although the absence of differences in control conditions rules out the use of non-mentalizing strategies, the human-like appearance of non-human agents may engage mentalizing processes to solve the task. Overall, results suggest that human-like robotic actions may be processed differently from humans’ and objects’ behavior. These findings inform our understanding of the relevance of an object’s physical features in triggering mentalizing abilities and its relevance for human–robot interaction.

Dynamic Cloth Manipulation Considering Variable Stiffness and Material Change Using Deep Predictive Model With Parametric Bias.

Dynamic manipulation of flexible objects such as fabric, which is difficult to modelize, is one of the major challenges in robotics. With the development of deep learning, we are beginning to see results in simulations and some actual robots, but there are still many problems that have not yet been tackled. Humans can move their arms at high speed using their flexible bodies skillfully, and even when the material to be manipulated changes, they can manipulate the material after moving it several times and understanding its characteristics. Therefore, in this research, we focus on the following two points: (1) body control using a variable stiffness mechanism for more dynamic manipulation, and (2) response to changes in the material of the manipulated object using parametric bias. By incorporating these two approaches into a deep predictive model, we show through simulation and actual robot experiments that Musashi-W, a musculoskeletal humanoid with a variable stiffness mechanism, can dynamically manipulate cloth while detecting changes in the physical properties of the manipulated object.

Body Calibration: Automatic Inter-Task Mapping between Multi-Legged Robots with Different Embodiments in Transfer Reinforcement Learning

Machine learning algorithms are effective in realizing the programming of robots that behave autonomously for various tasks. For example, reinforcement learning (RL) does not require supervision or data sets; the RL agent explores solutions by itself. However, RL requires a long learning time, particularly for actual robot learning situations. Transfer learning (TL) in RL has been proposed to address this limitation. TL realizes fast adaptation and decreases the problem-solving time by utilizing the knowledge of the policy, value function, and Q-function from RL. Taylor proposed TL using inter-task mapping that defines the correspondence between the state and action between the source and target domains. Inter-task mapping is defined based on human intuition and experience; therefore, the effect of TL may not be obtained. The difference in robot shapes for TL is similar to the cognition in the modification of human body composition, and automatic inter-task mapping can be performed by referring to the body representation that is assumed to be stored in the human brain. In this paper, body calibration is proposed, which refers to the physical expression in the human brain. It realizes automatic inter-task mapping by acquiring data modeled on a body diagram that illustrates human body composition and posture. The proposed method is evaluated in a TL situation from a computer simulation of RL to actual robot control with a multi-legged robot.