Abstract
The reconfigurable robot RSTAR (Rising Sprawl Tuned Autonomous Robot) is a newly developed crawling robot that can reconfigure its shape and shift the location of its center of mass. These features endow the RSTAR with inherent robustness and enhanced ability to overcome obstacles and crawl on a variety of terrains for a vast range of applications. However, defining the trajectories to fully exploit the robot’s capabilities is challenging, especially when complex maneuvers are required. Here, we show how reinforcement learning can be used to determine the optimal strategies to overcome three typical obstacles: squeezing through two adjacent obstacles, ducking underneath an obstacle and climbing over an obstacle. We detail the implementation of the Q learning algorithm in a simulation environment with a physical engine (UNITY™) to learn a feasible path in a minimum number of steps. The results show that the algorithm successfully charted a feasible trajectory in all cases. Comparing the trajectory found by the algorithm to trajectories devised by 12 human experts with discrete or continuous control showed that the algorithm trajectories were shorter than the expert trajectories. Experiments showing how the physical RSTAR robot can overcome different obstacles using the trajectories found in the simulation by the Q Learning algorithm are described and presented in the attached video.
Highlights
Miniature search and rescue ground robots that can be used in disaster areas have numerous advantages
We show how to apply reinforcement learning (RL) to exploit RSTAR’s advanced reconfiguration capabilities to climb over, duck underneath and squeeze through obstacles
KINEMATICS OF THE RSTAR The RSTAR is composed of a main rigid body and a pair of arms holding two sets of wheels and whegs [1]
Summary
Miniature search and rescue ground robots that can be used in disaster areas have numerous advantages. Yehezkel et al.: Overcoming Obstacles With a Reconfigurable Robot Using Reinforcement Learning Such maneuvers offline by running repetitive experiments in a physical or simulated environment. The autonomous learning of such trajectories requires taking complex dynamic considerations into account It must be done in setups that preserve the mechanical an electronic hardware of the robot during the learning process. Since highly advanced and complex mechanics are required Robots that have such capabilities, typically require intricate coordination of their degrees of freedom for performing motion. The inherent stability of the RSTAR and its decoupled degrees of freedom (the mechanisms and wheels can be actuated practically independently) allow for control policies with relatively large tolerances This makes it possible to use the much simpler discrete RL methods. System operation is compared to operation by human operators under a similar setup where the artificial learning mechanism does not have more information than the human operator, such that the results hold in the general case
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
