Tsukuba Challenge Research Articles

Person Searching Through an Omnidirectional Camera Using CNN in the Tsukuba Challenge

The Tsukuba Challenge is a competition, in which autonomous mobile robots run on a route set on a public road under a real environment. Their task includes not only simple running but also finding multiple specific persons at the same time. This study proposes a method that would realize person searching. While many person-searching algorithms use a laser sensor and a camera in combination, our method only uses an omnidirectional camera. The search target is detected using a convolutional neural network (CNN) that performs a classification of the search target. Training a CNN requires a great amount of data for which pseudo images created by composition are used. Our method is implemented in an autonomous mobile robot, and its performance has been verified in the Tsukuba Challenge 2017.

End-to-End Autonomous Mobile Robot Navigation with Model-Based System Support

Autonomous mobile robot navigation in real unmodified outdoor areas frequented by people on their business, children playing, fast running bicycles, and even robots, remains a difficult challenge. For eleven years, the Tsukuba Challenge Real World Robot Challenge (RWRC) has brought together robots, researchers, companies, government, and ordinary citizens, under the same outdoor space to push forward the limits of autonomous mobile robots. For the Tsukuba Challenge 2017 participation, our team proposed to study the problem of sensors-to-actuators navigation (also called End-to-End), this is, having the robot to navigate towards the destination on a complex path, not only moving straight but also turning at intersections. End-to-End (E2E) navigation was implemented using a convolutional neural network (CNN): the robot learns how to go straight, turn left, and turn right, using camera images and trajectory data. E2E network training and evaluation was performed at Nagoya University, on similar outdoor conditions to that of Tsukuba Challenge 2017 (TC2017). Even thought E2E was trained on a different environment and conditions, the robot successfully followed the designated trajectory in the TC2017 course. Learning how to follow the road no matter the environment is of the key attributes of E2E based navigation. Our E2E does not perform obstacle avoidance and can be affected by illumination and seasonal changes. Therefore, to improve safety and add fault tolerance measures, we developed an E2E navigation approach with model-based system as backup. The model-based system is based on our open source autonomous vehicle software adapted to use on a mobile robot. In this work we describe our approach, implementation, experiences and main contributions.

Developing a Remotely Operated Portable Mobile Robot

The Tsukuba Challenge is one of the few technical events where it is permitted to conduct demonstrations of autonomous mobile robots on a public road in a city. Thus, when limited trial runs are available several times per year, domestic and foreign teams must bring robots to the venue and conduct experiments using robots and observation instruments. For a team located far away from the venue, the cost of transporting equipment and staff each time becomes prohibitive. In this research, to reduce the cost of transporting robots and personnel, we design and develop a mobile robot platform that is easy to transport and deploy. Additionally, to allow remote experts to generate maps and routes without visiting the site, we construct a remote operating system that operates over the internet. In this paper, we describe the portable mobile robot platform and the software configuration needed for remote operation. Then, we report the results of our verification test.

Teaching-Playback Navigation Without a Consistent Map

This paper presents a teaching-playback navigation method that does not require a consistent map built using simultaneous localization and mapping (SLAM). Many open source projects related to autonomous navigation including SLAM have been made available recently; however, autonomous mobile robot navigation in large-scale environments is still difficult because it is difficult to build a consistent map. The navigation method presented in this paper uses several partial maps to represent an environment map. In other words, the complex mapping process is not necessary to begin autonomous navigation. In addition, the trajectory that the robot travels in the mapping phase can be directly used as a target path. As a result, teaching-playback autonomous navigation can be achieved without any off-line processes. We tested the navigation method using log data taken in the environment of the Tsukuba Challenge and the testing results show its performance. We provide source code for the navigation method, which includes modules required for autonomous navigation (https://github.com/NaokiAkai/AutoNavi).

Student Education Utilizing the Development of Autonomous Mobile Robot for Robot Competition

The purpose of our education approach is the giving the knowledge and the experience for the research activity to the freshman student who assigned the Lab. We think that “the interaction between the robot and the real world information which is observed by sensor” is important for the student education for student’s research in the Lab. Here, the interaction means how perform the robot based on the real world information which is observed by sensors. The development of the mobile robot which work in Robot competition (Tsukuba Challenge) requires “the interaction between the mobile robot and real world.” For this reason, we think that this activity (participating in the Tsukuba Challenge anf development of the mobile robot) is effective for the student education, and we educate the student by utilizing the development of the mobile robot for Tsukuba Challenge. Although our approach performed in short span, we achieve the good result. For example, the students achieves the research outcome by utilizing the learned skill, and submitted the paper to the academic conference. We verified that our education approach is effective for improvement the student education and motivation.

Special Issue on Real World Robot Challenge in Tsukuba – Autonomous Technology for Coexistence with Human Beings –

In this fourth of the “Special Issues on Real World Robot Challenge in Tsukuba,” we feature the control technology of autonomous robots. There is no guarantee that it will operate perfectly in a real-world environment even with the method already revealed. Participating robots in Tsukuba Challenge are required to carry out the assigned tasks under the prevailing weather conditions on the day of the events. Robots avoid oncoming pedestrians and obstacles in their path. In order to share the novel technology of the autonomous control method, this special issue presents a summary of the results of robots that participated in past Tsukuba Challenges. It is only thanks to the ongoing efforts of the organizers of Tsukuba Challenge and the enthusiasm on the part of the participants that we are able to present an issue such as this, and we are truly thankful to them. We also wish to thank the authors who submitted papers and articles for this issue, as well as our reviewers.

Generated Trajectory of Extended Lateral Guided Sensor Steering Mechanism for Steered Autonomous Vehicles in Real World Environments

This paper discusses the generated trajectory of an extended lateral guided sensor steering mechanism (SSM) method for a steered autonomous vehicle moving in a real world environment. In a previous study, an extended SSM was applied to the Smart Dump 9 and AR Chair robots for following preset waypoints on a map. These studies showed only the schematic idea of the method; the precise performance of the generated trajectory was not shown. This paper compares the Smart Dump 9 robot with a newly developed AR Skipper robot; these robots participated in the Tsukuba Challenge in 2015 and 2016, respectively. Finally, experimental data from the Tsukuba Challenge 2016 demonstrates the advantages of the extended SSM and developed control system.

Autonomous Mobile Robot Searching for Persons with Specific Clothing on Urban Walkway

In order for a robot to provide service in a real world environment, it has to navigate safely and recognize the surroundings. We have participated in Tsukuba Challenge to develop a robot with robust navigation and accurate object recognition capabilities. To achieve navigation, we have introduced the ROS packages, and the robot was able to navigate without major collisions throughout the challenge. For object recognition, we used both a laser scanner and camera to recognize a person in specific clothing, in real time and with high accuracy. In this paper, we evaluate the accuracy of recognition and discuss how it can be improved.

Robust and Accurate Monocular Vision-Based Localization in Outdoor Environments of Real-World Robot Challenge

This paper describes our approach to perform robust monocular camera metric localization in the dynamic environments of Tsukuba Challenge 2016. We address two issues related to vision-based navigation. First, we improved the coverage by building a custom vocabulary out of the scene and improving upon place recognition routine which is key for global localization. Second, we established possibility of lifelong localization by using previous year’s map. Experimental results show that localization coverage was higher than 90% for six different data sets taken in different years, while localization average errors were under 0.2 m. Finally, the average of coverage for data sets tested with maps taken in different years was of 75%.

ROS プラットフォームにおける移動ロボットの自律走行のための 動作獲得学習および走行シミュレーション

This paper describes a system for learning and autonomous navigation of mobile robots on ROS (Robot Operating System) platform. Applying learning of autonomous navigation in the ROS platform will increase availability for autonomous navigation in actual environments, because the ROS platform is often utilized for navigation system in actual environments such as Tsukuba Challenge. Especially, in this study, a learning system based on Deep Q-Networks, that is effective for learning tasks in high-dimensional state spaces, is introduces. At first, simple maze problems as simplified robot navigation environment are solved. Then, the maze problems are applied to simulated robot navigation tasks on ROS platform. Learning results show that the robot obtained a policy to reach any destinations in the maps of the ROS simulator.

Autonomous Mobile Robot Navigation Using Scene Matching with Local Features

[abstFig src='/00280006/13.jpg' width='300' text='Autonomous mobile robot' ] The validity of navigation using an autonomous traveling algorithm based on loose localization (the lax localization algorithm) for an autonomous mobile robot with view sequences [1] was studied, and autonomous travel of the robot in a general town environment with people walking was experimentally demonstrated. The autonomous mobile robot was assumed to be a general cart-type robot with steering wheels and fixed wheels, which was able to make a translational motion in the traveling direction and a rotational motion around the midpoint between the fixed wheels ( Fig. 1). The lax localization algorithm estimates only the distance in the traveling direction and the orientation of the cart, to make the robot follow the teaching path. The algorithm does not estimate the position in the transverse direction normal to the traveling direction. The cart orientation is estimated based on the direction in which a reference image can be viewed, and the position relative to the reference image in the robot traveling direction is estimated based the change in matching score. An autonomous travel experiment was conducted on the course of Tsukuba Challenge 2015 [a], and based on the estimation results, the robot could travel successfully in almost all the location of the course.

Special Issue on Real World Robot Challenge in Tsukuba – Autonomous Technology for Coexistence with Human Beings –

This issue is the third in a series on Real World Robot Challenge in Tsukuba. The Tsukuba Challenge has contributed much in establishing autonomous mobile robot control technology on outdoor walkways where robots must mingle with pedestrians and cyclists – not all of whom may be familiar with such robots. The rain on the day of the final run for the 2015 Tsukuba Challenge taught us valuable lessons in navigating robots in real environments. Since the 2013 Tsukuba Challenge, a new task was introduced in the second stage. This task consists of searching for human targets – a technological challenge for developing robots that are both mobile and useful. Our objective here is to share advanced control technology refined through experiments in the real-world environment of the Tsukuba Challenge. The Tsukuba Challenge is also providing a forum for technological education for university students studying robotics engineering and for technical exchange through open experiments. In this issue, we are pleased to present the control technology that this exchange has brought about. We would like to express our deep gratitude to the authors contributing to this issue and to the article reviewers who have helped make this all possible.

Human Detection by Fourier Descriptors and Fuzzy Color Histograms with Fuzzyc-Means Method

[abstFig src='/00280004/07.jpg' width='300' text='Result of specific person detection in Tsukuba Challenge' ] It is difficult to use histograms of oriented gradients (HOG) or other gradient-based features to detect persons in outdoor environments given that the background or scale undergoes considerable changes. This study involved the segmentation of depth images. Additionally, P-type Fourier descriptors were extracted as shape features from two-dimensional coordinates of a contour in the segmentation domains. With respect to the P-type Fourier descriptors, a person detector was created with the fuzzyc-means method (for general person detection). Furthermore, a fuzzy color histogram was extracted in terms of color features from the RGB values of the domain surface. With respect to the fuzzy color histogram, a detector of a person wearing specific clothes was created with the fuzzyc-means method (specific person detection). The study includes the following characteristics: 1) The general person detection requires less number of images used for learning and is robust against a change in the scale when compared to that in cases in which HOG or other methods are used. 2) The specific person detection gives results close to those obtained by human color vision when compared to the color indices such as RGB or CIEDE. This method was applied for a person search application at the Tsukuba Challenge, and the obtained results confirmed the effectiveness of the proposed method.

Recognition Method Applied to Smart Dump 9 Using Multi-Beam 3D LiDAR for the Tsukuba Challenge

[abstFig src='/00280004/03.jpg' width='300' text='Smart Dump 9 started at the Tsukuba Challenge 2015 final' ] The Tsukuba Challenge course includes a pedestrian road in which walkers, bicyclists, and mobile robots coexist. As a result, mobile robots encounter potentially dangerous situations when faced with moving bicycles. Navigating the challenge course involves locating target individuals in the search area and paying attention to the safety of bicyclists. Target individuals involve those who typically wear a cap and a refracted vest and are seated on chairs. This study proposes a method to identify pedestrians, bicyclists, and seated individuals by using a 3D LiDAR on Smart Dump 9. The SVM method was employed to identify the target seated individuals. An experiment was conducted on the challenge course to illustrate the advantages of the proposed method.

Monocular Vision-Based Localization Using ORB-SLAM with LIDAR-Aided Mapping in Real-World Robot Challenge

[abstFig src='/00280004/06.jpg' width='300' text='Monocular Visual Localization in Tsukuba Challenge 2015. Left: result of localization inside the map created by ORB-SLAM. Right: position tracking at starting point.' ] For the 2015 Tsukuba Challenge, we realized an implementation of vision-based localization based on ORB-SLAM. Our method combined mapping based on ORB-SLAM and Velodyne LIDAR SLAM, and utilized these maps in a localization process using only a monocular camera. We also apply sensor fusion method of odometer and ORB-SLAM from all maps. The combined method delivered better accuracy than the original ORB-SLAM, which suffered from scale ambiguities and map distance distortion. This paper reports on our experience when using ORB-SLAM for visual localization, and describes the difficulties encountered.

Development of Autonomous Robot with Simple Navigation System for Tsukuba Challenge 2015

[abstFig src='/00280004/01.jpg' width='300' text='MercuryMega (SICKLaser-Model)' ] This paper introduces the robot that was developed for Tsukuba Challenge 2015. One of the team’s design goals was to implement a simple sensor configuration and control algorithm. For example, the robot’s laser range finders (LRF) were of fixed type, and localization was accomplished using only a single LRF and no other sensor. Environment map matching was achieved using an algorithm that processed pyramid images using binary image processing to reduce computational cost. Shape information from the LRF and color information from a camera were combined to detect a signboard placed near a person. During an actual test in rainy weather, the robot ran the entire course and, detected two, out of a possible four, number of target persons.

Development of Autonomous Mobile Robot “MML-05” Based on i-Cart Mini for Tsukuba Challenge 2015

[abstFig src='/00280004/04.jpg' width='300' text='Autonomous mobile robots entered in the Tsukuba Challenge 2015' ] This paper describes a self-localization method for autonomous mobile robots entered in the Tsukuba Challenge 2015. One of the important issues in autonomous mobile robots is accurately estimating self-localization. An occupancy grid map, created manually before self-localization has typically been utilized to estimate the self-localization of autonomous mobile robots. However, it is difficult to create an accurate map of complex courses. We created an occupancy grid map combining local grid maps built using a leaser range finder (LRF) and wheel odometry. In addition, the self-localization of a mobile robot was calculated by integrating self-localization estimated by a map and matching it to wheel odometry information. The experimental results in the final run of the Tsukuba Challenge 2015 showed that the mobile robot traveled autonomously until the 600 m point of the course, where the occupancy grid map ended.

Environment Recognition for Autonomous Mobile Robots Using 3D Measurement Technologies

Environment recognition is a fundamental task for service robots to move autonomously and coexist in the same environment with people. A service robot uses data from environment in order to avoid obstacles, estimate its position, detect traversable roads, etc. LiDARs (Light Detection and Ranging) are among the dominant technologies for environment recognition, in particular 2D LiDARs are vastly used. However, it has been argued how for outdoor environments two-dimensional planar data is not sufficient, therefore there is an increasing demand for 3D LiDARs. Recently, small-lightweight 3D LiDARs for mobile robots have been developed. During the “Tsukuba Challenge”, where service robots probe their autonomous functions, it has been demonstrated how 3D LiDARs are effectively used outdoors on real public environments meant for people's living. We discuss the existing problems in 3D LiDARs and, as LiDAR manufacturer, we present the features we seek in prospect lasers to extend performance and capabilities.

Open Experiment of Autonomous Navigation of Mobile Robots in the City: Tsukuba Challenge 2014 and the Results

<div class=""abs_img""> <img src=""[disp_template_path]/JRM/abst-image/00270004/01.jpg"" width=""300"" /> Autonomous mobile robot in RWRC 2014</div> The Tsukuba Challenge, an open experiment for autonomous mobile robotics researchers, lets mobile robots travel in a real – and populated – city environment. Following the challenge in 2013, the mobile robots must navigate autonomously to their destination while, as the task of Tsukuba Challenge 2014, looking for and finding specific persons sitting in the environment. Total 48 teams (54 robots) seeking success in this complex challenge. </span>

Visual Navigation of a Wheeled Mobile Robot Using Front Image in Semi-Structured Environment

<div class=""abs_img""> <img src=""[disp_template_path]/JRM/abst-image/00270004/09.jpg"" width=""300"" /> Mobile robot with a stereo camera</div> Autonomous mobile robots has been an active research recently. In Japan, the Tsukuba Challenge is held annually since 2007 in order to realize autonomous mobile robots that coexist with human beings safely in society. Through technological incentives of such effort, laser range finder (LRF) based navigation has rapidly improved. A technical issue of these techniques is the reduction of the prior information because most of them require precise 3D model of the environment, that is poor in both maintainability and scalability. On the other hand, in spite of intensive studies on vision based navigation using cameras, no robot in the Challenge could achieve full camera navigation. In this paper, an image based control law to follow the road boundary is proposed. This method is a part of the topological navigation to reduce prior information and enhance scalability of the map. As the controller is designed based on the interaction model of the robot motion and image feature in the front image, the method is robust to the camera calibration error. The proposed controller is tested through several simulations and indoor/outdoor experiments to verify its performance and robustness. Finally, our results in Tsukuba Challenge 2014 using the proposed controller is presented. </span>