Tsukuba Challenge Research Articles

Special Issue on Autonomous Robotics Challenge

The Tsukuba Challenge and the Nakanoshima Robot Challenge are both technical challenges in which mobile robots run autonomously in real outdoor environments. They have been held almost every year since 2007 and 2018, respectively, and many robots have participated in these public experiments. The autonomous navigation of a mobile robot in the real world, that is, not on test tracks but in environments normally used by everyday people, poses a great many challenges. The robots have to deal with environments that change with the time of day, the weather, the season, etc., and they have to deal with unexpected stationary obstacles as well as moving ones, including people, bicycles, etc. These days, demonstration tests of delivery robots are being conducted in many places, but there are still many problems that need to be solved. In this special issue, we have gathered papers detailing the insights gained from running mobile robots in the two outdoor experiments, the Tsukuba Challenge and the Nakanoshima Robot Challenge. To run a robot with a high success rate in a real environment, it is very important to devise the robot configuration, the sensor data processing, and the behavior control based on the knowledge gained from many experiences. We hope that sharing the successes and failures in the papers in this special issue will lead to further technological improvements in the future.

Data Augmentation for Semantic Segmentation Using a Real Image Dataset Captured Around the Tsukuba City Hall

We are exploring the use of semantic scene understanding in autonomous navigation for the Tsukuba Challenge. However, manually creating a comprehensive dataset that covers various outdoor scenes with time and weather variations to ensure high accuracy in semantic segmentation is onerous. Therefore, we propose modifications to the model and backbone of semantic segmentation, along with data augmentation techniques. The data augmentation techniques, including the addition of virtual shadows, histogram matching, and style transformations, aim to improve the representation of variations in shadow presence and color tones. In our evaluation using images from the Tsukuba Challenge course, we achieved the highest accuracy by switching the model to PSPNet and changing the backbone to ResNeXt. Furthermore, the adaptation of shadow and histogram proved effective for critical classes in robot navigation, such as road, sidewalk, and terrain. In particular, the combination of histogram matching and shadow application demonstrated effectiveness for data not included in the base training dataset.

Path Planning Using a Flow of Pedestrian Traffic in an Unknown Environment

This paper outlines a path planning method for autonomous rovers navigating urban environments without prior mapping, with a particular focus on addressing the Tsukuba Challenge. Our approach utilizes observations of pedestrian and robot movement trajectories to construct path graphs for global path planning. We provide a detailed overview of the autonomous rover’s hardware and software system, as well as a comprehensive description of the path planning algorithm. Our methodology entails extracting and continuously tracking dynamic objects from LiDAR data, resulting in the creation of a path graph based on their observed trajectories. Subsequently, a path aligned with the desired direction is selected. Notably, in indoor experimental settings, our approach proves effective, as the rover successfully generates a path to the goal by closely monitoring and tracking pedestrian movements. In conclusion, this paper introduces a promising path planning methodology and suggests potential areas for further research in autonomous mobility within uncharted environments.

Dataset Creation for Semantic Segmentation Using Colored Point Clouds Considering Shadows on Traversable Area

Semantic segmentation, which provides pixel-wise class labels for an input image, is expected to improve the movement performance of autonomous robots significantly. However, it is difficult to train a good classifier for target applications; public large-scale datasets are often unsuitable. Actually, a classifier trained using Cityscapes is not enough accurate for the Tsukuba Challenge. To generate an appropriate dataset for the target environment, we attempt to construct a semi-automatic method using a colored point cloud obtained with a 3D scanner. Although some degree of accuracy is achieved, it is not practical. Hence, we propose a novel method that creates images with shadows by rendering them in the 3D space to improve the classification accuracy of actual images with shadows, for which existing methods do not output appropriate results. Experimental results using datasets captured around the Tsukuba City Hall demonstrate that the proposed method was superior when appropriate constraints were applied for shadow generation; the mIoU was improved from 0.358 to 0.491 when testing images were obtained at different locations.

Experimental Study of Seamless Switch Between GNSS- and LiDAR-Based Self-Localization

A self-localization method that can seamlessly switch positions and attitudes estimated using normal distributions transform (NDT) scan matching and a real-time kinematic global navigation satellite system (GNSS) is successfully developed. One of the issues encountered in this method is the sharing of global coordinates among the different estimation methods. Therefore, the three-dimensional environmental maps utilized in the NDT scan matching are created based on the planar Cartesian coordinate system used in the GNSS to obtain accurate information regarding the location, shape, and size of the actual terrain and geographic features. Consequently, seamlessly switching between different methods enables mobile robots to stably obtain accurate estimated positions and attitudes. An autonomous driving experiment is conducted using this self-localization method in the Tsukuba Challenge 2022, and the mobile robot completed a designated course involving more than 2 km in an urban area.

Practical Implementation of Visual Navigation Based on Semantic Segmentation for Human-Centric Environments

This study focuses on visual navigation methods for autonomous mobile robots based on semantic segmentation results. The challenge is to perform the expected actions without being affected by the presence of pedestrians. Therefore, we implemented a semantics-based localization method that is not affected by dynamic obstacles and a direction change method at intersections that functions even with coarse-grain localization results. The proposed method was evaluated through driving experiments in the Tsukuba Challenge 2022, where a 290 m run including 10 intersections was achieved in the confirmation run section.

Development of Autonomous Moving Robot Using Appropriate Technology for Tsukuba Challenge

We have been participating in the Tsukuba Challenge, an open experiment involving autonomous robots, since 2014. The technology of our robot has stabilized, and our robot has continued to win the Tsukuba Mayor Prize from 2018 to 2021 without changing the basic configuration of the body and navigation software. Here, we report the robot’s structure as the project’s current completed form. Our robot is designed with the policy of selecting the most rational technology (appropriate technology) to achieve the purpose, even if it is not the latest. For example, we used image-like two-dimensional data instead of a three-dimensional point cloud in map matching for robot positioning. For pedestrian signal recognition, which was required to perform an optional task, we did not use deep learning but rather conventional color image processing. These techniques are advantageous for balancing the execution time and accuracy required in the challenge.

Online Activities in Tsukuba Challenge 2020

Online Activities in Tsukuba Challenge 2020

Special Issue on Real World Robot Challenge in Tsukuba and Osaka

The Tsukuba Challenge is an open experiment of autonomous mobile robots in the real world. In its third stage since 2018, it is now to be held on a new course that starts at the Tsukuba City Hall. New tasks that require functions expected for autonomous travel in the real world have now been added, including passing checkpoints announced a day before the event, starting two vehicles simultaneously, traveling in an unmeasured environment, and strictly observing stop lines in the course. Also, in the spirit of the Tsukuba Challenge, the Nakanoshima Challenge, an open demonstration experiment project, has been held in the city of Osaka since 2018. As the only event in which autonomous mobile robots travel in the urban area of Osaka, the Nakanoshima Challenge is expected to identify new issues peculiar to autonomous navigation in real urban environments and to find solutions to them. This special issue includes a review paper on the Tsukuba Challenge, four research papers on the results of experiments done in the Tsukuba Challenge, four research papers related to the Nakanoshima Challenge, and three development reports. This special issue provides its readers with the frontline issues and the current status of development of autonomous mobile robots in real-world environments. We hope that the innovative efforts presented in this special issue will contribute to the development of science and industry.

Tsukuba Challenge 2019: Task Settings and Experimental Results

This paper overviews Tsukuba Challenge 2019. The Tsukuba Challenge is an experiment for autonomous navigation of mobile robots on public walkways. Navigation tasks through pedestrian paths in the city are given. Participating teams develop their own robot hardware and software. We describe the aim of the task settings and the analysis of the experimental results for all the teams. We studied the records of real-world experiments of Tsukuba Challenge 2019.

Proposal of Robot Software Platform with High Sustainability

At present, various robotics competitions are being held, including the Tsukuba Challenge. The purpose of participating in a robotics competition is to confirm what can be done with the current technology and to demonstrate new research results. Participating teams often use open source software (OSS) for path planning and autonomous navigation. OSS is advantageous in facilitating participation in robotics competitions. However, applying it to a new robot is difficult when a new research does not involve OSS. In addition, robot systems do not consist only of OSS, and the burden of developing and maintaining other systems is significant. To solve the above problems, a software platform that allows for the addition of research achievements of individual robots is desired. With such a platform, research elements that have already been developed can be shared without the need to develop a new system. This makes it easier to maintain and manage the system and increase its sustainability.

Visual Navigation Based on Semantic Segmentation Using Only a Monocular Camera as an External Sensor

The most popular external sensor for robots capable of autonomous movement is 3D LiDAR. However, cameras are typically installed on robots that operate in environments where humans live their daily lives to obtain the same information that is presented to humans, even though autonomous movement itself can be performed using only 3D LiDAR. The number of studies on autonomous movement for robots using only visual sensors is relatively small, but this type of approach is effective at reducing the cost of sensing devices per robot. To reduce the number of external sensors required for autonomous movement, this paper proposes a novel visual navigation scheme using only a monocular camera as an external sensor. The key concept of the proposed scheme is to select a target point in an input image toward which a robot can move based on the results of semantic segmentation, where road following and obstacle avoidance are performed simultaneously. Additionally, a novel scheme called virtual LiDAR is proposed based on the results of semantic segmentation to estimate the orientation of a robot relative to the current path in a traversable area. Experiments conducted during the course of the Tsukuba Challenge 2019 demonstrated that a robot can operate in a real environment containing several obstacles, such as humans and other robots, if correct results of semantic segmentation are provided.

Automatic Generation of Multidestination Routes for Autonomous Wheelchairs

To solve the problem of autonomously navigating multiple destinations, which is one of the tasks in the Tsukuba Challenge 2019, this paper proposes a method for automatically generating the optimal travel route based on costs associated with routes. In the proposed method, the route information is generated by playing back the acquired driving data to perform self-localization, and the self-localization log is stored. In addition, the image group of road surfaces is acquired from the driving data. The costs of routes are generated based on texture analysis of the road surface image group and analysis of the self-localization log. The cost-added route information is generated by combining the costs calculated by the two methods, and by assigning the combined costs to the route. The minimum-cost multidestination route is generated by conducting a route search using cost-added route information. Then, we evaluated the proposed method by comparing it with the method of generating the route using only the distance cost. The results confirmed that the proposed method generates travel routes that account for safety when the autonomous wheelchair is being driven.

A Survey of the Tsukuba Challenge 2019 for All Teams

A Survey of the Tsukuba Challenge 2019 for All Teams

Special Issue on Probabilistic Robotics and SLAM

Intelligent mobile robots need self-localization, map generation, and the ability to explore unknown environments autonomously. Probabilistic processing can be applied to overcome the problems of movement uncertainties and measurement errors. Probabilistic robotics and simultaneous localization and mapping (SLAM) technologies are therefore strongly related, and they have been the focus of many studies. As more and more practical applications are found for intelligent mobile robots, such as for autonomous driving and cleaning, the applicability of these techniques has been increasing. In this special issue, we provide a wide variety of very interesting papers ranging from studies and developments in applied SLAM technologies to fundamental theories for SLAM. There are five academic papers, one each on the following topics: first visit navigation, controls for following rescue clues, indoor localization using magnetic field maps, a new solution for self-localization using downhill simplex method, and object detection for long-term map management through image-based learning. In addition, in the next number, there will be a review paper by Tsukuba University’s Prof. Tsubouchi, who is famous for the Tsukuba Challenge and research related to mobile robotics. We editors hope this special issue will help readers to develop mobile robots and use SLAM technologies and probabilistic approaches to produce successful applications.

Tsukuba challenge simulator based on Unreal Engine 4

Tsukuba challenge simulator based on Unreal Engine 4

Tsukuba Challenge: Open Experiments for Autonomous Navigation of Mobile Robots in the City – Activities and Results of the First and Second Stages –

The Tsukuba Challenge is an open experiment for autonomous mobile robotics researchers who want to build small mobile robots capable of autonomously moving through real and populated pedestrian environments. The Tsukuba Challenge started in 2007 and has been run every year since then. Each year, the self-contained mobile robots of participated team are tasked with autonomously navigating more than 1 km of a given pedestrian pathway through the city. As of 2017, the final year of the second stage, a total of over 500 teams have taken a part in this challenge, by trying to develop their own robot hardware and software to complete the given task. In this paper, the basic concept and the history of Tsukuba Challenge are first explained, and then what has and has not achieved is discussed.

Special Issue on Real World Robot Challenge in Tsukuba

Tsukuba Challenge, which started in 2007, has contributed to the development of novel control technologies for autonomous navigation. The second stage of this challenge was completed in 2017, and now, the time is ripe for exploring new ideas and avenues. This is the fifth Special Issue on Real World Robot Challenge in Tsukuba, and it seems that the technological elements required for autonomous navigation are almost complete. However, it is obvious that such navigation capabilities are still at a significantly lower level of development compared to human capabilities. The need for automatic and self-driving vehicles has increased rapidly in recent years; many companies and researchers have been making great strides in research and development in the field of automatic driving. The Tsukuba Challenge pursues and encourages both student education and advanced research and development, focusing on automatic driving as an application technology. The essential capabilities required for a robot to reach the designated goal in the Tsukuba Challenge are self-localization and obstacles avoidance, and many studies have been conducted on these features. To complete the designated task, unification of these technologies and other qualities such as searching for persons, traveling on crosswalks, and recognizing traffic signals is required. This special issue concentrates on control technologies of autonomous mobile robots and expects to contribute toward future studies and development in this field.

Detection of Target Persons Using Deep Learning and Training Data Generation for Tsukuba Challenge

The Tsukuba Challenge is an open experiment competition held annually since 2007, and wherein the autonomous navigation robots developed by the participants must navigate through an urban setting in which pedestrians and cyclists are present. One of the required tasks in the Tsukuba Challenge from 2013 to 2017 was to search for persons wearing designated clothes within the search area. This is a very difficult task since it is necessary to seek out these persons in an environment that includes regular pedestrians, and wherein the lighting changes easily because of weather conditions. Moreover, the recognition system must have a light computational cost because of the limited performance of the computer that is mounted onto the robot. In this study, we focused on a deep learning method of detecting the target persons in captured images. The developed detection system was expected to achieve high detection performance, even when small-sized input images were used for deep learning. Experiments demonstrated that the proposed system achieved better performance than an existing object detection network. However, because a vast amount of training data is necessary for deep learning, a method of generating training data to be used in the detection of target persons is also discussed in this paper.

Tsukuba Challenge 2017 Dynamic Object Tracks Dataset for Pedestrian Behavior Analysis

Navigation in social environments, in the absence of traffic rules, is the difficult task at the core of the annual Tsukuba Challenge. In this context, a better understanding of the soft rules that influence social dynamics is key to improve robot navigation. Prior research attempts to model social behavior through microscopic interactions, but the resulting emergent behavior depends heavily on the initial conditions, in particular the macroscopic setting. As such, data-driven studies of pedestrian behavior in a fixed environment may provide key insight into this macroscopic aspect, but appropriate data is scarcely available. To support this stream of research, we release an open-source dataset of dynamic object trajectories localized in a map of 2017 Tsukuba Challenge environment. A data collection platform equipped with lidar, camera, IMU, and odometry repeatedly navigated the challenge’s course, recording observations of passersby. Using a background map, we localized ourselves in the environment, removed the static background from the point cloud data, clustered the remaining points into dynamic objects and tracked their movements over time. In this work, we present the Tsukuba Challenge Dynamic Object Tracks dataset, which features nearly 10,000 trajectories of pedestrians, cyclists, and other dynamic agents, in particular autonomous robots. We provide a 3D map of the environment used as global frame for all trajectories. For each trajectory, we provide at regular time intervals an estimated position, velocity, heading, and rotational velocity, as well as bounding boxes for the objects and segmented lidar point clouds. As additional contribution, we provide a discussion which focuses on some discernible macroscopic patterns in the data.