Metric Localisation for the NAO Robot

Abstract

We present a metric localisation approach for the NAO robot based on the methodology of depth estimation using optical flow in a frame-to-frame basis. We propose to convert optical flow into a 2-channel image from which images patches of \(60\times 60\) are extracted. Each patch is passed as input to a Convolutional Neural Network (CNN) with a regressor in the last layer, thus a depth value is estimated for such patch. A depth image is formed by putting together all the depth estimates obtained for each patch. The depth image is coupled with the RGB image and then passed to the well known ORB-SLAM system in its RGB-D version, this is, a visual simultaneous localisation and mapping approach that uses RGB and depth images to build a 3D map of the scene and use it to localise the camera. When using the depth images, the estimates are recovered with metric. Hence, the NAO’s position can be estimated in metres. Our approach aims at exploiting the walking motion of the robot, which produces image displacements in consecutive frames, and by taking advantage from the fact that the NAO’s walking motion could be programmed to be performed at constant speed. We mount a depth camera on the NAO’s head to produce a training dataset that associates patch RGB images with depth values. Then, a CNN can be trained to learn the patterns in between optical flow vectors and the scene depth. For evaluation, we use one of the in-built NAO’s camera. Our experiments show that this approach is feasible and could be exploited in applications where the NAO requires a localisation systems without depending on additional sensors or external localisation systems.KeywordsDepth estimationDeep learningCNNSLAMOptical flowNAO robot