Development of an Automatic Lawnmower with Real-Time Computer Vision for Obstacle Avoidance

Abstract

This work develops an automatic lawnmower (Auto-Lawnmower) using computer vision technology for obstacle avoidance. Several critical issues have been overcome in this development work, including the development of a simplified convolutional neural network (CNN) for decision making, and sufficiently large datasets needed to train the Auto-Lawnmower. The following key strategies are adapted to ensure necessary functionality and efficacy with minimum cost, considering possible mass production of Auto-Lawnmowers. First, we use at-time avoidance strategy: meaning the Auto-Lawnmower makes turns when it encounters an obstacle. This is possible because of the fact that a lawn mower usually can move at a very low speed (walking speed of a man). Second, we decided to use minimum necessary sensors, so as to make the system of our Auto-Lawnmower as simple as possible and hence can be very practical. A single monocular video camera is, therefore, used as the single sensor for both: (1) to generate real-time movie images of a large number of situations that the Auto-Lawnmower may encounter, which is labelled and builds up the training datasets; (2) to capture the real-time image when the trained Auto-Lawnmower is in action, to decide on either to avoid the obstacle or move forward while cutting the grass. A Law dataset with labels, called LawNet, has been established for training of CNNs. It collects a total of 168,542 labelled images taken from the perspective of our Auto-Lawnmower for lawns at the university campus. A concise CNN is created for high efficiency and trained with our LawNet to drive the Auto-Lawnmower. A prototype of Auto-Lawnmower is finally designed, built and tested for lawn mowing in real situations. It is found that it works well as designed, with minimum sensors (a single camera), and hence it has a good potential for mass production.