Plant growth information measurement based on object detection and image fusion using a smart farm robot

Abstract

Traditionally, vegetable and fruit production has relied on empirical and ambiguous decisions made by human farmers. To overcome this uncertainty in agriculture, smart farm robots have been widely studied in recent years. However, measuring growth information with robots remains a challenge because of the similarity in the appearance of the target plant and those around it. In this study, we propose a smart farm robot that accurately measures the growth information of a target plant based on object detection, image fusion, and data augmentation with fused images. The proposed smart farm robot uses an end-to-end real-time deep learning-based object detector that shows state-of-the-art performances. To distinguish the target plant from other plants with a higher accuracy and improved robustness than those of existing methods, we exploited image fusion using both RGB and depth images. In particular, the data augmentation, based on the fused RGB, and depth information, contributes to the precise measurement of growth information from smart farms, regardless of the high density of vegetables and fruits in these farms. We propose and evaluate a real-time measurement system to obtain precise target-plant growth information in precision agriculture. The code and models are publicly available on Github: https://github.com/kistvision/Plant_growth_measurement.