Tree Trunk Recognition in Orchard Autonomous Operations under Different Light Conditions Using a Thermal Camera and Faster R-CNN.

Ailian Jiang,Ryozo Noguchi,Tofael Ahamed

doi:10.3390/s22052065

Abstract

In an orchard automation process, a current challenge is to recognize natural landmarks and tree trunks to localize intelligent robots. To overcome low-light conditions and global navigation satellite system (GNSS) signal interruptions under a dense canopy, a thermal camera may be used to recognize tree trunks using a deep learning system. Therefore, the objective of this study was to use a thermal camera to detect tree trunks at different times of the day under low-light conditions using deep learning to allow robots to navigate. Thermal images were collected from the dense canopies of two types of orchards (conventional and joint training systems) under high-light (12–2 PM), low-light (5–6 PM), and no-light (7–8 PM) conditions in August and September 2021 (summertime) in Japan. The detection accuracy for a tree trunk was confirmed by the thermal camera, which observed an average error of 0.16 m for 5 m, 0.24 m for 15 m, and 0.3 m for 20 m distances under high-, low-, and no-light conditions, respectively, in different orientations of the thermal camera. Thermal imagery datasets were augmented to train, validate, and test using the Faster R-CNN deep learning model to detect tree trunks. A total of 12,876 images were used to train the model, 2318 images were used to validate the training process, and 1288 images were used to test the model. The mAP of the model was 0.8529 for validation and 0.8378 for the testing process. The average object detection time was 83 ms for images and 90 ms for videos with the thermal camera set at 11 FPS. The model was compared with the YOLO v3 with same number of datasets and training conditions. In the comparisons, Faster R-CNN achieved a higher accuracy than YOLO v3 in tree truck detection using the thermal camera. Therefore, the results showed that Faster R-CNN can be used to recognize objects using thermal images to enable robot navigation in orchards under different lighting conditions.

Highlights

Introduction published maps and institutional affilBetween 1995 and 2010, Japan’s agricultural labor force has gradually declined from4.14 to 2.39 million, and its average age increased from 59.1 years to 65.8 years [1]
The horizontal coordinate represents the real distance of the box, the vertical coordinate represents the distance value measured by the thermal camera, and
The horizontal coordinate represents the real distance of the box, the vertical coordinate represents the distance value measured by the thermal camera, and R2 represents the residual sum of squares, which represents the error between the real distance and the measured distance

Summary

Introduction

Introduction published maps and institutional affilBetween 1995 and 2010, Japan’s agricultural labor force has gradually declined from4.14 to 2.39 million, and its average age increased from 59.1 years to 65.8 years [1]. Between 1995 and 2010, Japan’s agricultural labor force has gradually declined from. 4.14 to 2.39 million, and its average age increased from 59.1 years to 65.8 years [1]. Agricultural robotics have the potential to support agricultural labor shortages and increase agricultural productivity in this critical stage of transformation [2]. In agricultural automation and robotics applications, vehicle navigation is important in outdoor environments, which are complex and uncertain compared to indoor conditions [3]. Satellite System (RTK-GNSS) with higher accuracy [4]. Orchard navigation is the most complex, and interruption of RTK-GNSS signals due to high and dense canopies is frequently reported [5]. As Japanese orchards have net and dense branches, GNSS signals may be affected, and many farmland orchards do not have base stations set up to use GNSS iations

Objectives

Methods

Results

Discussion

Conclusion