Real-time fruit detection using deep neural networks on CPU (RTFD): An edge AI application

Abstract

Intelligent harvesting can greatly reduce the manual labor for fruit harvesting and is one of the main application areas of intelligent agricultural machinery. Smart edge sensors must detect the growth status of tomatoes and strawberries in real-time for intelligent harvesting to be possible. Due to edge devices' relatively low image processing power, it remains challenging to implement deep learning algorithms for real-time detection in field applications. Therefore, scholars are strongly interested in utilizing computationally small, accurate, and quick models for edge harvesting machinery. In this research, we propose RTFD, a lightweight algorithm for edge CPU devices that detects fruit. Based on the PicoDet-S model, RTFD optimizes the efficiency of real-time detection for edge CPU computing devices by enhancing the model's structure, loss function and activation function. The experimental results demonstrate that the mAP of RTFD for tomato and strawberry datasets at various growth phases is 0.78 and 0.96, respectively, with a computational effort of 1.44 GFLOPs. RTFD improves mAP by 1.9% and 2.3% on the tomato and strawberry datasets, respectively, with minimal loss in computation and model parameters, making it more suitable for edge device deployment. Meanwhile, we proposed an approach for 8-bit quantization-aware training of the Conv2D and Linear layers of the model for improved edge deployment. This approach quantizes 32-bit floating-point values in the Conv2D and Linear layers into 8-bit integers, thereby reducing the model's size and accelerating detection speed. The testing findings indicate that the quantized RTFD model has a size of 1.33 MB and a CPU detection speed of around 11 FPS. Using the quantified trained model, we developed the Android application RTFD-CPU to evaluate the model's performance in test scenarios. RTFD-CPU was assessed on the smartphone Redmi K30pro (Qualcomm Snapdragon 865 CPU and 8 GB of RAM) with a detection speed of 19 FPS utilizing only the CPU. In summary, the proposed RTFD has great potential for intelligent picking machines, and the concept of redesigning the model structure, loss function, and activation function, as well as training by quantization to speed up the detection of deep neural networks, is anticipated to be successfully implemented in edge computing.