Deep Learning and Computer Vision for Estimating Date Fruits Type, Maturity Level, and Weight

Mohammed Algabri,Fahad Albogamy,Fattoh Abdu Alqershi,Mohammed Faisal,Hebah Elgibreen

doi:10.1109/access.2020.3037948

Abstract

According to the Food and Agriculture Organization, the world production of date fruits is 8,526,218 tons and around 1,302,859 tons in the Kingdom of Saudi Arabia (KSA) in 2018. There are several types of date fruits, and the most common in KSA are Barhi, Khalas, Meneifi, Naboot Saif, and Sullaj. Moreover, there are around five main maturity levels: Immature, Khalal, Khalal with Rutab, Pre-Tamar, and Tamar. Harvesting date fruits is performed according to its maturity level and type, which is a critical decision that significantly affects profit. In this paper, we propose a smart harvesting decision system to estimate date fruits type, maturity level, and weight using computer vision (CV) and deep learning (DL) techniques. The proposed system consists of three sub-systems: Dates maturity estimation system (DMES), type estimation system (DTES), and dates weight estimation system (DWES). We utilized four DL architectures, including ResNet, VGG-19, Inception-V3, and NASNet for both DMES and DTES and support vector machine (SVM) (regression and linear) for DWES. We evaluated the performance of the proposed system using the dataset collected by the Center of Smart Robotics Research. Using multiple performance metrics, DTES achieved maximum performance of 99.175% accuracy, an F1 score of 99.225%, 99.8% average precision, and 99.05% average recall. The maximum performance of DMES was 99.058% accuracy, F1 score of 99.34%, 99.64% average precision, and 99.08% average Recall. DWES achieved a maximum performance of 84.27% using SVM-Linear.

Highlights

AND BACKGROUNDDates are fruits of the date palm trees, which are famous for its high nutritional value and is a widespread summer fruit in the Arab world
In order to overcome the inability of estimating the date fruit type, maturity level, and weight per palm tree in the pre-harvesting stage, we proposed a computer vision (CV) and deep learning (DL) based smart system to predict date types (Barhi, Khalas, Meneifi, Naboot Saif, and Sullaj), date maturity levels (Immature, Khalal, Khalal with Rutab, PreTamar, and Tamar), and weight of date per palm tree in the pre-harvesting stage
We used ResNet, VGG-19, Inception-v3, and Neural Architecture Search Network (NASNet) models to train, evaluate and test dates type estimation system (DTES) to estimate five different dates types (Barhi, Khalas, Meneifi, Naboot Saif, and Sullaj), Dates maturity estimation system (DMES) to estimate the five maturity stage (Immature, Khalal, Khalal with Rutab, Pre-Tamar, and Tamar), and SVR to estimate the weight of date fruit

Summary

Introduction

AND BACKGROUNDDates are fruits of the date palm trees, which are famous for its high nutritional value and is a widespread summer fruit in the Arab world. Arabs relied on dates in their daily lives. Dates are the major fruit of the Kingdom of Saudi Arabia (KSA). According to the Saudi Arabian Ministry of Agriculture, date palm trees cover about 72% of the total. Based on the reports of the Food and Agriculture Organization (FAO), the global production of dates increased from 3,430,883 tons in 1990 to 8,526,218 tons in 2018 [1]. In KSA, dates production grew from around 527,881 tons in 1990 to 1,302,859 tons in 2018 [1]. The average production share of dates between 1994 and 2018 is 4,154,484 tons in Asia, 2,579,209 tons in Africa, 28,109 tons in the Americas, and 11,034.52 tons in Europe [1] (FIGURE 1)

Objectives

Results

Conclusion