Palm bunch grading technique using a multi-input and multi-label convolutional neural network

Abstract

Oil palm bunches, which are the raw materials for several oil palm products, are traded to a crude palm oil (CPO) factory or a middleman. Since high-oil-content palm bunches are desired to produce CPO, the factories and middlemen traditionally have several experts hired for grading FFBs. We have found that colour, shape and size of FFBs are three parameters that need to be considered for classification. Shape and colour data can be collected using RGB and IR sensors, while size data can be collected using IR and weight sensors. In this paper, we propose a deep learning-based model working with three sensors (RGB camera, infrared sensor, and a load cell) for classifying a palm bunch. The classifying framework of the algorithm is a multi-input and multi-label convolutional neural network. The multi-label classifies the 4 grades of FFBs (overripe, ripe, under ripe and unripe). Based on 1,575 images for 14 varieties of palm bunches collected from four trading sites, the sensors were installed, the algorithm was trained, validated, and tested with 70%, 20% and 10% of the overall datasets, respectively. The general images that are preprocessed after training model are applied and compared between original and preprocess images by Gradient-weighted Class Activation Mapping (Grad-CAM). The cross-entropy loss function was applied to investigate the probability error between the target and predicted label during the training process. Based on the testing datasets results, the proposed model achieves an accuracy of 90.26%, precision of 89.86%, recall of 89.54%, and an F1-score of 89.68%.