Image Fruit Research Articles

A General Image Super-Resolution Reconstruction Technique for Walnut Object Detection Model

Object detection models are commonly used in yield estimation processes in intelligent walnut production. The accuracy of these models in capturing walnut features largely depends on the quality of the input images. Without changing the existing image acquisition devices, this study proposes a super-resolution reconstruction module for drone-acquired walnut images, named Walnut-SR, to enhance the detailed features of walnut fruits in images, thereby improving the detection accuracy of the object detection model. In Walnut-SR, a deep feature extraction backbone network called MDAARB (multilevel depth adaptive attention residual block) is designed to capture multiscale information through multilevel channel connections. Additionally, Walnut-SR incorporates an RRDB (residual-in-residual dense block) branch, enabling the module to focus on important feature information and reconstruct images with rich details. Finally, the CBAM (convolutional block attention module) attention mechanism is integrated into the shallow feature extraction residual branch to mitigate noise in shallow features. In 2× and 4× reconstruction experiments, objective evaluation results show that the PSNR and SSIM for 2× and 4× reconstruction reached 24.66 dB and 0.8031, and 19.26 dB and 0.4991, respectively. Subjective evaluation results indicate that Walnut-SR can reconstruct images with richer detail information and clearer texture features. Comparative experimental results of the integrated Walnut-SR module show significant improvements in mAP50 and mAP50:95 for object detection models compared to detection results using the original low-resolution images.

Rapid detection of Yunnan Xiaomila based on lightweight YOLOv7 algorithm.

Real-time fruit detection is a prerequisite for using the Xiaomila pepper harvesting robot in the harvesting process. To reduce the computational cost of the model and improve its accuracy in detecting dense distributions and occluded Xiaomila objects, this paper adopts YOLOv7-tiny as the transfer learning model for the field detection of Xiaomila, collects images of immature and mature Xiaomila fruits under different lighting conditions, and proposes an effective model called YOLOv7-PD. Firstly, the main feature extraction network is fused with deformable convolution by replacing the traditional convolution module in the YOLOv7-tiny main network and the ELAN module with deformable convolution, which reduces network parameters while improving the detection accuracy of multi-scale Xiaomila targets. Secondly, the SE (Squeeze-and-Excitation) attention mechanism is introduced into the reconstructed main feature extraction network to improve its ability to extract key features of Xiaomila in complex environments, realizing multi-scale Xiaomila fruit detection. The effectiveness of the proposed method is verified through ablation experiments under different lighting conditions and model comparison experiments. The experimental results indicate that YOLOv7-PD achieves higher detection performance than other single-stage detection models. Through these improvements, YOLOv7-PD achieves a mAP (mean Average Precision) of 90.3%, which is 2.2%, 3.6%, and 5.5% higher than that of the original YOLOv7-tiny, YOLOv5s, and Mobilenetv3 models, respectively, the model size is reduced from 12.7 MB to 12.1 MB, and the model's unit time computation is reduced from 13.1 GFlops to 10.3 GFlops. The results shows that compared to existing models, this model is more effective in detecting Xiaomila fruits in images, and the computational complexity of the model is smaller.

Fruits Classification and Detection Application Using Deep Learning

Computer vision and image processing techniques are considered efficient tools for classifying various types of fruits and vegetables. In this paper, automated fruit classification and detection systems have been developed using deep learning algorithms. In this work, we used two datasets of colored fruit images. The first FIDS-30 dataset of 971 images with 30 distinct classes of fruits is publicly available. A major contribution of this work is to present a private dataset containing 761 images with eight categories of fruits, which have been collected and annotated by ourselves. In our work, the YOLOv3 deep learning object detection algorithm have been used for individual fruit detection across multiple classes, and ResNet50 and VGG16 techniques have been utilized for the final classification for the recognition of a single category of fruit in images. Next, we implemented the automatic fruit classification models with flask for the web framework. We got 86% and 85% accuracies from the public dataset with ResNet50 and VGG16, respectively. We achieved 99% accuracy with ResNet50 and 98% accuracy with the VGG16 model on the custom dataset. The domain adaptation approach is used in this work so that the proposed deep learning-based prediction model can cope with real-world problems of diverse domains. Finally, an Android smartphone application has been developed to classify and detect fruits with the camera in real-time. All the images uploaded from the Android device are automatically sent and consequently analyzed on the web, and finally, the processed data and results are returned to the smartphone. The custom dataset and implementation codes will be available after the manuscript has been accepted. The custom dataset can be found at: https://github.com/SumonAhmed334/dataset_fruit.

Sichuan Pepper Recognition in Complex Environments: A Comparison Study of Traditional Segmentation versus Deep Learning Methods

At present, picking Sichuan pepper is mainly undertaken by people, which is inefficient and presents the possibility of workers getting hurt. It is necessary to develop an intelligent robot for picking Sichuan peppers in which the key technology is accurate segmentation by means of mechanical vision. In this study, we first took images of Sichuan peppers (Hanyuan variety) in an orchard under various conditions of light intensity, cluster numbers, and image occlusion by other elements such as leaves. Under these various image conditions, we compared the ability of different technologies to segment the images, examining both traditional image segmentation methods (RGB color space, HSV color space, k-means clustering algorithm) and deep learning algorithms (U-Net convolutional network, Pyramid Scene Parsing Network, DeeplabV3+ convolutional network). After the images had been segmented, we compared the effectiveness of each algorithm at identifying Sichuan peppers in the various types of image, using the Intersection Over Union(IOU) and Mean Pixel Accuracy(MPA) indexes to measure success. The results showed that the U-Net algorithm was the most effective in the case of single front-lit clusters light without occlusion, with an IOU of 87.23% and an MPA of 95.95%. In multiple front-lit clusters without occlusion, its IOU was 76.52% and its MPA was 94.33%. Based on these results, we propose applicable segmentation methods for an intelligent Sichuan pepper-picking robot which can identify the fruit in images from various growing environments. The research showed good accuracy for the recognition and segmentation of Sichuan peppers, which suggests that this method can provide technical support for the visual recognition of a pepper-picking robot in the field.

FPN-Based Small Orange Fruit Detection From Farm Images With Occlusion

Fruit detection using deep learning is yielding very good performance, the goal of this work is to detect small fruits in images under these occlusion and overlapping conditions. The overlap among fruits and their occlusion can lead to false and missing detection, which decreases the accuracy and generalization ability of the model. Therefore, a small orange fruit recognition method based on improved Feature Pyramid Network was developed. To begin with, multi-scale feature fusion was used to fuse the detailed bottom features and high-level semantic features to detect small-sized orange to improve recognition rate. And then repulsion loss was used to take place of the original smooth L1 loss function. Besides, Soft non-maximum suppression was adopted to replace non-maximum suppression to screen the bounding boxes of orange to construct a recognition model of orange fruits. Finally, the network was trained and verified on the collected image data set. The results showed that compared with the traditional detection models, the mean average precision was improved from 79.7 to 82.8%.

Detection of Small Oranges Using YOLO v3 Feature Pyramid Mechanism

Existing approaches to fruit detection experience difficulty in detecting small fruits with low overall detection accuracy. The reasons why many detectors are unable to handle small fruits better are that fruit data sets are small, and they are not enough to train previous models of YOLO. Further, these models used in fruit detection are initialized by a pre-trained model and then fine-tuned on fruit data sets. The pre-trained model was trained on the ImageNet data set whose objects have a bigger scale than that of the fruits in the fruit pictures. Fruit detection being a fundamental task for automatic yield estimation, the goal is to detect all the fruits in images. YOLO-V3 uses multi-scale prediction to detect the final target, and its network structure is more complex. Thus, in this work, YOLO-V3 is used to predict bounding boxes on different scales and to make multi-scale prediction, thereby making YOLO-V3 more effective for detecting small targets. The feature pyramid mechanism integrates multi-scale feature information to improve the detection accuracy.

Prediction of the fruit development stage of sweet pepper (Capsicum annum var. annuum) by an ensemble model of convolutional and multilayer perceptron

An ensemble model of convolutional neural network (CNN) and multilayer perceptron (MLP) models was developed to detect sweet pepper ( Capsicum annuum var. annuum ) fruits in images and predict their development stages. The plants were grown in four rows in a greenhouse, and images were collected from each row. Plant environment and growth data were collected every minute and month, respectively. The fruit development stage was classified into immature, breaking, and mature stages with a CNN using images. The immature stage was internally divided into four stages with an MLP, so a total of six stages were classified using the CNN–MLP ensemble model. The plant growth and environmental data and the information from the CNN output were used for the MLP input. The average accuracy of the six stages was F1 score = 0.77 and IoU = 0.86. The ensemble model showed acceptable performance in predicting fruit development stages. The CNN-only model could classify the mature and breaking stages well, but the immature stages were not distinguished, while the MLP-only model could hardly classify the fruit stage except the immature stages. The most influential factors in classification were the data obtained from CNN and the plant growth and environment data. The ensemble models could help in appropriate labour allocation and strategic management by detecting individual fruits in images and predicting precise fruit development stages. • Ensemble of CNN and MLP models to predict fruit developmental stage of sweet pepper. • CNN-only and MLP-only models could not predict all six stages. • Ensemble model accurately detected fruits and predicted six developmental stages. • Model could be expanded to assist in labour allocation and strategic management.

Faster R-CNN With Classifier Fusion for Automatic Detection of Small Fruits

Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection. Note to Practitioners —This article was motivated by the problem of the detection of small fruits for yield estimation but it also applies to the detection of other small objects. Existing approaches to fruit detection experience difficulty in detecting small fruits and the overall detection accuracy suffers as a result. This article suggests a novel approach to detect fruits of all sizes by using a multiple classifier fusion strategy in the stage of fruit localization. In the learning of the classifiers, multilevel features of fruits are extracted. In addition, we achieve classifier diversity through the analysis of the correlation of classification probabilities. The diversity of classifiers also helps improve the performance of fruit localization. Using experiments on real fruit tree images, we show that the suggested approach is feasible on orange detection of various sites in different seasons. As a future work, we will integrate the suggested algorithm into a fruit yield estimation system and test it online.

A fruits recognition system based on a modern deep learning technique

The popular technology used in this innovative era is Computer vision for fruit recognition. Compared to other machine learning (ML) algorithms, deep neural networks (DNN) provide promising results to identify fruits in images. Currently, to identify fruits, different DNN-based classification algorithms are used. However, the issue in recognizing fruits has yet to be addressed due to similarities in size, shape and other features. This paper briefly discusses the use of deep learning (DL) for recognizing fruits and its other applications. The paper will also provide a concise explanation of convolution neural networks (CNNs) and the EfficientNet architecture to recognize fruit using the Fruit 360 dataset. The results show that the proposed model is 95% more accurate.

Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’

The performance of six existing deep learning architectures were compared for the task of detection of mango fruit in images of tree canopies. Images of trees (n = 1 515) from across five orchards were acquired at night using a 5 Mega-pixel RGB digital camera and 720 W of LED flood lighting in a rig mounted on a farm utility vehicle operating at 6 km/h. The two stage deep learning architectures of Faster R-CNN(VGG) and Faster R-CNN(ZF), and the single stage techniques YOLOv3, YOLOv2, YOLOv2(tiny) and SSD were trained both with original resolution and 512 × 512 pixel versions of 1 300 training tiles, while YOLOv3 was run only with 512 × 512 pixel images, giving a total of eleven models. A new architecture was also developed, based on features of YOLOv3 and YOLOv2(tiny), on the design criteria of accuracy and speed for the current application. This architecture, termed ‘MangoYOLO’, was trained using: (i) the 1 300 tile training set, (ii) the COCO dataset before training on the mango training set, and (iii) a daytime image training set of a previous publication, to create the MangoYOLO models ‘s’, ‘pt’ and ‘bu’, respectively. Average Precision plateaued with use of around 400 training tiles. MangoYOLO(pt) achieved a F1 score of 0.968 and Average Precision of 0.983 on a test set independent of the training set, outperforming other algorithms, with a detection speed of 8 ms per 512 × 512 pixel image tile while using just 833 Mb GPU memory per image (on a NVIDIA GeForce GTX 1070 Ti GPU) used for in-field application. The MangoYOLO model also outperformed other models in processing of full images, requiring just 70 ms per image (2 048 × 2 048 pixels) (i.e., capable of processing ~ 14 fps) with use of 4 417 Mb of GPU memory. The model was robust in use with images of other orchards, cultivars and lighting conditions. MangoYOLO(bu) achieved a F1 score of 0.89 on a day-time mango image dataset. With use of a correction factor estimated from the ratio of human count of fruit in images of the two sides of sample trees per orchard and a hand harvest count of all fruit on those trees, MangoYOLO(pt) achieved orchard fruit load estimates of between 4.6 and 15.2% of packhouse fruit counts for the five orchards considered. The labelled images (1 300 training, 130 validation and 300 test) of this study are available for comparative studies.

Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis

Detecting immature fruit in groves provides a promising benefit for growers to plan application of nutrients and estimate their yield and profit prior to harvesting. The goal of this study was to develop a robust algorithm to detect and count immature citrus fruit in images of the tree canopy. Images were all taken in low natural light conditions with a flashlight, and the green component of the colour images was used for further analysis. Local intensity maxima were detected and local binary pattern (LBP) features around them were extracted as an input of an ensemble classifier-RUSBoost. The positive predictions were considered as candidates and the hierarchical contour maps around them were extracted and fitted with Circular Hough Transform. The fitted circles were predicted as fruit targets if its radius were in a predetermined range. The algorithm was evaluated with a test set of 25 images, achieved 80.4% true positive rate and 82.3% precision rate, and F-measure was 81.3%. The good performance of occlusion tolerance of the proposed method was mainly coming from the robust LBP texture descriptor and hierarchical contour analysis (HCA) which used the pattern of light intensity distribution on fruit surface. This study proposed an innovative method to detect green fruit in images of trees only by using texture and intensity distribution.

Machine vision for counting fruit on mango tree canopies

Machine vision technologies hold the promise of enabling rapid and accurate fruit crop yield predictions in the field. The key to fulfilling this promise is accurate segmentation and detection of fruit in images of tree canopies. This paper proposes two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, and compares the use of these methods relative to existing techniques:—(i) a method based on K-nearest neighbour pixel classification and contour segmentation, and (ii) a method based on super-pixel over-segmentation and classification using support vector machines. The robustness of each algorithm was tested on multiple sets of images of mango trees acquired over a period of 3 years. These image sets were acquired under varying conditions (light and exposure), distance to the tree, average number of fruit on the tree, orchard and season. For images collected under the same conditions as the calibration images, estimated fruit numbers were within 16 % of actual fruit numbers, and the F1 measure of detection performance was above 0.68 for these methods. Results were poorer when models were used for estimating fruit numbers in trees of different canopy shape and when different imaging conditions were used. For fruit-background segmentation, K-nearest neighbour pixel classification based on colour and smoothness or pixel classification based on super-pixel over-segmentation, clustering of dense scale invariant feature transform features into visual words and bag-of-visual-word super-pixel classification using support vector machines was more effective than simple contrast and colour based segmentation. Pixel classification was best followed by fruit detection using an elliptical shape model or blob detection using colour filtering and morphological image processing techniques. Method results were also compared using precision–recall plots. Imaging at night under artificial illumination with careful attention to maintaining constant illumination conditions is highly recommended.