Apple Sorting Research Articles

In-field grading and sorting technology of apples: A state-of-the-art review

Apple is one of the most popular fruits. In-field grading and sorting of apples would enhance growers’ economic benefits by lowering production costs. This study reviews the key components and progress of the quality inspection algorithm for in-field grading and sorting of apples. Four key components (e.g., conveyor, imaging chamber, sorting actuator, and bin filler) are presented in detail, followed by summarizing the shortcomings of these components. The apple’s external (color, size, and defects) and internal quality inspection technologies, such as optical technologies of visible light, near-infrared (NIR), hyperspectral/multispectral imaging (HSI/MSI), and structured illumination (SI) were presented. Despite the excellent detection performance of emerging technologies (e.g., HSI, MSI, and SI), visible light is still dominantly used for in-filed grading. Challenges in getting information on the whole surface area of the apple, uneven lighting, machine size, throughput, and associated costs hamper the commercialization of apple in-field grading and sorting equipment. At present, more efforts should be devoted to internal quality inspection, by developing reliable, fast, and accurate detection equipment and algorithms. With the advancement of sensors and automation algorithms, as well as the emergence of mechanical systems that are suitable for in-field use, it is anticipated that the apple in-field grading and sorting equipment that inspects both external and internal quality will be realized and commercialized in the near future.

Raman spectroscopy as a tool for characterisation of quality parameters in Norwegian grown apples during ripening

This study shows for the first time the feasibility of Raman spectroscopy as a non-destructive method to follow the ripening process of apple fruits. Two different varieties of apples were studied: ‘Aroma’ and ‘Elstar’. By visual inspection, Raman spectra showed that the starch content was higher in ‘Elstar’ apples compared to ‘Aroma’. The degradation of starch over time could be detected in the Raman spectra, indicating that the method can be used to monitor the ripening process. The ripeness markers starch index, soluble solids content (SSC), and the sugars glucose, fructose and sucrose were determined with traditional destructive methods. Cross validated calibration models based on Raman spectroscopy were obtained for all quality parameters, and test set validation offered good results, with R2 in the range 0.4–0.86 for ‘Aroma’ and 0.4–0.95 for ‘Elstar’, respectively. The regression coefficients showed that the calibrations relied on Raman bands associated with starch and different sugars. The results suggest that Raman spectroscopy in the future could be used to determine the optimal time of harvesting and to sort apples into different degrees of ripeness.

Examination of an Image Identification System for Agricultural Machines Based on Machine Vision

As computer and mechanical automation technologies advance, machine vision-based non-destructive testing technology finds use in a multitude of domains. Non-destructive testing technologies can be used on apple sorting equipment to decrease apple damage while simultaneously increasing sorting efficiency. As a result, the apple sorting machine’s image identification system now incorporates machine vision technology. The automatic classification of apple grades is accomplished by gathering, processing, extracting, and computing the contour features of apple photographs using preset sorting levels. The automatic control system then sorts apples of different grades to designated locations, thus achieving the automation of apple sorting. Tests were run on the sorting machine’s image recognition system to confirm the solution’s viability. The outcomes demonstrate that the sorting machine can effectively classify fruit automatically based on their perimeter, which is important for fruit sorting automation.

New Insights into Recent Trends and Emerging Technologies in Apple Sorting

New Insights into Recent Trends and Emerging Technologies in Apple Sorting

Early Bruise Detection in Apple Based on an Improved Faster RCNN Model

Bruising is a common occurrence in apples that can lead to gradual fruit decay and substantial economic losses. Due to the lack of visible external features, the detection of early-stage bruising (occurring within 0.5 h) is difficult. Moreover, the identification of stems and calyxes is also important. Here, we studied the use of the short-wave infrared (SWIR) camera and the Faster RCNN model to enable the identification of bruises on apples. To evaluate the effectiveness of early bruise detection by SWIR bands compared to the visible/near-infrared (Vis/NIR) bands, a hybrid dataset with images from two cameras with different bands was used for validation. To improve the accuracy of the model in detecting apple bruises, calyxes, and stems, several improvements are implemented. Firstly, the Feature Pyramid Network (FPN) structure was integrated into the ResNet50 feature extraction network. Additionally, the Normalization-based Attention Module (NAM) was incorporated into the residual network, serving to bolster the attention of model towards detection targets while effectively mitigating the impact of irrelevant features. To reduce false positives and negatives, the Intersection over Union (IoU) metric was replaced with the Complete-IoU (CIoU). Comparison of the detection performance of the Faster RCNN model, YOLOv4P model, YOLOv5s model, and the improved Faster RCNN model, showed that the improved model had the best evaluation indicators. It achieved a mean Average Precision (mAP) of 97.4% and F1 score of 0.87. The results of research indicate that it is possible to accurately and effectively identify early bruises, calyxes, and stems on apples using SWIR cameras and deep learning models. This provides new ideas for real-time online sorting of apples for the presence of bruises.

Image Processing-Based System for Apple Sorting

This study sheds light on the evolution of the agricultural industry and highlights advances in production area. The salient recognition of fruit size and shape as critical quality parameters underscores the significance of the research. In response to this challenge, the research introduces specialized image processing techniques designed to streamline the sorting of apples in agricultural settings, specifically emphasizing accurate apple width estimation. A purpose-built machine was designed, featuring an enclosure box housing a cost-effective camera for the vision system and a chain conveyor for classifying Malus domestica Borkh kind apples. These goals were successfully achieved by implementing image preprocessing, segmentation, and measurement techniques to facilitate sorting. The proposed methodology classifies apples into three distinct classes, attaining an impressive accuracy of 94% in Class 1, 92% in Class 2, and 86% in Class 3. This represents an efficient and economical solution for apple classification and size estimation, promising substantial enhancements to sorting processes and pushing the boundaries of automation in the agricultural sector.

Разработка устройства сортировки яблок по сортам с использованием технологий сверточных нейронных сетей

The work is aimed at developing a convolutional neural network for automated sorting of apples by size, weight, and visual defects. Studies were carried out using an experimental installation, which was used to collect and markup the data array. The neural network was written in Python using the Tensorflow and Keras libraries. The created model exhibited an accuracy of 96.88 % in recognition results on delayed data. A sorter device model was designed to implement the trained algorithm in the sorting production cycle.

Impact of Storage on Nondestructive Detectability of Codling Moth Infestation in Apples

Highlights Codling moth responds differently to low temperature conditions during the supply chain. The length of storage and temperature determine the sensitivity of the codling moth to detection. Hyperspectral imaging sensor detectability of codling moth increased with temperature. Abstract. Different conditions during cold storage of codling moth (CM)-infested apples lead to different infestation levels, which affect overall product quality. In this study, the effects of postharvest storage duration (up to 20 weeks) and temperature (0°C, 4°C, and 10°C) on the detectability of CM-infested apples were investigated using the near-infrared (NIR) hyperspectral imaging (HSI) method (900–1700 nm). Fresh organic Gala apples were obtained directly from a commercial market and stored in a controlled environmental chamber at three temperatures for 20 weeks in two groups: control and CM-infested samples. Every four weeks, NIR hyperspectral images in reflectance mode were acquired directly for each set of samples. Machine learning models for the classification of CM-infested apples were developed based on the HSI data. The results revealed that storage duration and temperature had a significant effect on the performance of the classification models in the detection of CM-infested and control apples. Overall, the best classification rates were obtained for apples stored for 16 weeks, with accuracies of 97%, 94%, and 100% at storage temperatures of 0°C, 4°C, and 10°C, respectively. This study is critical for determining the effectiveness of HSI as a nondestructive method for sorting apples into classes based on CM infestation when stored under different conditions and duration, as in this study. Keywords: Apples, Codling moth, Detectability, Machine learning, Nondestructive method.

Applying the quick exposure check in the workstation design process, physical and virtual prototype assessment.

The Quick Exposure Check (QEC) assesses four major body parts and engages users in assessing some physical interactions relevant to design in task analysis. In this paper, we investigated the application of QEC as the ergonomic intervention to detect pre-production ergonomic design faults in the apple sorting machine by applying physical and virtual prototyping for three different tasks analysis divided into two phases (Task 1: Apple harvesting and preparation for sorting; Task 2: Sorting control and separation of waste fruits; Task 3: Transfer of separated apples). First, the QEC questionnaire was administered while Iranian participants interacted with the machine to detect abnormal posture. Second, we redesigned a concept of the machine and assessed it with QEC by a focus group. Before design, the high pressure in Task 1 is on the back (dynamic), shoulder/arm, and very high pressure in Task 2, and in Task 3 on the back (static), arm/shoulder/neck, making an uncomfortable situation for posture. After redesign, we observed decreased pressures on the back/shoulder/arm in Task 1 from high to medium, in Task 3 from very high to low, and also in Task 2, this was detectable decreasing from very high pressures on the back/shoulder/arm and the high pressures on the neck to medium. Prototyping with QEC demonstrated that accurate redesigning of the machine with concentration on shifting from static tasks to dynamic or conversely, and ease of access by adjusting dimensions according to anthropometry and auxiliary products, could reduce musculoskeletal disorders.

Development of a Graphic Interface Application for the Vision System of the Fruit Sorting Line

Introduction. At present, an intuitive graphical interface is an indispensable component of modern agricultural-oriented software products. Aim of the Article. The research is aimed at improving the efficiency of sorting apples by developing a graphical control interface for a vision system to recognize various defects and sort apples. Materials and Methods. The authors used methods of analysis, enumeration, comparison and synthesis of modern software solutions. Results. As a result of the research, there was created a graphical application of the software for the control unit of the machine vision system containing all the necessary tools for managing and optimizing costs when sorting apples into three commercial quality classes. Discussion and Conclusion. The graphical interface of the machine vision system was used in the line LSP-4 for sorting and packing apples. It was developed by Scientific and Practical Center NAS of Belarus for Agricultural Mechanization in 2020 and successfully passed state acceptance tests.

INFORMATION-MEASURING SYSTEM FOR CONTROL OF APPLES BY SIZE IN THEIR SORTING

The article provides information about the information-measuring system of the layout of the complex for sorting apples, which allows you to determine the maximum diameter of an apple when it moves along a roller conveyor. The control algorithm is based on the use of automated thresholding using the method of moments, as well as the Advanced Morphology operation. Studies have shown that the maximum diameter measurement error does not exceed 2 mm, and the influence of radial distortion is reduced to 0.17 % as a result of calibration.

VarietyQuest: Data-Driven Exploration of Apple Fruit Variety Classification Frameworks

The Kashmir valley is known for its breathtaking landscapes and the cultivation of a variety of fruits, with apples being one of the most popular. The Kashmiri apple has a distinct flavor and is widely appreciated for its quality, leading to its export to various destinations around the globe. The sorting and packaging of apples remain a significant challenge due to the lack of skilled labor and the sheer volume of apples produced in the season. As a result, a considerable amount of the harvest is either lost or damaged, leading to significant financial losses. To address this issue, automated classification systems for categorizing fruit varieties using machine learning-based techniques are being developed. Such systems could potentially lead to increased productivity while simultaneously reducing labor costs and errors in sorting and classification. This article provides an overview of the data-driven methodologies for the automated categorization of apple varieties, exploring both conventional and state-of-the-art approaches. The article also provides a concise discussion of the datasets employed in these frameworks. Our study identifies machine learning as a critical foundation for most Apple variety classification frameworks. Further, the absence of datasets for the Kashmiri apple variety is noted during the survey, highlighting the need for further research in this domain. Overall this research explores the automated classification and packaging systems, which can streamline the process and minimize losses while contributing to the growth of the Apple industry across the globe.

ОБОСНОВАНИЕ АЛГОРИТМА СОРТИРОВАНИЯ ПЛОДОВ СИСТЕМОЙ ТЕХНИЧЕСКОГО ЗРЕНИЯ

The standard for apples entering the retail trade in the Republic of Belarus provides for the division into three grades: the highest, the first and the second. The general scheme of automated sorting of fruits as a process consisting of their supply, classification and mechanical separation is considered in the article. Feeding of fruits is designed to solve the issues of flow formation and transportation of fruits, classification - detection and recognition of the structure of the fruit, decision making, and mechanical separation of the flow - impact and change of direction of fruit movement. Apples from the 2021 harvest of the Imant, Verbnoye, and Gloucester varieties were used for the study. The analysis of physical parameters showed that their sizes obey the law of normal distribution, while the minimum size of the fetus is 45 mm, and the largest is 95 mm, which inevitably leads to the formation of “twos” of fruits during the formation of the flow. An analysis of the probability of the appearance of possible “twos” of fruits, taking into account agrotechnical requirements, made it possible to substantiate a model of an automated process in which “twos” of fruits belong to a higher grade. A model for sorting apples by quality into 4 classes was proposed. The results of the research were applied in the development of a technological line for sorting and packing apples LSP-4 at the Scientific and Production Center of the National Academy of Sciences of Belarus for agricultural mechanization. The analysis of acceptance tests showed the accuracy of sorting fruits by size 75.4%, by size and the presence of defects - 73.1%.

Multi-Camera-Based Sorting System for Surface Defects of Apples.

In this paper, we addressed the challenges in sorting high-yield apple cultivars that traditionally relied on manual labor or system-based defect detection. Existing single-camera methods failed to uniformly capture the entire surface of apples, potentially leading to misclassification due to defects in unscanned areas. Various methods were proposed where apples were rotated using rollers on a conveyor. However, since the rotation was highly random, it was difficult to scan the apples uniformly for accurate classification. To overcome these limitations, we proposed a multi-camera-based apple sorting system with a rotation mechanism that ensured uniform and accurate surface imaging. The proposed system applied a rotation mechanism to individual apples while simultaneously utilizing three cameras to capture the entire surface of the apples. This method offered the advantage of quickly and uniformly acquiring the entire surface compared to single-camera and random rotation conveyor setups. The images captured by the system were analyzed using a CNN classifier deployed on embedded hardware. To maintain excellent CNN classifier performance while reducing its size and inference time, we employed knowledge distillation techniques. The CNN classifier demonstrated an inference speed of 0.069 s and an accuracy of 93.83% based on 300 apple samples. The integrated system, which included the proposed rotation mechanism and multi-camera setup, took a total of 2.84 s to sort one apple. Our proposed system provided an efficient and precise solution for detecting defects on the entire surface of apples, improving the sorting process with high reliability.

Evaluation of a new apple in-field sorting system for fruit singulation, rotation and imaging

Automated, computer-imaging based in-field grading and sorting of apples at the time of harvest has the potential to help growers achieve significant cost savings in postharvest storage and packing. Singulation and rotation of fruit are essential steps for a computer imaging-based grading/sorting system. We have recently developed a novel compact, low-cost in-field apple sorting system, which consists of pairs of multi-stage helical conveyance drives to singulate, rotate and advance apples so that they can be inspected by a computer vision unit for quality grading. This study was aimed at evaluating the singulating and rotating performance of the sorting system for automatic inspection and grading of apples, in terms of conveying speed, fruit size, and the percentage of apple exposure area for imaging. Two apple varieties (i.e., ‘Golden Delicious’ and ‘Delicious’) were used for experimental evaluations. To determine the rotational behavior of apples, each quadrant of the apple surface was painted with different colors. Images were acquired for the apples at 15 frames per second and then automatically segmented for the estimation of exposed areas in terms of pixels. It was found that the number of images acquired for each fruit varied from 24 to 9 images when the speed of the sorting system was changed from 1 to 3 apples per second per lane. The singulating and rotating mechanism (SRM) provided continuous and relatively even rotation of the apples while conveying, which was confirmed by cumulative percentages of the four colored areas of ‘Golden Delicious’ and ‘Delicious’ apples at the three sorting speeds. At the sorting speed of 1 apple/s, the entire surface of each fruit was imaged for more than 4 times, while at 3 apples/s, the surface of the apples was imaged for at least 1.4 times. Hence, the sorting system with 3 sorting lanes is able to perform in-field quality grading of apples at a throughput of 9 or more apples/s.

Automatic Detection of Small Sample Apple Surface Defects Using ASDINet.

The appearance quality of apples directly affects their price. To realize apple grading automatically, it is necessary to find an effective method for detecting apple surface defects. Aiming at the problem of a low recognition rate in apple surface defect detection under small sample conditions, we designed an apple surface defect detection network (ASDINet) suitable for small sample learning. The self-developed apple sorting system collected RGB images of 50 apple samples for model verification, including non-defective and defective apples (rot, disease, lacerations, and mechanical damage). First, a segmentation network (AU-Net) with a stronger ability to capture small details was designed, and a Dep-conv module that could expand the feature capacity of the receptive field was inserted in its down-sampling path. Among them, the number of convolutional layers in the single-layer convolutional module was positively correlated with the network depth. Next, to achieve real-time segmentation, we replaced the flooding of feature maps with mask output in the 13th layer of the network. Finally, we designed a global decision module (GDM) with global properties, which inserted the global spatial domain attention mechanism (GSAM) and performed fast prediction on abnormal images through the input of masks. In the comparison experiment with state-of-the-art models, our network achieved an AP of 98.8%, and a 97.75% F1-score, which were higher than those of most of the state-of-the-art networks; the detection speed reached 39ms per frame, achieving accuracy-easy deployment and substantial trade-offs that are in line with actual production needs. In the data sensitivity experiment, the ASDINet achieved results that met the production needs under the training of 42 defective pictures. In addition, we also discussed the effect of the ASDINet in actual production, and the test results showed that our proposed network demonstrated excellent performance consistent with the theory in actual production.

Development and Evaluation of a Small-Scale Apple Sorting Machine Equipped with a Smart Vision System

One of the most important matters in international trades for many local apple industries and auctions is accurate fruit quality classification. Defect recognition is a key in online computer-assisted apple sorting machines. Because of the cavity structure of the stem and calyx regions, the system tends to mistakenly treat them as true defects. Furthermore, there is no small-scale sorting machine with a smart vision system for apple quality classification where it is needed. Thus, the current study focuses on a highly accurate and feasible methodology for stem and calyx recognition based on Niblack thresholding and a machine learning technique using k-nearest neighbor (k-NN) classifiers associated with a locally designed small-scale apple sorting machine. To find an appropriate mode, the effects of different numbers of k and metric distances on stem and calyx region detection were evaluated. Results showed the effectiveness of the value of k and Euclidean distances in recognition accuracy. It is found that the 5-nearest neighbor classifier and the Euclidean distance using 80 training samples produced the best accuracy rates, at 100% for stem and 97.5% for calyx. The significance of the result is very promising in fabricating an advanced small-scale and low-cost sorting machine with a high accuracy for the horticultural industry.

Implementation of the HACCP System for Apple Juice Concentrate Based on Patulin Prevention and Control.

Patulin (PAT) is a toxic secondary metabolite produced by Aspergillus sp. and Penicillium sp., which acts as a contaminant of most apples and their products. The internationally recognized HACCP system is selected as the theoretical basis to more effectively reduce the PAT in apple juice concentrate (AJC). Through field investigation of apple juice concentrate (AJC) production enterprises, we collected 117 samples from 13 steps of AJC production, including whole apple, apple pulp, and apple juice. PAT contents were analyzed via high-performance liquid chromatography (HPLC) and compared with samples from the different production processes. The result demonstrated that the PAT content was significantly (p < 0.05) influenced by five processes, receipt of raw apples, sorting of raw apples, adsorption step, pasteurization, and aseptic filling. These processes were determined as the CCPs. Monitoring systems for maintaining CCPs within acceptable limits were established, and corrective actions were proposed in case a CCP was surpassed. Based on the above-identified CCPs, critical limits, and control methods (corrective actions), a HACCP plan related to the production process of AJC was established. This study provided important guidance for juice manufacturers wishing to effectively control the PAT content in their products.

Internal Quality Attributes and Sensory Characteristics of 'Ambrosia' Apples with Different Dry Matter Content after a Two-Week and a Ten-Week Air Storage at 1 °C.

This research was conducted to determine the compositional and textural characteristics and sensory profile of 'Ambrosia' apples with different dry matter content (DMC) as estimated using a Felix-750 Produce Quality Meter (Felix Instruments Inc., Camas, WA, USA). Fruits were harvested from a commercial orchard in Cawston and an experimental field in Summerland Research and Development Centre (SuRDC) in British Columbia, Canada, when the average absorbance difference index/coefficient of fruit skin δAbsorbance (δA) dropped under 0.45 ± 0.10. DMC levels were estimated after harvest at the blush/background transition zone for fruit categorization on 300 fruits from each location. Fruits were coded with an individual number and grouped in different DMC categories. The distribution of the estimated DMC levels obtained from two locations was different. The results indicate that DMC levels were strongly and positively correlated with the soluble solids content (SSC) of the fruit (r = 0.81). Sensory evaluations also demonstrated that apples in the lowest DMC category (12.5% ± 0.5 from Cawston) were considered the least sweet apples with the least overall flavour quality by panellists compared to the apples from the other DMC categories included in the sensory evaluations from the two locations. Panellists also perceived less-than-expected "fresh apple" and "tropical" flavours but more-than-expected "no flavour" and "bland" off flavour from the lowest-DMC-category apples. The non-destructive DMC measurements show a potential to be used to sort apples for SSC, sweetness and flavour; nevertheless, they were not related to firmness or textural attributes.

Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network

In order to realize on-line detection of defective apples on a two-lane fruit sorting machine, an inspection module was constructed using NIR cameras and a diffuse illumination chamber. A real-time apple defects inspection method was proposed based on YOLO V4 deep learning algorithm. The input images were generated by combining NIR images in three consecutive rubber roller stations. Channel pruning and layer pruning methods were used to simplify the YOLO V4 network and accelerate the detection speed. A non-maximum suppression (NMS) method based on L1 norm is proposed to remove redundant prediction box after fine-tuning the pruned network. The test results indicated that the model size and inference time of the pruning-based YOLO V4 network was decreased by 241.24 megabyte (MB) and 10.82 ms, respectively, and the mean average precision (mAP) was increased from 91.82% to 93.74%, compared with the YOLO V4 network before pruning. The pruning-based YOLO V4 network based on NIR images was not affected by the variation of skin color and suitable for detects identification of different cultivars including ‘Fuji’ apple covered in red-yellow striping and red blush, ‘Golden Delicious’, and ‘Granny Smith’, with the average detection accuracy of 93.9% at the on-line test assessing five fruit per second. The overall results showed that the proposed pruning-based YOLO V4 network combined with the developed inspection module, had great potential to be implemented in commercial fruit packing line for fruit defects identification.