Apple Color Research Articles

Classification of Apple Color and Deformity Using Machine Vision Combined with CNN

Accurately classifying the quality of apples is crucial for maximizing their commercial value. Deep learning techniques are being widely adopted for apple quality classification tasks, achieving impressive results. While existing research excels at classifying apple variety, size, shape, and defects, color and deformity analysis remain an under-explored area. Therefore, this study investigates the feasibility of utilizing convolutional neural networks (CNN) to classify the color and deformity of apples based on machine vision technology. Firstly, a custom-assembled machine vision system was constructed for collecting apple images. Then, image processing was performed to extract the largest fruit diameter from the 45 images taken for each apple, establishing an image dataset. Three classic CNN models (AlexNet, GoogLeNet, and VGG16) were employed with parameter optimization for a three-category classification task (non-deformed slice–red apple, non-deformed stripe–red apple, and deformed apple) based on apple features. VGG16 achieved the best results with an accuracy of 92.29%. AlexNet and GoogLeNet achieved 91.66% and 88.96% accuracy, respectively. Ablation experiments were performed on the VGG16 model, which found that each convolutional block contributed to the classification task. Finally, prediction using VGG16 was conducted with 150 apples and the prediction accuracy was 90.50%, which was comparable to or better than other existing models. This study provides insights into apple classification based on color and deformity using deep learning methods.

Influence of Foliar Spray of Plant Growth Regulator and Nutrients on Yield and Yield Attributing Traits on Guava (Psidium guajava L.) cv. Apple Colour

Influence of Foliar Spray of Plant Growth Regulator and Nutrients on Yield and Yield Attributing Traits on Guava (Psidium guajava L.) cv. Apple Colour

Effect of Foliar Feeding of Plant Growth Regulator and Nutrients on Phenological Attributes of Guava (Psidium guajava L.) cv Apple Colour

Guava, (Psidium guajava L.), small tropical tree or shrub of the family Myrtaceae, cultivated for its edible fruits. Guava trees are native to tropical America and are grown in tropical and subtropical areas worldwide. Nutrients and PGR play a crucial role in flowering and fruit production and their deficiency can significantly impact the yield, productivity, and quality of fruits. Therefore, keeping these points in view, the present investigation was carried out on a 10-year-old guava plant in a high-density plantation (3x3) at the Fruit Research Station Imaliya, Department of Horticulture, J.N.K.V.V., Jabalpur (M.P.) during the mrig-bahar season of 2022-23. The study employed a Factorial Randomised Block Design (FRBD) with 20 treatment combinations involving Plant Growth Regulators (PGR) like salicylic acid and various nutrients, replicated three times. Results indicated that the combination of salicylic acid (200 ppm) + borax (0.5%) significantly influenced several phenological parameters. This treatment resulted in the earliest flower bud initiation (18.33 days) after foliar application, the highest average number of flower buds per shoot (7.05), minimum days to 50% flowering (33.67 days), maximum average number of flowers per shoot (6.95), the highest fruit set percentage (92.18%), the highest fruit retention percentage (81.23%), and the lowest fruit drop percentage (18.77%). The application of salicylic acid at 200 ppm in conjunction with borax at 0.5% (S2 M4) followed by salicylic acid at 300 ppm with borax at 0.5% (S3 M4) demonstrated exceptional efficacy in significantly optimizing phenological parameters when compared to the control.

Coating with phenolic branched-chain fatty acid reduces Listeria innocua populations on apple fruit

An antimicrobial coating was produced by mixing phenolic branched-chain fatty acid (PBC-FA) with glycerol and a carboxymethyl cellulose solution (CMC) at pH 7. The resulting PBC-FA-CMC solution formed an emulsion with an average droplet size of 77 nm. The emulsion in the coating solution was stable for at least 30 days at 20 °C. The in vitro antimicrobial activity of the film formed from the PBC-FA emulsion was tested against a mixture of 3 strains of Listeria innocua (7 log CFU/mL). Film with a concentration of 1000 μg/mL of PBC-FA effectively reduced the population of L. innocua below the limit of detection (<1.48 log CFU/mL) in vitro. The effect of the 1000 μg/mL PBC-FA-CMC coating formulation was then evaluated against L. innocua inoculated on “Gala” apples. Results showed that compared with the non-coated control, the coating reduced L. innocua populations by ~2 log CFU/fruit and ~6 log CFU/fruit on the apple when enumerated on tryptic soy agar and selective media (PALCAM), respectively, indicating that PBC-FA applied as a coating on apples resulted in the sub-lethal injury of bacterial cells. When L. innocua was inoculated onto PBC-FA-coated apples, the L. innocua population decreased by ~4 log CFU/fruit during 14 days of shelf-life at 20 °C. The PBC-FA coating lowered the moisture loss but did not affect the color, firmness, or soluble solids content of apples during the 14-day at 20 °C. Overall, this study revealed that there is a potential that PBC-FA can be used as an antimicrobial coating to inactivate Listeria and preserve the quality of apples.

An Insight into the Hormonal Interplay Regulating Pigment Changes and Colour Development in the Peel of ‘Granny Smith’, ‘OPAL®’ and ‘Royal Gala’ Apples

AbstractThree apple cultivars with different peel colour and known ripening dynamics were examined throughout development and on-tree ripening to unravel how the hormonal crosstalk may influence colour changes. Our results indicate that the colour of ripe apples in ‘Granny Smith’ and ‘Opal®’ cultivars was primarily influenced by chlorophyll levels masking the contribution of other pigments, whilst the development of red colour in the ‘Royal Gala’ was not only due to an important loss of chlorophyll during ripening but also to anthocyanins accumulation. The interplay amongst indole-3-acetic acid (IAA), ethylene and abscisic acid (ABA) seemed to control pigment content during the later stages of development, indicating that variations in the hormonal profile contribute to colour differences in ripe apples. In ‘Granny Smith’ apples, basal levels of IAA, ethylene and ABA were associated with the inhibition of chlorophyll loss, which was responsible for its green colour, despite the presence of high carotenoid levels. In contrast, the peel of ‘Opal®’ and ‘Royal Gala’ apples underwent a degreening process facilitated by the IAA-mediated activation of ethylene metabolism, in a crosstalk facilitated by MdARF5. The accumulation of anthocyanins in the peel of ‘Royal Gala’ apples seemed to be regulated in an ethylene-dependent manner, with MdMYB1 acting as the mediator between ethylene signalling and the anthocyanins biosynthesis and vacuolar transport pathways. On the other hand, the decline in carotenoid content during fruit development in the three cultivars appeared to be regulated at the catabolic level through ABA and its derivatives and possibly also facilitated by ethylene-mediated mechanisms. Notably, chlorophyll levels, crucial for the observed colour changes, were likely controlled by MdGLK1, a transcription factor involved in chloroplast biogenesis, working in coordination with ethylene-dependent catabolic enzymes. Overall, understanding the cultivar-specific hormonal regulation of apple colour provides valuable insights for developing preharvest and postharvest strategies to ensure an optimal fruit colour aligned with consumer preferences.

New Progress in Intelligent Picking: Online Detection of Apple Maturity and Fruit Diameter Based on Machine Vision

In the realm of automated apple picking operations, the real-time monitoring of apple maturity and diameter characteristics is of paramount importance. Given the constraints associated with feature detection of apples in automated harvesting, this study proposes a machine vision-based methodology for the accurate identification of Fuji apples’ maturity and diameter. Firstly, maturity level detection employed an improved YOLOv5s object detection model. The feature fusion section of the YOLOv5s network was optimized by introducing the cross-level partial network module VoVGSCSP and lightweight convolution GSConv. This optimization aimed to improve the model’s multiscale feature information fusion ability while accelerating inference speed and reducing parameter count. Within the enhanced feature fusion network, a dual attention mechanism combining channel and spatial attention (GAM) was introduced to refine the color and texture feature information of apples and to increase spatial position feature weights. In terms of diameter determination, the contours of apples are obtained by integrating the dual features of color and depth images within the target boxes acquired using the maturity detection model. Subsequently, the actual area of the apple contour is determined by calculating the conversion relationship between pixel area and real area at the current depth value, thereby obtaining the diameter of the apples. Experimental results showed that the improved YOLOv5s model achieved an average maturity level detection precision of 98.7%. Particularly noteworthy was the detection accuracy for low maturity apples, reaching 97.4%, surpassing Faster R-CNN, Mask R-CNN, YOLOv7, and YOLOv5s models by 6.6%, 5.5%, 10.1%, and 11.0% with a real-time detection frame rate of 155 FPS. Diameter detection achieved a success rate of 93.3% with a real-time detection frame rate of 56 FPS and an average diameter deviation of 0.878 mm for 10 apple targets across three trials. Finally, the proposed method achieved an average precision of 98.7% for online detection of apple maturity level and 93.3% for fruit diameter features. The overall real-time inference speed was approximately 56 frames per second. These findings indicated that the method met the requirements of real-time mechanical harvesting operations, offering practical importance for the advancement of the apple industry.

CA-YOLOv5: A YOLO model for apple detection in the natural environment

Improving the effectiveness of harvesting robots requires quick and accurate apple detection in natural environments. The colour and shape features of apples are corrupted due to the reflected light and the incomplete coverage of the fruit bag, bringing difficulties to apple detection. To address this issue, the Coordinate Attention You Only Look Once version 5 (CA-YOLOv5) is designed to simultaneously detect bagged and unbagged apples in the natural environment. Firstly, 1525 apple images are collected from apple orchards to build a dataset. Secondly, to solve the reflected light problem, all C3 modules in the Backbone are substituted for Coordinate Attention modules which can improve the feature representation of objects. Finally, to solve the incomplete bagging problem, the Path Aggregation Network in the Neck is replaced by a Bidirectional Feature Pyramid Network which can better fuse the features of various sizes. The CA-YOLOv5 network reaches 82.7%, 89.8%, 48.6%, and 87.0% for recall, mAP@0.5, mAP@0.5:0.95, and F1 score, respectively, which is 2.3%,1.2%,1.9%, and 2.9% higher than the YOLOv5. The results reveal that CA-YOLOv5 has much superior detection performance than the original YOLOv5, and it can serve as a technical benchmark for the development of automatic orchard-picking robots.

Application of combined treatment of peracetic acid and ultraviolet-C for inactivating pathogens in water and on surface of apples

In this study, a combined treatment of peracetic acid (PAA) and 280 nm Ultraviolet-C (UVC) – Light emitting diode (LED) was applied for inactivating foodborne pathogens in water and apples. The combined treatment of PAA (50 ppm) and UVC-LED showed synergistic inactivation effects against Escherichia coli O157:H7 and Listeria monocytogenes in water. In mechanism analysis, PAA/UVC-LED treatment induced more lipid peroxidation, intracellular ROS, membrane, and DNA damage than a single treatment. Among them, membrane damage was the main synergistic inactivation mechanism of combination treatment. Cell rupture and shrink of both pathogens after PAA/UVC-LED treatment were also identified through scanning electron microscope (SEM) analysis. To examine inactivation of pathogens on the surface of apples by PAA, UVC-LED, and their combined treatment, a washing system (WS) was developed and used. Through applying the WS, PAA/UVC-LED treatment effectively inactivated two pathogens in washing solution and on the surface of apples below the detection limit (3.30 log CFU/2000 mL and 2.0 log CFU/apple) within 5 min. In addition, there was no significant difference in color or firmness of apples after PAA/UVC-LED treatment (p > 0.05).

Identification of spoilage yeasts in cashew apple (Annacardium occidentale L.) using MALDI-TOF MS

Cashew (Annacardium occidentale L.) is a hardy and golden crop of arid regions, and the pseudo fruit, cashew apple is a vital source of sugars and vitamins that is discarded as a waste during cashew nut processing. Cashew apples start fermenting soon after the harvest, and due to the presence of a wide spectrum of polyphenols and tannins, the change of colour of cashew apples leads to an unappealing marketability and storage stability. The present study is aimed to rapid identification of different spoilage yeasts and to identify them using MALDI-TOF Mass Spectroscopy. Spoilage yeasts like Candida krusei, C. tropicalis, Pichia norvegensis, Brettannomyces bruxellensis were the spoilage yeasts found to have high confidence score values of 2.00 - 3.00 with high consistency.

Effect of chemical pretreatment on quality attributes of the cashew apple.

Cashew apples, tropical pseudo fruit, are rich in bioactive compounds. It is still underutilized due to its high perishability and its astringent flavor. This study aims to extend its shelf life by chemical dip and dry method at the rural level. Inhibition of fruit-spoiling enzymes, such as polyphenol oxidase (PPO), peroxidase (POD), amylase, and cellulase, was a significant response in this method. Enzyme inhibition was carried out using chemicals: NaCl (1-10mM), CaCl2 (1-10mM), and ethylenediamine tetraacetic acid (0.1-1mM). The effect of chemical concentration and dipping time was studied using a full factorial method at three levels (-1, 0, and 1). The dipping time ranged from 60 to 180min, and chemical concentrations from 1 to 10mM were studied. Optimal treatment conditions were obtained as follows: NaCl concentration of 9.45mM, dipping time of 160min, and CaCl2 concentration of 7.8mM, dipping time of 160min. NaCl pretreatment showed maximum inhibition of PPO (>80%) and POD (>80%), whereas CaCl2 pretreatment showed maximum inhibition of amylase (60.58%) and cellulase (80.23%). Hence, to avoid postharvest losses, pretreatment with NaCl and CaCl2 was adequate to preserve the texture and color of cashew apples. PRACTICAL APPLICATION: Chemical pretreatment can prevent the postharvest losses of cashew apples. Inhibition of PPO, POD, amylase, and cellulase is vital in the shelf-life extension of cashew apples. Sodium chloride dip is a cost-effective method for increasing the storability of cashew apples.

Design of apple snacks – A study of the impact of calcium impregnation method on physicochemical properties and structure of apple tissues during convective drying

Calcium impregnation methods (at atmospheric pressure, AI, or under vacuum, VI) followed by air drying at 60 °C were used for producing healthy apple snacks. This study compared the effect of the impregnation method on the ultrastructure features and physicochemical properties (desorption isotherm, density, porosity and shrinkage, thermal transitions, rheological behaviour, colour) of fortified apples along convective drying. Calcium concentration in AI and VI dried apples were ≈ 3600 and ≈ 5200 ppm respectively, which implied, considering a portion of dried apple of 50 g, 18% and 26% of the recommended intake for adults. Calcium incorporation did not affect the desorption behaviour but induced browning of apple tissues. Drying resulted in a great increase in porosity of AI samples and a noticeable compaction of the VI tissues. Impregnated tissues at aw ≤ 0.47 exhibited distinctive mechanical properties: AI apples were characterized by very high stiffness and lower resistance to rupture, while VI apples appeared stronger and moderately stiff. Industrial relevance textThis study will help to develop large-scale commercial production of calcium fortified apple snacks with distinctive characteristics. High quality and content of nutrients, easy industrialization and storage and a long shelf life are important aspects of these healthy snacks. They constitute a valuable functional shelf-stable product, that also can help feeding the population during natural disasters and wars, as well as reducing food insecurity in vulnerable groups as part of the diet. On the other hand, the use of apples not meeting the organoleptic requirements for direct consumption, or not processed into juice, drink, or cider as raw material, looks of particular interest for diversifying the production within the framework of sustainable production.

Red-TE Homozygous Alleles of MdMYB10 Confer Full-Red Apple Fruit Skin in a High-Temperature Region

Apple is a major fruit crop grown worldwide and provides humans with an essential diet and health benefits. One of the health benefits is related to the accumulation of fruit anthocyanin, which also provides fruit with an attractive red colour. It is known that an MdMYB10 allele containing a transposable element (TE) insertion in its promoter (termed Red-TE allele) underlies anthocyanin accumulation and red colouration in the fruit skin of cultivated apples. However, the distribution of this Red-TE allele in wider Malus germplasm accessions is not clear. In this study, we showed that MdMYB10 RNA in fruit skin was specifically expressed from the Red-TE allele by using allele-specific expression analysis of transcriptome data. Apple cultivars and hybrids with homozygous Red-TE alleles showed stronger red colour than those with heterozygous alleles after analysing 65 cultivars and 337 hybrids. Furthermore, both hetero- and homozygous plants growing in the same high-temperature conditions had different colourations. However, the Red-TE allele was not detected in 16 wild apple accessions showing red skin, indicating that the red skin colour of these wild apples was not conferred by the Red-TE allele. These findings provide guidance for selecting cultivars able to develop consistent red colouration under high growth temperature conditions and open the opportunity for identifying novel genetic variants underpinning fruit red colouration in wild apple species.

A Novel Sodium Alginate-Carnauba Wax Film Containing Calcium Ascorbate: Structural Properties and Preservative Effect on Fresh-Cut Apples

In order to improve the mechanical properties, nutritional value and fresh-keeping ability of conventional sodium alginate edible composite membranes, a new type of edible composite film was prepared by adding water-blocking agent carnauba wax, plasticizer glycerin, antioxidant and nutritional enhancer sodium ascorbate on a basis of traditional sodium alginate composite film. In this study, the physical, mechanical and structural properties of different film components were investigated. The results showed the components did not simply combine, but produced interaction forces which improved the stability and mechanical properties of composite film. When the amount of calcium ascorbate was 0.4%, the water vapor transmittance of the composite film reached a minimum of 0.65 g·mm/(cm2·d·kPa), and the tensile strength and elongation at break reached the maximum, which were 398.64 MPa and 17.93%, respectively. Additionally, the sodium alginate-carnauba wax film exhibited better performance on the preservation of fresh-cut apples. Compared with other composite films, the color and hardness of fresh-cut apples coated with this composite film were better maintained, and the losses of titration acid content and soluble solid content were reduced. Moreover, the weight loss rate, increase in polyphenol oxidase activity and total colony count were inhibited. All results determined that the edible film has good application value in the field of fresh-cut fruit preservation, which provides a theoretical basis for further research on edible film.

Apple detection and instance segmentation in natural environments using an improved Mask Scoring R-CNN Model.

The accurate detection and segmentation of apples during growth stage is essential for yield estimation, timely harvesting, and retrieving growth information. However, factors such as the uncertain illumination, overlaps and occlusions of apples, homochromatic background and the gradual change in the ground color of apples from green to red, bring great challenges to the detection and segmentation of apples. To solve these problems, this study proposed an improved Mask Scoring region-based convolutional neural network (Mask Scoring R-CNN), known as MS-ADS, for accurate apple detection and instance segmentation in a natural environment. First, the ResNeSt, a variant of ResNet, combined with a feature pyramid network was used as backbone network to improve the feature extraction ability. Second, high-level architectures including R-CNN head and mask head were modified to improve the utilization of high-level features. Convolutional layers were added to the original R-CNN head to improve the accuracy of bounding box detection (bbox_mAP), and the Dual Attention Network was added to the original mask head to improve the accuracy of instance segmentation (mask_mAP). The experimental results showed that the proposed MS-ADS model effectively detected and segmented apples under various conditions, such as apples occluded by branches, leaves and other apples, apples with different ground colors and shadows, and apples divided into parts by branches and petioles. The recall, precision, false detection rate, and F1 score were 97.4%, 96.5%, 3.5%, and 96.9%, respectively. A bbox_mAP and mask_mAP of 0.932 and 0.920, respectively, were achieved on the test set, and the average run-time was 0.27 s per image. The experimental results indicated that the MS-ADS method detected and segmented apples in the orchard robustly and accurately with real-time performance. This study lays a foundation for follow-up work, such as yield estimation, harvesting, and automatic and long-term acquisition of apple growth information.

Transcriptome and Metabolome Analysis Reveals the Effect of Nitrogen-Potassium on Anthocyanin Biosynthesis in "Fuji" Apple.

Nitrogen (N) and potassium (K) have significant effects on apple peel color. To further understand the molecular mechanism of N-K regulation of apple color, we analyzed the apple peel under different N and K treatments using isotope labeling, transcriptomics, and metabolomics. Under high N treatments, fruit red color and anthocyanin content decreased significantly. High N decreased the 13C distribution rate and increased the Ndff values of fruits, while K increased the expression of MdSUTs and MdSOTs and promoted 13C transportation to fruits. Anthocyanin-targeted metabonomics and transcriptome analysis revealed that high N downregulated the expression of structural genes related to the anthocyanin synthesis pathway (MdPAL, Md4CL, MdF3H, MdANS, and MdUFGT) and their regulators (MdMYBs and MdbHLHs), and also decreased some metabolites contents. K alleviated this inhibition and seven anthocyanins were regulated by N-K. Our results improve the understanding of the synergistic regulation of apple fruit coloring by N-K.

Transcriptome analysis reveals candidate genes involved in nitrogen deficiency stress in apples

Nitrogen is one of the macroelements required for plant growth and development and the identification of candidate genes involved in nitrogen deficiency stress is of great importance to the sustainable development of agriculture. Here, we found that the color of apple leaves changed from dark green to yellow-green, the malondialdehyde (MDA) content, soluble protein content, and proline content significantly increased, the chlorophyll content significantly decreased in response to nitrate deficiency stress. According to the physiological and biochemical changes of apple leaves during nitrate deficiency stress, nitrogen deficiency stress was divided into two stages: early nitrogen deficiency stage (ES) and late nitrogen deficiency stage (LS). Transcriptome sequencing was performed in these two stress stages. 5773 differential expression genes (DEGs) were identified in the early nitrogen deficiency stress stage and 6130 DEGs were identified in the late nitrogen deficiency stress stage. Functional analysis of these DEGs revealed that a large number of DEGs were enriched in ‘porphyrin and chlorophyll metabolic’ pathways, the ‘photosynthesis’ pathway, the ‘photosynthesis-antenna protein’ pathway, and the ‘ABA', 'ETH', and 'JA' signal transduction pathways, and the metabolic networks of these pathways were constructed. In addition, overexpression of MdNAC4 weakened the tolerance of apple calli to nitrogen deficiency stress. Taken together, our results reveal possible pathways for apple adaptation to nitrogen deficiency stress and identify the function of MdNAC4, a key transcription factor regulating nitrogen deficiency stress, which enriches the molecular mechanism of apple adapting to a nitrogen deficiency environment.

Stability of phenolic compounds in apple and strawberry: Effect of different processing techniques in industrial set up

Different technologies commonly employed by the fruit processing industry affect the phenolic content. This study aimed to establish the extent to which different processing techniques and conditions, tested at an industrial scale, affect phenolics, color, and sensory attributes of fresh strawberries and apples. The effects of freezing, thermal treatments, and high-pressure processing were investigated. In strawberries, mild and standard thermal treatments showed similar patterns for most phenolic groups; an increase in proanthocyanidins, no change in ellagic acid conjugates, and a major decrease in flavonols and anthocyanins. In apples, mild treatments and high-pressure processing had similar effects in all phenolic groups, with increases in dihydrochalcones, hydroxycinnamics, and proanthocyanidins and decreases in flavonols. However, the standard thermal treatment increased flavonols and dihydrochalcones concentrations. This study shows that each fruit behaves differently, and both technology and processing conditions should be customized accordingly to preserve or even increase the phenolic content.

Electrohydrodynamic drying of fruit slices: Effect on drying kinetics, energy consumption, and product quality

The effect of electrohydrodynamic drying on the drying kinetics, energy consumption, and quality was studied for apple and strawberry slices with different thicknesses from 1 to 4 mm. Effective diffusivity and energy consumption were quantified via direct measurements, while visual quality attributes, such as color and shrinkage were determined through image analysis. It was found that effective diffusivity was independent of thickness, around 0.5×10-11 m2/s for apples and 0.24×10-11 m2/s for strawberries. The average specific energy consumption increased with the fruit thickness from 601.13 kJ/kg (1 mm) to 1433.83 kJ/kg (4 mm) for apple slices and from 1324.17 (3 mm) to 1756.71 kJ/kg (4 mm) for strawberry slices. The effect of EHD drying on apple color was significantly smaller than conventional hot-air drying. The impact of EHD drying on the shrinkage of apple slices was about 85%, and for strawberry slices, about 90%, which was higher than in hot-air drying. It can be concluded that EHD is effective for drying thin fruit slices due to the low energy consumption and better quality.

Quality changes and shelf-life prediction model of postharvest apples using partial least squares and artificial neural network analysis

The quality of postharvest apples is greatly affected by storage temperatures. In this paper, the sensory qualities, such as flavor, texture, color, and taste change of apples during storage at 4 °C and 20 °C were investigated. After correlation analysis, the partial least squares (PLS) and artificial neural network (ANN) techniques were used to build a shelf-life prediction model. The results showed that lower temperature storage can better maintain the color, flesh hardness, and release of volatile compounds of apples. The acidity of apples stored at 20 °C decreased much faster than that at 4 °C. The PLS models were successful in predicting the apple shelf life. When modeling using PLS with a single type index, the order of accuracy of the prediction model was texture, color, and flavor. As a nonlinear algorithm, the ANN model was also an effective predictive tool of apple shelf life at both temperatures.

Fusion of Mask RCNN and attention mechanism for instance segmentation of apples under complex background

It is important to precisely segment apples in an orchard during the growth period to obtain accurate growth information. However, the complex environmental factors and growth characteristics, such as fluctuating illumination, overlapping and occlusion of apples, the gradual change in the ground colour of apples from green to red, and the similarities between immature apples and background leaves, affect apple segmentation accuracy. The purpose of this study was to develop a precise apple instance segmentation method based on an improved Mask region-based convolutional neural network (Mask RCNN). An existing Mask RCNN model was improved by fusing an attention module into the backbone network to enhance its feature extraction ability. A combination of deformable convolution and the transformer attention with the key content only term was used as the attention module in this study. The experimental results showed that the improved Mask RCNN can accurately segment apples under various conditions, such as apples with shadows and different ground colours, overlapped apples, and apples occluded by branches and leaves. A recall, precision, F1 score, and segmentation mAP of 97.1%, 95.8%, 96.4% and 0.917, respectively, were achieved, and the average run-time on the test set was 0.25 s per image. Our method outperformed the two methods in comparison, indicating that it can accurately segment apples in the growth stage with a near real-time performance. This study lays the foundation for realizing accurate fruit detection and long-term automatic growth monitoring.