Mask R-CNN Model Research Articles

Deep Learning in Precision Agriculture: Artificially Generated VNIR Images Segmentation for Early Postharvest Decay Prediction in Apples.

Food quality control is an important task in the agricultural domain at the postharvest stage for avoiding food losses. The latest achievements in image processing with deep learning (DL) and computer vision (CV) approaches provide a number of effective tools based on the image colorization and image-to-image translation for plant quality control at the postharvest stage. In this article, we propose the approach based on Generative Adversarial Network (GAN) and Convolutional Neural Network (CNN) techniques to use synthesized and segmented VNIR imaging data for early postharvest decay and fungal zone predictions as well as the quality assessment of stored apples. The Pix2PixHD model achieved higher results in terms of VNIR images translation from RGB (SSIM = 0.972). Mask R-CNN model was selected as a CNN technique for VNIR images segmentation and achieved 58.861 for postharvest decay zones, 40.968 for fungal zones and 94.800 for both the decayed and fungal zones detection and prediction in stored apples in terms of F1-score metric. In order to verify the effectiveness of this approach, a unique paired dataset containing 1305 RGB and VNIR images of apples of four varieties was obtained. It is further utilized for a GAN model selection. Additionally, we acquired 1029 VNIR images of apples for training and testing a CNN model. We conducted validation on an embedded system equipped with a graphical processing unit. Using Pix2PixHD, 100 VNIR images from RGB images were generated at a rate of 17 frames per second (FPS). Subsequently, these images were segmented using Mask R-CNN at a rate of 0.42 FPS. The achieved results are promising for enhancing the food study and control during the postharvest stage.

Automatic fracture detection from the images of electrical image logs using Mask R-CNN

This paper presents a novel approach for automatically detecting fractures from electrical image logs using a state-of-the-art object detection and segmentation method, i.e., the Mask Regional Convolutional Neural Network (Mask R-CNN). First, we apply image processing algorithms of inpainting and augmentation to generate the basic dataset. The fractures in those processed images are properly identified and labeled. The processed images with the annotated fractures will be used as input data for the training and testing of the Mask R-CNN model. Subsequently, the input data are randomly split into training dataset, validation dataset, and testing dataset. The Mask R-CNN model is trained using the training and validation datasets. The performance and accuracy of the trained model is tested by comparing the predicted fractures from the testing dataset to the actual fractures that are identified manually. The testing results demonstrate that the trained Mask R-CNN model has precision and recall values of 96% and 92% in detecting fractures from the image logs. Last, we further validate the proposed approach by comparing the fractures detected using the developed Mask R-CNN model against those identified from three core samples. The comparison also indicates a high accuracy of the Mask R-CNN model.

In-Domain Transfer Learning Strategy for Tumor Detection on Brain MRI.

Transfer learning has gained importance in areas where there is a labeled data shortage. However, it is still controversial as to what extent natural image datasets as pre-training sources contribute scientifically to success in different fields, such as medical imaging. In this study, the effect of transfer learning for medical object detection was quantitatively compared using natural and medical image datasets. Within the scope of this study, transfer learning strategies based on five different weight initialization methods were discussed. A natural image dataset MS COCO and brain tumor dataset BraTS 2020 were used as the transfer learning source, and Gazi Brains 2020 was used for the target. Mask R-CNN was adopted as a deep learning architecture for its capability to effectively handle both object detection and segmentation tasks. The experimental results show that transfer learning from the medical image dataset was found to be 10% more successful and showed 24% better convergence performance than the MS COCO pre-trained model, although it contains fewer data. While the effect of data augmentation on the natural image pre-trained model was 5%, the same domain pre-trained model was measured as 2%. According to the most widely used object detection metric, transfer learning strategies using MS COCO weights and random weights showed the same object detection performance as data augmentation. The performance of the most effective strategies identified in the Mask R-CNN model was also tested with YOLOv8. Results showed that even if the amount of data is less than the natural dataset, in-domain transfer learning is more efficient than cross-domain transfer learning. Moreover, this study demonstrates the first use of the Gazi Brains 2020 dataset, which was generated to address the lack of labeled and qualified brain MRI data in the medical field for in-domain transfer learning. Thus, knowledge transfer was carried out from the deep neural network, which was trained with brain tumor data and tested on a different brain tumor dataset.

Vision-Based White Radish Phenotypic Trait Measurement with Smartphone Imagery

White radish is a nutritious and delectable vegetable that is enjoyed globally. Conventional techniques for monitoring radish growth are arduous and time-consuming, encouraging the development of novel methods for quicker measurements and greater sampling density. This research introduces a mathematical model working on high-resolution images to measure radish’s biophysical properties automatically. A color calibration was performed on the dataset using a color checker panel to minimize the impact of varying light conditions on the RGB images. Subsequently, a Mask-RCNN model was trained to effectively segment different components of the radishes. The observations of the segmented results included leaf length, leaf width, root width, root length, leaf length to width, root length to width, root shoulder color, and root peel color. The automated real-life measurements of these observations were then conducted and compared with actual results. The validation results, based on a set of white radish samples, demonstrated the models’ effectiveness in utilizing images for quantifying phenotypic traits. The average accuracy of the automated method was confirmed to be 96.2% when compared to the manual method.

Universal detection and segmentation of lymph nodes in multi-parametric MRI.

Reliable measurement of lymph nodes (LNs) in multi-parametric MRI (mpMRI) studies of the body plays a major role in the assessment of lymphadenopathy and staging of metastatic disease. Previous approaches do not adequately exploit the complementary sequences in mpMRI to universally detect and segment lymph nodes, and they have shown fairly limited performance. We propose a computer-aided detection and segmentation pipeline to leverage the T2 fat-suppressed (T2FS) and diffusion-weighted imaging (DWI) series from a mpMRI study. The T2FS and DWI series in 38 studies (38 patients) were co-registered and blended together using a selective data augmentation technique, such that traits of both series were visible in the same volume. A mask RCNN model was subsequently trained for universal detection and segmentation of 3D LNs. Experiments on 18 test mpMRI studies revealed that the proposed pipeline achieved a precision of [Formula: see text]%, sensitivity of [Formula: see text]% at 4 false positives (FP) per volume, and dice score of [Formula: see text]%. This represented an improvement of [Formula: see text]% in precision, [Formula: see text]% in sensitivity at 4FP/volume, and [Formula: see text]% in dice score, respectively, over current approaches evaluated on the same dataset. Our pipeline universally detected and segmented both metastatic and non-metastatic nodes in mpMRI studies. At test time, the input data used by the trained model could either be the T2FS series alone or a blend of co-registered T2FS and DWI series. Contrary to prior work, this eliminated the reliance on both the T2FS and DWI series in a mpMRI study.

Autonomous boundary inspection of Posidonia oceanica meadows using an underwater robot

Monitoring provides important information for the planning and execution of marine environment preservation operations. Posidonia oceanica is one of the principal bioindicators in Mediterranean coastal areas and regular monitoring activities play a crucial role in its conservation. However, an efficient observation of vast areas colonised with P. oceanica is extremely challenging and it currently requires tedious and time consuming diving activities. Autonomous Underwater Vehicles (AUVs) endowed with optical sensors could represent a viable solution in carrying out visual inspection surveys. Nevertheless, AUVs are usually programmed to perform pre-defined trajectories, which are not effective for seagrass monitoring applications, as meadows may be fragmented and their contours may be irregular. This work proposes a framework based on machine learning and computer vision that enables an AUV equipped with a down-looking camera to autonomously inspect the boundary of P. oceanica meadows to obtain an initial estimate of the meadow size. The proposed boundary inspection solution is composed of three main modules: (1) an image segmentation relying on a Mask R-CNN model to recognise P. oceanica in underwater images, (2) a boundary tracking strategy that generates guidance references to track P. oceanica contours, (3) a loop closure detector fusing visual and navigation information to identify when a meadow boundary has been completely explored. The image segmentation model and the visual part of the loop closure detection module were validated on real underwater images. The overall inspection framework was tested in a realistic simulation environment, using mosaics obtained from real images to replicate the actual monitoring scenarios. The results show that the proposed solution enables the AUV to autonomously accomplish the boundary inspection task of P. oceanica meadows, therefore representing an effective tool towards the conservation and protection of marine environments.

#6891 MACHINE LEARNING NEPHROPATHOLOGY DIAGNOSTICS OF THROMBOTIC MICROANGIOPATHIES

Abstract Background and Aims Thrombotic microangiopathies, comprising atypical hemolytic uremic syndrome (aHUS) and other diseases, can present with a broad clinical and histopathological spectrum. On our way to an evidence-base for the nephropathological work-up of TMAs, we have chosen a machine-learning approach, thus eliminating suboptimal reproducibility of descriptors for individual lesions in the three decisive compartments artery, arteriole and glomerulus with human experts. Here, we present our results for an end-to-end diagnostic system. Method We collected 50 random biopsies with TMAs of various etiologies (including aHUS, hypertension-associated, systemic sclerosis, anti-phospholipid antibody syndrome and others) and 50 biopsies with Mimickers (differential diagnoses of TMA), including severe hypertensive nephropathy, necrotising arteritis/arteriolitis, cryoglobulinemic vasculitis from the three participating centers Cologne, Weill-Cornell Medical Center, and Turin. Whole slide images (WSIs) from all four nephropathology stainings HE, PAS, trichrome and Jones were included in this study. We developed an instance segmentation Mask-RCNN model with a Swin Transformer (t) backbone on tissue crops detected using a lightweight variant of the U-Net segmentation architecture. For the classification model we used our own MorphSet++ set transformer architecture to process batches of EfficientNetv2s-encoded tissue crops entered in three separate compartment channels. Batches were chosen with Monte Carlo sampling or using our own soft Markov Chain Monte Carlo (MCMC) approach. Results of the classification model are reported with 5-fold internal cross-validation. Results Segmentation performance measured as mIOU, mAP, mAR, mF1, mAS for artery reached 0.565, 0.739, 0.679, 0.704, 0.995, for arteriole 0.342, 0.531, 0.488, 0.490, 0.996, for glomeruli 0.818, 0.880, 0.919, 0.896, 0.993. Classification accuracy reached 90% with no false positives for TMA. Missed cases of TMA could be salvaged by an experienced nephropathologist on the display of decisive compartment crops, which were selected using model confidence averaged across each sampling iteration. Conclusion We have designed and trained architectures capable of segmenting decisive compartments and diagnosing TMAs on renal biopsy sections. This will enable automatic analysis of clinicopathological datasets with TMA in large cohorts. Our ultimate goal is to use large cohorts from collaborating institutions for weakly supervised, case-level-annotated training of diagnostic, prognostic and theranostic classifiers.

KLASIFIKASI RAS ANJING MENGGUNAKAN METODE CONVOLUTIONAL NEURAL NETWORK DENGAN ARSITEKTUR VGG-16

Dogs are lovable animals that bring companionship and fun to any household, but things to consider before getting a dog are their daily needs including food, shelter, and animal care, as well as affection and physical and mental stimulation. This research was conducted to introduce dog breeds and how to care for each breed. This system is built using Mask R-CNN and Convolutional Neural Network (CNN), a deep learning architecture.
 Mask R-CNN model is used for detecting and cropping dog in images, trained using Microsoft Common Object of Context (MS COCO) dataset. CNN is used for classification of dog breeds in images and is trained using 17.513 images of 17 different breeds. Result for Mask R-CNN show the detection accuracy for dogs has 74% using test images, and CNN show the identification accuracy using test images has 82% accuracy, and for CNN using cropped images has 87% accuracy.

Instance Segmentation and Number Counting of Grape Berry Images Based on Deep Learning

In order to achieve accurate segmentation of each grape image per berry, we construct a dataset composed of red globe grape samples and use a two-stage “localization–segmentation” framework-based mask region convolutional neural network (Mask R-CNN) and one-stage “pixel classification without localization” framework-based You Only Look At CoefficienTs (YOLACT) and segmenting objects by locations (SOLO) models in the grape segmentation experiments. The optimal performance of the model Mask R-CNN was applied for further study. To address the problem of overlapping and occlusion causing inaccurate fruit detection in this model, the postprocessing algorithm of the Mask R-CNN model was improved by using the linear weighting method, and the experimental results were significantly improved. The model average precision (AP)0.50, AP0.75, the mean average precision (mAP), and the mean intersection of union (mIoU) improved by 1.98%, 2.72%, 4.30%, and 3.55%, respectively. The correlation coefficient was improved from 93.59% to 96.13% by using the improved Mask R-CNN to count the number of red globe grape berries, which also further illustrates that the fruit detection problem was well solved. Using the generalized method on untrained images of different grape varieties in different scenes also achieved good segmentation results. In this study, we provide a method for segmenting and counting grape berries that is useful for automating the grape industry.

Density estimation of the main structuring sessile species in underwater marine caves with a deep learning approach

Monitoring marine biodiversity is a challenge in some vulnerable and difficult-to-access habitats, such as underwater caves. Underwater caves are a great focus of biodiversity, concentrating a large number of species in their environment. However, most of the sessile species that live on the rocky walls are very vulnerable, and they are often threatened by different pressures. The use of these spaces as a destination for recreational divers can cause different impacts on the benthic habitat. In this work, we propose a methodology based on video recordings of cave walls and image analysis with deep learning algorithms to estimate the spatial density of structuring species in a study area. We propose a combination of automatic frame overlap detection, estimation of the actual extent of surface cover, and semantic segmentation of the main 10 species of corals and sponges to obtain species density maps. These maps can be the data source for monitoring biodiversity over time. In this paper, we analyzed the performance of three different semantic segmentation algorithms and backbones for this task and found that the Mask R-CNN model with the Xception101 backbone achieves the best accuracy, with an average segmentation accuracy of 82%.

A deep learning solution for particle size analysis in low resolution inline microscopy images based on generative adversarial network

We introduce a new strategy for image analysis of inline microscopy monitoring estimate particle size distribution using deep learning. The proposed method consists of two major components: First, a novel way to generate training image-label pairs with a high-level of credibility via a Cycle-consistent Generative Adversarial Network (CycleGAN), and second, a Mask-RCNN model trained with the generated data for the particle detection task. The proposed methodology eliminates the need for manual labeling in the training phase which is a labor-intensive step and can result in labeling errors given the fuzziness of these images. We studied the application of this strategy to images acquired with a particle vision and measurement (PVM) probe.The proposed methodology was applied to images of two particle morphologies with different sizes and concentrations. Our results showed that the proposed methodology could be inexpensively used to determine qualitative trends between crystal size distributions. This trend information is a very important aspect of crystallization process monitoring and is often enough to determine what is controlling the crystallization. Therefore, we see our approach as a step in the right direction to provide insights into the particularly challenging PVM inline microscopy monitoring process without the need for offline sampling.

Intelligent identification of livestock, a source of Schistosoma japonicum infection, based on deep learning of unmanned aerial vehicle images

To develop an intelligent recognition model based on deep learning algorithms of unmanned aerial vehicle (UAV) images, and to preliminarily explore the value of this model for remote identification, monitoring and management of cattle, a source of Schistosoma japonicum infection. Oncomelania hupensis snail-infested marshlands around the Poyang Lake area were selected as the study area. Image datasets of the study area were captured by aerial photography with UAV and subjected to augmentation. Cattle in the sample database were annotated with the annotation software VGG Image Annotator to create the morphological recognition labels for cattle. A model was created for intelligent recognition of livestock based on deep learning-based Mask R-convolutional neural network (CNN) algorithms. The performance of the model for cattle recognition was evaluated with accuracy, precision, recall, F1 score and mean precision. A total of 200 original UAV images were obtained, and 410 images were yielded following data augmentation. A total of 2 860 training samples of cattle recognition were labeled. The created deep learning-based Mask R-CNN model converged following 200 iterations, with an accuracy of 88.01%, precision of 92.33%, recall of 94.06%, F1 score of 93.19%, and mean precision of 92.27%, and the model was effective to detect and segment the morphological features of cattle. The deep learning-based Mask R-CNN model is highly accurate for recognition of cattle based on UAV images, which is feasible for remote intelligent recognition, monitoring, and management of the source of S. japonicum infection.

Automatic Spine Segmentation and Parameter Measurement for Radiological Analysis of Whole-Spine Lateral Radiographs Using Deep Learning and Computer Vision.

Radiographic examination is essential for diagnosing spinal disorders, and the measurement of spino-pelvic parameters provides important information for the diagnosis and treatment planning of spinal sagittal deformities. While manual measurement methods are the golden standard for measuring parameters, they can be time consuming, inefficient, and rater dependent. Previous studies that have used automatic measurement methods to alleviate the downsides of manual measurements showed low accuracy or could not be applied to general films. We propose a pipeline for automated measurement of spinal parameters by combining a Mask R-CNN model for spine segmentation with computer vision algorithms. This pipeline can be incorporated into clinical workflows to provide clinical utility in diagnosis and treatment planning. A total of 1807 lateral radiographs were used for the training (n = 1607) and validation (n = 200) of the spine segmentation model. An additional 200 radiographs, which were also used for validation, were examined by three surgeons to evaluate the performance of the pipeline. Parameters automatically measured by the algorithm in the test set were statistically compared to parameters measured manually by the three surgeons. The Mask R-CNN model achieved an average precision at 50% intersection over union (AP50) of 96.2% and a Dice score of 92.6% for the spine segmentation task in the test set. The mean absolute error values of the spino-pelvic parameters measurement results were within the range of 0.4° (pelvic tilt) to 3.0° (lumbar lordosis, pelvic incidence), and the standard error of estimate was within the range of 0.5° (pelvic tilt) to 4.0° (pelvic incidence). The intraclass correlation coefficient values ranged from 0.86 (sacral slope) to 0.99 (pelvic tilt, sagittal vertical axis).

Size Estimation for Shrimp Using Deep Learning Method

Shrimp farming has been becoming a new source of income for fishermen in South Korea. It is often necessary for fishers to measure the size of the shrimp for the purpose to understand the growth rate of the shrimp and to determine the amount of food put into the breeding pond. Traditional methods rely on humans, which has huge time and labor costs. This paper proposes a deep learning-based method for calculating the size of shrimps automatically. Firstly, we use fine-tuning techniques to update the Mask RCNN model with our farm data, enabling it to segment shrimps and generate shrimp masks. We then use skeletonizing method and maximum inscribed circle to calculate the length and width of shrimp, respectively. Our method is simple yet effective, and most importantly, it requires a small hardware resource and is easy to deploy to shrimp farms.

Enhance the Accuracy of Landslide Detection in UAV Images Using an Improved Mask R-CNN Model: A Case Study of Sanming, China.

Extracting high-accuracy landslide areas using deep learning methods from high spatial resolution remote sensing images is a hot topic in current research. However, the existing deep learning algorithms are affected by background noise and landslide scale effects during the extraction process, leading to poor feature extraction effects. To address this issue, this paper proposes an improved mask regions-based convolutional neural network (Mask R-CNN) model to identify the landslide distribution in unmanned aerial vehicles (UAV) images. The improvement of the model mainly includes three aspects: (1) an attention mechanism of the convolutional block attention module (CBAM) is added to the backbone residual neural network (ResNet). (2) A bottom-up channel is added to the feature pyramidal network (FPN) module. (3) The region proposal network (RPN) is replaced by guided anchoring (GA-RPN). Sanming City, China was selected as the study area for the experiments. The experimental results show that the improved model has a recall of 91.4% and an accuracy of 92.6%, which is 12.9% and 10.9% higher than the original Mask R-CNN model, respectively, indicating that the improved model is more effective in landslide extraction.

Detection and infected area segmentation of apple fire blight using image processing and deep transfer learning for site-specific management

Advanced sensing technologies and deep learning models are needed for automatic recognition of pathogens to protect trees in orchards. This study developed a fire blight disease detection and infected area segmentation system using image processing and deep learning approaches to automate the detection process in a complex apple orchard environment for site-specific management. Two types of images were acquired: multispectral images from an unmanned aerial vehicle (UAV) using a multispectral camera and red–greenblue (RGB) images from the ground using two different cameras. Multispectral images were preprocessed and used for image feature analysis by calculating vegetation indices, including excessive blue (ExB), normalized difference vegetation index (NDVI), green normalized difference vegetation index (GNDVI), red-edge normalized difference vegetation index (RENDVI), modified ratio vegetation index (RVI), and triangular blueness index (TBI). Vegetation indices were calculated from a total of 60 multispectral images (30 heathy and 30 fire blight infected). Results showed that RVI was most sensitive to fire blight infection among the six indices. A support vector machine model was used to classify unhealthy tree canopies. A Mask Region-Convolutional Neural Network (Mask R-CNN) based deep learning model was developed from RGB infected images. A total of 880 images were used for training, and 220 images were used for validation. Another 110 images were used for testing the trained Mask R-CNN model. A precision of 92.8 % and recall of 91.2 % were obtained by detecting the infected canopies using a ResNet-101 backbone and intersection over union (IoU) threshold of 0.7. The high precision demonstrates the effectiveness of Mask R-CNN for the identification and segmentation of fire blight infection in images taken in complex orchard conditions. These results prove the potential of this non-invasive sensing method in detecting disease in commercial fruit production for site-specific infected canopies removing.

Small-Target Detection Based on an Attention Mechanism for Apron-Monitoring Systems

Small-target detection suffers from the problems of low average precision and difficulties detecting targets from airport-surface surveillance videos. To address this challenge, this study proposes a small-target detection model based on an attention mechanism. First, a standard airport small-target dataset was established, where the absolute scale of each marked target meets the definition of a small target. Second, using the Mask Scoring R-CNN model as a baseline, an attention module was added to the feature extraction network to enhance its feature representation and improve the accuracy of its small-target detection. A multiscale feature pyramid fusion module was used to fuse more detailed shallow information according to the feature differences of diverse small targets. Finally, a more effective detection branch structure is proposed to improve detection accuracy. Experimental results verify the effectiveness of the proposed method in detecting small targets. Compared to the Mask R-CNN and Mask Scoring R-CNN models, the detection accuracy of the proposed method in two-pixel intervals with the lowest rate of small targets increased by 10%, 3.04% and 16%, 15.15%, respectively. The proposed method proved to have a higher accuracy and be more effective at small-target detection.

An Improved Mask RCNN Model for Segmentation of ‘Kyoho’ (Vitis labruscana) Grape Bunch and Detection of Its Maturity Level

The ‘Kyoho’ (Vitis labruscana) grape is one of the mainly fresh fruits; it is important to accurately segment the grape bunch and to detect its maturity level for the construction of an intelligent grape orchard. Grapes in the natural environment have different shapes, occlusion, complex backgrounds, and varying illumination; this leads to poor accuracy in grape maturity detection. In this paper, an improved Mask RCNN-based algorithm was proposed by adding attention mechanism modules to establish a grape bunch segmentation and maturity level detection model. The dataset had 656 grape bunches of different backgrounds, acquired from a grape growing environment of natural conditions. This dataset was divided into four groups according to maturity level. In this study, we first compared different grape bunch segmentation and maturity level detection models established with YoloV3, Solov2, Yolact, and Mask RCNN to select the backbone network. By comparing the performances of the different models established with these methods, Mask RCNN was selected as the backbone network. Then, three different attention mechanism modules, including squeeze-and-excitation attention (SE), the convolutional block attention module (CBAM), and coordinate attention (CA), were introduced to the backbone network of the ResNet50/101 in Mask RCNN, respectively. The results showed that the mean average precision (mAP) and mAP0.75 and the average accuracy of the model established with ResNet101 + CA reached 0.934, 0.891, and 0.944, which were 6.1%, 4.4%, and 9.4% higher than the ResNet101-based model, respectively. The error rate of this model was 5.6%, which was less than the ResNet101-based model. In addition, we compared the performances of the models established with MASK RCNN, adding different attention mechanism modules. The results showed that the mAP and mAP0.75 and the accuracy for the Mask RCNN50/101 + CA-based model were higher than those of the Mask RCNN50/101 + SE- and Mask RCNN50/101 + CBAM-based models. Furthermore, the performances of the models constructed with different network layers of ResNet50- and ResNet101-based attention mechanism modules in a combination method were compared. The results showed that the performance of the ResNet101-based combination with CA model was better than the ResNet50-based combination with CA model. The results showed that the proposed model of Mask RCNN ResNet101 + CA was good for capturing the features of a grape bunch. The proposed model has practical significance for the segmentation of grape bunches and the evaluation of the grape maturity level, which contributes to the construction of intelligent vineyards.

Investigation of a Machine learning methodology for the SKA pulsar search pipeline

The SKA pulsar search pipeline will be used for real time detection of pulsars. Modern radio telescopes, such as SKA will be generating petabytes of data in their full scale of operation. Hence, experience-based and data-driven algorithms are being investigated for applications, such as candidate detection. Here, we describe our findings from testing a state of the art object detection algorithm called Mask R-CNN to detect candidate signatures in the SKA pulsar search pipeline. We have trained the Mask R-CNN model to detect candidate images. A custom semi-auto annotation tool was developed and investigated to rapidly mark the regions of interest in large datasets. We have used a simulation dataset to train and build the candidate detection algorithm. A more detailed analysis is planned. This paper presents details of this initial investigation highlighting the future prospects.

Maize seedling information extraction from UAV images based on semi-automatic sample generation and Mask R-CNN model

ContextThe emergence rate and growth of maize seedlings are crucial for variety selection and farm managers; however, the complex planting environment and seedling morphological differences pose great challenges for seedling detection. ObjectiveThis study aims to rapidly quickly and accurately extract maize seedling information in the field environment based on UAV images with reduced labor cost. MethodsIn this paper, we proposed an automatic identification method for maize seedlings adapted to complex scenarios (different varieties and different seedling development stages) by fine-tuning the Mask R-CNN model. Aiming at the difficulty of obtaining the training data required by the deep learning algorithm, this paper proposes a semi-automatic labeling method for the deep learning sample data of maize seedlings. At last, we proposed a method to identify the locations of disrupted monopoly and extract seedling information such as coverage and seedling area uniformity, the mapping covers the whole experimental field. Results and conclusionsThis paper includes a discussion on the effect of real flight data and resampling data on model detection results. Indeed, the results show that the identification precision of the real flight data under the same resolution is lower than that of the resampled data. The detection precision of the model decreased as the spatial resolution decreased. To ensure AP@ 0.5IOU above 0.8, the minimum image spatial resolution is 2.1 cm. We finally selected a model which had the training data with a spatial resolution of 0.8 cm in 2019 and the average precision AP@0.5IOU was 0.887, the average accuracy of emergence rate monitoring in 2019 was 98.87 % and migration to 2020 is 95.70 %, 2021 is 98.77 %. SignificanceThis work can quickly and effectively extract maize seedlings, and provide accurate seedling information, which can provide support for timely supplementation and subsequent seed selection.