Instance Segmentation Algorithm Research Articles

IRSDD-YOLOv5: Focusing on the Infrared Detection of Small Drones

With the rapid growth of the global drone market, a variety of small drones have posed a certain threat to public safety. Therefore, we need to detect small drones in a timely manner so as to take effective countermeasures. At present, the method based on deep learning has made a great breakthrough in the field of target detection, but it is not good at detecting small drones. In order to solve the above problems, we proposed the IRSDD-YOLOv5 model, which is based on the current advanced detector YOLOv5. Firstly, in the feature extraction stage, we designed an infrared small target detection module (IRSTDM) suitable for the infrared recognition of small drones, which extracted and retained the target details to allow IRSDD-YOLOv5 to effectively detect small targets. Secondly, in the target prediction stage, we used the small target prediction head (PH) to complete the prediction of the prior information output via the infrared small target detection module (IRSTDM). We optimized the loss function by calculating the distance between the true box and the predicted box to improve the detection performance of the algorithm. In addition, we constructed a single-frame infrared drone detection dataset (SIDD), annotated at pixel level, and published an SIDD dataset publicly. According to some real scenes of drone invasion, we divided four scenes in the dataset: the city, sky, mountain and sea. We used mainstream instance segmentation algorithms (Blendmask, BoxInst, etc.) to train and evaluate the performances of the four parts of the dataset, respectively. The experimental results show that the proposed algorithm demonstrates good performance. The AP50 measurements of IRSDD-YOLOv5 in the mountain scene and ocean scene reached peak values of 79.8% and 93.4%, respectively, which are increases of 3.8% and 4% compared with YOLOv5. We also made a theoretical analysis of the detection accuracy of different scenarios in the dataset.

Extracting cow point clouds from multi-view RGB images with an improved YOLACT++ instance segmentation

Obtaining shape features from a farm environment is the basis for animal size measurement, body condition scoring, weight prediction, and individual identification. To improve the robustness of point cloud segmentation and fully utilize 2D (Two dimension) and 3D (Three dimension) information for segmenting animal shape data, a novel pixel-level segmentation method of cow point clouds was proposed by fusing real images with high resolution and point clouds. First, the 3D point clouds of cows were generated from the multi-view images by using the Structure from Motion (SfM) algorithm, and the corresponding relationship between the point clouds and the real image was established by the camera interior and exterior parameters. Second, for 2D image segmentation, the improved, single-stage instance segmentation algorithm YOLACT++ (You Only Look At CoefficienTs) was applied to improve the segmentation accuracy of cow body regions, and the average precision of multi-view images was 85.9%. Last, in point cloud segmentation, the image segmentation results of the cows were mapped to 3D point clouds by the correspondence between point clouds and images, and the visual points of different viewpoints were fused to obtain the 3D point cloud of the target cow. The average relative error of truncated areas between the proposed method and manual segmentation of the point cloud was 2.18%, and the change in the truncated area caused by the increase in images was less than 5% with the appropriate number of images (20–40). The results demonstrated the effectiveness of the proposed method in 3D point cloud segmentation of cows and its potential application in animal shape acquisition.

C1M2: a universal algorithm for 3D instance segmentation, annotation, and quantification of irregular cells.

Cell instance segmentation is a fundamental task for many biological applications, especially for packed cells in three-dimensional (3D) microscope images that can fully display cellular morphology. Image processing algorithms based on neural networks and feature engineering have enabled great progress in two-dimensional (2D) instance segmentation. However, current methods cannot achieve high segmentation accuracy for irregular cells in 3D images. In this study, we introduce a universal, morphology-based 3D instance segmentation algorithm called Crop Once Merge Twice (C1M2), which can segment cells from a wide range of image types and does not require nucleus images. C1M2 can be extended to quantify the fluorescence intensity of fluorescent proteins and antibodies and automatically annotate their expression levels in individual cells. Our results suggest that C1M2 can serve as a tissue cytometry for 3D histopathological assays by quantifying fluorescence intensity with spatial localization and morphological information.

Detecting the overfilled status of domestic and commercial bins using computer vision

As the amount of waste being produced globally is increasing, there is a need for more efficient waste management solutions to accommodate this expansion. The first step in waste management is the collection of bins or containers. Each bin truck in a fleet is assigned a collection route. As the bin trucks have a finite amount of storage for waste, accepting overfilled bins may result in filling this storage before the end of the collection route. This creates inefficiencies as a second bin truck is needed to finish the collection route if the original becomes full. Currently, the recording and tracking of overfilled bins is a manual process, requiring the bin truck operator to undertake this task, resulting in longer collection route durations. To create a more efficient and automated process, computer vision methods are considered for the task of detecting the bin status. Video footage from a commercial collection route for two bin types, automated side loader (ASL) and front-end loader (FEL), was utilized to create appropriate computer vision datasets for the task of fully supervised object detection and instance segmentation. Selected state-of-the-art object detection and instance segmentation algorithms were used to investigate their performances on this proprietary dataset. A mean average precision (mAP) score of 0.8 or greater was achieved with each model, reflecting the effectiveness of using computer vision as a tool to automate the process of recording overfilled bins.

Low-value recyclable waste identification based on NIR feature analysis and RGB-NIR fusion

Low-value recyclable waste is one of the main components of solid waste, which is challenging to recycle because of its low density, small size, multiple categories, and large quantity. However, the primarily used spectral-based sorting is expensive, so incineration and landfill are still the main disposal methods in developing countries. In order to reduce the cost of sorting equipment, improve the accuracy and efficiency of identification, and facilitate the promotion of automatic sorting, this paper combines RGB and limited NIR data to realize cost-effective sorting. Firstly, an RGB-NIR instance segmentation dataset of white, color, and transparent PE, PP, and PET plastic flakes was produced, which are the main components of low-value recyclable waste. Then PCA and ANOVA were used to analyze the characteristics of the amplitude and band ratio of NIR data, where six characteristic bands were obtained. After that, the effect of the multi-channel NIR instance segmentation algorithm was tested based on 12 combinations of 6 bands, and the F1-score of the two better band combinations (1193, 1218, 1666 nm and 1134, 1193, 1218, 1349, 1398, 1666 nm) were 0.965 and 0.980, respectively. Finally, the selected 3-band NIR data and RGB data were mixed to create a fusion identification algorithm, which achieved the best material recognition accuracy (F1-score = 0.981) and color recognition accuracy (F1-score = 0.972). Moreover, the 9-categories comprehensive F1-score of the RGB-NIR fusion model is 0.961, which shows that it can recognize both material and color recognition.

Recognition and segmentation of teeth and mandibular nerve canals in panoramic dental X-rays by Mask RCNN

This study proposes a method based on the instance segmentation algorithm Mask RCNN for recognizing and segmenting 32 teeth and 2 mandibular nerve canals in panoramic dental X-rays. Compared with Faster RCNN, Mask RCNN has an additional semantic segmentation branch, which could directly obtain the position of teeth and the information about mandibular nerve canals in this study. At the same time, Mask RCNN introduces ROI Align, making Mask segmentation more accurate. A total of 120 training sets, 40 validation sets, and 120 test sets were used in the training and testing process. Forty of the test sets were used to test the full-tooth panoramic dental X-rays, and 40 were used for the edentulous panoramic dental X-rays, with 40 for the mandibular canals. The average precision, recall, and F1-score values of the full teeth were 96.59 %, 96.4 %, and 96.5 %, respectively. The Pm value of the defined edentulous panoramic dental X-rays was 95.41 %, and the average precision, recall, and F1-score values of mandibular canals were 76.05 %, 83.70 %, and 78.10 %, respectively. The results showed that the algorithm could effectively identify each tooth, including missing teeth and mandibular nerve canals in panoramic dental X-rays, solving the problem of too many oral structures in panoramic dental X-rays, which makes it difficult to determine the overall oral state. This study will help other researchers promote the research on artificial intelligence-assisted diagnosis in the oral field.

A Pig Mass Estimation Model Based on Deep Learning without Constraint.

The body mass of pigs is an essential indicator of their growth and health. Lately, contactless pig body mass estimation methods based on computer vision technology have gained attention thanks to their potential to improve animal welfare and ensure breeders' safety. Nonetheless, current methods require pigs to be restrained in a confinement pen, and no study has been conducted in an unconstrained environment. In this study, we develop a pig mass estimation model based on deep learning, capable of estimating body mass without constraints. Our model comprises a Mask R-CNN-based pig instance segmentation algorithm, a Keypoint R-CNN-based pig keypoint detection algorithm and an improved ResNet-based pig mass estimation algorithm that includes multi-branch convolution, depthwise convolution, and an inverted bottleneck to improve accuracy. We constructed a dataset for this study using images and body mass data from 117 pigs. Our model achieved an RMSE of 3.52 kg on the test set, which is lower than that of the pig body mass estimation algorithm with ResNet and ConvNeXt as the backbone network, and the average estimation speed was 0.339 s·frame-1 Our model can evaluate the body quality of pigs in real-time to provide data support for grading and adjusting breeding plans, and has broad application prospects.

Abstract 5379: Deep learning enables label-free tracking of heterogeneous subpopulations

Abstract Observing and quantifying the proliferation of subpopulations in cancer is key to understanding how heterogeneous groups of cells interact and respond to therapy, their environment, and each other. Prior research has demonstrated that cell-state properties such as metastatic potential and genotype perturbation are encoded in cellular morphology and can be identified with various machine-learning approaches. This encoding spans multiple imaging modalities such as brightfield, phase contrast, and stained whole-slide images, and phenotype prediction can be accomplished using both classical and deep machine learning methods. Here we show that not only do these prediction capabilities extend to transcriptomic subpopulations, but that they can be used to track these populations in high throughput longitudinal live-cell imaging experiments. Using single-cell RNA sequencing, we observed that, among untreated MDA-MB-231 triple negative breast cancer cells, there exist two transcriptomically distinct populations of cells. Using fluorescence-activated cell sorting based on the differentially expressed surface marker ESAM, we isolated these subpopulations and fluorescently labeled them with mCherry or GFP depending on their transcriptomic cluster. Cells were then grown in monoculture or coculture and imaged every 4 hours at 20x resolution. To identify the associated transcriptomic cluster, we trained an instance segmentation algorithm, Mask R-CNN, to both segment and classify cells. We find that, despite being derived from the same cell line, these phenotypes can be predicted using phase contrast images alone. These results demonstrate that cellular phenotypes manifested by distinctive RNA expression signatures can now be surveilled in a high-throughput manner across multiple samples and conditions without further RNA sequencing or biomarker labeling. We anticipate that this methodology of high throughput tracking will be applicable to other heterogeneous subpopulations, such as isolated therapy-resistant or sensitive cells. Tracking transcriptomically distinct populations using only high-throughput imaging will increase the granularity of population analysis and enable more rapid assessment of cancer cell evolutionary dynamics. Citation Format: Tyler Jost, Andrea Gardner, Amy Brock. Deep learning enables label-free tracking of heterogeneous subpopulations. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5379.

318 Designing and Developing a Novel Deep Computer Vision Platform for Intraoperative Prediction and Analytics

INTRODUCTION: Robust artificial intelligence (AI)-based surgical video analysis platforms could lead to novel insights for intraoperative guidance. METHODS: Microvascular decompression surgeries were recorded using a Storz endoscope. A dataset of 2611 frames from 4 operative videos were segmented to create ground truth images. A sparse labeling paradigm was used, and training data comprised only 3% of the total video frames. Surgical anatomy including the brain stem, cerebellum, cranial nerves, vascular structures and surgical instruments were annotated. We developed a custom deep learning framework built on top of a state-of-the-art instance segmentation algorithm SOLOv2, the baseline. Model was trained for each video and pre-training was transferred across videos. Mean average precision(mAP) was computed within and across videos as an evaluation metric. Lastly, a novel metric to quantify arterial pulsation-induced nerve deformation is introduced, and compared before and after Teflon sponge placement. RESULTS: Our model consistently outperforms the baseline, with an average mAP of 63.75 for within video, and shows feasibility on novel patient video test set with mAP of 40.13 using few-shot learning. For the novel test video, our pulsatility index during compression is 9.5 whereas after decompression is 6.0, indicating successful dampening of artery-nerve pulsation after sponge insertion. CONCLUSIONS: In a sparse labeling paradigm, we design and develop a custom deep computer vision-based instance segmentation architecture to predict and track anatomical structures and surgical objects with high accuracy. We create a novel metric, the “pulsatility index”, which is able to quantify the nerve-artery interface for the first time. Once correlated with outcome measures, real-time AI-based video feed analysis may have the transformative potential to re-define intraoperative standard of care.

Precise Human Removal and Inpainting Using Mask RCNN and LaMa

Sometimes people are not supposed to be in a photo for various purposes, but this is usually unavoidable. Therefore, in the postprocessing of the image, it can be solved by removing people from the picture without affecting the coherence and naturalness of the object and background in the photo. We propose a human removal method based on image instance segmentation and image inpainting. Firstly, we send an image into the image instance segmentation algorithm to obtain a mask covering the unwanted parts of the picture. Then we do dilatation on this mask to expand the mask region. Finally, the inpainting algorithm will take the image and the processed mask and produce an inpainted image with no human and feel natural.

Marine ship instance segmentation by deep neural networks using a global and local attention (GALA) mechanism.

Marine ships are the transport vehicle in the ocean and instance segmentation of marine ships is an accurate and efficient analysis approach to achieve a quantitative understanding of marine ships, for example, their relative locations to other ships or obstacles. This relative spatial information is crucial for developing unmanned ships to avoid crashing. Visible light imaging, e.g. using our smartphones, is an efficient way to obtain images of marine ships, however, so far there is a lack of suitable open-source visible light datasets of marine ships, which could potentially slow down the development of unmanned ships. To address the problem of insufficient datasets, here we built two instance segmentation visible light datasets of marine ships, MariBoats and MariBoatsSubclass, which could facilitate the current research on instance segmentation of marine ships. Moreover, we applied several existing instance segmentation algorithms based on neural networks to analyze our datasets, but their performances were not satisfactory. To improve the segmentation performance of the existing models on our datasets, we proposed a global and local attention mechanism for neural network models to retain both the global location and semantic information of marine ships, resulting in an average segmentation improvement by 4.3% in terms of mean average precision. Therefore, the presented new datasets and the new attention mechanism will greatly advance the marine ship relevant research and applications.

Research on Instance Segmentation Algorithm of Greenhouse Sweet Pepper Detection Based on Improved Mask RCNN

The fruit quality and yield of sweet peppers can be effectively improved by accurately and efficiently controlling the growth conditions and taking timely corresponding measures to manage the planting process dynamically. The use of deep-learning-based image recognition technology to segment sweet pepper instances accurately is an important means of achieving the above goals. However, the accuracy of the existing instance segmentation algorithms is seriously affected by complex scenes such as changes in ambient light and shade, similarity between the pepper color and background, overlap, and leaf occlusion. Therefore, this paper proposes an instance segmentation algorithm that integrates the Swin Transformer attention mechanism into the backbone network of a Mask region-based convolutional neural network (Mask RCNN) to enhance the feature extraction ability of the algorithm. In addition, UNet3+ is used to improve the mask head and segmentation quality of the mask. The experimental results show that the proposed algorithm can effectively segment different categories of sweet peppers under conditions of extreme light, sweet pepper overlap, and leaf occlusion. The detection AP, AR, segmentation AP, and F1 score were 98.1%, 99.4%, 94.8%, and 98.8%, respectively. The average FPS value was 5, which can be satisfied with the requirement of dynamic monitoring of the growth status of sweet peppers. These findings provide important theoretical support for the intelligent management of greenhouse crops.

Research on Optimization of 3D Printing Manufacturing Technology Based on Segmentation Algorithm

3D printing technology is a relatively mature processing technology. In actual production, external support needs to be added to ensure the smooth progress of the processing process. Tree support is a new type of external support method. Combined with instance segmentation technology in artificial intelligence, it can better solve the problems of collapse, deformation, and lack of precision in the 3D printing process and improve production efficiency. For the feature loss problem in the printing model, a new instance segmentation algorithm is adopted, including deepening the feature extraction architecture and using the nonlinear activation function Mish. For the feature reconstruction after instance segmentation, this paper optimizes and improves the classical concealment method, including the spatial range concealment method Z-buffer algorithm and the area scan line algorithm of the plane range concealment method to realize the projection mapping transformation of the 3D object model by combining coordinate refinement and Z-buffer algorithm.

Instance Segmentation Algorithm Based on Semantic Alignment and Graph Node Interaction

Instance Segmentation Algorithm Based on Semantic Alignment and Graph Node Interaction

Automated Construction Site Monitoring Based on Improved YOLOv8-seg Instance Segmentation Algorithm

Automated Construction Site Monitoring Based on Improved YOLOv8-seg Instance Segmentation Algorithm

Ship hull number detection and recognition under sparse samples

目的 舰船舷号检测识别是海面态势感知的关键技术，精准的舷号检测识别对海洋权益保护具有重要意义。但目前没有公开数据提供支持。为此，本文先构建了一个真实场景下的稀疏舰船舷号数据集（sparse ship hullnumber dataset in real scene，SSHN-RS），包含3 004幅舰船图像，共计11 328个舷号字符，覆盖了多国、各类、水平、倾斜、背景简单、背景复杂、光线不佳和被遮挡的舰船舷号样本，是一个具有挑战性的数据集。基于SSHN-RS，开展舰船舷号检测识别研究，其主要难点在于：1）样本稀疏，模型容易过拟合；2）舷号字符分布密集，网络难以充分提取各字符特征；3）部分字符存在嵌套区域和相似区域，网络会识别出大量冗余结果。针对上述难点，提出了一种基于多视角渐进式上下文解耦的舰船舷号检测识别算法。方法 首先，引入一个固定中心和最大化面积的随机透视变换技术，在不增加样本数量的前提下扩充舷号姿态，实现了数据增广，提升了模型的泛化能力；其次，提出了一个渐进式上下文解耦技术，先通过依次擦除舷号各字符生成一系列新样本，再利用特征提取网络提取和融合各样本的多尺度特征，不仅减少字符上下文信息对特征学习的干扰，而且再次增广了数据；最后，在测试阶段，提出了一个掩码间扰动抑制技术，先根据预测结果采用与渐进式上下文解耦技术类似的方法生成新样本并重新进行预测，再引入一个1维非极大值抑制技术去除预测结果中错误的冗余字符，输出最佳检测识别结果，进一步优化网络性能。结果 在SSHN-RS上采用主流实例分割算法进行定性和定量评估。在定量评估上，本文算法舷号的检测精确率、召回率、F值和识别率分别可达0.985 4，0.957 6，0.971 3，0.901 8，均优于其他算法。相比指标排名第2的算法，分别提高了4.51%，3.45%，3.97%，8.83%；在定性评估上，本文算法更适合舰船舷号检测识别任务，检测识别性能更高。此外，本文算法可以泛化到其他实例分割算法中，以经典算法Mask RCNN（mask region based convolutional neural network）为例，加入本文算法各模块后，各指标分别提升了9.82%，6.04%，7.80%，6.73%。结论 本文算法可以解决舷号检测识别任务中因样本稀疏、舷号分布密集、部分字符存在嵌套和相似性带来的问题，在主观和客观上均取得了最先进的性能，并且具有通用性。SSHN-RS可通过https://github.com/Bingchuan897/SSHN-RS获取。;Objective Ship hull number detection and recognition can be as the key technologies for marine awareness. It is essential for the preservation of maritime rights and interests. However，it is required to data-driven researches in support of ship hull number detection and recognition. Therefore，we develop a sparse ship hull number dataset in real scene （SSHN-RS），which contains 3 004 images with a total of 11 328 hull numbers. The challenging SSHN-RS dataset is featured of ship hull numbers of various countries，hull numbers of various types，horizontal hull numbers，inclined hull numbers，hull numbers with complex background，hull numbers with simple background，poorly illuminated hull numbers and partially occluded hull numbers. We carry out SSHN-RS-related research on ship hull number detection and recognition. The main challenges are required to be resolved on three aspects as following：1）the hull number samples are sparse， which causes over-fitting of the network，2）the features of hull number are densely distributed，which is challenged to learn some of the hull number characteristics fully，and 3）some hull number have its nested areas and a high degree of similarity，which is costly for large number of redundant results. Method To resolve these problems mentioned above，we demonstrate a ship hull number detection and recognition algorithm in terms of multi-view progressive context decoupling. First，a random perspective transformation technology with fixed center and maximized area is illustrated. To realize data augmentation and improve the generalization ability of the model，the hull number spatial attitude is extended without increasing the number of samples. Second，a progressive context decoupling technology is proposed. A series of new samples are first generated by sequentially erasing each character of the hull number，and the feature extraction network is then used to extract and fuse the multi-scale features of each sample. It can reduce the influence of feature-contextual information on feature learning and rich the data expansion. It can improve the feature expression ability effectively as well. Finally，in the testing stage，to generate new samples，and inputs the new samples into the testing network for prediction， an inter mask disturbance suppression technology is first focused on a method similar to the progressive context decoupling technology. At the same time，a one-dimensional non-maximum suppression technology is introduced that it can mainly process the hull number recognition results of each sample twice. To suppress the redundant mask in the detection and recognition results effectively，it can make each character of the hull number of all samples correspond to a recognition result. For the testing network，the output has only a set of optimal results. However，there is hull number noisy recognition in some samples. Therefore，the hull number recognition results of all samples are added on the original images，and the second one-dimensional non-maximum suppression technology is performed and processed on it. It can suppress the noises in some samples and outputs a set of optimal results. The post-processing module can optimize the detection and recognition performance further. Result The comparative experiment is mainly carried out on SSHN-RS. First，we conduct evaluation analysis on the general instance segmentation algorithms. Multi-view progressive context decoupling-based detection precision，recall，f-score and recognition rate of the ship number detection and recognition algorithm can reach 0. 985 4，0. 957 6， 0. 971 3 and 0. 901 8，which are improved by 4. 51%，3. 45%，3. 97% and 8. 83% respectively compared to the secondranked method. Second，the ablation experiments on the ship hull number detection and recognition algorithm are carried out in terms of multi-view progressive context decoupling. The experimental results show that the performance of ship hull number detection and recognition is improved whether each technology is used solely or mixed. Finally，to validate its effectiveness，we apply some popular modules of the ship hull number detection and recognition algorithm based on multiview progressive context decoupling to other general instance segmentation algorithms. Taking the classic algorithm mask region based convolutional neural network（Mask RCNN）as an example，the indexes have been improved by 9. 82%， 6. 04%，7. 80% and 6. 73% of each after our algorithm module is added. Conclusion Our SSHN-RS contains rich and effective ship hull number information，which can provide data support for ship hull number detection and recognition. The experimental results show that the multi-view progressive context decoupling-based ship hull number detection and recognition algorithm can optimize the detection and recognition performance of hull numbers due to sparse samples，dense character distribution，nested areas，and characters-between high similarity. This algorithm can be generalized to other deep learning based segmentation algorithms. The benchmarks-relevant are provided as a basis for future research on ship hull number detection and recognition. The dataset is available at https://github.com/Bingchuan897/SSHN-RS.

Medical image instance segmentation: from candidate region to no candidate region

In recent years, the task of object detection and segmentation in medical image is the research hotspot and difficulty in the field of image processing. Instance segmentation provides instance-level labels for different objects belonging to the same class, so it is widely used in the field of medical image processing. In this paper, medical image instance segmentation was summarized from the following aspects: First, the basic principle of instance segmentation was described, the instance segmentation models were classified into three categories, the development context of the instance segmentation algorithm was displayed in two-dimensional space, and six classic model diagrams of instance segmentation were given. Second, from the perspective of the three models of two-stage instance segmentation, single-stage instance segmentation and three-dimensional (3D) instance segmentation, we summarized the ideas of the three types of models, discussed the advantages and disadvantages, and sorted out the latest developments. Third, the application status of instance segmentation in six medical images such as colon tissue image, cervical image, bone imaging image, pathological section image of gastric cancer, computed tomography (CT) image of lung nodule and X-ray image of breast was summarized. Fourth, the main challenges in the field of medical image instance segmentation were discussed and the future development direction was prospected. In this paper, the principle, models and characteristics of instance segmentation are systematically summarized, as well as the application of instance segmentation in the field of medical image processing, which is of positive guiding significance to the study of instance segmentation.

Instance segmentation using semi-supervised learning for fire recognition

Fire disaster brings enormous danger to the safety of human life and property, and it is important to identify the fire situation in time through image processing technology. The current instance segmentation algorithms suffer from problems such as inadequate fire images and annotations, low recognition accuracy, and slow inference speed for fire recognition tasks. In this paper, we propose a semi-supervised learning-based fire instance segmentation method based on deep learning image processing technology. We used a lightweight version of the SOLOv2 network and optimized the network structure to improve accuracy. We propose a semi-supervised learning method based on fire features. To reduce the negative impact of error pseudo-labels on the model training, the pseudo-labels are matched by the color and morphological features of flames and smoke at the pseudo-label generation stage, and some images are screened for strong image enhancement before entering the next round of training for the student model. We further exploit the potential of the model with a limited dataset and improve the model accuracy without affecting the inference efficiency of the model. Experiments show that our proposed algorithm can successfully improve the accuracy of fire instance segmentation with good inference speed.

Semi-Supervised Instance-Segmentation Model for Feature Transfer Based on Category Attention.

In the task of image instance segmentation, semi-supervised instance segmentation algorithms have received constant research attention over recent years. Among these algorithms, algorithms based on transfer learning are better than algorithms based on pseudo-label generation in terms of segmentation performance, but they can not make full use of the relevant characteristics of source tasks. To improve the accuracy of these algorithms, this work proposes a semi-supervised instance segmentation model AFT-Mask (attention-based feature transfer Mask R-CNN) based on category attention. The AFT-Mask model takes the result of object-classification prediction as "attention" to improve the performance of the feature-transfer module. In detail, we designed a migration-optimization module for connecting feature migration and classification prediction to enhance segmentation-prediction accuracy. To verify the validity of the AFT-Mask model, experiments were conducted on two types of datasets. Experimental results show that the AFT-Mask model can achieve effective knowledge transfer and improve the performance of the benchmark model on semi-supervised instance segmentation.

High-throughput phenotyping of nematode cysts.

The beet cyst nematode Heterodera schachtii is a plant pest responsible for crop loss on a global scale. Here, we introduce a high-throughput system based on computer vision that allows quantifying beet cyst nematode infestation and measuring phenotypic traits of cysts. After recording microscopic images of soil sample extracts in a standardized setting, an instance segmentation algorithm serves to detect nematode cysts in these images. In an evaluation using both ground truth samples with known cyst numbers and manually annotated images, the computer vision approach produced accurate nematode cyst counts, as well as accurate cyst segmentations. Based on such segmentations, cyst features could be computed that served to reveal phenotypical differences between nematode populations in different soils and in populations observed before and after the sugar beet planting period. The computer vision approach enables not only fast and precise cyst counting, but also phenotyping of cyst features under different conditions, providing the basis for high-throughput applications in agriculture and plant breeding research. Source code and annotated image data sets are freely available for scientific use.