Pixel-level Labels Research Articles

HybridMatch: Semi-Supervised Facial Landmark Detection via Hybrid Heatmap Representations

Facial landmark detection is an essential task in face-processing techniques. Traditional methods however require expensive pixel-level labels. Semi-supervised facial landmark detection has been explored as an alternative but previous approaches only focus on training-oriented issues (e.g., noisy pseudo-labels in the semi-supervised learning), neglecting task-oriented issues (i.e., the quantization error in the landmark detection). We argue that semi-supervised landmark detectors should resolve the two technical issues simultaneously. Through a simple experiment, we found that task- and training-oriented solutions may negatively influence each other, thus eliminating their negative interactions is important. To this end, we devise a new heatmap regression framework via hybrid representation, namely HybridMatch.We utilize both 1-D and 2-D heatmap representations. Here, the 1-D and 2-D heatmap help alleviate the task-oriented and the training-oriented issues, respectively. To exploit the advantages of our hybrid representation, we introduce curriculum learning; relying more on the 2-D heatmap at the early training stage and gradually increasing the effects of the 1-D heatmap. By resolving the two issues simultaneously, we can capture more precise landmark points than existing methods with only a few annotated data. Extensive experiments show that HybridMatch achieves state-of-the-art performance on three benchmark datasets, especially showing 26.3% NME improvement over the existing method in the 300-W <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">full</i> set at 5% data ratio. Surprisingly, our method records a comparable performance, 5.04 ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">challenging</i> set in the 300-W) to the fully-supervised facial landmark detector 5.03. The remarkable performance of HybridMatch shows its potential as a practical alternative to the fully-supervised model.

Co-Saliency Detection Guided by Group Weakly Supervised Learning

The detection results of many existing co-saliency detection methods are easily interfered by the unrelated salient objects, which have similar appearance characteristics to co-salient objects. Therefore, mining the inter-saliency cues which contain the common category information of multiple related images is the core of co-saliency detection. To address above concern, a novel group weakly supervised learning induced co-saliency detection (GWSCoSal) model is proposed in this paper. First of all, a novel group class activation maps (GCAM) network is constructed and trained through a group weakly supervised learning scheme, which adopts the common category of a group of related images as the ground truth. The GCAM produced by the trained GCAM network are considered as the inter-saliency cues, which can only highlight the regions covered by the objects with common category. Afterwards, the GCAM are integrated into a feature pyramid networks (FPN) based backbone trained by the pixel-level labels to infer the co-saliency maps. The group weakly supervised and the pixel-level learning are jointly implemented for end-to-end training of GWSCoSal model. The comprehensive comparisons with 13 state-of-the-art methods demonstrate that, our GWSCoSal model can detect the co-salient objects more accurately under the condition of being interfered by the similar unrelated salient objects, and the overall performance of which has achieved the level of state-of-the-art methods. The ablation study of our GWSCoSal model validates the effectiveness of proposed GCAM network.

Weakly supervised semantic segmentation via Graph RecalibratiOn with Scaling Weight uNit

Weakly supervised semantic segmentation (WSSS) has garnered considerable attention for its efficacy in generating pixel-level labels using weak labels. Class activation maps (CAMs) are utilized by WSSS to generate pseudo-masks from image-level labels. However, these CAMs primarily focus on the most discriminative features of an object, while less discriminative features may be ignored or unidentified. Due to co-occurring pixels, it may also be impossible to distinguish between the foreground and background. In this paper, we propose a method referred to as Graph RecalibratiOn with Scaling Weight uNit (GROWN) to address these challenges. It illustrates the relation between local and global features by utilizing graph structure. Adaptively representing the image’s semantic features is possible by scaling weights that aggregate contextual features. The proposed method successfully captures long-range dependencies and extracts contextual features to improve the pseudo-mask quality. As a result, the proposed method can predict pixel-level labels effectively. The datasets PASCAL VOC 2012 and MS COCO were utilized in the experiments. GROWN outperforms state-of-the-art WSSS methods that employ image-level labels, as demonstrated by the results.

Improving Synthetic to Realistic Semantic Segmentation With Parallel Generative Ensembles for Autonomous Urban Driving

Semantic segmentation is paramount for autonomous vehicles to have a deeper understanding of the surrounding traffic environment and enhance safety. Deep neural networks (DNNs) have achieved remarkable performances in semantic segmentation. However, training such a DNN requires a large amount of labeled data at the pixel level. In practice, it is a labor-intensive task to manually annotate dense pixel-level labels. To tackle the problem associated with a small amount of labeled data, deep domain adaptation (DDA) methods have recently been developed to examine the use of synthetic driving scenes so as to significantly reduce the manual annotation cost. Despite remarkable advances, these methods, unfortunately, suffer from the generalizability problem that fails to provide a holistic representation of the mapping from the source image domain to the target image domain. In this article, we, therefore, develop a novel ensembled DDA to train models with different upsampling strategies, discrepancy, and segmentation loss functions. The models are, therefore, complementary with each other to achieve better generalization in the target image domain. Such a design does not only improves the adapted semantic segmentation performance but also strengthens the model reliability and robustness. Extensive experimental results demonstrate the superiorities of our approach over several state-of-the-art methods.

Weakly-supervised detection of AMD-related lesions in color fundus images using explainable deep learning

Background and ObjectivesAge-related macular degeneration (AMD) is a degenerative disorder affecting the macula, a key area of the retina for visual acuity. Nowadays, AMD is the most frequent cause of blindness in developed countries. Although some promising treatments have been proposed that effectively slow down its development, their effectiveness significantly diminishes in the advanced stages. This emphasizes the importance of large-scale screening programs for early detection. Nevertheless, implementing such programs for a disease like AMD is usually unfeasible, since the population at risk is large and the diagnosis is challenging. For the characterization of the disease, clinicians have to identify and localize certain retinal lesions. All this motivates the development of automatic diagnostic methods. In this sense, several works have achieved highly positive results for AMD detection using convolutional neural networks (CNNs). However, none of them incorporates explainability mechanisms linking the diagnosis to its related lesions to help clinicians to better understand the decisions of the models. This is specially relevant, since the absence of such mechanisms limits the application of automatic methods in the clinical practice. In that regard, we propose an explainable deep learning approach for the diagnosis of AMD via the joint identification of its associated retinal lesions. MethodsIn our proposal, a CNN with a custom architectural setting is trained end-to-end for the joint identification of AMD and its associated retinal lesions. With the proposed setting, the lesion identification is directly derived from independent lesion activation maps; then, the diagnosis is obtained from the identified lesions. The training is performed end-to-end using image-level labels. Thus, lesion-specific activation maps are learned in a weakly-supervised manner. The provided lesion information is of high clinical interest, as it allows clinicians to assess the developmental stage of the disease. Additionally, the proposed approach allows to explain the diagnosis obtained by the models directly from the identified lesions and their corresponding activation maps. The training data necessary for the approach can be obtained without much extra work on the part of clinicians, since the lesion information is habitually present in medical records. This is an important advantage over other methods, including fully-supervised lesion segmentation methods, which require pixel-level labels whose acquisition is arduous. ResultsThe experiments conducted in 4 different datasets demonstrate that the proposed approach is able to identify AMD and its associated lesions with satisfactory performance. Moreover, the evaluation of the lesion activation maps shows that the models trained using the proposed approach are able to identify the pathological areas within the image and, in most cases, to correctly determine to which lesion they correspond. ConclusionsThe proposed approach provides meaningful information—lesion identification and lesion activation maps—that conveniently explains and complements the diagnosis, and is of particular interest to clinicians for the diagnostic process. Moreover, the data needed to train the networks using the proposed approach is commonly easy to obtain, what represents an important advantage in fields with particularly scarce data, such as medical imaging.

Mixed-UNet: Refined class activation mapping for weakly-supervised semantic segmentation with multi-scale inference

Deep learning techniques have shown great potential in medical image processing, particularly through accurate and reliable image segmentation on magnetic resonance imaging (MRI) scans or computed tomography (CT) scans, which allow the localization and diagnosis of lesions. However, training these segmentation models requires a large number of manually annotated pixel-level labels, which are time-consuming and labor-intensive, in contrast to image-level labels that are easier to obtain. It is imperative to resolve this problem through weakly-supervised semantic segmentation models using image-level labels as supervision since it can significantly reduce human annotation efforts. Most of the advanced solutions exploit class activation mapping (CAM). However, the original CAMs rarely capture the precise boundaries of lesions. In this study, we propose the strategy of multi-scale inference to refine CAMs by reducing the detail loss in single-scale reasoning. For segmentation, we develop a novel model named Mixed-UNet, which has two parallel branches in the decoding phase. The results can be obtained after fusing the extracted features from two branches. We evaluate the designed Mixed-UNet against several prevalent deep learning-based segmentation approaches on our dataset collected from the local hospital and public datasets. The validation results demonstrate that our model surpasses available methods under the same supervision level in the segmentation of various lesions from brain imaging.

Segmentation-based Decision Networks for Steel Surface Defect Detection

&lt;p&gt;With the advent of the Industrial 4.0 era, deep learning has been continuously applied to the task of surface defect detection, and effective progress has been made. However, the limited number of training samples and high labelling costs are considerable obstacles to the vigorous development of this task. Thus, we explore the use of different numbers of labels with various accuracies during training to achieve the maximum detection accuracy with the lowest cost. Our proposed method includes improved segmentation and decision networks. An attention mechanism is integrated into the segmentation subnetwork. Moreover, atrous convolutions are used in the segmentation and decision subnetworks. In addition, the original loss function is improved. Several experiments are carried out on the Severstal Steel Defect dataset collected in Germany, and the results show that each component improves the detection accuracy by 1% to 2%. Finally, when we add an appropriate number of pixel-level labels in the weakly supervised learning mode, the detection accuracy reaches that of the fully supervised mode with a significantly reduced annotation cost.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

High-Performance Pixel-Level Grasp Detection Based on Adaptive Grasping and Grasp-Aware Network

Machine vision-based planar grasping detection is challenging due to uncertainty about object shape, pose, size, etc. Previous methods mostly focus on predicting discrete gripper configurations, and may miss some ground-truth grasp postures. In this article, a pixel-level grasp detection method is proposed, which uses deep neural network to predict pixel-level gripper configurations on RGB images. First, a novel oriented arrow representation model (OAR-model) is introduced to represent the gripper configuration of parallel-jaw and three-fingered gripper, which can partly improve the applicability to different grippers. Then, the adaptive grasping attribute model is proposed to adaptively represent the grasping attribute of objects, for resolving angle conflicts in training and simplifying pixel-level labeling. Lastly, the adaptive feature fusion and grasp-aware network (AFFGA-Net) is proposed to predict pixel-level OAR-models on RGB images. AFFGA-Net improves the robustness in unstructured scenarios by using hybrid atrous spatial pyramid and adaptive decoder connected in sequence. On the public Cornell dataset and actual objects, our structure achieves 99.09% and 98.0% grasp detection accuracy, respectively. In over 2400 robotic grasp trials, our structure achieves an average success rate of 98.77% in single-object scenarios and 93.69% in cluttered scenarios. Moreover, AFFGA-Net completes a grasp detection pipeline within 15 ms.

Anti-Interference From Noisy Labels: Mean-Teacher-Assisted Confident Learning for Medical Image Segmentation.

Manually segmenting medical images is expertise-demanding, time-consuming and laborious. Acquiring massive high-quality labeled data from experts is often infeasible. Unfortunately, without sufficient high-quality pixel-level labels, the usual data-driven learning-based segmentation methods often struggle with deficient training. As a result, we are often forced to collect additional labeled data from multiple sources with varying label qualities. However, directly introducing additional data with low-quality noisy labels may mislead the network training and undesirably offset the efficacy provided by those high-quality labels. To address this issue, we propose a Mean-Teacher-assisted Confident Learning (MTCL) framework constructed by a teacher-student architecture and a label self-denoising process to robustly learn segmentation from a small set of high-quality labeled data and plentiful low-quality noisy labeled data. Particularly, such a synergistic framework is capable of simultaneously and robustly exploiting (i) the additional dark knowledge inside the images of low-quality labeled set via perturbation-based unsupervised consistency, and (ii) the productive information of their low-quality noisy labels via explicit label refinement. Comprehensive experiments on left atrium segmentation with simulated noisy labels and hepatic and retinal vessel segmentation with real-world noisy labels demonstrate the superior segmentation performance of our approach as well as its effectiveness on label denoising.

Global weighted average pooling network with multilevel feature fusion for weakly supervised brain tumor segmentation

Medical image segmentation plays a vital role in computer-aided diagnosis and intelligent medical treatment. It can preprocess medical images to help doctors better diagnose diseases. Class activation map (CAM) is an important technology in weakly supervised segmentation, which can achieve image segmentation without pixel-level label training. This technology can well meet the needs of medical image segmentation. However, CAM obtaining is still unperfect due to global average pooling (GAP). GAP will cause important and non-important regions to be given equal attention during the training process. So, CAM cannot demarcate the boundary of the target regions well. In order to solve this problem, a global weighted average pooling network fusing the grayscale information of medical images is proposed. The proposed network can solve the problem that GAP has the same concern for important regions and non-important regions of the feature map, because the different weights can be learned for different positions of the feature map before the GAP in the proposed model. At the same time, because of the grayscale difference between the tumor area and the non-tumor area in the brain tumor image, the low-level grayscale information of the medical image is fused with the high-level semantic information extracted by the network to learn the weights. This operation gives full play to the advantages of feature maps of different levels. The experiment results on the popular medical image dataset BraTS2019 show that the proposed method can well improve the performance of CAM and help CAM fit the boundaries of objects. Meanwhile, in the DSC evaluation, the proposed method achieves a score of 64.1%, which is a 4.6% improvement over a recent research method.

Weakly supervised skin lesion segmentation based on spot‐seeds guided optimal regions

Automatic skin lesion segmentation is the most critical and relevant task in computer-aided skin cancer diagnosis. Methods based on convolutional neural networks (CNNs) are mainly used in current skin lesion segmentation. The requirement of huge pixel-level labels is a significant obstacle to achieve semantic segmentation of skin lesion by CNNs. In this paper, a novel weakly supervised framework for skin lesion segmentation is presented, which generates high-quality pixel-level annotations and optimizes the segmentation network. A hierarchical image segmentation algorithm can predict a boundary map for training images. Then, the optimal regions of candidate hierarchical levels are selected. Afterward, Superpixels-CRF built on the optimal regions is guided by spot seeds to propagate information from spot seeds to unlabeled regions, resulting in high-quality pixel-level annotations. Using these high-quality pixel-level annotations, a segmentation network can be trained and segmentation masks can be predicted. To iteratively optimize the segmentation network, the predicted segmentation masks are refined and the segmentation network are retrained. Comparative experiments demonstrate that the proposed segmentation framework reduces the gap between weakly and fully supervised skin lesion segmentation methods, and achieves state-of-the-art performance while reducing human labeling efforts.

PGNet: Positioning Guidance Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Images

Semantic segmentation of very-high-resolution (VHR) remote sensing images plays an important role in the intelligent interpretation of remote sensing since it predicts pixel-level labels to the images. Although many semantic segmentation methods of VHR remote sensing images have emerged recently and achieved good results, it is still a challenging task because the objects of VHR remote sensing images show large intra-class and small inter-class variations, and their size varies in a large range. Therefore, we proposed a novel semantic segmentation framework for VHR remote sensing images, called Positioning Guidance Network (PGNet), which consists of the feature extractor, a positioning guiding module (PGM), and a self-multiscale collection module (SMCM). First, the PGM can extract long-range dependence and global context information with the help of the transformer architecture and effectively transfer them to each pyramid-level feature, thus effectively improving the segmentation effectiveness between different semantic objects. Secondly, the SMCM we designed can effectively extract multi-scale information and generate high-resolution feature maps with high-level semantic information, thus helping to segment objects in small and varying sizes. Without bells and whistles, the mIoU scores of the proposed PGNet on the iSAID dataset and ISPRS Vaihingn dataset are 1.49% and 2.40% higher than FactSeg, respectively.

RGB-D mutual guidance for semi-supervised defocus blur detection

Defocus blur detection (DBD) aims to detect and locate defocus features in visual scenes. While available fully supervised DBD methods improve detection accuracy, they rely on large-scale handcrafted pixel-level labels and single-mode images, which makes them expensive and error-prone. In this work, we explore the use of depth information in a semi-supervised DBD method for the first time. Different from previous approaches, we generate a pair of reversible defocused homogeneous regions in weakly supervised mutual guidance networks (MGMs) to provide weak semantic guidance for this task. In a strongly supervised mutual attention network, depth information and RGB features are used to learn the defocus blur homogeneous region from the ground truth (GT). Meanwhile, depth information is extracted using a depth estimation network to guide the defocus location and provide a strong prior for the weakly supervised part. The experimental results show that our network outperform the available fully supervised methods in DBD, and provides new inspiration for research on semi-supervised robust and RGB-D multi-modal DBD.

Semi-supervised COVID-19 CT image segmentation using deep generative models

BackgroundA recurring problem in image segmentation is a lack of labelled data. This problem is especially acute in the segmentation of lung computed tomography (CT) of patients with Coronavirus Disease 2019 (COVID-19). The reason for this is simple: the disease has not been prevalent long enough to generate a great number of labels. Semi-supervised learning promises a way to learn from data that is unlabelled and has seen tremendous advancements in recent years. However, due to the complexity of its label space, those advancements cannot be applied to image segmentation. That being said, it is this same complexity that makes it extremely expensive to obtain pixel-level labels, making semi-supervised learning all the more appealing. This study seeks to bridge this gap by proposing a novel model that utilizes the image segmentation abilities of deep convolution networks and the semi-supervised learning abilities of generative models for chest CT images of patients with the COVID-19.ResultsWe propose a novel generative model called the shared variational autoencoder (SVAE). The SVAE utilizes a five-layer deep hierarchy of latent variables and deep convolutional mappings between them, resulting in a generative model that is well suited for lung CT images. Then, we add a novel component to the final layer of the SVAE which forces the model to reconstruct the input image using a segmentation that must match the ground truth segmentation whenever it is present. We name this final model StitchNet.ConclusionWe compare StitchNet to other image segmentation models on a high-quality dataset of CT images from COVID-19 patients. We show that our model has comparable performance to the other segmentation models. We also explore the potential limitations and advantages in our proposed algorithm and propose some potential future research directions for this challenging issue.

Classification of myocardial fibrosis in DE-MRI based on semi-supervised semantic segmentation and dual attention mechanism

ObjectiveIt is essential to utilize cardiac delayed-enhanced magnetic resonance imaging (DE-MRI) to diagnose cardiovascular disease. By segmenting myocardium DE-MRI images, it provides critical information for the evaluation and treatment of myocardial infarction. As a consequence, it is vital to investigate the segmentation and classification technique of myocardial DE-MRI. MethodsFirstly, an end-to-end minimally supervised and semi-supervised semantic DE-MRI myocardial fibrosis segmentation framework is proposed, which combines image classification and semantic segmentation branches based on the self-attention mechanism. Following that, a residual hole network fused with the dual attention mechanism was built, and a double attention metabolic pathway classification method for cardiac fibrosis in DE-MRI images was developed. ResultsBy adding pixel-level labels to an extra 40 training images, the segmentation model may enhance semantic segmentation performance by 2.6 percent (from 61.2 percent to 63.8 percent). When the number of pixel-level labels is increased to 80, semi-supervised feature extraction increases by 4.7 percent when compared to weakly guided semantic segmentation. Adding an attention mechanism to the critical network DRN (Deep Residual Network) can increase the classifier's performance by a small amount. Experiments revealed that the models worked effectively. ConclusionThis paper investigates the segmentation and classification of cardiac fibrosis in DE-MRI data using a semi-supervised semantic segmentation and dual attention mechanism, dealing with the issue that existing segmentation algorithms have difficulty segmenting myocardial fibrosis tissue. In the future, we can consider optimizing the design of the attention module to reduce the module computation.

Reliable Label-Supervised Pixel Attention Mechanism for Weakly Supervised Building Segmentation in UAV Imagery

Building segmentation for Unmanned Aerial Vehicle (UAV) imagery usually requires pixel-level labels, which are time-consuming and expensive to collect. Weakly supervised semantic segmentation methods for image-level labeling have recently achieved promising performance in natural scenes, but there have been few studies on UAV remote sensing imagery. In this paper, we propose a reliable label-supervised pixel attention mechanism for building segmentation in UAV imagery. Our method is based on the class activation map. However, classification networks tend to capture discriminative parts of the object and are insensitive to over-activation; therefore, class activation maps cannot directly guide segmentation network training. To overcome these challenges, we first design a Pixel Attention Module that captures rich contextual relationships, which can further mine more discriminative regions, in order to obtain a modified class activation map. Then, we use the initial seeds generated by the classification network to synthesize reliable labels. Finally, we design a reliable label loss, which is defined as the sum of the pixel-level differences between the reliable labels and the modified class activation map. Notably, the reliable label loss can handle over-activation. The preceding steps can significantly improve the quality of the pseudo-labels. Experiments on our home-made UAV data set indicate that our method can achieve 88.8% mIoU on the test set, outperforming previous state-of-the-art weakly supervised methods.

Confidence-and-Refinement Adaptation Model for Cross-Domain Semantic Segmentation

With the rapid development of convolutional neural networks (CNNs), significant progress has been achieved in semantic segmentation. Despite the great success, such deep learning approaches require large scale real-world datasets with pixel-level annotations. However, considering that pixel-level labeling of semantics is extremely laborious, many researchers turn to utilize synthetic data with free annotations. But due to the clear domain gap, the segmentation model trained with the synthetic images tends to perform poorly on the real-world datasets. Unsupervised domain adaptation (UDA) for semantic segmentation recently gains an increasing research attention, which aims at alleviating the domain discrepancy. Existing methods in this scope either simply align features or the outputs across the source and target domains or have to deal with the complex image processing and post-processing problems. In this work, we propose a novel multi-level UDA model named Confidence-and-Refinement Adaptation Model (CRAM), which contains a confidence-aware entropy alignment (CEA) module and a style feature alignment (SFA) module. Through CEA, the adaptation is done locally via adversarial learning in the output space, making the segmentation model pay attention to the high-confident predictions. Furthermore, to enhance the model transfer in the shallow feature space, the SFA module is applied to minimize the appearance gap across domains. Experiments on two challenging UDA benchmarks “GTA5-to-Cityscapes” and “SYNTHIA-to-Cityscapes” demonstrate the effectiveness of CRAM. We achieve comparable performance with the existing state-of-the-art works with advantages in simplicity and convergence speed.

Joint weakly and fully supervised learning for surface defect segmentation from images

Surface defect segmentation from industrial images based on deep learning has been rapidly developed in recent years. However, the related methods depend heavily on a large number of costly pixel-level labels. This paper introduces a method that uses abundant weakly supervised images and a few fully supervised images to reduce the labeling expense and obtain an excellent segmentation result. First, we propose a novel weakly supervised semantic segmentation method, the multistage squeeze-and-excitation (SE)-augmented adaptive Lp norm class activation map (multistage SALN-CAM), which adopts both position-level and channel-level attention mechanisms. Based on multistage SALN-CAM, a Siamese network is designed to jointly train fully supervised and weakly supervised images simultaneously with the help of an auxiliary cross-field and cross-attention (CFCA) network that maps features from the classification field to the segmentation field with a cross-attention mechanism. Finally, we retrain a fully supervised segmentation model using images with pixel-level labels or pseudo-pixel-level labels generated by the Siamese network. In our experiments, based on the Severstal: Steel Defect Detection dataset, our method, in which 25% of the images (2512) have pixel-level labels and 75% (7544) of the images have class-level labels, obtains an mDice of 94.01% on the test set. This result completely surpasses that of the fully supervised method, which uses 4775 images; thus, our method reduces the labeling expense by 37%.

Deep Neural Networks Learn Meta-Structures from Noisy Labels in Semantic Segmentation

How deep neural networks (DNNs) learn from noisy labels has been studied extensively in image classification but much less in image segmentation. So far, our understanding of the learning behavior of DNNs trained by noisy segmentation labels remains limited. In this study, we address this deficiency in both binary segmentation of biological microscopy images and multi-class segmentation of natural images. We generate extremely noisy labels by randomly sampling a small fraction (e.g., 10%) or flipping a large fraction (e.g., 90%) of the ground truth labels. When trained with these noisy labels, DNNs provide largely the same segmentation performance as trained by the original ground truth. This indicates that DNNs learn structures hidden in labels rather than pixel-level labels per se in their supervised training for semantic segmentation. We refer to these hidden structures in labels as meta-structures. When DNNs are trained by labels with different perturbations to the meta-structure, we find consistent degradation in their segmentation performance. In contrast, incorporation of meta-structure information substantially improves performance of an unsupervised segmentation model developed for binary semantic segmentation. We define meta-structures mathematically as spatial density distributions and show both theoretically and experimentally how this formulation explains key observed learning behavior of DNNs.

A pixel-level coarse-to-fine image segmentation labelling algorithm

Fine segmentation labelling tasks are time consuming and typically require a great deal of manual labor. This paper presents a novel method for efficiently creating pixel-level fine segmentation labelling that significantly reduces the amount of necessary human labor. The proposed method utilizes easily produced multiple and complementary coarse labels to build a complete fine label via supervised learning. The primary label among the coarse labels is the manual label, which is produced with simple contours or bounding boxes that roughly encompass an object. All others coarse labels are complementary and are generated automatically using existing algorithms. Fine labels can be rapidly created during the supervised learning of such coarse labels. In the experimental study, the proposed technique achieved a fine label IOU (intersection of union) of 92% in segmenting our newly constructed bean field dataset. The proposed method also achieved 95% and 92% mean IOU when tested on publicly available agricultural CVPPP and CWFID datasets, respectively. Our proposed method of segmentation also achieved a mean IOU of 81% when it was tested on our newly constructed paprika disease dataset, which includes multiple categories.