Foreground Classes Research Articles

Novel color space representation extracted by NMF to segment a color image

This paper considers the task of separating pixels in color image into background and foreground classes. Using the machine learning technique known as Nonnegative Matrix Factorization, data pertaining to different color channels – selected by color spaces – are combined, and a novel space representation is extracted.The novel representation of the image includes additional information, namely “metacolor”, which could be related to foreground and background and adopted to improve binary segmentation of the investigated image. In both qualitative and quantitative experiments, the use of novel color space representation produces some improvements in the binary segmentation results when it compared to those obtained applying common simpler thresholding algorithms directly to the original image.

Weakly supervised instance segmentation via class double-activation maps and boundary localization

Weakly supervised instance segmentation based on image-level class labels has recently gained much attention, in which the primary key step is to generate the pseudo labels based on class activation maps (CAMs). Most methods adopt binary cross-entropy (BCE) loss to train the classification model. However, since BCE loss is not class mutually exclusive, activations among classes occur independently. Thus, not only do foreground classes are wrongly activated as background, but also incorrect activations among confusing classes are occurred in the foreground. To solve this problem, we propose the Class Double-Activation Map, called Double-CAM. Firstly, the vanilla CAM is extracted from the multi-label classifier and then fused with the output feature map of backbone. The enhanced feature map of each class is fed into the single-label classification branch with softmax cross-entropy (SCE) loss and entropy minimization module, from which the more accurate Double-CAM is extracted. It refines the vanilla CAM to improve the quality of pseudo labels. Secondly, to mine object edge cues from Double-CAM, we propose the Boundary Localization (BL) module to synthesize boundary annotations, so as to provide constraints for label propagation more explicitly without adding additional supervision. The quality of pseudo masks is also improved substantially with the addition of BL module. Finally, the generated pseudo labels are used to train fully supervised instance segmentation networks. The evaluations on VOC and COCO datasets show that our method achieves excellent performance, outperforming mainstream weakly supervised segmentation methods at the same supervisory level, even those that depend on stronger supervision.

An Analysis of Loss Functions for Heavily Imbalanced Lesion Segmentation.

Heavily imbalanced datasets are common in lesion segmentation. Specifically, the lesions usually comprise less than 5% of the whole image volume when dealing with brain MRI. A common solution when training with a limited dataset is the use of specific loss functions that rebalance the effect of background and foreground voxels. These approaches are usually evaluated running a single cross-validation split without taking into account other possible random aspects that might affect the true improvement of the final metric (i.e., random weight initialisation or random shuffling). Furthermore, the evolution of the effect of the loss on the heavily imbalanced class is usually not analysed during the training phase. In this work, we present an analysis of different common loss metrics during training on public datasets dealing with brain lesion segmentation in heavy imbalanced datasets. In order to limit the effect of hyperparameter tuning and architecture, we chose a 3D Unet architecture due to its ability to provide good performance on different segmentation applications. We evaluated this framework on two public datasets and we observed that weighted losses have a similar performance on average, even though heavily weighting the gradient of the foreground class gives better performance in terms of true positive segmentation.

Dual Branch Multi-Level Semantic Learning for Few-Shot Segmentation.

Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples in support images. Although progress has been made recently by combining prototype-based metric learning, existing methods still face two main challenges. First, various intra-class objects between the support and query images or semantically similar inter-class objects can seriously harm the segmentation performance due to their poor feature representations. Second, the latent novel classes are treated as the background in most methods, leading to a learning bias, whereby these novel classes are difficult to correctly segment as foreground. To solve these problems, we propose a dual-branch learning method. The class-specific branch encourages representations of objects to be more distinguishable by increasing the inter-class distance while decreasing the intra-class distance. In parallel, the class-agnostic branch focuses on minimizing the foreground class feature distribution and maximizing the features between the foreground and background, thus increasing the generalizability to novel classes in the test stage. Furthermore, to obtain more representative features, pixel-level and prototype-level semantic learning are both involved in the two branches. The method is evaluated on PASCAL- 5i 1 -shot, PASCAL- 5i 5 -shot, COCO- 20i 1 -shot, and COCO- 20i 5 -shot, and extensive experiments show that our approach is effective for few-shot semantic segmentation despite its simplicity.

A Multi-View Integrated Ensemble for the Background Discrimination of Semi-Supervised Semantic Segmentation

The key to semi-supervised semantic segmentation is to assign the appropriate pseudo-label to the pixels of unlabeled images. Recently, various approaches to consistency-based training and the filtering of reliable pseudo-labels have shown remarkable results. Nonetheless, there are still issues to be addressed. We find that recent approaches have specific problems in common. In pseudo-labels for training unlabeled images, we confirm that false foreground class pseudo-labels are mostly caused by background class confusion, not confusion between different foreground classes. To solve this problem, we propose a foreground and background discrimination model for semi-supervised semantic segmentation. Our proposed model is trained using a novel approach called multi-view integrated ensemble (MVIE) via output perturbation. Experimental results in various partition protocols show that our approach outperforms the existing state of the art (SOTA) in binary prediction on unlabeled data, and the segmentation model trained with the help of our model outperforms existing models.

Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation.

This study investigates barely-supervised medical image segmentation where only few labeled data, i.e., single-digit cases are available. We observe the key limitation of the existing state-of-the-art semi-supervised solution cross pseudo supervision is the unsatisfactory precision of foreground classes, leading to a degenerated result under barely-supervised learning. In this paper, we propose a novel Compete-to-Win method (ComWin) to enhance the pseudo label quality. In contrast to directly using one model's predictions as pseudo labels, our key idea is that high-quality pseudo labels should be generated by comparing multiple confidence maps produced by different networks to select the most confident one (a compete-to-win strategy). To further refine pseudo labels at near-boundary areas, an enhanced version of ComWin, namely, ComWin+, is proposed by integrating a boundary-aware enhancement module. Experiments show that our method can achieve the best performance on three public medical image datasets for cardiac structure segmentation, pancreas segmentation and colon tumor segmentation, respectively. The source code is now available at https://github.com/Huiimin5/comwin.

Addressing class imbalance in deep learning for acoustic target classification

AbstractAcoustic surveys provide important data for fisheries management. During the surveys, ship-mounted echo sounders send acoustic signals into the water and measure the strength of the reflection, so-called backscatter. Acoustic target classification (ATC) aims to identify backscatter signals by categorizing them into specific groups, e.g. sandeel, mackerel, and background (as bottom and plankton). Convolutional neural networks typically perform well for ATC but fail in cases where the background class is similar to the foreground class. In this study, we discuss how to address the challenge of class imbalance in the sampling of training and validation data for deep convolutional neural networks. The proposed strategy seeks to equally sample areas containing all different classes while prioritizing background data that have similar characteristics to the foreground class. We investigate the performance of the proposed sampling methodology for ATC using a previously published deep convolutional neural network architecture on sandeel data. Our results demonstrate that utilizing this approach enables accurate target classification even when dealing with imbalanced data. This is particularly relevant for pixel-wise semantic segmentation tasks conducted on extensive datasets. The proposed methodology utilizes state-of-the-art deep learning techniques and ensures a systematic approach to data balancing, avoiding ad hoc methods.

An Object Detection Method Using Probability Maps for Instance Segmentation to Mask Background

This paper proposes a two-step detector called segmented object detection, whose performance is improved by masking the background region. Previous single-stage object detection methods suffer from the problem of imbalance between foreground and background classes, where the background occupies more regions in the image than the foreground. Thus, the loss from the background is firmly incorporated into the training. RetinaNet addresses this problem with Focal Loss, which focuses on foreground loss. Therefore, we propose a method that generates probability maps using instance segmentation in the first step and feeds back the generated maps as background masks in the second step as prior knowledge to reduce the influence of the background and enhance foreground training. We confirm that the detector can improve the accuracy by adding instance segmentation information to both the input and output rather than only to the output results. On the Cityscapes dataset, our method outperforms the state-of-the-art methods.

“Working women demand peace and food”: Gender and class in the East London Federation of Suffragettes' food politics

AbstractIn this article, we foreground how gender, class and feminism underpin the aims and modes of mobilization of the food politics of a British first wave suffragette organization, the East London Federation of Suffragettes. Our discussion shows how upper and middle‐class suffragettes excluded working‐class women and marginalized their political gendered classed interests, raising questions about feminist cross‐class solidarities today. We focus on three of East London Federation of the Suffragettes quite different modes of mobilization: embodied protests, radical welfare community organizing and food protest writing, all of which foreground class politics. In discussing three quite distinct modes of mobilization, we highlight the rich package of strategies they created and the different classed identities and struggles in these. We show the diversity of gendered and classed social roles around which the women politicked, as workers, mothers, housewives and consumers. The historical focus enables us to “see” the activities and identifications over time to understand and map their range and dynamics. Moreover, suffragette politics have a “longtail” and continue to influence feminist politics and thinking, but the working‐class mobilizing and food politics have been much less recognized and yet offer potential insights for feminist activism, including the cost‐of‐living crisis today.

A Foreground Prototype-Based One-Shot Segmentation of Brain Tumors.

The potential for enhancing brain tumor segmentation with few-shot learning is enormous. While several deep learning networks (DNNs) show promising segmentation results, they all take a substantial amount of training data in order to yield appropriate results. Moreover, a prominent problem for most of these models is to perform well in unseen classes. To overcome these challenges, we propose a one-shot learning model to segment brain tumors on brain magnetic resonance images (MRI) based on a single prototype similarity score. With the use of recently developed few-shot learning techniques, where training and testing are carried out utilizing support and query sets of images, we attempt to acquire a definitive tumor region by focusing on slices containing foreground classes. It is unlike other recent DNNs that employed the entire set of images. The training of this model is carried out in an iterative manner where in each iteration, random slices containing foreground classes of randomly sampled data are selected as the query set, along with a different random slice from the same sample as the support set. In order to differentiate query images from class prototypes, we used a metric learning-based approach based on non-parametric thresholds. We employed the multimodal Brain Tumor Image Segmentation (BraTS) 2021 dataset with 60 training images and 350 testing images. The effectiveness of the model is evaluated using the mean dice score and mean IoU score. The experimental results provided a dice score of 83.42 which was greater than other works in the literature. Additionally, the proposed one-shot segmentation model outperforms the conventional methods in terms of computational time, memory usage, and the number of data.

Query semantic reconstruction for background in few-shot segmentation

Few-shot segmentation (FSS) aims to segment unseen classes using a few annotated samples. Typically, a prototype representing the foreground class is extracted from annotated support image(s) and is matched to features representing each pixel in the query image. However, models learnt in this way are insufficiently discriminatory, and often produce false positives: misclassifying background pixels as foreground. Some FSS methods try to address this issue by using the background in the support image(s) to help identify the background in the query image. However, the backgrounds of these images are often quite distinct, and hence, the support image background information is uninformative. This article proposes a method, QSR, that extracts the background from the query image itself, and as a result is better able to discriminate between foreground and background features in the query image. This is achieved by modifying the training process to associate prototypes with class labels including known classes from the training data and latent classes representing unknown background objects. This class information is then used to extract a background prototype from the query image. To successfully associate prototypes with class labels and extract a background prototype that is capable of predicting a mask for the background regions of the image, the machinery for extracting and using foreground prototypes is induced to become more discriminative between different classes. Experiments achieves state-of-the-art results for both 1-shot and 5-shot FSS on the PASCAL-5i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5^{i}$$\\end{document} and COCO-20i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$20^{i}$$\\end{document} dataset. As QSR operates only during training, results are produced with no extra computational complexity during testing.

A novel smoke detection algorithm based on improved mixed Gaussian and YOLOv5 for textile workshop environments

AbstractThere are a lot of flammable materials in the textile workshop, and once a fire occurs, it will cause property damage and casualties. At present, smoke detection in textile workshops mainly relies on temperature‐sensing smoke sensors with low detection rate and poor real‐time performance, which cannot meet the task of smoke detection in complex environments. Therefore, this paper proposes an improved mixed Gaussian and YOLOv5 smoke detection algorithm for textile workshops. In order to reduce the interference of static background in smoke detection, an improved gaussian mixture algorithm is used to extract suspected smoke areas in video by using the dynamic characteristics of smoke. Then, an adaptive attention module is added to the feature pyramid infrastructure of the YOLOv5 target detection network to improve the multi‐scale target recognition ability. In addition, the focal loss function is used to reduce the impact of background and foreground class imbalances on the detection results. The experimental results show that the detection accuracy of the proposed method is 94.7%, and the average detection speed is 66.7 FPS. By comparing with the existing state‐of‐the‐art algorithms, the detection capability of this method has been significantly improved. At the same time, it has high real‐time performance and detection accuracy in smoke detection in textile workshops.

FI-WSOD: Foreground Information Guided Weakly Supervised Object Detection

Existing solutions for weakly supervised object detection (WSOD) generally follow the multiple instance learning (MIL) paradigm to formulate WSOD as a multi-class classification problem over a set of region proposals. However, without the supervision signal of ground-truth boxes, the training objective of multi-class classification makes the detectors devote main efforts to finding the most common pattern of each class, as the common pattern is always the most discriminative evidence for classification. In addition, although learning from distinguishing multiple foreground classes, the detectors can still ignore to differentiate foreground regions from the background ones, which causes false alarm in prediction. These two points account for the limited localization capability of MIL-based WSOD methods. To this end, we propose foreground information guided WSOD (FI-WSOD), a novel framework that introduces an extra foreground-background binary classification (F-BBC) sub-task to the original MIL-based WSOD paradigm. Particularly, the proposed FI-WSOD ameliorates object detectors at both training and inference stages. At the training stage, the involvement of F-BBC task not only improves the feature representation of the network, but also provides extra information from the foreground-background perspective. By leveraging the learnt foreground information, a Foreground Guided Self-Training (FGST) module is further proposed to filter out noisy samples, and to mine representative seeds from the remaining proposals. Moreover, a Multi-Seed Training strategy is performed to reduce the impact of noisy labels when training the self-training networks in FGST. At the inference stage, the F-BBC results are utilized to update the initial multi-class classification scores, which are further integrated with FGST results for better performance. We have conducted extensive experiments on the prevalent Pascal VOC 2007, Pascal VOC 2012 and MSCOCO datasets, and report a series of state-of-the-art records achieved by our proposed framework.

Attention-Assisted Adversarial Model for Cerebrovascular Segmentation in 3D TOF-MRA Volumes.

Cerebrovascular segmentation in time-of-flight magnetic resonance angiography (TOF-MRA) volumes is essential for a variety of diagnostic and analytical applications. However, accurate cerebrovascular segmentation in 3D TOF-MRA is faced with multiple issues, including vast variations in cerebrovascular morphology and intensity, noisy background, and severe class imbalance between foreground cerebral vessels and background. In this work, a 3D adversarial network model called A-SegAN is proposed to segment cerebral vessels in TOF-MRA volumes. The proposed model is composed of a segmentation network A-SegS to predict segmentation maps, and a critic network A-SegC to discriminate predictions from ground truth. Based on this model, the aforementioned issues are addressed by the prevailing visual attention mechanism. First, A-SegS is incorporated with feature-attention blocks to filter out discriminative feature maps, though the cerebrovascular has varied appearances. Second, a hard-example-attention loss is exploited to boost the training of A-SegS on hard samples. Further, A-SegC is combined with an input-attention layer to attach importance to foreground cerebrovascular class. The proposed methods were evaluated on a self-constructed voxel-wise annotated cerebrovascular TOF-MRA segmentation dataset, and experimental results indicate that A-SegAN achieves competitive or better cerebrovascular segmentation results compared to other deep learning methods, effectively alleviating the above issues.

Global Mean Square Error Separation Loss

Class imbalance problem greatly affects the accuracy of deep-learning-based object detectors. To weaken its influence, this paper proposes a global mean square error separation loss (GMSES Loss). GMSES Loss assigns a factor to each example, and then enhances the learning of hard-to-learn examples by reducing the factors of easy-to-learn examples. Meanwhile, the global mean square error separation method is introduced to enhance the learning of foreground classes. To verify the effectiveness of our algorithm, experiments are conducted with YOLOv4 and RetinaNet as the baselines, respectively. The results have shown that our loss function can improve the performance of one-stage object detectors without adding any extra hyperparameters.

Improved YOLOv3 Model for Workpiece Stud Leakage Detection

In this study, a deep convolutional neural network based on an improved You only look once version 3 (YOLOv3) is proposed to improve the accuracy and real-time detection of small targets in complex backgrounds when detecting leaky weld studs on an automotive workpiece. To predict stud locations, the prediction layer of the model increases from three layers to four layers. An image pyramid structure obtains stud feature maps at different scales, and shallow feature fusion at multiple scales obtains stud contour details. Focal loss is added to the loss function to solve the imbalanced sample problem. The reduced weight of simple background classes allows the algorithm to focus on foreground classes, reducing the number of missed weld studs. Moreover, K-medians algorithm replaces the original K-means clustering to improve model robustness. Finally, an image dataset of car body workpiece studs is built for model training and testing. The results reveal that the average detection accuracy of the improved YOLOv3 model is 80.42%, which is higher than the results of Faster R-CNN, single-shot multi-box detector (SSD), and YOLOv3. The detection time per image is just 0.32 s (62.8% and 23.8% faster than SSD and Faster R-CNN, respectively), fulfilling the requirement for stud leakage detection in real-world working environments.

Segmentation of kidney mass using AgDenseU-Net 2.5D model

The Kidney and Kidney Tumor Segmentation Challenge 2021 (KiTS21) released a kidney CT dataset with 300 patients. Unlike KiTS19, KiTS21 provided a cyst category. Therefore, the segmentation of kidneys, tumors, and cysts will be able to assess the complexity and aggressiveness of kidney mass. Deep learning models can save medical resources, but 3D models still have some disadvantages, such as the high cost of computing resources. This paper proposes a scheme that saves computing resources and achieves the segmentation of kidney mass in two steps. First, we preprocess the kidney volume data using the automatic down-sampling method of 3D images, reducing the volume while preserving the feature information. Second, we finely segment kidneys, tumors, and cysts using the AgDenseU-Net (Attention gate DenseU-Net) 2.5D model. KiTS21 proposed using Hierarchical Evaluation Classes (HECs) to compute a metric for the superset: the HEC of kidney considers kidneys, tumors, and cysts as the foreground to compute segmentation performance; the HEC of kidney mass considers both tumor and cyst as the foreground classes; the HEC of tumor considers tumor as the foreground only. For KiTS21, our model achieved a dice score of 0.971 for the kidney, 0.883 for the mass, and 0.815 for the tumor. In addition, we also tested segmentation results without HECs, and our model achieved a dice score of 0.950 for the kidney, 0.878 for the tumor, and 0.746 for the cyst. The results demonstrate that the method proposed in this paper can be used as a reference for kidney tumor segmentation.

Distinguishing foreground and background alignment for unsupervised domain adaptative semantic segmentation

Unsupervised domain adaptive semantic segmentation uses the knowledge learned from the labeled source domain dataset to guide the segmentation of the target domain. However, this domain migration method will cause a large inter-domain difference due to the different feature distributions between the source domain and the target domain. We use the self-supervised learning method to generate pseudo labels for the target domain, so that the corresponding pixels are directly aligned with the source domain according to the segmentation loss. Through observation, it is found that the spatial distribution of the background class in the source domain and the target domain has a small difference, and the appearance of the same class of the foreground class will also be quite different. We use the method of distinguishing alignment between foreground and background classes. We understand that acquiring the rich space and channel information on the feature map during the convolution process is essential for fine-grained semantic segmentation. Therefore, in order to obtain the dependency relationship between the channels of the feature map and the spatial position information, we use a channel and spatial parallel attention module. This module enables the network to select and amplify valuable space and channel information from the global information and suppress useless information. In addition, we introduce focal loss to solve the problem of class imbalance in the data set. Experiments show that our method achieves better segmentation performance in unsupervised domain adaptive semantic segmentation. • We use self-supervised learning to generate pseudo-labels for the target domain. • We distinguish and align the foreground and background classes. • We use parallel attention module to capture the space and channel information. • We add focal loss to the overall loss to reduce the impact of class imbalance.

Deep Learning-Based Segmentation of Peach Diseases Using Convolutional Neural Network.

Peach diseases seriously affect peach yield and people’s health. The precise identification of peach diseases and the segmentation of the diseased areas can provide the basis for disease control and treatment. However, the complex background and imbalanced samples bring certain challenges to the segmentation and recognition of lesion area, and the hard samples and imbalance samples can lead to a decline in classification of foreground class and background class. In this paper we applied deep network models (Mask R-CNN and Mask Scoring R-CNN) for segmentation and recognition of peach diseases. Mask R-CNN and Mask Scoring R-CNN are classic instance segmentation models. Using instance segmentation model can obtain the disease names, disease location and disease segmentation, and the foreground area is the basic feature for next segmentation. Focal Loss can solve the problems caused by difficult samples and imbalance samples, and was used for this dataset to improve segmentation accuracy. Experimental results show that Mask Scoring R-CNN with Focal Loss function can improve recognition rate and segmentation accuracy comparing to Mask Scoring R-CNN with CE loss or comparing to Mask R-CNN. When ResNet50 is used as the backbone network based on Mask R-CNN, the segmentation accuracy of segm_mAP_50 increased from 0.236 to 0.254. When ResNetx101 is used as the backbone network, the segmentation accuracy of segm_mAP_50 increased from 0.452 to 0.463. In summary, this paper used Focal Loss on Mask R-CNN and Mask Scoring R-CNN to generate better mAP of segmentation and output more detailed information about peach diseases.

Anomaly detection-inspired few-shot medical image segmentation through self-supervision with supervoxels.

Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on a limited number of slices, we argue that this ad-hoc solution is insufficient to capture the background properties. Motivated by this, and the observation that the foreground class (e.g., one organ) is relatively homogeneous, we propose a novel anomaly detection-inspired approach to few-shot medical image segmentation in which we refrain from modeling the background explicitly. Instead, we rely solely on a single foreground prototype to compute anomaly scores for all query pixels. The segmentation is then performed by thresholding these anomaly scores using a learned threshold. Assisted by a novel self-supervision task that exploits the 3D structure of medical images through supervoxels, our proposed anomaly detection-inspired few-shot medical image segmentation model outperforms previous state-of-the-art approaches on two representative MRI datasets for the tasks of abdominal organ segmentation and cardiac segmentation.