Supervision For Segmentation Research Articles

Multi-task learning framework to predict the status of central venous catheter based on radiographs

Hospital patients can have catheters and lines inserted during the course of their admission to give medicines for the treatment of medical issues, especially the central venous catheter (CVC). However, malposition of CVC will lead to many complications, even death. Clinicians always detect the status of the catheter to avoid the above issues via X-ray images. To reduce the workload of clinicians and improve the efficiency of CVC status detection, a multi-task learning framework for catheter status classification based on the convolutional neural network (CNN) is proposed. The proposed framework contains three significant components which are modified HRNet, multi-task supervision including segmentation supervision and heatmap regression supervision as well as classification branch. The modified HRNet maintaining high-resolution features from the start to the end can ensure to generation of high-quality assisted information for classification. The multi-task supervision can assist in alleviating the presence of other line-like structures such as other tubes and anatomical structures shown in the X-ray image. Furthermore, during the inference, this module is also considered as an interpretation interface to show where the framework pays attention to. Eventually, the classification branch is proposed to predict the class of the status of the catheter. A public CVC dataset is utilized to evaluate the performance of the proposed method, which gains 0.823 AUC (Area under the ROC curve) and 82.6% accuracy in the test dataset. Compared with two state-of-the-art methods (ATCM method and EDMC method), the proposed method can perform best.

High-resolution feature based central venous catheter tip detection network in X-ray images.

Hospital patients can have catheters and lines inserted during the course of their admission to give medicines for the treatment of medical issues, especially the central venous catheter (CVC). However, malposition of CVC will lead to many complications, even death. Clinicians always detect the malposition based on position detection of CVC tip via X-ray images. To reduce the workload of the clinicians and the percentage of malposition occurrence, we propose an automatic catheter tip detection framework based on a convolutional neural network (CNN). The proposed framework contains three essential components which are modified HRNet, segmentation supervision module, and deconvolution module. The modified HRNet can retain high-resolution features from start to end, ensuring the maintenance of precise information from the X-ray images. The segmentation supervision module can alleviate the presence of other line-like structures such as the skeleton as well as other tubes and catheters used for treatment. In addition, the deconvolution module can further increase the feature resolution on the top of the highest-resolution feature maps in the modified HRNet to get a higher-resolution heatmap of the catheter tip. A public CVC Dataset is utilized to evaluate the performance of the proposed framework. The results show that the proposed algorithm offering a mean Pixel Error of 4.11 outperforms three comparative methods (Ma's method, SRPE method, and LCM method). It is demonstrated to be a promising solution to precisely detect the tip position of the catheter in X-ray images.

Learning class-agnostic masks with cross-task refinement for weakly supervised semantic segmentation

Weakly supervised semantic segmentation (WSSS) commonly relies on Class Activation Mapping (CAM) to produce pseudo semantic labels using image-level annotations. However, because CAM maps often form sparse object regions with poor boundaries, they cannot provide sufficient segmentation supervision. Because off-the-shelf saliency maps can provide rich object boundaries that can be leveraged to improve semantic segmentation, we propose to jointly learn semantic segmentation and class-agnostic masks by using image-level annotations and off-the-shelf saliency maps as supervision. We also propose a cross-task label refinement mechanism, which takes advantage of the learned class-agnostic masks and semantic segmentation masks, to refine the pseudo labels and provide more accurate supervision to both tasks. Moreover, we introduce a new normalization method for CAM to generate more complete class-specific localization maps. The improved CAM maps complement our learned class-agnostic masks, leading to high-quality pseudo semantic segmentation labels. Extensive experiments demonstrate the effectiveness of the proposed approach, with state-of-the-art WSSS results established on PASCAL VOC 2012 and MS COCO.

Chest X-Ray Diagnostic Quality Assessment: How Much Is Pixel-Wise Supervision Needed?

Chest X-ray is an important imaging method for the diagnosis of chest diseases. Chest radiograph diagnostic quality assessment is vital for the diagnosis of the disease because unqualified radiographs have negative impacts on doctors' diagnosis and thus increase the burden on patients due to the re-acquirement of the radiographs. So far no algorithms and public data sets have been developed for chest radiograph diagnostic quality assessment. Towards effective chest X-ray diagnostic quality assessment, we analyze the image characteristics of four main chest radiograph diagnostic quality issues, i.e. Scapula Overlapping Lung, Artifact, Lung Field Loss, and Clavicle Unflatness. Our experiments show that general image classification methods are not competent for the task because the detailed information used for quality assessment by radiologists cannot be fully exploited by deep CNNs and image-level annotations. Then we propose to leverage a multi-label semantic segmentation framework to find the problematic regions, and then classify the quality issues based on the results of segmentation. However, subsequent classification is often negatively affected by certain small segmentation errors. Therefore, we propose to estimate a distance map that measures the distance from a pixel to its nearest segment, and use it to force the prediction of semantic segmentation more holistic and suitable for classification. Extensive experiments validate the effectiveness of our semantic-segmentation-based solution for chest X-ray diagnostic quality assessment. However, general segmentation-based algorithms requires fine pixel-wise annotations in the era of deep learning. In order to reduce reliance on fine annotations and further validate how important pixel-wise annotations are, weak supervision for segmentation is applied, and demonstrates its ability close to that of full supervision. Finally, we present ChestX-rayQuality, a chest radiograph data set, which comprises 480 frontal-view chest radiographs with semantic segmentation annotations and four labels of quality issue. Also, other 1212 chest radiographs with limited annotations are imported to validate our algorithms and arguments on larger data set. These two data set will be made publicly available.

Weakly-supervised semantic segmentation with saliency and incremental supervision updating

Weakly-supervised semantic segmentation aims at tackling the dense labeling task using weak supervision so as to reduce human annotation efforts. For weakly-supervised semantic segmentation using only image-level annotation, we propose a novel model of Learning with Saliency and Incremental Supervision Updating (LSISU), in which both the guidances of saliency prior and class information are jointly used and the segmentation supervision is dynamically updated. In the proposed LSISU, we present an image saliency objective complementary to classification loss, by which the trained weakly-supervised deep network can effectively deal with object co-occurrence problem. Meanwhile, we make full use of the class-wise pooling strategy to generate initial mask estimation of high quality. Given an initial annotation, a segmentation network is learned along with incremental supervision updating, which plays a role of region expansion and corrects the falsely estimated supervision for training images. The incremental supervision updating is performed on the fly and involves repeated usage of a fully connected conditional random field algorithm. LSISU achieves superior segmentation performance in terms of mIoU metric on benchmark datasets, which are 62.5% on the PASCAL VOC 2012 test set and 30.1% on the COCO val set.

Multi‐level feature fusion network for crowd counting

Crowd counting has become a noteworthy vision task due to the needs of numerous practical applications, but it remains challenging. State-of-the-art methods generally estimate the density map of the crowd image with the high-level semantic features of various deep convolutional networks. However, the absence of low-level spatial information may result in counting errors in the local details of the density map. To this end, a novel framework named Multi-level Feature Fusion Network (MFFN) for single image crowd counting is proposed. The proposed MFFN, which is constructed in an encoder–decoder fashion, incorporates semantic and spatial information for generating high-resolution density maps of input crowd images. Skip connections are developed between the encoder and the decoder so that low-level spatial information and high-level semantic features can be combined by element-wise addition. In addition, a dense dilated convolution block is placed behind the encoder, extracting multi-scale context features to guide feature fusion by a channel attention mechanism. The model is trained by multi-task learning; semantic segmentation supervision is introduced to enhance feature representation. Extensive experiments are conducted on three crowd counting datasets (ShanghaiTech, UCF_CC_50, UCF-QNRF), and the results show that MFFN outperforms state-of-the-art methods. In addition, sufficient ablation studies are performed to verify the effectiveness of each component in our proposed method.

An 8-layer residual U-Net with deep supervision for segmentation of the left ventricle in cardiac CT angiography

Accurate segmentation of left ventricle (LV) is a fundamental step in evaluation of cardiac function. Cardiac CT angiography (CCTA) has become an important clinical diagnostic method for cardio-vascular disease (CVD) due to its non-invasive, short exam time, and low cost. To obtain the segmentation of the LV in CCTA scans, we present a deep learning method based on an 8-layer residual U-Net with deep supervision. Based on the original 4-layer U-Net, our method deepened the network to eight layers, which increased the fitting capacity of the network, thus greatly improved its LV recognition capability. Residual blocks were incorporated to optimize the network from the increased depth. Auxiliary paths as deep supervision were introduced to supervise the intermediate information to improve the segmentation quality. In this study, we collected CCTA scans of 100 patients. Eighty patients with 1600 discrete slices were used to train the LV segmentation and the remaining 20 patients with 400 discrete slices were used for testing our method. An interactive graph cut algorithm was utilized reliably to annotate the LV reference standard that was further confirmed by cardiologists. Online data augmentation was performed in the training process to improve the generalization and robustness of our method. Compared with the segmentation results from the original U-Net and FC-DenseNet56 with Dice similarity coefficient (DSC) of 0.878±0.230 and 0.897±0.189, respectively, our method demonstrated higher segmentation accuracy and robustness for varying LV shape, size, and contrast, achieving DSC of 0.927±0.139. Without online data augmentation, our method resulted in inferior performance with DSC of 0.911±0.170. In addition, compared with the provided results from other existing studies in the LV segmentation of cardiac CT images, our method achieved a competitive performance for the LV segmentation. The proposed 8-layer residual U-Net with deep supervision accurately and efficiently segments the LV in CCTA scans. This method has potential advantages to be a reliable segmentation method and useful for the evaluation of cardiac function in the future study.

Graininess-Aware Deep Feature Learning for Robust Pedestrian Detection.

In this paper, we propose a graininess-aware deep feature learning method for pedestrian detection. Unlike most existing methods which utilize the convolutional features without explicit distinction, we appropriately exploit multiple convolutional layers and dynamically select most informative features. Specifically, we train a multi-scale pedestrian attention via pixel-wise segmentation supervision to efficiently identify the pedestrian of particular scales. We encodes the fine-grained attention map into the feature maps of the detection layers to guide them to highlight the pedestrians of specific scale and avoid the background interference. The graininess-aware feature maps generated with our attention mechanism are more focused on pedestrians, and in particular on the small-scale and occluded targets. We further introduce a zoom-in-zoom-out module to enhances the features by incorporating local details and context information. Extensive experimental results on five challenging pedestrian detection benchmarks show that our method achieves very competitive or even better performance with the state-of-the-arts and is faster than most existing approaches.