Semantic Edge Detection Research Articles

Category-agnostic semantic edge detection by measuring neural representation randomness

Edge detection plays a fundamental role in computer vision tasks and gains wide applications. In particular, semantic edge detection recently draws more attention due to the high demand for a fine-grained understanding of visual scenes. However, detecting high-level semantic edges hidden in visual scenes is quite challenging. Existing semantic edge detection methods focus on category-aware semantic edges and require elaborate category annotations. Instead, we first propose the category-agnostic semantic edge detection task without additional semantic category annotations. To achieve this goal, we propose to utilize only edge position annotations and leverage the information randomness of semantic edges. Specifically, we align semantic edge positions to the ground truth by maximizing randomness on edge regions and minimizing randomness on non-edge regions in the training process. In the inference process, we first obtain neural representations by the trained network, and then generate semantic edges by measuring neural randomness. We evaluate our method by comparisons with alternative methods on two well-known datasets: Cityscapes (Cordts et al., 2016) and SBD (Hariharan et al., 2014). The results demonstrate our superiority over the alternatives, which is more significant under weak annotations. We also provide comprehensive mechanism studies to verify the generalizability, rationality, and validity of our working mechanism.

Toward Accurate Human Parsing Through Edge Guided Diffusion.

Existing human parsing frameworks commonly employ joint learning of semantic edge detection and human parsing to facilitate the localization around boundary regions. Nevertheless, the parsing prediction within the interior of the part contour may still exhibit inconsistencies due to the inherent ambiguity of fine-grained semantics. In contrast, binary edge detection does not suffer from such fine-grained semantic ambiguity, leading to a typical failure case where misclassification occurs inner the part contour while the semantic edge is accurately detected. To address these challenges, we develop a novel diffusion scheme that incorporates guidance from the detected semantic edge to mitigate this problem by propagating corrected classified semantics into the misclassified regions. Building upon this diffusion scheme, we present an Edge Guided Diffusion Network (EGDNet) for human parsing, which can progressively refine the parsing predictions to enhance the accuracy and coherence of human parsing results. Moreover, we design a horizontal-vertical aggregation to exploit inherent correlations among body parts along both the horizontal and vertical axes, which aims at enhancing the initial parsing results. Extensive experimental evaluations on various challenging datasets demonstrate the effectiveness of the proposed EGDNet. Remarkably, our EGDNet shows impressive performances on six benchmark datasets, including four human body parsing datasets (LIP, CIHP, ATR, and PASCAL-Person-Part), and two human face parsing datasets (CelebAMask-HQ and LaPa).

TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency

This paper addresses the task of semantic segmentation in computer vision, aiming to achieve precise pixel-wise classification. We investigate the joint training of models for semantic edge detection and semantic segmentation, which has shown a promise. However, implicit cross-task consistency learning in multitask networks is limited. To address this, we propose a novel “decoupled cross-task consistency loss” that explicitly enhances cross-task consistency. Our semantic segmentation network, TriangleNet, achieves a substantial 2.88% improvement over the Baseline in mean Intersection over Union (mIoU) on the Cityscapes test set. Notably, TriangleNet operates at 77.4% mIoU/46.2 FPS on Cityscapes, showcasing real-time inference capabilities at full resolution. With multiscale inference, performance is further enhanced to 77.8%. Furthermore, TriangleNet consistently outperforms the Baseline on the FloodNet dataset, demonstrating its robust generalization capabilities. The proposed method underscores the significance of multitask learning and explicit cross-task consistency enhancement for advancing semantic segmentation and highlights the potential of multitasking in real-time semantic segmentation.

Mask Guidance Pyramid Network for Overlapping Cervical Cell Edge Detection

An important indicator of cervical cancer diagnosis is to calculate the proportion of diseased cells and cancer cells, so it is necessary to segment cells and judge the cell status. The existing methods are difficult to deal with the segmentation of overlapping cells. In order to solve this problem, we put forward such a hypothesis by reading a large number of literature, that is, image segmentation and edge measurement tasks have unity in high-level features. To prove this hypothesis, in this paper, we focus on the complementary between overlapping cervical cell edge information and cell object information to get higher accuracy cell edge detection results. Specifically, we present a joint multi-task learning framework for overlapping cell edge detection by the mask guidance pyramid network. The main component of the framework is the Mask Guidance Module (MGM), which integrates two tasks and stores the shared latent semantics to interact in the two tasks. For semantic edge detection, we propose the novel Refinement Aggregated Module (RAM) fusion to promote semantic edges. Finally, to improve the edge pixel accuracy, the edge consistency constraint loss function is introduced to our model training. Our extensive experiments have proved that our method outperforms other edge detection efforts.

SE-Calib: Semantic Edge-Based LiDAR–Camera Boresight Online Calibration in Urban Scenes

Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in earth observation and robotic applications. Although boresight parameters can be obtained through pre-calibration with artificial targets, unforeseen movement of sensors during data collection can lead to significant errors in the boresight parameters. To address this issue, we propose SE-Calib, an automatic and target-free online boresight calibration method for LiDAR-Camera systems. SE-Calib firstly extracts semantic edge features from both point clouds and images simultaneously using the 3D semantic segmentation (3D-SS) and 2D semantic edge detection (2D-SED) methods. The boresight parameters are then optimized with an adaptive solver and maximizing the Soft Semantic Response Consistency Metric (SSRCM) scores iteratively. The SSRCM is designed to evaluate the coherence of cross-modular semantic edge features, and a confidence function is proposed to filter out unreliable optimization results. Experiments conducted on challenging urban datasets show an average boresight error of 0.206 degrees (2.47 pixels in reprojection error), demonstrating the effectiveness and robustness of the proposed method.

All-Higher-Stages-In Adaptive Context Aggregation for Semantic Edge Detection

Convolutional Neural Networks (CNNs) can reveal local variation details and multi-scale spatial context in images via low-to-high stages of feature expression; effective fusion of these raw features is key to Semantic Edge Detection (SED). The methods available in the field generally fuse features across stages in a position-aligned mode, which cannot satisfy the requirements of diverse semantic context in categorizing different pixels. In this paper, we propose a deep framework for SED, the core of which is a new multi-stage feature fusion structure, called All-HiS-In ACA (All-Higher-Stages-In Adaptive Context Aggregation). All-HiS-In ACA can adaptively select semantic context from all higher-stages for detailed features via a cross-stage self-attention paradigm, and thus can obtain fused features with high-resolution details for edge localization and rich semantics for edge categorization. In addition, we develop a non-parametric Inter-layer Complementary Enhancement (ICE) module to supplement clues at each stage with their counterparts in adjacent stages. The ICE-enhanced multi-stage features are then fed into the All-HiS-In ACA module. We also construct an Object-level Semantic Integration (OSI) module to further refine the fused features by enforcing the consistency of the features within the same object. Extensive experiments demonstrate the superior performance of the proposed method over state-of-the-art works.

Semantic-Edge-Supervised Single-Stage Detector for Oriented Object Detection in Remote Sensing Imagery

In recent years, significant progress has been made in arbitrary-oriented object detection. Different from natural images, object detection in aerial images remains its problems and challenges. Current feature enhancement strategies in this field mainly focus on enhancing the local critical response of the target while ignoring the target’s contextual information, which is indispensable for detecting remote sensing targets in complex backgrounds. In this paper, we innovatively combine semantic edge detection with arbitrary-oriented object detection and propose a feature enhancement network base on a semantic edge supervision module (SES) that realizes an attention-like mechanism in three dimensions of space, channel, and pyramid level. It helps the network pay attention to the edge features of targets at multiple scales to obtain more regression clues. Furthermore, to solve the problem of dense objects with different directions in remote sensing images, we propose a rotation-invariant spatial pooling pyramid (RISPP) to extract the features of objects from multiple orientations. Based on the two feature enhancement modules, we named the network SE2-Det; extensive experiments on large public datasets of aerial images (DOTA and UCAS-AOD) validate our approach’s effectiveness and demonstrate our detector’s superior performance.

Improvement in design and training of feature pyramid network for contour refinement

• A simplified feature pyramid architecture, called Edge Pyramid Network, applied for the first time to edge detection. • A recursive fine-tuning strategy introducing a new weighted negative loss to detect accurate and thin edges. • State-of-the-art results on the benchmark datasets for semantic and category-agnostic edge detection. Semantic edge detection is a challenging problem which aims to produce multiple edge maps corresponding to different object categories. The design and the training of the related network are the two most powerful levers to optimize its accuracy. To this end, we first refine existing semantic edge detection architectures with low and high level features inside our Edge Pyramid Network, which already outperforms state-of-the-art networks on popular benchmark datasets. Then, we apply an ingenious fine-tuning strategy, which incrementally improves edge detection accuracy through a few alternated training cycles, using various loss functions to perform thick and thin edge detection. We observe that applying separately any of these refinements outperforms such networks as SEAL and STEAL for category-aware edge detection. More remarkable is the fact that these two refinements provide accumulated benefits on the edge detection performance (+0.85% and +1.78% on MF-scores compared to STEAL on SBD and Cityscapes datasets respectively). This improvement is visible in the edge maps in which EPN architecture results in smoother edges and the fine-tuning strategy is responsible for thinner edges. Furthermore, we show that EPN architecture achieves competitive performance (ODS F-measure of 0.828) against the state-of-the-art category-agnostic edge detection networks on the BSDS500 dataset.

Study on the 3D point cloud semantic segmentation method of fusion semantic edge detection

With the continuous development of deep learning, semantic segmentation, as the basis of 3D scene understanding, has also been widely used in 3D point clouds. Semantic segmentation based on the point cloud has obvious advantages due to its rich data. Aiming at the problems of unclear segmentation target and unclear edge in point cloud semantic segmentation, a 3D point cloud semantic segmentation algorithm integrating edge detection was proposed. Firstly, complete global semantic features are obtained by the mainstream 3D point cloud semantic segmentation framework. Then, semantic edge detection is used to extract edge semantic features from the point cloud. Finally, the fusion module fuses the semantic features belonging to the same object to make the segmentation target more accurate. In addition, a dual semantic loss function is used to produce semantic segmentation results with better boundaries. Experimental results show that the improved algorithm has better precision than KPConv in S3DIS and ScanNet datasets and better segmentation performance.

Semantic Edge Detection with Diverse Deep Supervision

Semantic edge detection (SED), which aims at jointly extracting edges as well as their category information, has far-reaching applications in domains such as semantic segmentation, object proposal generation, and object recognition. SED naturally requires achieving two distinct supervision targets: locating fine detailed edges and identifying high-level semantics. Our motivation comes from the hypothesis that such distinct targets prevent state-of-the-art SED methods from effectively using deep supervision to improve results. To this end, we propose a novel fully convolutional neural network using diverse deep supervision within a multi-task framework where bottom layers aim at generating category-agnostic edges, while top layers are responsible for the detection of category-aware semantic edges. To overcome the hypothesized supervision challenge, a novel information converter unit is introduced, whose effectiveness has been extensively evaluated on SBD and Cityscapes datasets.

Precise Extraction of Buildings from High-Resolution Remote-Sensing Images Based on Semantic Edges and Segmentation

Building extraction is a basic task in the field of remote sensing, and it has also been a popular research topic in the past decade. However, the shape of the semantic polygon generated by semantic segmentation is irregular and does not match the actual building boundary. The boundary of buildings generated by semantic edge detection has difficulty ensuring continuity and integrity. Due to the aforementioned problems, we cannot directly apply the results in many drawing tasks and engineering applications. In this paper, we propose a novel convolutional neural network (CNN) model based on multitask learning, Dense D-LinkNet (DDLNet), which adopts full-scale skip connections and edge guidance module to ensure the effective combination of low-level information and high-level information. DDLNet has good adaptability to both semantic segmentation tasks and edge detection tasks. Moreover, we propose a universal postprocessing method that integrates semantic edges and semantic polygons. It can solve the aforementioned problems and more accurately locate buildings, especially building boundaries. The experimental results show that DDLNet achieves great improvements compared with other edge detection and semantic segmentation networks. Our postprocessing method is effective and universal.

Shearlets as feature extractor for semantic edge detection: the model-based and data-driven realm.

Semantic edge detection has recently gained a lot of attention as an image-processing task, mainly because of its wide range of real-world applications. This is based on the fact that edges in images contain most of the semantic information. Semantic edge detection involves two tasks, namely pure edge detection and edge classification. Those are in fact fundamentally distinct in terms of the level of abstraction that each task requires. This fact is known as the distracted supervision paradox and limits the possible performance of a supervised model in semantic edge detection. In this work, we will present a novel hybrid method that is based on a combination of the model-based concept of shearlets, which provides probably optimally sparse approximations of a model class of images, and the data-driven method of a suitably designed convolutional neural network. We show that it avoids the distracted supervision paradox and achieves high performance in semantic edge detection. In addition, our approach requires significantly fewer parameters than a pure data-driven approach. Finally, we present several applications such as tomographic reconstruction and show that our approach significantly outperforms former methods, thereby also indicating the value of such hybrid methods for biomedical imaging.

Multi-scale spatial context-based semantic edge detection

The good fusion of multi-scale features obtained by Convolutional neural networks (CNNs) is key to semantic edge detection; however, obtaining fusion is challenging. This paper presents a Multi-scale Spatial Context-based deep network for Semantic Edge Detection (MSC-SED). Different from state-of-the-art methods, MSC-SED gradually fuses multi-scale low-to-high level CNN features in an end-to-end architecture. This fusion structure obtains rich multi-scale features while enhancing the details of higher-level features. Beside the overall structure, we present the following two functional modules: the Context Aggregation Module (CAM) and Location-Aware fusion Module (LAM). The CAM helps to enrich context in features at each stage, before and after fusion. The LAM helps to selectively integrate lower-level features. The proposed method outperforms state-of-the-art approaches in terms of both the edge quality and the accuracy of edge categorization on both the SBD and Cityscapes datasets.

SEMEDA: Enhancing segmentation precision with semantic edge aware loss

Per-Pixel Cross entropy (PPCE) is a commonly used loss on semantic segmentation tasks. However, it suffers from a number of drawbacks. Firstly, PPCE only depends on the probability of the ground truth class since the latter is usually one-hot encoded. Secondly, PPCE treats all pixels independently and does not take the local structure into account. While perceptual losses (e.g. matching prediction and ground truth in the embedding space of a pre-trained VGG network) would theoretically address these concerns, it does not constitute a practical solution as segmentation masks follow a distribution that differs largely from natural images. In this paper, we introduce a SEMantic EDge-Aware strategy (SEMEDA) to solve these issues. Inspired by perceptual losses, we propose to match the ’probability texture’ of predicted segmentation mask and ground truth through a proxy network trained for semantic edge detection on the ground truth masks. Through thorough experimental validation on several datasets, we show that SEMEDA steadily improves the segmentation accuracy with negligible computational overhead and can be added with any popular segmentation networks in an end-to-end training framework.

An accurate and efficient multi-category edge detection method

Edge detection plays an important role in image processing. With the development of deep learning, the accuracy of edge detection has been greatly improved, and people have more requirements for edge detection tasks. Most edge detection algorithms are binary edge detection methods, but there are usually multiple categories of edges in an image. In this paper, we present an accurate multi-category edge detection network, the richer category-aware semantic edge detection network (R-CASENet). In order to make full use of convolutional neural network’s powerful feature expression capabilities, we attempt to use more information from feature maps for edge feature extraction and classification. Using the ResNet101 network as the backbone, firstly we merge the building blocks in different composite blocks and down-sample to obtain the feature maps. Then we fuse the feature maps in different composite blocks to obtain the final fused classifier. Experimental results show that R-CASENet can achieve state-of-the-art performance on the large SBD dataset. Furthermore, to get precise one-pixel width edges, we also propose an edge refinement network (ERN) structure. The proposed scheme is an end-to-end method and the proposed ERN can reduce redundant points and improve computational efficiency, especially for further image processing.

Prominent edge detection with deep metric expression and multi-scale features

Edge detection is one of today’s hottest computer vision issues with widely applications. It is beneficial for improving the capability of many vision systems, such as semantic segmentation, salient object detection and object recognition. Deep convolution neural networks (CNNs) recently have been employed to extract robust features, and have achieved a definite improvement. However, there is still a long run to study this hotspot with the main reason that CNNs-based approaches may cause the edges thicker. To address this problem, a novel semantic edge detection algorithm using multi-scale features is proposed. Our model is deep symmetrical metric learning network, which includes 3 key parts. Firstly, the deep detail layer, as a preprocessing layer and a guide module, is employed to remove some low-frequency information and still maintain the edge. Secondly, the deep encoder-decoder networks extract multi-scale features of original image, integrated for complementing information among each level feature. Finally, metric learning is introduced to generate a metric space used to predict edge result. It is easy to distinguish different categories, such as edge space and object space. Simulations and comparisons on benchmark datasets demonstrate the proposed algorithm is superior to the others through visual and quantitative evaluation, and specifically, the score of ODS reachs 0.788.