Object Boundaries Research Articles

Robust unsupervised domain adaptation by retaining confident entropy via edge concatenation

The generalization capability of unsupervised domain adaptation can mitigate the need for extensive pixel-level annotations to train semantic segmentation networks by training models on synthetic data as a source with computer-generated annotations. Entropy-based adversarial networks are proposed to improve source domain prediction; however, they disregard significant external information, such as edges, which have the potential to identify and distinguish various objects within an image accurately. To address this issue, we introduce a novel approach to domain adaptation, leveraging the synergy of internal and external information within entropy-based adversarial networks. In this approach, we enrich the discriminator network with edge-predicted probability values within this innovative framework to enhance the clarity of class boundaries. Furthermore, we devised a probability-sharing network that integrates diverse information for more effective segmentation. Incorporating object edges addresses a pivotal aspect of unsupervised domain adaptation that has frequently been neglected in the past—the precise delineation of object boundaries. Conventional unsupervised domain adaptation methods usually center around aligning feature distributions and may not explicitly model object boundaries. Our approach effectively bridges this gap by offering clear guidance on object boundaries, thereby elevating the quality of domain adaptation. Our approach undergoes rigorous evaluation on the established unsupervised domain adaptation benchmarks, specifically in adapting SYNTHIA → Cityscapes and SYNTHIA → Mapillary. Experimental results show that the proposed model attains better performance than state-of-the-art methods. The superior performance across different unsupervised domain adaptation scenarios highlights the versatility and robustness of the proposed method.

Self-supervised Semantic Segmentation: Consistency over Transformation.

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, , which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model's ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches.

Saliency as Pseudo-Pixel Supervision for Weakly and Semi-Supervised Semantic Segmentation.

Existing studies on semantic segmentation using image-level weak supervision have several limitations, including sparse object coverage, inaccurate object boundaries, and co-occurring pixels from non-target objects. To overcome these challenges, we propose a novel framework, an improved version of Explicit Pseudo-pixel Supervision (EPS++), which learns from pixel-level feedback by combining two types of weak supervision. Specifically, the image-level label provides the object identity via the localization map, and the saliency map from an off-the-shelf saliency detection model offers rich object boundaries. We devise a joint training strategy to fully utilize the complementary relationship between disparate information. Notably, we suggest an Inconsistent Region Drop (IRD) strategy, which effectively handles errors in saliency maps using fewer hyper-parameters than EPS. Our method can obtain accurate object boundaries and discard co-occurring pixels, significantly improving the quality of pseudo-masks. Experimental results show that EPS++ effectively resolves the key challenges of semantic segmentation using weak supervision, resulting in new state-of-the-art performances on three benchmark datasets in a weakly supervised semantic segmentation setting. Furthermore, we show that the proposed method can be extended to solve the semi-supervised semantic segmentation problem using image-level weak supervision. Surprisingly, the proposed model also achieves new state-of-the-art performances on two popular benchmark datasets.

Locally Adaptive Channel Attention-Based Spatial–Spectral Neural Network for Image Deblurring

Recently, due to the rapid development of deep neural networks in the field of computer vision, many studies have been conducted in the field of image deblurring. However, previous methods cannot attain satisfactory deblurring performance in object boundaries and local regions containing image details in restored results. This paper proposes a new method that uses a locally adaptive channel attention module for a spectral–spatial network to resolve the problem of single-image deblurring. Unlike existing methods, our proposed method consists of a spectral restorer and spatial restorer that adopt a locally adaptive attention mechanism in the spectral–spatial domains. In addition, unlike a conventional spectral–spatial network that only considers the magnitude of the frequency coefficients, the proposed method uses both the magnitude and phase of the frequency coefficients in the training stage. Our locally adaptive channel attention spectral–spatial network can focus on informative channels that are closely related to blur artifacts. Specifically, the spatial restorer, which guides the intensity of the blur image to fit the ground truth by exploring the interdependencies of feature channels, can efficiently restore scene characteristics while the spectral restorer, which guides the magnitude and phase of the blur image’s frequency coefficients to fit those of the ground truth by exploring the interdependencies of feature channels, fine-tunes the details of structures. The experimental results show that the proposed method outperformed the deblurring results compared to benchmark methods in terms of both qualitative evaluation and quantitative metrics.

An efficient geometric partitioning mode by adaptive blending method for screen content coding

AbstractGeometric partitioning mode (GPM) was newly adopted in Versatile Video Coding (VVC) to increase the partitioning precision of the arbitrary boundaries of actual objects. Note that the blending process of the GPM is not suitable for screen content with various sharp and strong edges. To alleviate this drawback, three new GPM matrices are introduced, and an efficient GPM is proposed to utilize four different GPM matrices and implicitly determine one of them depending on the discontinuity and gradients of the two predictors. The proposed method achieves an average BD‐rate reduction of 1.2% and 1.6% without additional computational complexity for random access and low‐delay B configurations, respectively, over the VVC test model (VTM18.0).

CenterPoint-SE: A Single-Stage Anchor-Free 3-D Object Detection Algorithm With Spatial Awareness Enhancement

Real-time and accurate 3-D object detection is one of the foundational technologies for environmental perception in autonomous vehicles. However, the existing second-stage anchor-based 3-D object detection algorithms have high accuracy, but they are challenging in terms of computation complexity and latency. Due to poor perception of spatial features, the accuracy of the existing single-stage anchor-free detection algorithms with low latency are difficult to be implemented into autonomous vehicles. Therefore, we focus on enhancing the spatial perception ability of the anchor-free detection network based on CenterPoints. In this paper, we propose a single-stage anchor-free 3-D object detector CenterPoint-Space-Enhancement (CenterPoint-SE) algorithm and construct an efficient 3-D backbone network to extract fine-grained spatial geometric features by introducing a spatial attention mechanism and residual structure. At the same time, a powerful spatial semantic feature fusion module, the enhancement of feature fusion (EF-Fusion), is designed. In addition, we add a lightweight IoU prediction branch to improve the algorithm’s perception of various object sizes. Finally, we add a foreground point segmentation auxiliary training branch to enable the 3-D backbone to obtain object boundary features. We use the ONCE dataset to train and validate the proposed model, and the results showed that the proposed CenterPoint-SE achieves 70.33 mAP and an inference speed of 17.15 FPS, outperforming other methods.

Local image fitting-based active contour for vector-valued images

<span>Variational active contour seeks to segment or extract desired object boundaries for further analysis. The model can be divided into global segmentation and selective segmentation. Selective segmentation, which focuses on segmenting a particular object, is preferable to the global model. Recently, a number of selective segmentation models have been developed to precisely extract an object on grayscale images. Nevertheless, if the input image is vector-valued (colour), these models merely convert it to a grayscale image, resulting in data loss owing to the reduction in image dimension. Furthermore, they may have poor segmentation performance due to the intensity inhomogeneous images. Therefore, a new model on variational selective active contour for segmenting vector-valued images has been proposed that incorporates the concepts of local image fitting and distance-based fitting terms into a variational minimization energy functional. Moreover, a Gaussian function was used as a regularizer to replace the computationally expensive Total Variation term. Then, the proposed model’s Euler Lagrange equation has been provided to solve the model. When segmenting an object in inhomogeneous intensity images, the result of the proposed model was about 30% more accurate based on the Jaccard value and about 3 times faster than other existing methods.</span>

Implementing the edge enhancement with vortex filter in both linear and nonlinear optics

The edge enhancement technique, as an effective method to represent the boundary of objects, plays an important role in image processing. Among them, the vortex filtering, which is based on the radial Hilbert transformation, has been paid great attention due to its ability to achieve isotropic and anisotropic edge enhancement. Recent years have witnessed a growing interest in the nonlinear vortex filter to skillfully realize the visualization of the object edge under invisible light irradiation. In this paper, we start from reviewing the achievements have been made with the vortex filtering technique in linear optics, and then discussed the recent processes of the scalar and vector vortex filter in nonlinear optics. We hope that the nonlinear optical vortex filter can motivate some promising applications in biological edge imaging with visible light-sensitive specimens.

Color and luminance processing in V1 complex cells and artificial neural networks.

Object recognition by natural and artificial visual systems benefits from the identification of object boundaries. A useful cue for the detection of object boundaries is the superposition of luminance and color edges. To gain insight into the suitability of this cue for object recognition, we examined convolutional neural network models that had been trained to recognize objects in natural images. We focused specifically on units in the second convolutional layer whose activations are invariant to the spatial phase of a sinusoidal grating. Some of these units were tuned for a nonlinear combination of color and luminance, which is broadly consistent with a role in object boundary detection. Others were tuned for luminance alone, but very few were tuned for color alone. A literature review reveals that V1 complex cells have a similar distribution of tuning. We speculate that this pattern of sensitivity provides an efficient basis for object recognition, perhaps by mitigating the effects of lighting on luminance contrast polarity. The absence of a contrast polarity-invariant representation of chromaticity alone suggests that it is redundant with other representations.

Visual Ranging Based on Object Detection Bounding Box Optimization

Faster and more accurate ranging can be achieved by combining the object detection technique based on deep learning with conventional visual ranging. However, changes in scene, uneven lighting, fuzzy object boundaries and other factors may result in a non-fit phenomenon between the detection bounding box and the object. The pixel spacing between the detection bounding box and the object can cause ranging errors. To reduce pixel spacing, increase the degree of fit between the object detection bounding box and the object, and improve ranging accuracy, an object detection bounding box optimization method is proposed. Two evaluation indicators, WOV and HOV, are also proposed to evaluate the results of bounding box optimization. The experimental results show that the pixel width of the bounding box is optimized by 1.19~19.24% and the pixel height is optimized by 0~12.14%. At the same time, the ranging experiments demonstrate that the optimization of the bounding box improves the ranging accuracy. In addition, few practical monocular range measurement techniques can also determine the distance to an object whose size is unknown. Therefore, a similar triangle ranging technique based on height difference is suggested to measure the distance to items of unknown size. A ranging experiment is carried out based on the optimization of the detecting bounding box, and the experimental results reveal that the ranging relative error within 6 m is between 0.7% and 2.47%, allowing for precise distance measurement.

DASANet: A 3D Object Detector with Density-and-Sparsity Feature Aggregation

In the field of autonomous driving and robotics, 3D object detection is a difficult, but important task. To improve the accuracy of detection, LiDAR, which collects the 3D point cloud of a scene, is updated constantly. But the density of the collected 3D points is low, and its distribution is unbalanced in the scene, which influences the accuracy of 3D object detectors in regards to object location and identification. Although corresponding high-resolution scene images from cameras can be used as supplemental information, poor fusion strategies can result in decreased accuracy compared with that of LiDAR-point-only detectors. Thus, to improve the detection performance for the classification, localization, and even boundary location of 3D objects, a two-stage detector with density-and-sparsity feature aggregation, called DASANet, is proposed in this paper. In the first stage, dense pseudo point clouds are generated with images from cameras and are used to obtain the initial proposals. In the second stage, two novel feature aggregation modules are designed to fuse LiDAR point information and pseudo point information, which refines the semantic and detailed representation of the feature maps. To supplement the semantic information of the highest-scale LiDAR features for object localization and classification, a triple differential information supplement (TDIS) module is presented to extract the LiDAR-pseudo differential features and enhance them in spatial, channel, and global dimensions. To increase the detailed information of the LiDAR features for object boundary location, a Siamese three-dimension coordinate attention (STCA) module is presented to extract stable LiDAR and pseudo point cloud features with a Siamese encoder and fuse these features using a three-dimension coordinate attention. Experiments using the KITTI Vision Benchmark Suite demonstrate the improved performance of our DASANet in regards to the localization and boundary location of objects. The ablation studies demonstrate the effectiveness of the TDIS and the STCA modules.

Real-Time Cargo Loading Identification Based on Self-Powered Camera

In practice, the statistics of cargo loading capacity needs to be realized by counting the loading actions. Object detection algorithm may not meet the requirements of such tasks under the influence of environmental factors. And the current temporal action localization methods may ignore the spatial information of objects and identify the inaccurate action time boundary of multiple objects in the cargo loading task. To fill this technical gap, this paper designs a cargo loading identification framework. The core of this framework is the Target Area Rise-descend (TAR) algorithm, which is based on the object detection model and uses the regularity of object location and object size to realize real-time recognition of the cargo loading process. We ensure the real-time and robustness of the algorithm by some means like nap mechanism and time regularity. The garbage truck recycling task is taken as an example to complete the elaboration of the proposed method. Experiments show that the accuracy of the proposed method is higher than 99%, and the real-time performance is higher than 20fps on Neural Network Computing Unit (NPU). Considering the limitations of intelligent application scenarios and resources, self-powered camera is applied. For mechanical motion similar to the scene in this paper, the proposed algorithm can use extremely low computing power to capture the motion process and achieve data statistics.

From coarse to fine: a deep 3D probability volume contours framework for tumour segmentation and dose painting in PET images.

With the increasing integration of functional imaging techniques like Positron Emission Tomography (PET) into radiotherapy (RT) practices, a paradigm shift in cancer treatment methodologies is underway. A fundamental step in RT planning is the accurate segmentation of tumours based on clinical diagnosis. Furthermore, novel tumour control methods, such as intensity modulated radiation therapy (IMRT) dose painting, demand the precise delineation of multiple intensity value contours to ensure optimal tumour dose distribution. Recently, convolutional neural networks (CNNs) have made significant strides in 3D image segmentation tasks, most of which present the output map at a voxel-wise level. However, because of information loss in subsequent downsampling layers, they frequently fail to precisely identify precise object boundaries. Moreover, in the context of dose painting strategies, there is an imperative need for reliable and precise image segmentation techniques to delineate high recurrence-risk contours. To address these challenges, we introduce a 3D coarse-to-fine framework, integrating a CNN with a kernel smoothing-based probability volume contour approach (KsPC). This integrated approach generates contour-based segmentation volumes, mimicking expert-level precision and providing accurate probability contours crucial for optimizing dose painting/IMRT strategies. Our final model, named KsPC-Net, leverages a CNN backbone to automatically learn parameters in the kernel smoothing process, thereby obviating the need for user-supplied tuning parameters. The 3D KsPC-Net exploits the strength of KsPC to simultaneously identify object boundaries and generate corresponding probability volume contours, which can be trained within an end-to-end framework. The proposed model has demonstrated promising performance, surpassing state-of-the-art models when tested against the MICCAI 2021 challenge dataset (HECKTOR).

Hyperspectral image classification Based on weakened Laplacian pyramid and guided filtering

ABSTRACT In recent years, methods of spatial-spectral feature extraction have been widely used in hyperspectral image (HSI) classification and have achieved good performance. However, the ways are often unable to distinguish effectively the boundary of the adjacent ground objects. Multi-scale feature fusion alleviates the problems to a certain extent, but in the case of limited samples, the previous methods often fail to achieve the desired effect. In this paper, we propose a multi-scale hyperspectral image classification method based on weakened Laplacian pyramid and guided filtering to nicely distinguish the boundary of the different ground objects. This structure first obtains multi-scale information by constructing a weakened Laplacian pyramid, and weakens the magnification of Gaussian blur in the upsampling process. While obtaining multi-scale images, it can not only retain enough image information to expand feature samples but also emphasize edge information in the edge extraction of the Laplacian pyramid. Then, the weakened Laplacian pyramid feature map is used as the guiding image of the guiding filter, which can better preserve the edge features and further improve the classification performance of HSI. Extensive experiments are carried out on four HSI data sets. The results show that our method is superior in classification accuracy and computational cost.

Learning to refine object boundaries

Existing Deep-Learning-based contour detectors suffer from the issues of the sharpness and the correctness of their predictions, which often need offline post-processing to sharpen the results as well as improve model performance. In this work, we present a novel method that can learn to refine object contour in training and directly output crisp object boundaries in inference. To this end, we first introduce a keypoint-focal loss that draws point-based attention to the isolated contour annotations. It allows an edge detector to jointly optimize the appearance thickness and the localization accuracy of predictions in the training procedure. Moreover, we present a regularization loss to further improve the performance of an edge detector. Lastly, we present the Contour Transformer model for precisely localizing object boundaries in images. We repurpose and integrate a Transformer-style hyper module into an encoder–decoder network, effectively aggregating global contextual information on high-level features and significantly enhancing the discriminative power for classifying foreground/background pixels. We train and test our Attention and Contour Transformer detector (ACTD) on four widely adopted datasets, i.e., BSDS500, NYUD, Multi-Cue, and RoadNet. The proposed method achieves an ODS F-score of 0.826 on BSDS500 and an ODS F-score of 0.783 on NYUD, outperforming previous top detectors.

MSCSA-Net: Multi-Scale Channel Spatial Attention Network for Semantic Segmentation of Remote Sensing Images

Although deep learning-based methods for semantic segmentation have achieved prominent performance in the general image domain, semantic segmentation for high-resolution remote sensing images remains highly challenging. One challenge is the large image size. High-resolution remote sensing images can have very high spatial resolution, resulting in images with hundreds of millions of pixels. This makes it difficult for deep learning models to process the images efficiently, as they typically require large amounts of memory and computational resources. Another challenge is the complexity of the objects and scenes in the images. High-resolution remote sensing images often contain a wide variety of objects, such as buildings, roads, trees, and water bodies, with complex shapes and textures. This requires deep learning models to be able to capture a wide range of features and patterns to segment the objects accurately. Moreover, remote sensing images can suffer from various types of noise and distortions, such as atmospheric effects, shadows, and sensor noises, which can also increase difficulty in segmentation tasks. To deal with the aforementioned challenges, we propose a new, mixed deep learning model for semantic segmentation on high-resolution remote sensing images. Our proposed model adopts our newly designed local channel spatial attention, multi-scale attention, and 16-piece local channel spatial attention to effectively extract informative multi-scale features and improve object boundary discrimination. Experimental results with two public benchmark datasets show that our model can indeed improve overall accuracy and compete with several state-of-the-art methods.

Assessment of Techniques for Detection of Transient Radio-Frequency Interference (RFI) Signals: A Case Study of a Transient in Radar Test Data

The authors present a case study of the investigation of a transient signal that appeared in the testing of a radar receiver. The characteristics of the test conditions and data are first discussed. The authors then proceed to outline the methods for detecting and analyzing transients in the data. For this, they consider several methods based on modern signal processing and evaluate their utility. The initial method used for identifying transients is based on computer vision techniques, specifically, thresholding spectrograms into binary images, morphological processing, and object boundary extraction. The authors also consider deep learning methods and methods related to optimal statistical detection. For the latter approach, since the transient in this case was chirp-like, the method of maximum likelihood is used to estimate its parameters. Each approach is evaluated, followed by a discussion of how the results could be extended to analysis and detection of other types of transient radio-frequency interference (RFI). The authors find that computer vision, deep learning, and statistical detection methods are all useful. However, each is best used at different stages of the investigation when a transient appears in data. Computer vision is particularly useful when little is known about the transient, while traditional statistically optimal detection can be quite accurate once the structure of the transient is known and its parameters estimated.

Multi-temporal urban semantic understanding based on GF-2 remote sensing imagery: from tri-temporal datasets to multi-task mapping

ABSTRACT High resolution satellite images are becoming increasingly available for urban multi-temporal semantic understanding. However, few datasets can be used for land-use/land-cover (LULC) classification, binary change detection (BCD) and semantic change detection (SCD) simultaneously because classification datasets always have one time phase and BCD datasets focus only on the changed location, ignoring the changed classes. Public SCD datasets are rare but much needed. To solve the above problems, a tri-temporal SCD dataset made up of Gaofen-2 (GF-2) remote sensing imagery (with 11 LULC classes and 60 change directions) was built in this study, namely, the Wuhan Urban Semantic Understanding (WUSU) dataset. Popular deep learning based methods for LULC classification, BCD and SCD are tested to verify the reliability of WUSU. A Siamese-based multi-task joint framework with a multi-task joint loss (MJ loss) named ChangeMJ is proposed to restore the object boundaries and obtains the best results in LULC classification, BCD and SCD, compared to the state-of-the-art (SOTA) methods. Finally, a large spatial-scale mapping for Wuhan central urban area is carried out to verify that the WUSU dataset and the ChangeMJ framework have good application values.

Making Sensation: Sculpturing Bamboo by Joko Avianto

The creation of visual art has often been considered to be the privilege of the artist, where art is confined to the purpose and intention of its creation, and the assessment of art becomes a series of explanations that restrain the meaning of an artistic expression. From the perspective of Deleuze-Guattari, art is a monument of sensation that transcends the boundaries of subject and object, and it stands as a statement about itself. The relationship between the artist-subject and material-object used in the process of creating visual art is therefore in the domain of visual practice as a kind of thinking and acting strategy. Joko Avianto’s three-dimensional works exhibited in Frankfurt, Germany, and Yokohama, Japan, are examples of Deleuze-Guattari’s explanation of monuments of sensations in their embodiment. Joko Avianto creates a distinctive and subjective method of artistic action in his creative process, transforming perception into percept and affection into affect, and affirming the state of becoming. Art, visual art, and art practice are mechanisms that allow them to express an ethico-aesthetic paradigm. This paradigm reconnects each existence and becoming of the subject, through aesthetic experience, to the re-emerging new world. Thus, what makes art special is its capability to both transcend and surpass the limits to rediscover or restore the boundless.

Coupled material point and level set methods for simulating soils interacting with rigid objects with complex geometry

The material point method (MPM) is often used to simulate soils that interact with (nearly) rigid objects, such as structures, machines, or rocks. Yet MPM simulations of such problems are quite challenging when the objects have complex shapes. In this paper, we propose an efficient approach for incorporating geometrically complex rigid objects into MPM modeling. The proposed approach leverages the level set method, which can efficiently delineate arbitrary surface geometry, to represent the boundary of a discrete object. For coupling the level set object with the MPM domain, a robust algorithm is developed on the basis of contact mechanics. Through numerical examples of varied complexity, we verify the proposed approach and demonstrate its ability to efficiently simulate challenging problems wherein soils interact with complex rigid objects such as debris-resisting baffles, a vehicle wheel, and basal terrain.