Object Contour Research Articles

THE CURVE ARC AS A STRUCTURE ELEMENT OF AN OBJECT CONTOUR IN THE IMAGE TO BE RECOGNIZED

Context. The proposed article relates to the field of visual information processing in a computer environment, more precisely to the determination the parameters of the interest object in the image, in particular, the contour of the interest object In most cases, the contour of an object is a simply connected sequence of curve arcs.
 Objective. The purpose and subject of the study is to find and to propose such a definition of the digital curve arc, as the most important element of the object contour in the recognizable image, which does not contradict modern neurophysiological conceptions about visual perception, and to recognize the object contour as a sequence of the digital curve arcs.
 Method. The representation of the image in the form of a structural model is used, one of the structural elements of which is the contour of the object, consisting of digital curve arcs. Also, the image is considered as a cellular complex which corresponds to modern ideas about human visual perception.
 Results. The new definition for arc of a digital curve as a sequence of digital straight segments is proposed, which does not contradict to modern concepts of neurophysiology. In contrast to the known definitions of a curve arc, the proposed definition of a digital curve arc makes it possible to determine the start and end points of the arc. According to the description of the contour of an object as a simply connected closed sequence of line segments, it is proposed to construct a description of the contour as a sequence of arcs of digital curves.
 Conclusions. The use of the proposed definition of the digital curve arc in image processing makes it possible to recognize the contour of an object in an image and present it in a form close to visual perception. For best results, the use of variable resolution in image processing algorithms is recommended.

Distractor-aware Event-based Tracking.

Event cameras, or dynamic vision sensors, have recently achieved success from fundamental vision tasks to high-level vision researches. Due to its ability to asynchronously capture light intensity changes, event camera has an inherent advantage to capture moving objects in challenging scenarios including objects under low light, high dynamic range, or fast moving objects. Thus event camera are natural for visual object tracking. However, the current event-based trackers derived from RGB trackers simply modify the input images to event frames and still follow conventional tracking pipeline that mainly focus on object texture for target distinction. As a result, the trackers may not be robust dealing with challenging scenarios such as moving cameras and cluttered foreground. In this paper, we propose a distractor-aware event-based tracker that introduces transformer modules into Siamese network architecture (named DANet). Specifically, our model is mainly composed of a motion-aware network and a target-aware network, which simultaneously exploits both motion cues and object contours from event data, so as to discover motion objects and identify the target object by removing dynamic distractors. Our DANet can be trained in an end-to-end manner without any post-processing and can run at over 80 FPS on a single V100. We conduct comprehensive experiments on two large event tracking datasets to validate the proposed model. We demonstrate that our tracker has superior performance against the state-of-the-art trackers in terms of both accuracy and efficiency.

Multimodal Fused Deep Learning Networks for Domain Specific Image Similarity Search

The exponential increase in data over the past few years, particularly in images, has led to more complex content since visual representation became the new norm. E-commerce and similar platforms maintain large image catalogues of their products. In image databases, searching and retrieving similar images is still a challenge, even though several image retrieval techniques have been proposed over the decade. Most of these techniques work well when querying general image databases. However, they often fail in domain-specific image databases, especially for datasets with low intraclass variance. This paper proposes a domain-specific image similarity search engine based on a fused deep learning network. The network is comprised of an improved object localization module, a classification module to narrow down search options and finally a feature extraction and similarity calculation module. The network features both an offline stage for indexing the dataset and an online stage for querying. The dataset used to evaluate the performance of the proposed network is a custom domain-specific dataset related to cosmetics packaging gathered from various online platforms. The proposed method addresses the intraclass variance problem with more precise object localization and the introduction of top result reranking based on object contours. Finally, quantitative and qualitative experiment results are presented, showing improved image similarity search performance.

A Novel Variable Stiffness and Tunable Bending Shape Soft Robotic Finger Based on Thermoresponsive Polymers

In this study, a soft robotic finger with variable stiffness and tunable bending shape capability is proposed. The finger employs shape memory polymers (SMP) as the variable stiffness element and polyimide (PI) electrothermal films as the flexible heaters for segmental heating. SMP and PI electrothermal films constitute the variable stiffness structure. The SMP part exhibits a reversible transition from glass state to rubber state during the heating/cooling process. The whole transition process results in an elastic modulus range of three orders of magnitude. The finger can obtain multiple bending shapes and adapt to the outer contours of diverse objects because of the segmental stiffness modulation capability of the variable stiffness structure. Supercoiled polymer artificial muscles (SCPAMs) are used as actuators for the finger. SMPs and SCPAMs are thermoresponsive and can be activated by thermal heating with electricity, making the whole finger structure compact. Experimental results show that the stiffness of the finger at room temperature (25 °C) is 10.54 times that at 70 °C when the actuators are on. The performance of the finger is further validated via the sequential motion demonstration and grasping tests with variable stiffness and bending shape control capabilities, showing the proposed design’s potential in soft grippers and robotic design.

Multi-Generator Discriminator Network Using Texture-Edge Information

In the proposed paper, a parallel structure type Generative Adversarial Network (GAN) using edge and texture information is proposed. In the existing GAN-based model, many learning iterations had to be given to obtaining an output that was somewhat close to the original data, and noise and distortion occurred in the output image even when learning was performed. To solve this problem, the proposed model consists of two generators and three discriminators to propose a network in the form of a parallel structure. In the network, each edge information and texture information were received as inputs, learning was performed, and each character was combined and outputted through the Combine Discriminator. Through this, edge information and distortion of the output image were improved even with fewer iterations than DCGAN, which is the existing GAN-based model. As a result of learning on the network of the proposed model, a clear image with improved contour and distortion of objects in the image was output from about 50,000 iterations.

Новый оператор выделения контуров объектов железнодорожной инфраструктуры

Presents a new operator for selecting the contours of railway infrastructure objects in images. The study focuses on the development of successive stages of the algorithm, including smoothing with a Gaussian filter, adaptive partitioning to regions and image binarization, as well as suppression of non-contour elements. The operator demonstrates efficiency in improving the accuracy of contouring of railway objects such as trains, cars, platforms and signals. The proposed methods to process images, remove noise and select objects, thus increasing the overall quality of recognition. The simulation results show that the new operator contributes to a significant improvement in the accuracy of fault detection compared to alternative methods.

TCIANet: Transformer-Based Context Information Aggregation Network for Remote Sensing Image Change Detection

Change detection based on remote sensing data is an important method to detect the earth surface changes. With the development of deep learning, convolutional neural networks have excelled in the field of change detection. However, the existing neural network models are susceptible to external factors in the change detection process, leading to pseudo change and missed detection in the detection results. In order to better achieve the change detection effect and improve the ability to discriminate pseudo-change, this paper proposes a new method, namely, transformer-based context information aggregation network (TCIANet) for remote sensing image change detection. First, we use a filter-based visual tokenizer (FVT) to segment each temporal feature map into multiple visual semantic tokens. Second, the addition of the progressive sampling vision transformer (PS-ViT) not only effectively excludes the interference of irrelevant changes, but also uses the transformer encoder to obtain compact spatio-temporal context information in the token set. Then, the tokens containing rich semantic information are fed into the pixel space and the transformer decoder is used to acquire pixel-level features. In addition, we use the feature fusion module (FFM) to fuse low-level semantic feature information to complete the extraction of coarse contour information of the changed region. Then the semantic relationships between object regions and contours are captured by contour-graph reasoning module (CGRM) to obtain feature maps with complete edge information. Finally, the prediction model is used to discriminate the change of feature information and generate the final change map. Numerous experimental results show that our method has more obvious advantages in visual effect and quantitative evaluation than other methods.

Rethinking and Improving Few-Shot Segmentation From a Contour-Aware Perspective

Existing few-shot segmentation approaches basically adopt the idea of comparing the semantic prototype vector of the query image and support images, and then obtaining the segmentation result. However, recent studies have shown that a single feature vector in feature map cannot accurately represent pixel-level categories, thus leading to poor segmentation of object boundary and semantic ambiguity. To address this common problem, we propose a novel contour-aware network (CTANet) for few-shot segmentation in this paper. Unlike the usual practice of classifying each pixel separately, CTANet regards all pixels within the same contour as a whole, which can take advantage of the internal consistency of objects to obtain a more accurate representation of category information. To obtain the accurate object contour, our network consists of a contour generation module and a contour refinement module, where the former exploits multiple levels of features to generate a primary contour map and the latter learns to refine the primary contour map. Furthermore, a novel contour-aware mixed loss is proposed to fuse the common BCE loss and our contour-aware loss to supervise the training process on two levels, pixel-level and contour-level. Extensive experiments demonstrate that our CTANet achieves a new state-of-the-art performance on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$ \text{PASCAL-5}^{i}$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$ \text{COCO-20}^{i}$</tex-math></inline-formula> . Hopefully, our new perspective could provide more clues for future research on few-shot segmentation. Our code is freely available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hardtogetA/CTANet</uri> .

Generative Steganography via Auto-Generation of Semantic Object Contours

As a promising technique of resisting steganalysis detection, generative steganography usually generates a new image driven by secret information as the stego-image. However, it generally encodes secret information as entangled features in a non-distribution-preserving manner for the stego-image generation, which leads to two common issues: 1) limited accuracy of information extraction, and 2) low security in feature-domain. To address the above issues, we propose a generative steganographic framework via auto-generation of semantic object contours, in which a given secret message is encoded as the disentangled features, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., object-contours, in a distribution-preserving manner for the stego-image generation. In this framework, we propose a contour generative adversarial nets (CtrGAN) consisting of a contour-generator and a contour-discriminator, which are adversarially trained with reinforcement learning. To realize the generative steganography, by using the contour-generator of the trained CtrGAN, a contour point selection (CPS)-based encoding strategy is designed to encode the secret message as the contours. Then, the BicycleGAN is employed to transform the generated contours to the corresponding stego-image. Extensive experiments demonstrate the proposed steganographic approach achieves superior performance in the aspects of information extraction accuracy, especially under common image attacks, and feature-domain security, compared to the state-of-the-arts.

Prosthetic finger for fingertip tactile sensing via flexible chromatic optical waveguides.

Building prosthetics indistinguishable from human limbs to accurately receive and transmit sensory information to users not only promises to radically improve the lives of amputees, but also shows potential in a range of robotic applications. Currently, a mainstream approach is to embed electrical or optical sensors with force/thermal sensing functions on the surface or inside of prosthetic fingers. Compared with electrical sensing technologies, tactile sensors based on stretchable optical waveguides have the advantages of easy fabrication, chemical safety, environmental stability, and compatibility with prosthetic structural materials. However, so far, research has mainly focused on the perception of finger joint motion or external press, and there is still a lack of study on optical sensors with fingertip tactile capabilities (such as texture, hardness, slip detection, etc.). Here we report a 3D printing prosthetic finger with flexible chromatic optical waveguides implanted at the fingertip. The finger achieves distributed displacement/force sensing detection, and exhibits high sensitivity, fast response and good stability. The finger can be used to conduct active sensory experiments, and the detection parameters include object contour, hardness, slip direction and speed, temperature, etc. Finally, exploratory research on identifying and manipulating objects is carried out with this finger. The developed prosthetic finger can artificially recreate touch perception and realize complex functions such as note-writing analysis and braille recognition.

Arbitrary Style Transfer With Fused Convolutional Block Attention Modules

The advancement of deep learning has rendered image style transfer a progressively intricate subject matter. The proposed solution aims to tackle the limitations of current methods in retaining the content image object contours and avoiding blurred image boundaries and mismatched color matching after stylization. To achieve this, an arbitrary-style transfer network is introduced, which leverages the attention mechanism. The network comprises an encoder-decoder module, a convolutional block attention module (CBAM), and an adaptive attention normalization network (AdaAttN) module. The CBAM attention mechanism is presented as an extension of the AdaAttN network, with the aim of producing stylized images that exhibit both global and local style coordination. This is achieved by leveraging long-range dependencies in the image. Additionally, a novel loss function, referred to as the structural similarity loss, is proposed to enhance the consistency of the generated images with respect to the underlying content structure. Finally, a new local feature loss is introduced to further enhance the visual quality of the stylized images at a local level. The study involved conducting style transfer training on a dataset comprising 82,783 real images and 81,446 artistic images. Furthermore, an additional set of 1,000 resultant images, generated from 100 real photos and 10 artistic portraits, was utilized for testing purposes. The study compares the experimental outcomes with four contemporary-style transfer techniques. Additionally, the efficacy of the CBAM module and SSIM loss function is demonstrated through ablation experiments. The findings of the experiment demonstrate that the network proposed has the ability to effectively adapt to the local style and can adeptly correspond the semantically proximate style features to the content features, thereby preserving superior spatial consistency.

Dual Cross-Attention for Video Object Segmentation via Uncertainty Refinement

In this paper, we propose a novel approach to video object segmentation where dual streams consisting of a shared network and a special network are designed to constitute the feature memory of history frames. Cues of spatial position and time stamp are explicitly explored to learn the context for each frame in the video sequence. Self-attention and cross-attention are simultaneously exploited to extract more powerful features for segmentation. In contrast to STM and its variants, the proposed dual cross-attention performs in both appearance space and semantic space such that the derived features are more distinctive and then robust to similar overlapping objects. During decoding for segmentation, a local refinement technique is designed for the uncertain boundaries to obtain more precise and smooth object contours. Experimental results on the challenging benchmark datasets DAVIS-2016, DAVIS-2017, and YouTube-VOS demonstrate the effectiveness of our proposed approach to video object segmentation.

Chain code strategy for lossless storage and transfer of segmented binary medical data

Obtaining a 3D medical visualization is a tedious process requiring several processing steps (such as segmentation) and assigning various rendering parameters (such as color and opacity). Current systems use video/image exporting or snapshots to save results. Such vendor-dependent tools not only prevent the possibility of further interactions but also creates additional large-size data that is problematic to store in PACS over time and hard to transfer for teleradiology applications. To overcome, alternative strategies propose a representation of the visualizations, which only store segmentation masks that contains the binary form of segmented data. Unfortunately, existing compression methods are limited to effectively compress the volumetric data. In this study, lossless storage of binary segmented data is effectively performed by two newly-proposed chain code approaches. Particularly-two novel contributions are presented: 1. The dictionary of normalized angle difference is improved as a new chain symbol coding procedure, namely normalized angle difference, by adding new symbols to the dictionary aiming to generate a low-entropy symbol sequence for medical volumes. 2. A new volumetric approach that utilizes 26 symbols to encode volumetric data is developed. Each slice is visited, and the contour of the segmented object is codified such that eight different vectors for each slice (pointing to one of the four faces of each voxel, plus four towards one of its edges) are obtained. The developed methods are tested on diverse volumetric segmented data and compared to existing standards. It is shown that the proposed methods outperform well-established techniques.

Application of YOLOX deep learning model for automated object detection on thermograms

A method of automating the data analysis of thermal imaging systems in the field of safety control is proposed. It has been established that today video surveillance technologies have a number of disadvantages that can be eliminated by using thermal imaging cameras. Analysis of infrared images can be automated in order to reduce percentage of false positives and increase the efficiency of thermal imaging video surveillance systems. A high level of interference, unclear object contours and low image resolution are real problems in automating the object detecting process on thermographic images. The traditional and promising methods of thermograms analysis and approaches that can lead to creating the automated thermal video surveillance systems are discussed. It is proposed to use deep learning, which in recent years has proven itself as an effective way of image analysis. The study is based on review of existing works, as methods of automating object detection process on thermograms. It is proposed to use YOLOX as a deep learning model. This model has one of the best quality indicators and speed processing input parameters on standard datasets. FLIR’s Thermal Starter annotated set of thermal images is used to train the model. The value of mAP at the level 55% is obtained according the results of model training for recognizing 4 classes of objects on thermograms. Different advantages and disadvantages of this development are analyzed. Ways of further improvement of the neural network method of automation of thermal imaging safety control systems have been determined.

Recurrent neural circuits for perceptual grouping

Neurons in the visual cortex are sensitive to context: Responses to stimuli presented within their classical receptive fields (CRFs) are modulated by stimuli in their surrounding extra-classical receptive fields (eCRFs). However, the circuits underlying these contextual effects are not well understood, and little is known about how these circuits drive perception during everyday vision. We tackle these questions by approximating circuit-level eCRF models with a differentiable discrete-time recurrent neural network that is trainable with gradient-descent. After optimizing model synaptic connectivity and dynamics for object contour detection in natural images, the neural-circuit model rivals human observers on the task with far better sample efficiency than state-of-the-art computer vision approaches. Notably, the model also exhibits CRF and eCRF phenomena typically associated with primate vision. The model’s ability to accurately detect object contours also critically depends on these effects, and these contextual effects are not found in ablated versions of the model. Finally, we derive testable predictions about the neural mechanisms responsible for contextual integration and illustrate their importance for accurate and efficient perceptual grouping.

Semi-supervised learning approach for localization and pose estimation of texture-less objects in cluttered scenes

3D object recognition and 6D pose estimation are crucial and fundamental endeavours for industrial assembly line automation such as robotic controlled pick-and-place. While the problem on textured objects is extensively studied, it is still an open research topic for texture-less industrial parts, e.g, solid cylinder and hollow tube, which are symmetric and appear similar in shapes from many viewing perspectives, causing pose ambiguity. Also, the industrial assembly line environment is usually cluttered and the captured data is noisy, which makes this task even more challenging. In this paper, we propose a novel object localization and pose estimation technique using RGB images and depth maps of industrial assembly parts. Our segmentation model is fully morphological and unsupervised for localizing the region of interest containing the target object extracted from the depth map. Our segmentation technique is effective in the presence of partial occlusion, multiple objects, and cluttered scenes. We use a model based approach for object recognition based on Stochastic Gradient Descent trained on features of Histogram of Oriented Gradients (HOG) and invariant moments of the region of interest containing the target object. We generate synthetic training images automatically from the CAD models of the industrial parts. We use a contour matching strategy based on Dynamic Time Warping (DTW) algorithm to estimate the optimal 6D pose of the object from a set of candidates. Experimental results show that our proposed approach competes and demonstrates advantages on the challenging T-LESS dataset compared to the state-of-the-art methods. • We introduce a strategy for generating multiple viewpoint synthetic images from a CAD model. • We introduce a non-supervised clustering process for object segmentation based on a depth map that does not require a training stage. • We demonstrate how different HOG feature parameters can be tuned to achieve better prediction and performance. • We present a new real-time strategy for object pose estimation using Dynamic Time Warping (DTW) between the target object contour and best prior candidates. • We show that our method achieves better time performance than other works for pose estimation.

A combined deformable model and medical transformer algorithm for medical image segmentation

Deep learning–based segmentation models usually require substantial data, and the model usually suffers from poor generalization due to the lack of training data and inefficient network structure. We proposed to combine the deformable model and medical transformer neural network on the image segmentation task to alleviate the aforementioned problems. The proposed method first employs a statistical shape model to generate simulated contours of the target object, and then the thin plate spline is applied to create a realistic texture. Finally, a medical transformer network was constructed to segment three types of medical images, including prostate MR image, heart US image, and tongue color images. The segmentation accuracy of the three tasks achieved 89.97%, 91.90%, and 94.25%, respectively. The experimental results show that the proposed method improves medical image segmentation performance.Graphical abstract

Multi-Branch Adaptive Hard Region Mining Network for Urban Scene Parsing of High-Resolution Remote-Sensing Images

Scene parsing of high-resolution remote-sensing images (HRRSIs) refers to parsing different semantic regions from the images, which is an important fundamental task in image understanding. However, due to the inherent complexity of urban scenes, HRRSIs contain numerous object classes. These objects present large-scale variation and irregular morphological structures. Furthermore, their spatial distribution is uneven and contains substantial spatial details. All these features make it difficult to parse urban scenes accurately. To deal with these dilemmas, in this paper, we propose a multi-branch adaptive hard region mining network (MBANet) for urban scene parsing of HRRSIs. MBANet consists of three branches, namely, a multi-scale semantic branch, an adaptive hard region mining (AHRM) branch, and an edge branch. First, the multi-scale semantic branch is constructed based on a feature pyramid network (FPN). To reduce the memory footprint, ResNet50 is chosen as the backbone, which, combined with the atrous spatial pyramid pooling module, can extract rich multi-scale contextual information effectively, thereby enhancing object representation at various scales. Second, an AHRM branch is proposed to enhance feature representation of hard regions with a complex distribution, which would be difficult to parse otherwise. Third, the edge-extraction branch is introduced to supervise boundary perception training so that the contours of objects can be better captured. In our experiments, the three branches complemented each other in feature extraction and demonstrated state-of-the-art performance for urban scene parsing of HRRSIs. We also performed ablation studies on two HRRSI datasets from ISPRS and compared them with other methods.

A Method for Extracting Building Information from Remote Sensing Images Based on Deep Learning.

Semantic segmentation of remote sensing images is an important issue in remote sensing tasks. Existing algorithms can extract information more accurately, but it is difficult to capture the contours of objects and further reveal the interaction information between different objects in the image. Therefore, a deep learning-based method for extracting building information from remote sensing images is proposed. First, the deep learning semantic segmentation model DeepLabv3+ and Mixconv2d are combined, and convolution kernels of different sizes are used for feature recognition. Then, the regularization method based on Rdrop Loss improves the accuracy and efficiency of contour capture for objects of different resolutions, and at the same time improves the consistency of dataset fitting. Finally, the proposed remote sensing image information extraction method is verified based on the self-built dataset. The experimental results show that the proposed algorithm can effectively improve the algorithm efficiency and result accuracy, and has good segmentation performance.

UDNet: Uncertainty-aware deep network for salient object detection

Most of the existing deep learning based salient object detection (SOD) models adopt multi-level feature fusion strategies, and have achieved remarkable progress. However, current SOD models still suffer from the uncertainty dilemma in predicting salient probabilities of the pixels surrounding the contour of salient objects. To solve this issue, we propose a novel uncertainty-aware SOD model, where multiple supervision signals, i.e., internal contour uncertainty map, saliency map and external contour uncertainty map, are used to guide the network to not only focus on the pixels in the salient object but also shift its partial attention to the pixels surrounding the contour of salient objects. Furthermore, we introduce a new feature interaction module to aggregate internal contour uncertainty features, saliency features and external contour uncertainty features in the decoding stage, aiming to enhance the model’s ability in dealing with the “uncertain” pixels. Extensive experiments on four public benchmark datasets demonstrate the superiority of the proposed method over the existing state-of-the-art SOD methods. Furthermore, the proposed method shows better attribute-based performance on the SOC dataset, suggesting that the proposed model can also handle challenging scenarios in SOD.