Structure Information Of Objects Research Articles

SOA: Seed point offset attention for indoor 3D object detection in point clouds

Three-dimensional object detection plays a pivotal role in scene understanding and holds significant importance in various indoor perception applications. Traditional methods based on Hough voting are susceptible to interference from background points or neighboring objects when casting votes for the target’s center from each seed point. Moreover, fixed-size set abstraction modules may result in the loss of structural information for large objects. To address these challenges, this paper proposes a three-dimensional object detection model based on seed point offset attention. The objective of this model is to enhance the model’s resilience to voting noise interference and alleviate feature loss for large-scale objects. Specifically, a seed point offset tensor is first defined, and then the offset tensor self-attention network is employed to learn the weights between votes, thereby establishing a correlation between the voting semantic features and the object structural information. Furthermore, an object surface perception module is introduced, which incorporates detailed features of local object surfaces into global feature representations through vote backtracking and surface mapping. Experimental results indicate that the model achieved excellent performance on the ScanNet-V2 (mAP@0.5, 60.3%) and SUN RGB-D (mAP@0.5, 64.0%) datasets, respectively improving by 2.6% (mAP@0.5) and 5.4% (mAP@0.5) compared to VoteNet.

Collaborative compensative transformer network for salient object detection

Salient object detection (SOD) is of high significance for various computer vision applications but is a challenging task due to the complicated scenes in real-world images. Most state-of-the-art SOD methods aim to build long-range dependency for improving global contrast modeling in complicated scenes. However, most of them suffer from the prior assumption of treating image patches as visual tokens for building long-range dependency. This is because this assumption leads to localizing salient regions with uncertain boundaries due to the lost object structure information. In this paper, to address this issue, we re-construct the prior assumption of treating both patches and superpixels as visual tokens for building long-range dependency, which takes into account the properties of superpixels and patches in preserving detailed structural-aware information and local context information, respectively. Based on the re-constructed prior assumption, we propose a Collaborative Compensative Transformer Network (CCTNet) for the SOD task. CCTNet firstly alternates the computation within the same kind of vision tokens and among different vision tokens to build their dependencies. By this means, the relationship between multi-level global context and detailed structure representation can be explicitly modeled for consistent semantic and object structure understanding. Then, CCTNet performs feature joint decoding for SOD by fusing the complementary global context and detailed structure for locating objects with certain boundaries. Extensive experiments were conducted to validate the effectiveness of the proposed modules. Furthermore, the experiments on ten benchmark datasets demonstrated the state-of-the-art performance of CCTNet on both RGB and RGB-D SOD.

Multiplatform Bundle Adjustment Method Supported by Object Structural Information

Multiplatform Bundle Adjustment Method Supported by Object Structural Information

DGRN: Image super-resolution with dual gradient regression guidance

In single image super-resolution (SISR), deep neural networks learn mainly nonlinear functions to obtain promising high-resolution (HR) images. However, there are usually two undesired limitations to recovered images of existing SR methods. First, since this task is typically an ill-posed problem, recovering the structural information for the SR process is usually sharp edges but distorted. Second, since this task generally has an extremely large space for the mapping function, learning this mapping from low-resolution (LR) to HR images is typically difficult. For the most part, SR models usually suffer from the lack of structural information for the objects in images and poor performance. To address the above issues, we propose a dual gradient regression scheme in the first stage of the network to recover the structure information of objects in images and the corresponding loss function to learn the difference between structural maps. In the second stage of the network, we restore HR gradient maps by the first stage to provide additional structural information for the second stage, further constraining the structure of the SR image and reducing the space of the possible functions. Extensive experiments demonstrate our results outperform the state-of-the-art SR methods in the SSIM, and achieve better visual effects than the state-of-the-art SR methods.

Occluded prohibited object detection in X-ray images with global Context-aware Multi-Scale feature Aggregation

Prohibited Object Detection (POD) in X-ray images plays an important role in protecting public safety. Automatic and accurate POD is required to relieve the working pressure of security inspectors. However, the existing methods cannot obtain a satisfactory detection accuracy, and especially, the problem of object occlusion also has not been solved well. Therefore, in this paper, according to the specific characteristics of X-ray images as well as low-level and high-level features of Convolutional Neural Network (CNN), different feature enhancement strategies have been elaborately designed for occluded POD. First, a learnable Gabor convolutional layer is designed and embedded into the low layer of the network to enhance the network's capability to capture the edge and contour information of object. A Spatial Attention (SA) mechanism is then designed to weight the output features of the Gabor convolutional layer to enhance the spatial structure information of object and suppress the background noises simultaneously. For the high-level features, Global Context Feature Extraction (GCFE) module is proposed to extract multi-scale global contextual information of object. And, a Dual Scale Feature Aggregation (DSFA) module is proposed to fuse these global features with those of another layer. To verify the effectiveness of the proposed modules, they are embedded into typical one-stage and two-stage object detection frameworks, i.e., Faster R-CNN and YOLO v5L, obtaining POD-F and POD-Y methods, respectively. The proposed methods have been extensively evaluated on three publicly available benchmark datasets, namely SIXray, OPIXray and WIXray. The experimental results show that, compared with existing methods, the proposed POD-Y method can achieve a state-of-the-art detection accuracy. And POD-F can also achieve a competitive detection performance among the two-stage detection methods.11https://github.com/Open-my-paper-code/POD.

Weakly Supervised RGB-D Salient Object Detection With Prediction Consistency Training and Active Scribble Boosting.

RGB-D salient object detection (SOD) has attracted increasingly more attention as it shows more robust results in complex scenes compared with RGB SOD. However, state-of-the-art RGB-D SOD approaches heavily rely on a large amount of pixel-wise annotated data for training. Such densely labeled annotations are often labor-intensive and costly. To reduce the annotation burden, we investigate RGB-D SOD from a weakly supervised perspective. More specifically, we use annotator-friendly scribble annotations as supervision signals for model training. Since scribble annotations are much sparser compared to ground-truth masks, some critical object structure information might be neglected. To preserve such structure information, we explicitly exploit the complementary edge information from two modalities (i.e., RGB and depth). Specifically, we leverage the dual-modal edge guidance and introduce a new network architecture with a dual-edge detection module and a modality-aware feature fusion module. In order to use the useful information of unlabeled pixels, we introduce a prediction consistency training scheme by comparing the predictions of two networks optimized by different strategies. Moreover, we develop an active scribble boosting strategy to provide extra supervision signals with negligible annotation cost, leading to significant SOD performance improvement. Extensive experiments on seven benchmarks validate the superiority of our proposed method. Remarkably, the proposed method with scribble annotations achieves competitive performance in comparison to fully supervised state-of-the-art methods.

Accurate Localization of Oil Tanks in Remote Sensing Images via FGMRST-Based CNN

Object localization is an important application of remote sensing images and the basis of information extraction. The acquired accuracy is the key factor to improve the accuracy of object structure information inversion. The floating roof oil tank is a typical cylindrical artificial object, and its top cover fluctuates up and down with the change in oil storage. Taking the oil tank as an example, this study explores the localization by combining the traditional feature parameter method and convolutional neural networks (CNNs). In this study, an improved fast radial symmetry transform (FRST) algorithm called fast gradient modulus radial symmetry transform (FGMRST) is proposed and an approach based on FGMRST combined with CNN is proposed. It effectively adds the priori of circle features to the calculation process. Compared with only using CNN, it achieves higher precision localization with fewer network layers. The experimental results based on SkySat data show that the method can effectively improve the calculation accuracy and efficiency of the same order of magnitude network, and by increasing the network depth, the accuracy still has a significant improvement.

Automated detection and counting of Artemia using U-shaped fully convolutional networks and deep convolutional networks

The brine shrimp Artemia is a widely used cost-effective diet in aquaculture. In many Artemia studies, e.g., in a quality assessment of Artemia hatching, an automated method for detecting and counting the Artemia objects in images would be highly desired. However, there are few such works in literature. Moreover, it is very challenging to separate Artemia objects that are highly adjacent. In this paper, we propose a two-stage method for Artemia detection and counting, combining a target marker proposal network with a target classification network. In the first stage, the marker proposal network is implemented using U-shaped fully convolutional networks. This module can indicate target candidates, separate adjacent objects and obtain the object structural information simultaneously. In the second stage, using deep convolutional networks, we design a classifier to classify the target candidates into categories or label as a non-target, thereby obtaining the Artemia detection and counting results. Furthermore, an Artemia detection and counting dataset is collected to train and test the proposed method. Experimental results confirm that the proposed method can accurately detect and count the Artemia objects that have high degrees of adjacency in images, outperforming an ad hoc method based on hand-crafted features and the state-of-the-art YOLO-v3 method.

PA-MVSNet: Sparse-to-Dense Multi-View Stereo With Pyramid Attention

Multi-view based 3D reconstruction aims to obtain 3D structure information of objects in space through two-dimensional images. In this paper, we propose a new multi-view stereo network that can robustly reconstruct the scene. To enhance the feature representation ability of Point-MVSNet, a pyramid attention module is introduced. Specifically, we exploit the attention mechanism for the multi-scale feature pyramid to capture larger receptive fields and richer information. Instead of constructing a feature pyramid as the input, results of the pyramid attention module at different scales are directly used for the next layer. The network eventually generates a high-quality depth estimation for 3D reconstruction from sparse to dense by an iterative refinement schedule. Experiments have been performed to evaluate 3D reconstruction quality by comparison with existing state-of-the-art methods on the DTU dataset. The experimental results indicate our method performs the best in overall quality compared with previous methods, proving the effectiveness of our method. In the end, we use the data collected by mobile devices to implement 3D reconstruction with a combination of traditional and learning-based methods, providing ideas for the 3D reconstruction technology on mobile devices.

EPGNet: Enhanced Point Cloud Generation for 3D Object Detection.

Three-dimensional object detection from point cloud data is becoming more and more significant, especially for autonomous driving applications. However, it is difficult for lidar to obtain the complete structure of an object in a real scene due to its scanning characteristics. Although the existing methods have made great progress, most of them ignore the prior information of object structure, such as symmetry. So, in this paper, we use the symmetry of the object to complete the missing part in the point cloud and then detect it. Specifically, we propose a two-stage detection framework. In the first stage, we adopt an encoder–decoder structure to generate the symmetry points of the foreground points and make the symmetry points and the non-empty voxel centers form an enhanced point cloud. In the second stage, the enhanced point cloud is input into the baseline, which is an anchor-based region proposal network, to generate the detection results. Extensive experiments on the challenging KITTI benchmark show the effectiveness of our method, which has better performance on both 3D and BEV (bird’s eye view) object detection compared with some previous state-of-the-art methods.

PolSAR Image Classification with Lightweight 3D Convolutional Networks

Convolutional neural networks (CNNs) have become the state-of-the-art in optical image processing. Recently, CNNs have been used in polarimetric synthetic aperture radar (PolSAR) image classification and obtained promising results. Unlike optical images, the unique phase information of PolSAR data expresses the structure information of objects. This special data representation makes 3D convolution which explicitly modeling the relationship between polarimetric channels perform better in the task of PolSAR image classification. However, the development of deep 3D-CNNs will cause a huge number of model parameters and expensive computational costs, which not only leads to the decrease of the interpretation speed during testing, but also greatly increases the risk of over-fitting. To alleviate this problem, a lightweight 3D-CNN framework that compresses 3D-CNNs from two aspects is proposed in this paper. Lightweight convolution operations, i.e., pseudo-3D and 3D-depthwise separable convolutions, are considered as low-latency replacements for vanilla 3D convolution. Further, fully connected layers are replaced by global average pooling to reduce the number of model parameters so as to save the memory. Under the specific classification task, the proposed methods can reduce up to 69.83% of the model parameters in convolution layers of the 3D-CNN as well as almost all the model parameters in fully connected layers, which ensures the fast PolSAR interpretation. Experiments on three PolSAR benchmark datasets, i.e., AIRSAR Flevoland, ESAR Oberpfaffenhofen, EMISAR Foulum, show that the proposed lightweight architectures can not only maintain but also slightly improve the accuracy under various criteria.

Single Image Snow Removal Using Sparse Representation and Particle Swarm Optimizer

Images are often corrupted by natural obscuration (e.g., snow, rain, and haze) during acquisition in bad weather conditions. The removal of snowflakes from only a single image is a challenging task due to situational variety and has been investigated only rarely. In this article, we propose a novel snow removal framework for a single image, which can be separated into a sparse image approximation module and an adaptive tolerance optimization module. The first proposed module takes the advantage of sparsity-based regularization to reconstruct a potential snow-free image. An auto-tuning mechanism for this framework is then proposed to seek a better reconstruction of a snow-free image via the time-varying inertia weight particle swarm optimizers in the second proposed module. Through collaboration of these two modules iteratively, the number of snowflakes in the reconstructed image is reduced as generations progress. By the experimental results, the proposed method achieves a better efficacy of snow removal than do other state-of-the-art techniques via both objective and subjective evaluations. As a result, the proposed method is able to remove snowflakes successfully from only a single image while preserving most original object structure information.

USING EDGECONV TO IMPROVE 3D OBJECT DETECTION FROM RGB-D DATA

Abstract. In this paper, we proposed a novel 3D deep learning model for object localization and object bounding boxes estimation. To increase the detection efficiency of small objects in the large scale scenes, the local neighbourhood geometric structure information of objects has been taken into the Edgeconv model, which can operate the original point clouds. We evaluated the 3D bounding box with high resolution in the RGB-D dataset and acquired stable effectiveness even under the sparse points and the strong occlusion. The experimental results indicate that our method achieved the higher mean average precision and better IOU of bounding boxes in SUN RGB-D dataset and KITTI benchmark.

Visual object tracking with multi-scale superpixels and color-feature guided kernelized correlation filters

While discriminative correlation filter based tracking algorithms have achieved competitive results and demonstrated successfully, there still remain some challenges, such as handling the scenarios of scale variation, fast motion, etc. A region of interest with a fixed size, which is usually used by discriminative correlation filter based tracking algorithms to train correlation filter and track object, makes the trackers hard to deal with the challenges of fast motion and scale variation. It also restricts the use of object structure information. In this paper, we propose a Multi-Scale Superpixels and Color Feature Guided Kernelized Correlation Filters (MSSCF-KCF) to deal with the problems mentioned above. Firstly, we treat tracking procedure as optimizing the combination of components of an object, and propose a multi-scale superpixel method to segment object image based on a proposed global confidence mask which determines the center and size of object patches automatically. Then, KCF is embedded into Bayesian filter framework, in which the proposed color guided confidence map is viewed as an observation model, to expand the tracking range and improve the performance when handling fast motion. Finally, we propose a center distance matrix of each object patch to use structure information of object. A novel min–max criterion as well as drop-out strategy is used to search the optimal combination and estimate the state of object, which also helps the tracker to cope with scale variation. Besides, monitoring update strategy is proposed to monitor tracking operation and adjust parameters dynamically. The proposed algorithm is tested on a self-made dataset and some challenging sequences with a novel evaluation criterion. The results demonstrate the feasibility and effectiveness compared with the state-of-the-art methods.

Deterministic object tracking using Gaussian ringlet and directional edge features

Challenges currently existing for intensity-based histogram feature tracking methods in wide area motion imagery (WAMI) data include object structural information distortions, background variations, and object scale change. These issues are caused by different pavement or ground types and from changing the sensor or altitude. All of these challenges need to be overcome in order to have a robust object tracker, while attaining a computation time appropriate for real-time processing. To achieve this, we present a novel method, Directional Ringlet Intensity Feature Transform (DRIFT), which employs Kirsch kernel filtering for edge features and a ringlet feature mapping for rotational invariance. The method also includes an automatic scale change component to obtain accurate object boundaries and improvements for lowering computation times. We evaluated the DRIFT algorithm on two challenging WAMI datasets, namely Columbus Large Image Format (CLIF) and Large Area Image Recorder (LAIR), to evaluate its robustness and efficiency. Additional evaluations on general tracking video sequences are performed using the Visual Tracker Benchmark and Visual Object Tracking 2014 databases to demonstrate the algorithms ability with additional challenges in long complex sequences including scale change. Experimental results show that the proposed approach yields competitive results compared to state-of-the-art object tracking methods on the testing datasets.

Object-Oriented Island Land Cover Change Detection by Iteratively Reweighted Multivariate Statistical Analysis

ABSTRACTIn studies of changes in island geographic information, a key topic is island land cover change detection using multitemporal remote sensing images; such information can provide good reference for island exploitation, marine resource conservation, marine economic development, and national defense building. This study proposes a fast and easy-to-implement algorithm of object-oriented change detection based on iteratively reweighted multivariate statistical analysis with high-resolution remote sensing images and vector data, improving effects of normal multivariate alternative detection by combining object-oriented technology with probability density function of chi-square distribution. This method builds vectors in feature space with spectral, texture, and spatial structure information of objects segmented based on boundaries and categories of features in vector data. By feature optimization and iterative extension to multivariate statistical analysis for change detection, change regions can be extracted. Experiments on Zhaoshu Island area show that the proposed method is highly effective and accurate in detection of change regions.

Three-Dimensional Representations of Objects in Dorsal Cortex are Dissociable from Those in Ventral Cortex.

An established conceptualization of visual cortical function is one in which ventral regions mediate object perception while dorsal regions support spatial information processing and visually guided action. This division has been contested by evidence showing that dorsal regions are also engaged in the representation of object shape, even when actions are not required. The critical question is whether these dorsal, object-based representations are dissociable from ventral representations, and whether they play a functional role in object recognition. We examined the neural and behavioral profile of patients with impairments in object recognition following ventral cortex damage. In a functional magnetic resonanace imaging experiment, the blood oxygen level-dependent response in the ventral, but not dorsal, cortex of the patients evinced less sensitivity to object 3D structure compared with that of healthy controls. Consistently, in psychophysics experiments, the patients exhibited significant impairments in object perception, but still revealed residual sensitivity to object-based structural information. Together, these findings suggest that, although in the intact system there is considerable crosstalk between dorsal and ventral cortices, object representations in dorsal cortex can be computed independently from those in ventral cortex. While dorsal representations alone are unable to support normal object perception, they can, nevertheless, support a coarse description of object structural information.

Robust tensor subspace learning for anomaly detection

Background modeling plays an important role in many applications of computer vision such as anomaly detection and visual tracking. Most existing algorithms for learning appearance model are vector-based methods without maintaining the 2D spatial structure information of objects in an image. To this end, a robust tensor subspace learning algorithm is developed for background modeling which can capture the appearance changes through adaptively updating the tensor subspace. In the tensor framework, the spatial structure information is maintained and utilized for feature extraction of objects. Then by incorporating the robust scheme, we can weight individual pixel of an image to reduce the influence of outliers on background modeling. Furthermore an incremental algorithm for the robust tensor subspace learning is proposed to adapt to the variation of appearance model. The experimental results illustrate the effectiveness of the proposed robust learning algorithm for anomaly detection.

TWin support tensor machines for MCs detection

Tensor representation is useful to reduce the overfitting problem in vector-based learning algorithm in pattern recognition. This is mainly because the structure information of objects in pattern analysis is a reasonable constraint to reduce the number of unknown parameters used to model a classifier. In this paper, we generalize the vector-based learning algorithm TWin Support Vector Machine (TWSVM) to the tensor-based method TWin Support Tensor Machines (TWSTM), which accepts general tensors as input. To examine the effectiveness of TWSTM, we implement the TWSTM method for Microcalcification Clusters (MCs) detection. In the tensor subspace domain, the MCs detection procedure is formulated as a supervised learning and classification problem, and TWSTM is used as a classifier to make decision for the presence of MCs or not. A large number of experiments were carried out to evaluate and compare the performance of the proposed MCs detection algorithm. By comparison with TWSVM, the tensor version reduces the overfitting problem.

High-resolution, wide-field object reconstruction with synthetic aperture Fourier holographic optical microscopy

We utilize synthetic-aperture Fourier holographic microscopy to resolve micrometer-scale microstructure over millimeter-scale fields of view. Multiple holograms are recorded, each registering a different, limited region of the sample object's Fourier spectrum. They are "stitched together" to generate the synthetic aperture. A low-numerical-aperture (NA) objective lens provides the wide field of view, and the additional advantages of a long working distance, no immersion fluids, and an inexpensive, simple optical system. Following the first theoretical treatment of the technique, we present images of a microchip target derived from an annular synthetic aperture (NA = 0.61) whose area is 15 times that due to a single hologram (NA = 0.13); they exhibit a corresponding qualitative improvement. We demonstrate that a high-quality reconstruction may be obtained from a limited sub-region of Fourier space, if the object's structural information is concentrated there.