Object Extraction Research Articles

Mapping Paddy Rice with Sentinel-1/2 and Phenology-, Object-Based Algorithm—A Implementation in Hangjiahu Plain in China Using GEE Platform

In tropical/subtropical monsoon regions, accurate rice mapping is hampered by the following factors: (1) The frequent occurrence of clouds in such areas during the rice-growing season interferes strongly with optical remote sensing observations; (2) The agro-landscape in such regions is fragmented and scattered. Rice maps produced using low spatial resolution data cannot well delineate the detailed distribution of rice, while pixel-based mapping using medium and high resolutions has significant salt-and-pepper noise. (3) The cropping system is complex, and rice has a rotation schedule with other crops. Therefore, the Phenology-, Object- and Double Source-based (PODS) paddy rice mapping algorithm is implemented, which consists of three steps: (1) object extraction from multi-temporal 10-m Sentinel-2 images where the extracted objects (fields) are the basic classification units; (2) specifying the phenological stage of transplanting from Savitzky–Golay filtered enhanced vegetation index (EVI) time series using the PhenoRice algorithm; and (3) the identification of rice objects based on flood signal detection from time-series microwave and optical signals of the Sentinel-1/2. This study evaluated the potential of the combined use of the Sentinel-1/2 mission on paddy rice mapping in monsoon regions with the Hangzhou-Jiaxin-Huzhou (HJH) plain in China as the case study. A cloud computing approach was used to process the available Sentinel-1/2 imagery from 2019 and MODIS images from 2018 to 2020 in the HJH plain on the Google Earth Engine (GEE) platform. An accuracy assessment showed that the resultant object-based paddy rice map has a high accuracy with a producer (user) accuracy of 0.937 (0.926). The resultant 10-m paddy rice map is expected to provide unprecedented detail, spatial distribution, and landscape patterns for paddy rice fields in monsoon regions.

Fusion of Infrared and Visible Images Based on Infrared Object Extraction

The ideal fused results of infrared and visible images, should contain the important infrared objects, and preserve the visible textural detail information as much as possible. The fused images are more consistent with human visual perception effect. For this purpose, a novel infrared and visible image fusion framework is proposed. Under the guidance of the model, the source images are decomposed into largescale edge, small-scale textural detail and coarse-scale base level information. Among which, the large-scale edge information contains the main infrared features, on this basis, the infrared image is further segmented into the object, transition and background regions by OTSU multi-threshold segmentation algorithm. In the end, the fused weights for the decomposed sub-information are determined by the segmented results, so that, the infrared object information can be effectively injected into the fused image, and the important visible textural detail information can be preserved as much as possible in the fused image. Experimental results show that, the proposed method can not only highlight the infrared objects, but also preserve the visual information in the visible image as much as possible. The fused results are superior to the commonly used representative fusion methods, both in subjective perception and objective evaluation.

Automated Extraction and Presentation of Data Practices in Privacy Policies

AbstractPrivacy policies are documents required by law and regulations that notify users of the collection, use, and sharing of their personal information on services or applications. While the extraction of personal data objects and their usage thereon is one of the fundamental steps in their automated analysis, it remains challenging due to the complex policy statements written in legal (vague) language. Prior work is limited by small/generated datasets and manually created rules. We formulate the extraction of fine-grained personal data phrases and the corresponding data collection or sharing practices as a sequence-labeling problem that can be solved by an entity-recognition model. We create a large dataset with 4.1k sentences (97k tokens) and 2.6k annotated fine-grained data practices from 30 real-world privacy policies to train and evaluate neural networks. We present a fully automated system, called PI-Extract, which accurately extracts privacy practices by a neural model and outperforms, by a large margin, strong rule-based baselines. We conduct a user study on the effects of data practice annotation which highlights and describes the data practices extracted by PI-Extract to help users better understand privacy-policy documents. Our experimental evaluation results show that the annotation significantly improves the users’ reading comprehension of policy texts, as indicated by a 26.6% increase in the average total reading score.

A Review of Tree Species Classification Based on Airborne LiDAR Data and Applied Classifiers

Remote sensing techniques, developed over the past four decades, have enabled large-scale forest inventory. Light Detection and Ranging (LiDAR), as an active remote sensing technology, allows for the acquisition of three-dimensional point clouds of scanned areas, as well as a range of features allowing for increased performance of object extraction and classification approaches. As many publications have shown, multiple LiDAR-derived metrics, with the assistance of classification algorithms, contribute to the high accuracy of tree species discrimination based on data obtained by laser scanning. The aim of this article is to review studies in the species classification literature which used data collected by Airborne Laser Scanning. We analyzed these studies to figure out the most efficient group of LiDAR-derived features in species discrimination. We also identified the most powerful classification algorithm, which maximizes the advantages of the derived metrics to increase species discrimination performance. We conclude that features extracted from full-waveform data lead to the highest overall accuracy. Radiometric features with height information are also promising, generating high species classification accuracies. Using random forest and support vector machine as classifiers gave the best species discrimination results in the reviewed publications.

Updated Object Extraction in Underground Facility based on Centroid

Updated Object Extraction in Underground Facility based on Centroid

HADEM-MACS: a hybrid approach for detection and extraction of objects in movement by multimedia autonomous computer systems

Nowadays, we assist in the multiplication of applications that need to exactly identify objects crossing a field watched by cameras. Multimedia information thus becomes an inescapable medium for the validation of applications such as identification, localisation and object tracking. This gives rise to many processing methods that, after collecting multimedia data (images, videos), continue with a pre-processing in order to reduce noise, finally it finishes with processing in order to extract objects, more precisely the form of object that capture our domain of interest. The first stage consists of detecting objects in movement in the scene. This detection passes through a background modelling. Model based on mixture of Gaussian is commonly used and it is classified in the categories of parametric approaches. However, many other approaches exist today in literature each of them having its advantages and drawbacks. We present in this paper a hybrid approach of movement detection which combines an improved version of mixture of Gaussian with a neural network-based approach.

Contour Extraction of Individual Cattle From an Image Using Enhanced Mask R-CNN Instance Segmentation Method

In animal husbandry, the traceability of individual cattle, their health information, and performance records greatly depend on computer vision and image processing-based approaches. However, some of these approaches perform below expectations in obtaining real-time information about individual cattle. No doubt, inaccurate segmentation and incomplete extraction of each cattle object from an image are notable contributory factors. As accurate segmentation is a prerequisite for obtaining real-time information about individual cattle, and since the algorithm of Mask R-CNN relies on the algorithm of simultaneous localization and mapping (SLAM), for the construction of the semantic map, which sometimes exchanges image background for the foreground, there is a need to enhance the available approaches towards achieving precision animal husbandry. To achieve this, an enhanced Mask R-CNN instance segmentation method is proposed to support indistinct boundaries and irregular shapes of cattle bodies. The methods employed in the research are in multiple folds: (1) Pre-enhancement of the image using generalized color Fourier descriptors (GCFD); (2) Provision of optimal filter size that was smaller than ResNet101 (the backbone of Mask R-CNN) for the extraction of smaller and composite features; (3) Utilization of multiscale semantic features using region proposals; (4) A fully connected layer of existing Mask R-CNN integrated with a sub-network for enhanced segmentation and (5) Post-enhancement of the image using Grabcut. Experiments on the datasets of cattle images produced better results when compared to other state-of-the-art methods with 0.93 mAP.

Research on Adaptive Component Analysis Method of Spectral Reflectance Reconstruction

Spectral reflectance reconstruction is the key technology of multi-spectral color reproduction, and it solves the exact color information restoring of original scene to provide color information support for high-fidelity reproduction. The current mainstream principal component analysis method is suitable for information reconstruction of simple objects and smooth objects, and the independent component analysis method is adaptive for color main component extraction of complex objects or scenes. Integrating the advantages of these two methods and imported blind source signal estimation theory, this study highlights the adaptive component analysis method for spectral reflectance reconstruction. Firstly it clarified the reconstruction principle and method of adaptive component analysis methods, and then it carried on the spectral reflectance reconstruction test by selecting the typical color lumps of Finland University “AOTF Munsell Color Matt” spectrum dataset. The results showed the reconstruction precision was higher and the spectral matching skewness index was very small (less than 0.020 basic), besides the reconstruction efficiency was higher and the method adaptability was stronger. Moreover, this study provided a new theoretical interpretation for Color Constancy Theory of human vision.

КОМПЛЕКТ ЖЕЛЕЗНЫХ ИЗДЕЛИЙ ПЕРИОДА РАННЕГО СРЕДНЕВЕКОВЬЯ ИЗ ЗАРИНСКОГО РАЙОНА АЛТАЙСКОГО КРАЯ

The Zarinsky District in archaeological terms is one of the least studied territory of the Altai Territory. Systematic surveys were rarely carried out there. However, this region is important for the study of ancient and medieval history, as it is located on the border of Western and Southern Siberia. Recently, information about archaeological finds that were obtained using metal detectors has become frequent. The article examines the situation recorded by the competent authorities associated with the partial destruction of a historical and cultural monument of federal significance and the illegal extraction of iron objects from it. The study of the confiscated artifacts was carried out on the basis of an official request from the Directorate of the FSB of Russia in the Altai Territory. Detailed descriptions of artifacts (bit, adze, two arrowheads) are given, which are supplemented by detailed illustrations. It was concluded that the found set belongs to the period of the early Middle Ages and may be associated with the Srostki culture. Keywords: Altai Territory, Zarinsky District, Starodrachenino, archaeological site, earlier Middle Ages, barrow, bit, adze, arrowhead

Image Co-Skeletonization via Co-Segmentation.

Recent advances in the joint processing of a set of images have shown its advantages over individual processing. Unlike the existing works geared towards co-segmentation or co-localization, in this article, we explore a new joint processing topic: image co-skeletonization, which is defined as joint skeleton extraction of the foreground objects in an image collection. It is well known that object skeletonization in a single natural image is challenging, because there is hardly any prior knowledge available about the object present in the image. Therefore, we resort to the idea of image co-skeletonization, hoping that the commonness prior that exists across the semantically similar images can be leveraged to have such knowledge, similar to other joint processing problems such as co-segmentation. Moreover, earlier research has found that augmenting a skeletonization process with the object's shape information is highly beneficial in capturing the image context. Having made these two observations, we propose a coupled framework for co-skeletonization and co-segmentation tasks to facilitate shape information discovery for our co-skeletonization process through the co-segmentation process. While image co-skeletonization is our primary goal, the co-segmentation process might also benefit, in turn, from exploiting skeleton outputs of the co-skeletonization process as central object seeds through such a coupled framework. As a result, both can benefit from each other synergistically. For evaluating image co-skeletonization results, we also construct a novel benchmark dataset by annotating nearly 1.8 K images and dividing them into 38 semantic categories. Although the proposed idea is essentially a weakly supervised method, it can also be employed in supervised and unsupervised scenarios. Extensive experiments demonstrate that the proposed method achieves promising results in all three scenarios.

Object Detection-Tracking Algorithm for Unmanned Surface Vehicles Based on a Radar-Photoelectric System

Object tracking is an important basis for the autonomous navigation of unmanned surface vehicles. However, several problems still must be addressed for a wide applicating of object tracking in unmanned surface vehicles. First, if multiple objects of the same classification exist in the same field of view, then stable extraction of an object is difficult. Second, in an environment with a complex background and large changes in object shape, the tracking accuracy is low, and object tracking errors and tracking loss can easily occur. Third, much time is required to detect a high-resolution real-time video stream, not meeting the delay requirement of the photoelectric servo stable tracking. To resolve these problems, this paper proposes an object detection-tracking algorithm based on a radar-photoelectric system. The algorithm combines an object detection algorithm with an object tracking algorithm and involves the following steps. First, a first-frame object extraction algorithm is used to extract the tracking object from the first frame. Second, a region of interest (ROI)-prediction algorithm is used to predict ROIs and detect objects in these ROIs. This algorithm can effectively solve the above problems in marine tests. When multiple objects of the same classification exist in the same field of view, the algorithm can extract the radar-guided object stably. When faced with a complex background and a large change in object shape, the algorithm substantially improves the accuracy and robustness of object tracking. Compared with the conventional object detection algorithm, the time consumption of this algorithm is reduced by 25.8%.

Shape-Preserving Stereo Object Remapping via Object-Consistent Grid Warping.

Viewing various stereo images under different viewing conditions has escalated the need for effective object-level remapping techniques. In this paper, we propose a new object spatial mapping scheme, which adjusts the depth and size of the selected object to match user preference and viewing conditions. Existing warping-based methods often distort the shape of important objects or cannot faithfully adjust the depth/size of the selected object due to improper warping such as local rotations. In this paper, by explicitly reducing the transformation freedom degree of warping, we propose an optimization model based on axis-aligned warping for object spatial remapping. The proposed axis-aligned warping based optimization model can simultaneously adjust the depths and sizes of selected objects to their target values without introducing severe shape distortions. Moreover, we propose object consistency constraints to ensure the size/shape of parts inside a selected object to be consistently adjusted. Such constraints improve the size/shape adjustment performance while remaining robust to some extent to incomplete object extraction. Experimental results demonstrate that the proposed method achieves high flexibility and effectiveness in adjusting the size and depth of objects compared with existing methods.

Extraction of Real-World Objects in the Environment through Computer Vision

Extraction of Real-World Objects in the Environment through Computer Vision

A Study on Updated Object Detection and Extraction of Underground Information

A Study on Updated Object Detection and Extraction of Underground Information

Objective Extraction of Evoked Event-Related Oscillation from Time-Frequency Representation of Event-Related Potentials

Evoked event-related oscillations (EROs) have been widely used to explore the mechanisms of brain activities for both normal people and neuropsychiatric disease patients. In most previous studies, the calculation of the regions of evoked EROs of interest is commonly based on a predefined time window and a frequency range given by the experimenter, which tends to be subjective. Additionally, evoked EROs sometimes cannot be fully extracted using the conventional time-frequency analysis (TFA) because they may be overlapped with each other or with artifacts in time, frequency, and space domains. To further investigate the related neuronal processes, a novel approach was proposed including three steps: (1) extract the temporal and spatial components of interest simultaneously by temporal principal component analysis (PCA) and Promax rotation and project them to the electrode fields for correcting their variance and polarity indeterminacies, (2) calculate the time-frequency representations (TFRs) of the back-projected components, and (3) compute the regions of evoked EROs of interest on TFRs objectively using the edge detection algorithm. We performed this novel approach, conventional TFA, and TFA-PCA to analyse both the synthetic datasets with different levels of SNR and an actual ERP dataset in a two-factor paradigm of waiting time (short/long) and feedback (loss/gain) separately. Synthetic datasets results indicated that N2-theta and P3-delta oscillations can be stably detected from different SNR-simulated datasets using the proposed approach, but, by comparison, only one oscillation was obtained via the last two approaches. Furthermore, regarding the actual dataset, the statistical results for the proposed approach revealed that P3-delta was sensitive to the waiting time but not for that of the other approaches. This study manifested that the proposed approach could objectively extract evoked EROs of interest, which allows a better understanding of the modulations of the oscillatory responses.

Semantic Structured Image Coding Framework for Multiple Intelligent Applications

Fast-growing intelligent media processing applications demand efficient processing throughout the processing chain from the edge to the cloud, and the complexity bottleneck usually lies in the parallel decoding of multiple-channel compressed bitstreams before analyzing. This occurs because the traditional media coding scheme generates a binary stream without a semantic structure, which is unable to be operated directly at the bitstream level to support different tasks such as classification, recognition, detection, etc. Therefore, in this article, we propose a learning-based semantically structured image coding (SSIC) framework to generate a semantically structured bitstream (SSB), where each part of the bitstream represents a specific object and can be directly used for the aforementioned intelligent tasks. Specifically, we integrate an object location extraction module into the compression framework to locate and align objects in the feature domain. Then, each object together with the background is compressed separately and reorganized to form a structured bitstream to enable the analysis or reconstruction of specific objects directly from partial bitstream. Furthermore, in contrast to existing learning-based compression schemes that train the specific model for a specific bitrate, we share most of the model parameters among various bitrates to significantly reduce the model size for variable-rate compression. The experimental results demonstrate the effectiveness of the proposed coding scheme whose compression performance is comparable to existing image coding schemes, where intelligent tasks such as classification and pose estimation can be directly performed on a partial bitstream without performance degradation, significantly reducing the complexity for analyzing tasks.

Pole-Like Object Extraction and Pole-Aided GNSS/IMU/LiDAR-SLAM System in Urban Area.

Vision-based sensors such as LiDAR (Light Detection and Ranging) are adopted in the SLAM (Simultaneous Localization and Mapping) system. In the 16-beam LiDAR aided SLAM system, due to the difficulty of object detection by sparse laser data, neither the grid-based nor feature point-based solution can avoid the interference of moving objects. In an urban environment, the pole-like objects are common, invariant and have distinguishing characteristics. Therefore, it is suitable to bring more robust and reliable positioning results as auxiliary information in the process of vehicle positioning and navigation. In this work, we proposed a scheme of a SLAM system using a GNSS (Global Navigation Satellite System), IMU (Inertial Measurement Unit) and LiDAR sensor using the position of pole-like objects as the features for SLAM. The scheme combines a traditional preprocessing method and a small scale artificial neural network to extract the pole-like objects in environment. Firstly, the threshold-based method is used to extract the pole-like object candidates from the point cloud, and then, the neural network is applied for training and inference to obtain pole-like objects. The result shows that the accuracy and recall rate are sufficient to provide stable observation for the following SLAM process. After extracting the poles from the LiDAR point cloud, their coordinates are added to the feature map, and the nonlinear optimization of the front end is carried out by utilizing the distance constraints corresponding to the pole coordinates; then, the heading angle and horizontal plane translation are estimated. The ground feature points are used to enhance the elevation, pitch and roll angle accuracy. The performance of the proposed navigation system is evaluated through field experiments by checking the position drift and attitude errors during multiple two-min mimic GNSS outages without additional IMU motion constrain such as NHC (Nonholonomic Constrain). The experimental results show that the performance of the proposed scheme is superior to that of the conventional feature point grid-based SLAM with the same back end, especially in congested crossroads where slow-moving vehicles are surrounded and pole-like objects are rich in the environment. The mean plane position error during two-min GNSS outages was reduced by 38.5%, and the root mean square error was reduced by 35.3%. Therefore, the proposed pole-like feature-based GNSS/IMU/LiDAR SLAM system can fuse condensed information from those sensors effectively to mitigate positioning and orientation errors, even in a short-time GNSS denied environment.

Efficient point cloud segmentation approach using energy optimization with geometric features for 3D scene understanding.

Efficient and quick extraction of unknown objects in cluttered 3D scenes plays a significant role in robotics tasks such as object search, grasping, and manipulation. This paper describes a geometric-based unsupervised approach for the segmentation of cluttered scenes into objects. The proposed method first over-segments the raw point clouds into supervoxels to provide a more natural representation of 3D point clouds and reduce the computational cost with a minimal loss of geometric information. Then the fully connected local area linkage graph is used to distinguish between planar and nonplanar adjacent patches. Then the initial segmentation is completed utilizing the geometric features and local surface convexities. After the initial segmentation, many subgraphs are generated, each of which represents an individual object or part of it. Finally, we use the plane extracted from the scene to refine the initial segmentation result under the framework of global energy optimization. Experiments on the Object Cluttered Indoor Dataset dataset indicate that the proposed method can outperform the representative segmentation algorithms in terms of weighted overlap and accuracy, while our method has good robustness and real-time performance.

Comparison of Different U-Net Models for Building Extraction from High-Resolution Aerial Imagery

Building extraction from high-resolution aerial imagery plays an important role in geospatial applications such as urban planning, telecommunication, disaster monitoring, navigation, updating geographic databases, and urban dynamic monitoring. Automatic building extraction is a challenging task, as the buildings in different regions have different spectral and geometric properties. Therefore, the classical image processing techniques are not sufficient for automatic building extraction from high-resolution aerial imagery applications. Deep learning and semantic segmentation models, which have gained popularity in recent years, have been used for automatic object extraction from high-resolution images. U-Net model, which was originally developed for biomedical image processing, was used for building extraction. The encoder part of the U-Net model has been modified with Vgg16, InceptionResNetV2, and DenseNet121 convolutional neural networks. Therefore, building extraction was performed using Vgg16 U-Net, InceptionResNetV2 U-Net, and DenseNet121 U-Net models. In the fourth method, the results obtained from each U-Net model were combined in order to obtain the final result by maximum voting. This study aims to compare the performance of these four methods in building extraction from high-resolution aerial imagery. Images of Chicago from the Inria Aerial Image Labeling Dataset were used in the study. The images used have 0.3 m spatial resolution, 8-bit radiometric resolution and 3-band (red, green, and blue bands). Images consist of 36 tiles and they were divided into image subsets of 512x512 pixels. Thus, a total of 2715 image subsets were formed. 80% of the image subsets (2172 image subset) were used as training and 20% (543 image subset) as testing. To evaluate the accuracy of methods, the F1 score of the building class was employed. The F1 scores for building class have been calculated as 0.866, 0.860, 0.856, and 0.877 on test images for U-Net Vgg16, U-Net InceptionResNetV2, U-Net DenseNet121, and majority voting method, respectively.

Maximum a posterior based level set approach for image segmentation with intensity inhomogeneity

Intensity inhomogeneity is an unavoidable obstacle in image segmentation, which causes inaccuracy in object extraction. Generally, the approach to tackling intensity inhomogeneity is constructing a bias field descriptor which may lead to corruption of image intensity. In this paper, we propose a novel level set model based on maximum a posterior principle. To properly collect the objectslocal information, the proposed method utilizes Gaussian distribution to model the conditional probability of image intensity within specific patches. To construct the prior information, we then model the intensity inhomogeneity as Gaussian distribution whose mean is 1 and whose variance is the same as image intensity. Finally, the maximum a posterior based energy functional combined local image information and adequate prior information is defined. In addition, our method can be adopted and transformed into the state-of-the-art methods. To validate its effectiveness and performance, we compare our method with popular deep learning methods and classical level set methods. The roubstness analysis of initial contour, noise and intensity bias is given. Experimental results show our method achieves outstanding adaptability and stability.