Hierarchical Region-based Representation Research Articles

Multi-modal RGB-D Image Segmentation from Appearance and Geometric Depth Maps

Classical image segmentation algorithms exploit the detection of similarities and discontinuities of different visual cues to define and differentiate multiple regions of interest in images. However, due to the high variability and uncertainty of image data, producing accurate results is difficult. In other words, segmentation based just on color is often insufficient for a large percentage of real-life scenes. This work presents a novel multi-modal segmentation strategy that integrates depth and appearance cues from RGB-D images by building a hierarchical region-based representation, i.e., a multi-modal segmentation tree (MM-tree). For this purpose, RGB-D image pairs are represented in a complementary fashion by different segmentation maps. Based on color images, a color segmentation tree (C-tree) is created to obtain segmented and over-segmented maps. From depth images, two independent segmentation maps are derived by computing planar and 3D edge primitives. Then, an iterative region merging process can be used to locally group the previously obtained maps into the MM-tree. Finally, the top emerging MM-tree level coherently integrates the available information from depth and appearance maps. The experiments were conducted using the NYU-Depth V2 RGB-D dataset, which demonstrated the competitive results of our strategy compared to state-of-the-art segmentation methods. Specifically, using test images, our method reached average scores of 0.56 in Segmentation Covering and 2.13 in Variation of Information.

Object recognition in hyperspectral images using Binary Partition Tree representation

In this work, an image representation based on Binary Partition Tree is proposed for object detection in hyperspectral images. This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure, which succeeds in presenting: (i) the decomposition of the image in terms of coherent regions and (ii) the inclusion relations of the regions in the scene. Hence, the BPT representation defines a search space for constructing a robust object identification scheme. Spatial and spectral information are integrated in order to analyze hyperspectral images with a region-based perspective. For each region represented in the BPT, spatial and spectral descriptors are computed and the likelihood that they correspond to an instantiation of the object of interest is evaluated. Experimental results demonstrate the good performances of this BPT-based approach.

Hyperspectral image segmentation using a new spectral unmixing-based binary partition tree representation.

The binary partition tree (BPT) is a hierarchical region-based representation of an image in a tree structure. The BPT allows users to explore the image at different segmentation scales. Often, the tree is pruned to get a more compact representation and so the remaining nodes conform an optimal partition for some given task. Here, we propose a novel BPT construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts. Linear spectral unmixing consists of finding the spectral signatures of the materials present in the image (endmembers) and their fractional abundances within each pixel. The proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error. Results are presented on real hyperspectral data sets with different contexts and resolutions.

Depth order estimation for video frames using motion occlusions

This study proposes a system to estimate the depth order of regions belonging to a monocular image sequence. For each frame, the regions are ordered according to their relative depth using information from the previous and following frames. The algorithm estimates occlusions relying on a hierarchical region-based representation of the image by means of a binary tree. This representation is used to define the final depth order partition which is obtained through an energy minimisation process. Finally, to achieve a global and consistent depth ordering, a depth order graph is constructed and used to eliminate contradictory local cues. The system is evaluated and compared with the state-of-the-art figure/ground labelling systems showing very good results.

Hyperspectral Image Representation and Processing With Binary Partition Trees

The optimal exploitation of the information provided by hyperspectral images requires the development of advanced image-processing tools. This paper proposes the construction and the processing of a new region-based hierarchical hyperspectral image representation relying on the binary partition tree (BPT). This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure. Hence, the BPT succeeds in presenting: 1) the decomposition of the image in terms of coherent regions, and 2) the inclusion relations of the regions in the scene. Based on region-merging techniques, the BPT construction is investigated by studying the hyperspectral region models and the associated similarity metrics. Once the BPT is constructed, the fixed tree structure allows implementing efficient and advanced application-dependent techniques on it. The application-dependent processing of BPT is generally implemented through a specific pruning of the tree. In this paper, a pruning strategy is proposed and discussed in a classification context. Experimental results on various hyperspectral data sets demonstrate the interest and the good performances of the BPT representation.

Processing Multidimensional SAR and Hyperspectral Images With Binary Partition Tree

The current increase of spatial as well as spectral resolutions of modern remote sensing sensors represents a real opportunity for many practical applications but also generates important challenges in terms of image processing. In particular, the spatial correlation between pixels and/or the spectral correlation between spectral bands of a given pixel cannot be ignored. The traditional pixel-based representation of images does not facilitate the handling of these correlations. In this paper, we discuss the interest of a particular hierarchical region-based representation of images based on binary partition tree (BPT). This representation approach is very flexible as it can be applied to any type of image. Here both optical and radar images will be discussed. Moreover, once the image representation is computed, it can be used for many different applications. Filtering, segmentation, and classification will be detailed in this paper. In all cases, the interest of the BPT representation over the classical pixel-based representation will be highlighted.

Connected operators

Connected operators are filtering tools that act by merging elementary regions called flat zones. Connecting operators cannot create new contours nor modify their position. Therefore, they have very good contour-preservation properties and are capable of both low-level filtering and higher-level object recognition. This article gives an overview on connected operators and their application to image and video filtering. There are two popular techniques used to create connected operators. The first one relies on a reconstruction process. The operator involves first a simplification step based on a "classical" filter and then a reconstruction process. In fact, the reconstruction can be seen as a way to create a connected version of an arbitrary operator. The simplification effect is defined and limited by the first step. The examples we show include simplification in terms of size or contrast. The second strategy to define connected operators relies on a hierarchical region-based representation of the input image, i.e., a tree, computed in an initial step. Then, the simplification is obtained by pruning the tree, and, third, the output image is constructed from the pruned tree. This article presents the most important trees that have been used to create connected operators and also discusses important families of simplification or pruning criteria. We also give a brief overview on efficient implementations of the reconstruction process and of tree construction. Finally, the possibility to define and to use nonclassical notions of connectivity is discussed and illustrated.

Trajectory Tree as an Object-Oriented Hierarchical Representation for Video

This paper presents the trajectory tree as a hierarchical region-based representation for video sequences. Motion, as well as spatial features from multiple frames are used to generate a set of temporal regions structured within a hierarchy of scale and motion coherency. The resulting representations offer a global description of the entire video sequence and enhance semantic analysis potential. A multiscale segmentation strategy is proposed whereby region-merging criteria of progressively greater complexity are used to define partition layers of increasing aptitude for object detection. A novel data structure, called the trajectory adjacency graph, is defined for the long-term analysis of partition sequences. Furthermore, mechanisms for assessing connectivity, verifying temporal continuity, and proposing merging operations based on color, affine, and translational motion homogeneity characteristics over the entire sequence are also introduced. Finally, as demonstrated through experimental results, the trajectory tree offers a concise yet versatile support for video object segmentation, description and retrieval tasks.