Hierarchical Segmentation Research Articles

RULE-BASED CLASSIFICATION OF A HYPERSPECTRAL IMAGE USING MSSC HIERARCHICAL SEGMENTATION

Abstract. The Hierarchical SEGmentation (HSEG) algorithm, which combines region object finding with region object clustering, has given good performances for hyperspectral image analysis. This technique produces at its output a hierarchical set of image segmentations. The automated selection of a single segmentation level is often necessary. In this paper, we propose to use spectral-spatial classifiers at the marker selection procedure, each of them combining the results of a pixel-wise classification and a segmentation map. Then, a novel marker-based HSEG algorithm (that is called Multiple Spectral-Spatial Classifier-HSEG (MSSC-HSEG)) is applied, resulting in a segmentation map. The segmentation results are then used in a rule-based classification using spectral, geometric, textural, and contextual information. The experimental results, presented for a hyperspectral airborne image, demonstrate that the proposed approach yields accurate segmentation and classification maps, when compared to previously classification techniques.

The function of Notch signalling in segment formation in the crustacean Daphnia magna (Branchiopoda)

Ten years ago we showed for the first time that Notch signalling is required in segmentation in spiders, indicating the existence of similar mechanisms in arthropod and vertebrate segmentation. However, conflicting results in various arthropod groups hampered our understanding of the ancestral function of Notch in arthropod segmentation. Here we fill a crucial data gap in arthropods and analyse segmentation in a crustacean embryo. We analyse the expression of homologues of the Drosophila and vertebrate segmentation genes and show that members of the Notch signalling pathway are expressed at the same time as the pair-rule genes. Furthermore, inactivation of Notch signalling results in irregular boundaries of the odd-skipped-like expression domains and affects the formation of segments. In severe cases embryos appear unsegmented. We suggest two scenarios for the function of Notch signalling in segmentation. The first scenario agrees with a segmentation clock involving Notch signalling, while the second scenario discusses an alternative mechanism of Notch function which is integrated into a hierarchical segmentation cascade.

IMPROVING SEMANTIC UPDATING METHOD ON 3D CITY MODELS USING HYBRID SEMANTIC-GEOMETRIC 3D SEGMENTATION TECHNIQUE

Abstract. Cities and urban areas entities such as building structures are becoming more complex as the modern human civilizations continue to evolve. The ability to plan and manage every territory especially the urban areas is very important to every government in the world. Planning and managing cities and urban areas based on printed maps and 2D data are getting insufficient and inefficient to cope with the complexity of the new developments in big cities. The emergence of 3D city models have boosted the efficiency in analysing and managing urban areas as the 3D data are proven to represent the real world object more accurately. It has since been adopted as the new trend in buildings and urban management and planning applications. Nowadays, many countries around the world have been generating virtual 3D representation of their major cities. The growing interest in improving the usability of 3D city models has resulted in the development of various tools for analysis based on the 3D city models. Today, 3D city models are generated for various purposes such as for tourism, location-based services, disaster management and urban planning. Meanwhile, modelling 3D objects are getting easier with the emergence of the user-friendly tools for 3D modelling available in the market. Generating 3D buildings with high accuracy also has become easier with the availability of airborne Lidar and terrestrial laser scanning equipments. The availability and accessibility to this technology makes it more sensible to analyse buildings in urban areas using 3D data as it accurately represent the real world objects. The Open Geospatial Consortium (OGC) has accepted CityGML specifications as one of the international standards for representing and exchanging spatial data, making it easier to visualize, store and manage 3D city models data efficiently. CityGML able to represents the semantics, geometry, topology and appearance of 3D city models in five well-defined Level-of-Details (LoD), namely LoD0 to LoD4. The accuracy and structural complexity of the 3D objects increases with the LoD level where LoD0 is the simplest LoD (2.5D; Digital Terrain Model (DTM) + building or roof print) while LoD4 is the most complex LoD (architectural details with interior structures). Semantic information is one of the main components in CityGML and 3D City Models, and provides important information for any analyses. However, more often than not, the semantic information is not available for the 3D city model due to the unstandardized modelling process. One of the examples is where a building is normally generated as one object (without specific feature layers such as Roof, Ground floor, Level 1, Level 2, Block A, Block B, etc). This research attempts to develop a method to improve the semantic data updating process by segmenting the 3D building into simpler parts which will make it easier for the users to select and update the semantic information. The methodology is implemented for 3D buildings in LoD2 where the buildings are generated without architectural details but with distinct roof structures. This paper also introduces hybrid semantic-geometric 3D segmentation method that deals with hierarchical segmentation of a 3D building based on its semantic value and surface characteristics, fitted by one of the predefined primitives. For future work, the segmentation method will be implemented as part of the change detection module that can detect any changes on the 3D buildings, store and retrieve semantic information of the changed structure, automatically updates the 3D models and visualize the results in a userfriendly graphical user interface (GUI).

SMOR: A Semantic Multi-View Object Representation System in 2D Image Sequences

In this work, we propose a framework for segmenting and tracking objects in a static scene using 2D image sequences taken from different viewpoints. The system includes the use of a set of simple algorithms to segment, identify, group and track the semantic objects modeled as a set of regions with compatible surface features. The integration of these algorithms for design of the system is the main contribution of our work. Previous works have considered this issue for the sequences of the static scenes containing a single object or dynamic scenes of multiple objects from a single shot. We are studying this issue for multiple objects from different shots. We present a hierarchical segmentation of image sequences into shots, regions, and objects. Shots are detected using inter-frame information related to camera motion, the regions are identified using the color information in each frame, and the objects are formed by adjacent regions with compatible shape and form. Then we track the segmented regions/objects throughout the sequence using motion information and color constancy. The result of the system is a multi-view representation of objects of a static scene. Comparisons to demonstrate the performance of the systems is provided and discussed.

Selective Search Method for Object Localization and Detection using Wavelets and Hierarchical Segmentations

This article proposes a selective search method for object localization in natural images by applying image multi-segmentation, image scaling, and heuristics. The method increases the number of generated windows that delimitate the area of an object with an accuracy superior to 50%. Over-segmentation is applied on original size images in order to locate small objects, and it is also applied over scaled images because these can still be over-segmented. This process produces less regions on areas with many textures. The over-segmentation was applied using the CIE Luv color model, and using the H and the I channels of the HSI model. The proposed method is category independent and allows the location of objects with heterogeneous characteristics by using heuristics and hierarchical segmentation. The proposed method produces 9, 366 windows per image covering 96.78% of the objects in the PASCAL VOC 2007 test image collection, increasing in 0.8% the localization results reported in the state of the art.

Learning Full Pairwise Affinities for Spectral Segmentation

Segmenting a single image into multiple coherent groups remains a challenging task in the field of computer vision. Particularly, spectral segmentation which uses the global information embedded in the spectrum of a given image's affinity matrix is a major trend in image segmentation. This paper focuses on the problem of efficiently learning a full range of pairwise affinities gained by integrating local grouping cues for spectral segmentation. We first construct a sparse multilayer graph whose nodes are both the pixels and the oversegmented regions obtained by an unsupervised segmentation algorithm. By applying the semi-supervised learning strategy to this graph, the intra and interlayer affinities between all pairs of nodes can be estimated without iteration. These pairwise affinities are then applied into the spectral segmentation algorithms. In this paper, two types of spectral segmentation algorithms are introduced: $(K)$-way segmentation and hierarchical segmentation. Our algorithms provide high-quality segmentations which preserve object details by directly incorporating the full-range connections. Moreover, since our full affinity matrix is defined by the inverse of a sparse matrix, its eigendecomposition can be efficiently computed. The experimental results on the BSDS and MSRC image databases demonstrate the superiority of our segmentation algorithms in terms of relevance and accuracy compared with existing popular methods.

Microphase Separation Facilitating and Stabilizing Hierarchical Segment Self-Assembly of Combined Main-Chain/Side-Chain Liquid Crystalline Polymer in Diblock Copolymer

A series of diblock copolymers (PS-b-PBBHCSs) composed of polystyrene (PS) and poly(2,5-bis{[6-(4-butoxy-4′-oxybiphenyl)hexyl]oxycarbonyl}styrene) (PBBHCS) were successfully synthesized via atom transfer radical polymerization, with their chemical structures confirmed by various characterization techniques. While PS is a conventional coil polymer, PBBHCS is a novel combined main-chain/side-chain liquid crystalline polymer (MCSCLCP) which can exhibit hierarchically ordered structures. Small-angle X-ray scattering results show that the diblock copolymers with different compositions could present lamellar and rectangular cylinder morphologies before the order–disorder transition. Combining the differential scanning calorimetry and X-ray scattering results, we find that the PBBHCS block in the confined space of microphase separation is still able to exhibit its liquid crystalline (LC) structures with the rectangular main-chain scaffold on the nanometer scale and the side-chain ordering on the subnanometer sca...

New evaluation metrics for mesh segmentation

3D model segmentation avails to skeleton extraction, shape partial matching, shape correspondence, texture mapping, shape deformation, and shape annotation. Many excellent solutions have been proposed in the last decade. How to efficiently evaluate these methods and impartially compare their performances are important issues. Since the Princeton segmentation benchmark has been proposed, their four representative metrics have been extensively adopted to evaluate segmentation algorithms. However, comparison to only a fixed ground-truth is problematic because objects have many semantic segmentations, hence we propose two novel metrics to support comparison with multiple ground-truth segmentations, which are named Similarity Hamming Distance (SHD) and Adaptive Entropy Increment (AEI). SHD is based on partial similarity correspondences between automatic segmentation and ground-truth segmentations, and AEI measures entropy change when an automatic segmentation is added to a set of different ground-truth segmentations. A group of experiments demonstrates that the metrics are able to provide relatively higher discriminative power and stability when evaluating different hierarchical segmentations, and also provide an effective evaluation more consistent with human perception.

Global–local optimizations by hierarchical cuts and climbing energies

Hierarchical segmentation is a multi-scale analysis of an image and provides a series of simplifying nested partitions. Such a hierarchy is rarely an end by itself and requires external criteria or heuristics to solve problems of image segmentation, texture extraction and semantic image labelling. In this theoretical paper we introduce a novel framework: hierarchical cuts, to formulate optimization problems on hierarchies of segmentations. Second we provide the three important notions of h-increasing, singular, and scale increasing energies, necessary to solve the global combinatorial optimization problem of partition selection and which results in linear time dynamic programs. Common families of such energies are summarized, and also a method to generate new ones is described. Finally we demonstrate the application of this framework on problems of image segmentation and texture enhancement.

DISCRETE TOPOLOGY BASED HIERARCHICAL SEGMENTATION FOR EFFICIENT OBJECT-BASED IMAGE ANALYIS: APPLICATION TO OBJECT DETECTION IN HIGH RESOLUTION SATELLITE IMAGES

Abstract. With rapid developments in satellite and sensor technologies, there has been a dramatic increase in the availability of high resolution (HR) remotely sensed images. Hence, the ability to collect images remotely is expected to far exceed our capacity to analyse these images manually. Consequently, techniques that can handle large volumes of data are urgently needed. In many of today's multiscale techniques the underlying representation of objects is still pixel-based, i.e. object entities are still described/accessed via pixelbased descriptors, thereby creating a bottleneck when processing large volumes of data. Also, these techniques do not yet leverage the topological and contextual information present in the image. We propose a framework for Discrete Topology based hierarchical segmentation, addressing both the algorithms and data structures that will be required. The framework consists of three components: 1) Conversion to dart-based representation, 2) Size-Constrained-Region Merging to generate multiple segmentations, and 3) Update of two sparse arrays SIGMA and LAMBDA which together encode the topology of each region in the hierarchy. The results of our representation are demonstrated both on a synthetic and a real high resolution images. Application of this representation to objectdetection is also discussed.

Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarse-to-fine deformable model

Precise segmentation and identification of thoracic vertebrae is important for many medical imaging applications though it remains challenging due to the vertebra’s complex shape and varied neighboring structures. In this paper, a new method based on learned bone-structure edge detectors and a coarse-to-fine deformable surface model is proposed to segment and identify vertebrae in 3D CT thoracic images. In the training stage, a discriminative classifier for object-specific edge detection is trained using steerable features and statistical shape models for 12 thoracic vertebrae are also learned. For the run-time testing, we design a new coarse-to-fine, two-stage segmentation strategy: subregions of a vertebra first deform together as a group; then vertebra mesh vertices in a smaller neighborhood move group-wise to progressively drive the deformable model towards edge response maps by optimizing a probability cost function. In this manner, the smoothness and topology of vertebrae shapes are guaranteed. This algorithm performs successfully with reliable mean point-to-surface errors 0.95±0.91mm on 40 volumes. Consequently a vertebra identification scheme is also proposed via mean surface mesh matching. We achieve a success rate of 73.1% using a single vertebra, and over 95% for 8 or more vertebra which is comparable or slightly better than state-of-the-art [5].

Simultaneous segmentation of prostatic zones using Active Appearance Models with multiple coupled levelsets

In this work we present an improvement to the popular Active Appearance Model (AAM) algorithm, that we call the Multiple-Levelset AAM (MLA). The MLA can simultaneously segment multiple objects, and makes use of multiple levelsets, rather than anatomical landmarks, to define the shapes. AAMs traditionally define the shape of each object using a set of anatomical landmarks. However, landmarks can be difficult to identify, and AAMs traditionally only allow for segmentation of a single object of interest. The MLA, which is a landmark independent AAM, allows for levelsets of multiple objects to be determined and allows for them to be coupled with image intensities. This gives the MLA the flexibility to simulataneously segmentation multiple objects of interest in a new image.In this work we apply the MLA to segment the prostate capsule, the prostate peripheral zone (PZ), and the prostate central gland (CG), from a set of 40 endorectal, T2-weighted MRI images. The MLA system we employ in this work leverages a hierarchical segmentation framework, so constructed as to exploit domain specific attributes, by utilizing a given prostate segmentation to help drive the segmentations of the CG and PZ, which are embedded within the prostate. Our coupled MLA scheme yielded mean Dice accuracy values of .81, .79 and .68 for the prostate, CG, and PZ, respectively using a leave-one-out cross validation scheme over 40 patient studies. When only considering the midgland of the prostate, the mean DSC values were .89, .84, and .76 for the prostate, CG, and PZ respectively.

Impurity profile analysis of drug products containing acetylsalicylic acid: a chemometric approach

Abstract In this work attention is focused on impurity profile analysis in combination with infrared spectroscopy and chemometric methods. This approach is considered as an alternative to generally complex and time-consuming classic analytical techniques such as liquid chromatography. Various strategies for constructing descriptive models able to identify relations among drug impurity profiles hidden in multivariate chromatographic data sets are also presented and discussed. The hierarchical (cluster analysis) and non-hierarchical segmentation algorithms (k-means method) and principal component analysis are applied to gain an overview of the similarities and dissimilarities among impurity profiles of acetylsalicylic acid formulations. A tree regression algorithm based on infrared spectra is used to predict the relative content of impurities in the drug products investigated. Satisfactory predictive abilities of the models derived indicate the possibility of implementing them in the quality control of drug products.

Local Mutual Information for Dissimilarity-Based Image Segmentation

Connective segmentation based on the definition of a dissimilarity measure on pairs of adjacent pixels is an appealing framework to develop new hierarchical segmentation methods. Usually, the dissimilarity is fully determined by the intensity values of the considered pair of adjacent pixels, so that it is independent of the values of the other image pixels. In this paper, we explore dissimilarity measures depending on the overall image content encapsulated in its local mutual information and show its invariance to information preserving transforms. This is investigated in the framework of the connective segmentation and constrained connectivity paradigms and leads to the concept of dependent connectivities. An efficient probability estimator based on depth functions is proposed to handle multi-dimensional images. Experiments conducted on hyper-spectral and multi-angular remote sensing images highlight the robustness of the proposed approach.

Discovering focal regions of slightly-aggregated sparse signals

The characteristic aspects of dynamic distortions on a lengthy time series of i.i.d. pure noise when embedded with slightly-aggregating sparse signals are summarized into a significantly shorter recurrence time process of a chosen extreme event. We first employ the Kolmogorov---Smirnov statistic to compare the empirical recurrence time distribution with the null geometry distribution when no signal being present in the original time series. The power of such a hypothesis testing depends on varying degrees of aggregation of sparse signals: from a completely random distribution of singletons to batches of various sizes on the entire temporal span. We demonstrate the Kolmogorov---Smirnov statistic capturing the dynamic distortions due to slightly-aggregating sparse signals better than does Tukey's Higher Criticism statistic even when the batch size is as small as five. Secondly, after confirming the presence of signals in the pure noise time series, we apply the hierarchical factor segmentation (HFS) algorithm again based on the recurrence time process to compute focal segments that contain a significantly higher intensity of signals than do the rest of the temporal regions. In a computer experiment with a given fixed number of signals, the focal segments identified by the HFS algorithm afford many folds of signal intensity which also critically depend on the degree of aggregation of sparse signals. This ratio information can facilitate better sensitivity, equivalent to a smaller false discovery rate, if the signal-discovering protocol implemented within the computed focal regions is different from that used outside of the focal regions. We also numerically compute the specificity as the total number of signals contained in the computed collection of focal regions, which indicates the inherent difficulty in the task of sparse signal discovery.

Conditional Toggle Mappings: Principles and Applications

We study a class of mathematical morphology filters to operate conditionally according to a set of pixels marked by a binary mask. The main contribution of this paper is to provide a general framework for several applications including edge enhancement and image denoising, when it is affected by salt-and-pepper noise. We achieve this goal by revisiting shock filters based on erosions and dilations and extending their definition to take into account the prior definition of a mask of pixels that should not be altered. New definitions for conditional erosions and dilations leading to the concept of conditional toggle mapping. We also investigate algebraic properties as well as the convergence of the associate shock filter. Experiments show how the selection of appropriate methods to generate the masks lead to either edge enhancement or salt-and-pepper denoising. A quantitative evaluation of the results demonstrates the effectiveness of the proposed methods. Additionally, we analyse the application of conditional toggle mapping in remote sensing as pre-filtering for hierarchical segmentation.

Hierarchical Remote Sensing Image Analysis via Graph Laplacian Energy

Segmentation and classification are important tasks in remote sensing image analysis. Recent research shows that images can be described in hierarchical structure or regions. Such hierarchies can produce the state-of-the-art segmentations and can be used in the classification. However, they often contain more levels and regions than required for an efficient image description, which may cause increased computational complexity. In this letter, we propose a new hierarchical segmentation method that applies graph Laplacian energy as a generic measure for segmentation. It reduces the redundancy in the hierarchy by an order of magnitude with little or no loss of performance. In the classification stage, we apply local self-similarity feature to capture the internal geometric layouts of regions in an image. By incorporating advantages from both semantic hierarchical segmentation and local geometric region description, we have achieved better performance than those from the methods being compared. In the experimental section, we validate the effectiveness of our method by showing results on QuickBird and GeoEye-1 image data sets.

A Spatially-Constrained Color–Texture Model for Hierarchical VHR Image Segmentation

This letter presents a novel spatially-constrained color–texture model for hierarchical segmentation of very high resolution images. The segmentation starts with an initial partition, where the image is partitioned into many homogeneous regions. Then, the regions are regarded as node sets of a region adjacency graph, in which the distances of each pair of adjacent regions are calculated combining color and textural features with spatial constraint. Finally, a stepwise optimized region merging process is applied to obtain hierarchical segmentation results. Experiments and comparisons by using different satellite images are carried out to demonstrate the encouraging performance as well as the high efficiency of the proposed method.

View Transformation Based on a Single Outdoor Image

Abstract An automatic approach for view transformation based on a single outdoor image is proposed in this paper. First, the hierarchical segmentation method is conducted to segment an outdoor image into several meaningful regions and each region is labelled as sky, ground or standing object. Then, different methods are used to estimate each region's depth according to its label. After that, the obtained depth information is utilized to create a new view image after any rotation, translation and pitch. Finally, the image inpainting work for the missing colour region is accomplished using its neighbour's colour. Extensive experiments show the proposed approach not only improves the accuracy of view transformation, but also performs well even for images with occlusion phenomena.

Automated hierarchical segmentation of high- resolution remote sensing imagery with introduced relaxation factors

Automated hierarchical segmentation of high- resolution remote sensing imagery with introduced relaxation factors