Hierarchical Segmentation Research Articles

Combining segmentation and attention: a new foveal attention model.

Artificial vision systems cannot process all the information that they receive from the world in real time because it is highly expensive and inefficient in terms of computational cost. Inspired by biological perception systems, artificial attention models pursuit to select only the relevant part of the scene. On human vision, it is also well established that these units of attention are not merely spatial but closely related to perceptual objects (proto-objects). This implies a strong bidirectional relationship between segmentation and attention processes. While the segmentation process is the responsible to extract the proto-objects from the scene, attention can guide segmentation, arising the concept of foveal attention. When the focus of attention is deployed from one visual unit to another, the rest of the scene is perceived but at a lower resolution that the focused object. The result is a multi-resolution visual perception in which the fovea, a dimple on the central retina, provides the highest resolution vision. In this paper, a bottom-up foveal attention model is presented. In this model the input image is a foveal image represented using a Cartesian Foveal Geometry (CFG), which encodes the field of view of the sensor as a fovea (placed in the focus of attention) surrounded by a set of concentric rings with decreasing resolution. Then multi-resolution perceptual segmentation is performed by building a foveal polygon using the Bounded Irregular Pyramid (BIP). Bottom-up attention is enclosed in the same structure, allowing to set the fovea over the most salient image proto-object. Saliency is computed as a linear combination of multiple low level features such as color and intensity contrast, symmetry, orientation and roundness. Obtained results from natural images show that the performance of the combination of hierarchical foveal segmentation and saliency estimation is good in terms of accuracy and speed.

Scale Object Selection (SOS) through a hierarchical segmentation by a multi-spectral per-pixel classification

In high resolution multispectral optical data, the spatial detail of the images are generally smaller than the dimensions of objects, and often the spectral signature of pixels is not directly representative of classes we are interested in. Thus, taking into account the relations between groups of pixels becomes increasingly important, making object-oriented approaches preferable. In this work several scales of detail within an image are considered through a hierarchical segmentation approach, while the spectral information content of each pixel is accounted by a per-pixel classification. The selection of the most suitable spatial scale for each class is obtained by merging the hierarchical segmentation and the per-pixel classification through the Scale Object Selection (SOS) algorithm. The SOS algorithm starts processing data from the highest level of the hierarchical segmentation, which has the least amount of spatial detail, down to the last segmentation map. At each segmentation level, objects are assigned to a specific class whenever the percentage of pixels belonging to the latter, according to a pixel-based procedure, exceeds a predefined threshold, thereby automatically selecting the most appropriate spatial scale for the classification of each object. We apply our method to multispectral, panchromatic and pan-sharpened QuickBird images, considering two different test cases: a region on the Etna volcano (Italy), imaged at a 2.4m resolution, and an area close to the town of Balakot (Pakistan), imaged at a 0.6m resolution. On both test-cases the proposed approach enhanced the classification accuracy with respect to the single-segmentation per-pixel classification approach. A detailed analysis of the benefits achieved using the hierarchical segmentation with respect to a single segmentation is reported.

Segmentation of 3D Meshes Using p-Spectral Clustering.

In this paper, we propose a new approach to get the optimal segmentation of a 3D mesh as a human can perceive using the minima rule and spectral clustering. This method is fully unsupervised and provides a hierarchical segmentation via recursive cuts. We introduce a new concept of the adjacency matrix based on cognitive studies. We also introduce the use of one-spectral clustering which leads to the optimal Cheeger cut value.

Learning Layouts for Single-Page Graphic Designs.

This paper presents an approach for automatically creating graphic design layouts using a new energy-based model derived from design principles. The model includes several new algorithms for analyzing graphic designs, including the prediction of perceived importance, alignment detection, and hierarchical segmentation. Given the model, we use optimization to synthesize new layouts for a variety of single-page graphic designs. Model parameters are learned with Nonlinear Inverse Optimization (NIO) from a small number of example layouts. To demonstrate our approach, we show results for applications including generating design layouts in various styles, retargeting designs to new sizes, and improving existing designs. We also compare our automatic results with designs created using crowdsourcing and show that our approach performs slightly better than novice designers.

Robust method for infrared small-target detection based on Boolean map visual theory.

In this paper, we present an infrared small target detection method based on Boolean map visual theory. The scheme is inspired by the phenomenon that small targets can often attract human attention due to two characteristics: brightness and Gaussian-like shape in the local context area. Motivated by this observation, we perform the task under a visual attention framework with Boolean map theory, which reveals that an observer's visual awareness corresponds to one Boolean map via a selected feature at any given instant. Formally, the infrared image is separated into two feature channels, including a color channel with the original gray intensity map and an orientation channel with the orientation texture maps produced by a designed second order directional derivative filter. For each feature map, Boolean maps delineating targets are computed from hierarchical segmentations. Small targets are then extracted from the target enhanced map, which is obtained by fusing the weighted Boolean maps of the two channels. In experiments, a set of real infrared images covering typical backgrounds with sky, sea, and ground clutters are tested to verify the effectiveness of our method. The results demonstrate that it outperforms the state-of-the-art methods with good performance.

Evaluation of pixel- and object-based approaches for mapping wild oat (Avena sterilis) weed patches in wheat fields using QuickBird imagery for site-specific management

This paper compares of pixel- and object-based techniques for mapping wild oat weed patches in wheat fields using multi-spectral QuickBird satellite imagery for site-specific weed management. The research was conducted at two levels: (1) at the field level, on 11 and 15 individual infested wheat fields in 2006 and 2008, respectively, and (2) on a broader level, by analysing the entire 2006 and 2008 images. To evaluate the wild oat patches mapping at the field level, both pixel- and object-based image analyses were tested with six classification algorithms: Parallelepipeds (P), Mahalanobis Distance (MD), Maximum Likelihood (ML), Spectral Angle Mapper (SAM), Support Vector Machine (SVM) and Decision Tree (DT). The results showed that weed patches could be accurately detected with both analyses obtaining global accuracies between 80% and 99% for most of the fields. The MD and SVM classifiers were the most accurate for both the pixel- and object-based images from 2006 and 2008, respectively. In the broad-scale analysis, all of the wheat fields were identified in the imagery using a multiresolution hierarchical segmentation based on two scales. The first segmentation scale was classified using the MD and ML algorithms to discriminate wheat fields from other land uses. Accuracies greater than 85% were obtained for MD and 88% for ML for both imagery. A hierarchical analysis was then performed with the second segmentation scale, increasing the accuracies to 93% and 91% for 2006 and 2008 imagery, respectively. Finally, based on the most accurate results obtained in the field-level study, pixel-based classifications using the MD, ML and SVM algorithms were applied to the wheat fields identified. The results of these broad-level analyses showed that wild oat patches were accurately discriminated in all the wheat fields present in the entire images with accuracies greater than 91% for all the classifiers tested.

WE‐E‐17A‐06: Assessing the Scale of Tumor Heterogeneity by Complete Hierarchical Segmentation On MRI

Purpose:In many cancers, intratumoral heterogeneity exists in vascular and genetic structure. We developed an algorithm which uses clinical imaging to interrogate different scales of heterogeneity. We hypothesize that heterogeneity of perfusion at large distance scales may correlate with propensity for disease recurrence. We applied the algorithm to initial diagnosis MRI of rhabdomyosarcoma patients to predict recurrence.Methods:The Spatial Heterogeneity Analysis by Recursive Partitioning (SHARP) algorithm recursively segments the tumor image. The tumor is repeatedly subdivided, with each dividing line chosen to maximize signal intensity difference between the two subregions. This process continues to the voxel level, producing segments at multiple scales. Heterogeneity is measured by comparing signal intensity histograms between each segmented region and the adjacent region. We measured the scales of contrast enhancement heterogeneity of the primary tumor in 18 rhabdomyosarcoma patients. Using Cox proportional hazards regression, we explored the influence of heterogeneity parameters on relapse‐free survival (RFS). To compare with existing methods, fractal and Haralick texture features were also calculated.Results:The complete segmentation produced by SHARP allows extraction of diverse features, including the amount of heterogeneity at various distance scales, the area of the tumor with the most heterogeneity at each scale, and for a given point in the tumor, the heterogeneity at different scales. 10/18 rhabdomyosarcoma patients suffered disease recurrence. On contrast‐enhanced MRI, larger scale of maximum signal intensity heterogeneity, relative to tumor diameter, predicted for shorter RFS (p=0.05). Fractal dimension, fractal fit, and three Haralick features did not predict RFS (p=0.09‐0.90).Conclusion:SHARP produces an automatic segmentation of tumor regions and reports the amount of heterogeneity at various distance scales. In rhabdomyosarcoma, RFS was shorter when the primary tumor exhibited larger scale of heterogeneity on contrast‐enhanced MRI. If validated on a larger dataset, this imaging biomarker could be useful to help personalize treatment.

Web-based Visualization and Query of semantically segmented multiresolution 3D Models in the Field of Cultural Heritage

Abstract. Many important Cultural Heritage sites have been studied over long periods of time by different means of technical equipment, methods and intentions by different researchers. This has led to huge amounts of heterogeneous "traditional" datasets and formats. The rising popularity of 3D models in the field of Cultural Heritage in recent years has brought additional data formats and makes it even more necessary to find solutions to manage, publish and study these data in an integrated way. The MayaArch3D project aims to realize such an integrative approach by establishing a web-based research platform bringing spatial and non-spatial databases together and providing visualization and analysis tools. Especially the 3D components of the platform use hierarchical segmentation concepts to structure the data and to perform queries on semantic entities. This paper presents a database schema to organize not only segmented models but also different Levels-of-Details and other representations of the same entity. It is further implemented in a spatial database which allows the storing of georeferenced 3D data. This enables organization and queries by semantic, geometric and spatial properties. As service for the delivery of the segmented models a standardization candidate of the OpenGeospatialConsortium (OGC), the Web3DService (W3DS) has been extended to cope with the new database schema and deliver a web friendly format for WebGL rendering. Finally a generic user interface is presented which uses the segments as navigation metaphor to browse and query the semantic segmentation levels and retrieve information from an external database of the German Archaeological Institute (DAI).

Two tree-based methods for the waterfall

The waterfall transform is a hierarchical segmentation technique based on the watershed transform from the field of mathematical morphology. Watershed-based techniques are useful in numerous fields ranging from image segmentation to cell-and-portal generation for games. The waterfall helps mitigate the problem of over-segmentation that commonly occurs when applying the basic watershed transform. It can also be used as a core part of a method for constructing image partition forests, a tree-based, multi-scale representation of an image. The best existing method for the waterfall is fast and effective, but our experience has been that it is not as straightforward to implement as might be desired. Furthermore, it does not deal consistently with the issue of non-minimal plateaux. This paper therefore proposes two new tree-based methods for the waterfall. Both are easier to implement than the existing state-of-the-art, and in our implementations, both were faster by a constant factor. The Simplified Waterfall (SW) method focuses on simplicity and ease of implementation; the Balanced Waterfall (BW) method focuses on robust handling of non-minimal plateaux. We perform experiments on both 2D and 3D images to contrast the new methods with each other and with the existing state-of-the-art, and show that both achieve a noticeable speed-up whilst producing similar results.

The fast computation of hierarchic image segmentation for uncummulative segments union criterions while processing ice situation data

The method of an acceleration of algorithms for hierarchic image segmentation is proposed. The algorithm is applied when functionality of the decision rule does not need segments features recalculation on an each iteration.

Single Stock Dynamics on High-Frequency Data: From a Compressed Coding Perspective

High-frequency return, trading volume and transaction number are digitally coded via a nonparametric computing algorithm, called hierarchical factor segmentation (HFS), and then are coupled together to reveal a single stock dynamics without global state-space structural assumptions. The base-8 digital coding sequence, which is capable of revealing contrasting aggregation against sparsity of extreme events, is further compressed into a shortened sequence of state transitions. This compressed digital code sequence vividly demonstrates that the aggregation of large absolute returns is the primary driving force for stimulating both the aggregations of large trading volumes and transaction numbers. The state of system-wise synchrony is manifested with very frequent recurrence in the stock dynamics. And this data-driven dynamic mechanism is seen to correspondingly vary as the global market transiting in and out of contraction-expansion cycles. These results not only elaborate the stock dynamics of interest to a fuller extent, but also contradict some classical theories in finance. Overall this version of stock dynamics is potentially more coherent and realistic, especially when the current financial market is increasingly powered by high-frequency trading via computer algorithms, rather than by individual investors.

A Framework for Land Cover Classification Using Discrete Return LiDAR Data: Adopting Pseudo-Waveform and Hierarchical Segmentation

Acquiring current, accurate land-use information is critical for monitoring and understanding the impact of anthropogenic activities on natural environments. Remote sensing technologies are of increasing importance because of their capability to acquire information for large areas in a timely manner, enabling decision makers to be more effective in complex environments. Although optical imagery has demonstrated to be successful for land cover classification, active sensors, such as light detection and ranging (LiDAR), have distinct capabilities that can be exploited to improve classification results. However, utilization of LiDAR data for land cover classification has not been fully exploited. Moreover, spatial-spectral classification has recently gained significant attention since classification accuracy can be improved by extracting additional information from the neighboring pixels. Although spatial information has been widely used for spectral data, less attention has been given to LiDAR data. In this work, a new framework for land cover classification using discrete return LiDAR data is proposed. Pseudo-waveforms are generated from the LiDAR data and processed by hierarchical segmentation. Spatial features are extracted in a region-based way using a new unsupervised strategy for multiple pruning of the segmentation hierarchy. The proposed framework is validated experimentally on a real dataset acquired in an urban area. Better classification results are exhibited by the proposed framework compared to the cases in which basic LiDAR products such as digital surface model and intensity image are used. Moreover, the proposed region-based feature extraction strategy results in improved classification accuracies in comparison with a more traditional window-based approach.

Multiagent Object-Based Classifier for High Spatial Resolution Imagery

Object-based classification, including object-based segmentation and classification, has been applied for the classification of high spatial resolution imagery due to the increase in the spatial resolution and the limited spectral resolution. Because of the independent design of the object-based segmentation and classification in many of the traditional object-based classification methods, additional work is required to select the appropriate segmentation algorithms to match the classification algorithms. The object-based segmentation algorithms, e.g., the fractal net evolution approach (FNEA), have been successfully utilized to provide the homogeneous regions, and are the basis of object-based classification. However, the traditional FNEA algorithm is greatly influenced by the global control strategy of the region-growing procedure. In addition, the existing object classification methods take little account of the object context information, which is important for high spatial-resolution image interpretation. To improve the accuracy of the object-based classification, in this paper, a multiagent object-based classification framework (MAOCF) for high-resolution remote sensing imagery is proposed. The proposed approach avoids the issue of segmentation algorithm selection by unifying the processing of object-based segmentation and classification through the use of a 4-tuple agent model. In the uniform framework, a multiagent object-based segmentation (MAOS) algorithm is proposed to optimally control the procedure of object merging. In addition, a MAOC is proposed to utilize the contextual information from the surrounding objects by taking advantage of the benefits of a multiagent system, e.g., strong interaction, high flexibility, and parallel global control capability. Due to the characteristics of a multiagent system, MAOCF has the potential for a parallel computing ability. Three experiments with different types of images were performed to evaluate the performance of MAOS and MAOC in comparison to other segmentation and classification algorithms: 1) mean-shift segmentation; 2) FNEA; 3) recursive hierarchical segmentation; and 4) the majority voting object-based classification method. The experimental results demonstrate that MAOS and MAOC give a stable performance with high spatial resolution remote-sensing imagery, and are competitive with the other methods.

Co-Saliency Detection Based on Hierarchical Segmentation

Co-saliency detection, an emerging and interesting issue in saliency detection, aims to discover the common salient objects in a set of images. This letter proposes a hierarchical segmentation based co-saliency model. On the basis of fine segmentation, regional histograms are used to measure regional similarities between region pairs in the image set, and regional contrasts within each image are exploited to evaluate the intra-saliency of each region. On the basis of coarse segmentation, an object prior for each region is measured based on the connectivity with image borders. Finally, the global similarity of each region is derived based on regional similarity measures, and then effectively integrated with intra-saliency map and object prior map to generate the co-saliency map for each image. Experimental results on two benchmark datasets demonstrate the better co-saliency detection performance of the proposed model compared to the state-of-the-art co-saliency models.

Hierarchical Image Segmentation Using a Combined Geometrical and Feature Based Approach

This paper presents a fully automatic segmentation algorithm based on geometrical and local attributes of color images. This method incorporates a hierarchical assessment scheme into any general segmentation algorithm for which the segmentation sensitivity can be changed through parameters. The parameters are varied to create different segmentation levels in the hierarchy. The algorithm examines the consistency of segments based on local features and their relationships with each other, and selects segments at different levels to generate a final segmentation. This adaptive parameter variation scheme provides an automatic way to set segmentation sensitivity parameters locally according to each region's characteristics instead of the entire image. The algorithm does not require any training dataset. The geometrical attributes can be defined by a shape prior for specific applications, i.e. targeting objects of interest, or by one or more general constraint(s) such as boundaries between regions for non-specific applications. Using mean shift as the general segmentation algorithm, we show that our hierarchical approach generates segments that satisfy geometrical properties while conforming with local properties. In the case of using a shape prior, the algorithm can cope with partial occlusions. Evaluation is carried out on the Berkeley Segmentation Dataset and Benchmark (BSDS300) (general natural images) and on geo-spatial images (with specific shapes of interest). The F-measure for our proposed algorithm, i.e. the harmonic mean between precision and recall rates, is 64.2% on BSDS300, outperforming the same segmentation algorithm in its standard non-hierarchical variant.

Hierarchical Segmentation of Multitemporal RADARSAT-2 SAR Data Using Stationary Wavelet Transform and Algebraic Multigrid Method

The objective of this paper is to develop a new effective method for hierarchical segmentation of multitemporal ultrafine-beam synthetic aperture radar (SAR) data in urban areas. Multitemporal RADARSAT-2 ultrafine-beam highresolution horizontal transmit and horizontal receive-Synthetic Aperture Radar (HH-SAR) images acquired in the rural-urban fringe of the Greater Toronto Area during the summer of 2008 are selected for this research. Stationary wavelet transform (SWT) and algebraic multigrid (AMG) method are proposed for segmentation of SAR data. SWT is applied for decomposition of multitemporal SAR images in image preprocessing. The hierarchical and matrix-based AMG method is applied for segmentation. A pyramid of fine-to-coarse grids is constructed by iteration of selecting representative pixels and calculating the interpolation matrix between a fine-level grid and a coarse-level grid. When the pyramid is completed, segments are determined by a top-down scanning based on the interpolation matrices. The AMG techniques provide a complete hierarchical segmentation of SAR data. The experimental results show that our method produces higher accuracy than eCognition.

Hierarchical classification of complex landscape with VHR pan-sharpened satellite data and OBIA techniques

Land-cover/land-use thematic maps are a major need in urban and country planning. This paper demonstrates the capabilities of Object Based Image Analysis in multi-scale thematic classification of a complex sub-urban landscape with simultaneous presence of agricultural, residential and industrial areas using pan-sharpened very high resolution satellite imagery. The classification process was carried out step by step through the creation of different hierarchical segmentation levels and exploiting spectral, geometric and relational features. The framework returned a detailed land-cover/land-use map with a Cohen's kappa coefficient of 0.84 and an overall accuracy of 85%.

Fast Hierarchical Segmentation of High-Resolution Remote Sensing Image with Adaptive Edge Penalty

Fast Hierarchical Segmentation of High-Resolution Remote Sensing Image with Adaptive Edge Penalty

A Hierarchical Segmentation Approach towards Roads and Slopes for Collapse Recognition

Color image processing is widely used in Intelligent Transport System, but seldom used in recognition of roads and slopes collapse. The application can reduce time and efforts. And the roads and slopes segmentation is the first and key step of the recognition system, which is a challenging and difficult problem. One of the problems is the presence of different types of roads and slopes. In this paper, we propose a novel framework for segmenting road images in a hierarchical manner that can separate the following objects: road and slopes with or without collapse, sky, road signs, cars, buildings and vegetation from the images. Then the Region of Interests (ROIs), i.e. the roads and slopes, are obtained with the geometrical, location of the objects and statistical color features which are extracted based on L*a*b color space and Gabor filter. According to combination K-means clustering with region merging, connected-component algorithm and morphological operation, the roads and slopes are segmented. The hierarchical approach does not assume the roads are present in the same type and assume the road images can be captured from arbitrary angles. The experiments show that the approach in this paper can achieve a satisfied result on various road images.

An unsupervised method for dominant colour region segmentation in yarn‐dyed fabrics

This paper presents a novel unsupervised approach to detect dominant colour regions standing out conspicuously in yarn-dyed fabric images. For a dominant colour region of a yarn-dyed fabric, measured by an imaging system, its individual yarn has an irregular three-dimensional shape resulting in significant colour difference among pixels of the yarn. This difference leads to difficulty in segmenting yarns into dominant colour regions. A probabilistic model is proposed in this study to associate the colour of a dominant colour region with the colours of its yarns. Based on this model, the colour histograms of a dominant colour region are first estimated from those of yarns in a yarn-dyed fabric image. Then, a hierarchical segmentation structure is devised to detect dominant colour regions in the image. Experimental results show that the proposed approach achieves satisfactory performance for dominant colour region segmentation in yarn-dyed fabric images, with high computational efficiency.