Hierarchical Segmentation Research Articles

Analysis of segmentation ontology reveals the similarities and differences in connectivity onto L2/3 neurons in mouse V1

Quantitatively comparing brain-wide connectivity of different types of neuron is of vital importance in understanding the function of the mammalian cortex. Here we have designed an analytical approach to examine and compare datasets from hierarchical segmentation ontologies, and applied it to long-range presynaptic connectivity onto excitatory and inhibitory neurons, mainly located in layer 2/3 (L2/3), of mouse primary visual cortex (V1). We find that the origins of long-range connections onto these two general cell classes—as well as their proportions—are quite similar, in contrast to the inputs on to a cell type in L6. These anatomical data suggest that distal inputs received by the general excitatory and inhibitory classes of neuron in L2/3 overlap considerably.

Urban crystallization and the morphogenesis of urban territories

ABSTRACT We develop the perspective of crystallization as a way to shed a light on the morphogenesis and stabilization of urban territories. We start by describing crystallization as a consolidation of a visible and singular order that establishes certain privileged directions of growth and breaks spatial and temporal symmetries. We then illuminate how crystallization processes unfold at different scales, going through a series of historical cases, from the stabilization of urban regions, to iconic places such as Times Square, New York, and on to large scale linear or path crystals, such the Turia Riverbed Park in Valencia building on these cases, we then discuss crystallization as a phenomenon requiring metastability, and how this metastability relates to different ways and forms of territorial stabilization. Finally, we discuss how crystallization, by making certain figures and directions more salient than others, also plays an important part in the emergence of new scales and in the processes of urban rescaling, that is, how crystallization also contributes to a hierarchical segmentation of the urban environment.

Identification and Evaluation of Urban Construction Waste with VHR Remote Sensing Using Multi-Feature Analysis and a Hierarchical Segmentation Method

With rapid urbanization, the disposal and management of urban construction waste have become the main concerns of urban management. The distribution of urban construction waste is characterized by its wide range, irregularity, and ease of confusion with the surrounding ground objects, such as bare soil, buildings, and vegetation. Therefore, it is difficult to extract and identify information related to urban construction waste by using the traditional single spectral feature analysis method due to the problem of spectral confusion between construction waste and the surrounding ground objects, especially in the context of very-high-resolution (VHR) remote sensing images. Considering the multi-feature analysis method for VHR remote sensing images, we propose an optimal method that combines morphological indexing and hierarchical segmentation to extract the information on urban construction waste in VHR images. By comparing the differences between construction waste and the surrounding ground objects in terms of the spectrum, geometry, texture, and other features, we selected an optimal feature subset to improve the separability of the construction waste and other objects; then, we established a classification model of knowledge rules to achieve the rapid and accurate extraction of construction waste information. We also chose two experimental areas of Beijing to validate our algorithm. By using construction waste separability quality evaluation indexes, the identification accuracy of construction waste in the two study areas was determined to be 96.6% and 96.2%, the separability indexes of the construction waste and buildings reached 1.000, and the separability indexes of the construction waste and vegetation reached 1.000 and 0.818. The experimental results show that our method can accurately identify the exposed construction waste and construction waste covered with a dust screen, and it can effectively solve the problem of spectral confusion between the construction waste and the bare soil, buildings, and vegetation.

Multi-Faceted Hierarchical Image Segmentation Taxonomy (MFHIST)

An abundance of various segmentation techniques are available in the literature, that cater to wide range of image understanding applications. The paper proposes a unified way of systematic categorization of the research work on image segmentation called Multi-Faceted Hierarchical Image Segmentation Taxonomy (MFHIST), which consist of six facets presented in a hierarchical manner - scope, requirement, control, feature, image representation and approach specifications. Every scope is exemplified with research works from the literature for better understanding. The paper gives an illustration of populating MFHIST, to provide the reader a quick grasp of few important state-of-art image segmentation research works and their adaptations. As a case study, the illustrations display a limited version to uncover the journey of basic to modern adaptations in the areas region based segmentation approach, such as Markov Random Fields, Spectral Clustering, Active Contour Model, Mean Shift Clustering. The other segmentation approaches have not been considered here, owing to the enormous volume of works in the past four to five decades and limitation in articulating all of them using MFHIST. The performance analysis of the algorithms using quantitative metrics is not in the present scope and will be considered in future version of MFHIST.

Efficient Hierarchical Multi-Object Segmentation in Layered Graphs

Abstract We propose a novel efficient seed-based method for the multi-object segmentation of images based on graphs, named Hierarchical Layered Oriented Image Foresting Transform (HLOIFT). It uses a tree of the relations between the image objects, with each node in the tree representing an object. Each tree node may contain different individual high-level priors of its corresponding object and defines a weighted digraph, named as layer. The layer graphs are then integrated into a hierarchical graph, considering the hierarchical relations of inclusion and exclusion. A single energy optimization is performed in the hierarchical layered weighted digraph leading to globally optimal results satisfying all the high-level priors. The experimental evaluations of HLOIFT, on medical, natural, and synthetic images, indicate promising results comparable to the related baseline methods that include structural information, but with lower computational complexity. Compared to the hierarchical segmentation by the min-cut/max-flow algorithm, our approach is less restrictive, leading to globally optimal results in more general scenarios, and has a better running time.

A novel method for spectral-spatial classification of hyperspectral images with a high spatial resolution

In this research, a new method for spectral-spatial classification of hyperspectral images based on hierarchical segmentation algorithm is introduced. Among the various spectral-spatial classification algorithms, the marker-based hierarchical segmentation algorithm has so far achieved the best results in combination with the support vector machine (SVM) classification algorithm. In the proposed method adopted in this research, the dimensions of the hyperspectral image were reduced via the minimum noise fraction (MNF) algorithm. Then, five spatial/texture features of wavelet transform, Gabor transform, mean, entropy, and contrast were extracted from the obtained bands. Later on, the multi-layer perceptron (MLP) neural network and SVM classification algorithms were applied to the obtained spectral and texture features, and their results were combined. The resulting map was then used to select the markers and to combine them with the marker-based hierarchical segmentation algorithm using the majority voting rule. The proposed method was applied to three hyperspectral images, Pavia, Telops, and Washington DC Mall. The results of the experiments demonstrate the superiority of the proposed method over the initial marker-based hierarchical algorithm. Quantitatively, it was better by 8, 12, and 9% for the Pavia, Telops, and Washington DC Mall datasets regarding the Kappa coefficient parameter, respectively.

Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation

For compression fracture detection and evaluation, an automatic X-ray image segmentation technique that combines deep-learning and level-set methods is proposed. Automatic segmentation is much more difficult for X-ray images than for CT or MRI images because they contain overlapping shadows of thoracoabdominal structures including lungs, bowel gases, and other bony structures such as ribs. Additional difficulties include unclear object boundaries, the complex shape of the vertebra, inter-patient variability, and variations in image contrast. Accordingly, a structured hierarchical segmentation method is presented that combines the advantages of two deep-learning methods. Pose-driven learning is used to selectively identify the five lumbar vertebrae in an accurate and robust manner. With knowledge of the vertebral positions, M-net is employed to segment the individual vertebra. Finally, fine-tuning segmentation is applied by combining the level-set method with the previously obtained segmentation results. The performance of the proposed method was validated by 160 lumbar X-ray images, resulting in a mean Dice similarity metric of 91.60±2.22%. The results show that the proposed method achieves accurate and robust identification of each lumbar vertebra and fine segmentation of individual vertebra.

Superpixel Based Hierarchical Segmentation for Color Image

In this letter, we propose a hierarchical segmentation (HS) method for color images, which can not only maintain the segmentation accuracy, but also ensure a good speed. In our method, HS adopts the fuzzy simple linear iterative clustering (Fuzzy SLIC) to obtain an over-segmentation result. Then, HS uses the fast fuzzy C-means clustering (FFCM) to produce the rough segmentation result based on superpixels. Finally, HS takes the non-iterative K-means clustering using priority queue (KPQ) to refine the segmentation result. In the validation experiments, we tested our method and compared it with state-of-the-art image segmentation methods on the Berkeley (BSD500) benchmark under different types of noise. The experiment results show that our method outperforms state-of-the-art techniques in terms of accuracy, speed and robustness.

Fast Multiscale Superpixel Segmentation for SAR Imagery

Superpixel segmentation is essential to the rapid information extraction from synthetic aperture radar (SAR) imagery. In this letter, we propose a fast multiscale superpixel segmentation method based on the minimum spanning tree (MST), which can generate all scales of superpixels accurately in real time. Therefore, our method has the ability to segment SAR imagery with different scales efficiently and is meaningful for applications that require different levels of SAR image details. Experimental results on two real SAR images demonstrate that our proposed superpixel segmentation method can capture the image information of different levels, resulting in better hierarchical segmentation performance in comparison with other state-of-the-art methods.

Transition-Aware Detection of Modes of Locomotion and Transportation Through Hierarchical Segmentation

Recognizing human daily activities with motion sensors data, specifically, modes of locomotion and transportation provides important contextual information that enhances the effectiveness of mobile applications. For instance, in assisted living or sports monitoring it is essential to log driving or running episodes. Previous studies in this field have utilized a fixed-size windowing technique for segmenting the sequential data of sensors. While segmenting signals into larger windows provides more information about the signal for classifiers, it increases misclassification rate when a transition occurs between the activities (i.e., multi-class windows). This will lead to inaccurate detection and logging of the activities of interest. To identify the exact time of transition from one to another activity as precisely as possible, this article proposes a fast and efficient hierarchical search algorithm that finds the exact sample at which transition occurs. This search algorithm can be applied to any activity recognition model with various lengths of segmentation window. To further improve the performance, we propose a new structure of 2D signal inputs to be used with 2D convolutional neural networks (CNN), which improves the ability of the CNN in capturing patterns underlying in inter-axes correlations. Experimental results show that the proposed transition detection method improves the F1-score by 28% compared to using fixed-size windowing approach for multi-class windows. In addition, the proposed method is 20 times faster than the naive search.

Multi-scale region composition of hierarchical image segmentation

Hierarchical image segmentation is a prominent trend in the literature as a way to improve the segmentation quality. Generally, meaningful objects in an image are described by segments from different scales. Thus, one may spend extra effort on searching for the best representation of objects in the hierarchical segmentation result. In this paper, a novel algorithm is proposed to optimally select the segmentation scale, which leads to a composite segmentation as the output. To this end, the quality of regions from different scales of the hierarchical segmentation is evaluated. Then, a graphical model is constructed as a set of nodes. The weights among nodes are computed according to the segmentation quality of regions in multiple levels. In order to optimize the labeling of each node in the graph, the composition process is performed twice with two sampling intervals. Comprehensive experiments are conducted on different datasets for popular hierarchical image segmentation algorithms. The results show that the output of the proposed algorithm can improve the quality of hierarchical segmentation in a single scale at a low cost of computation.

SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGERY USING OBJECT-BASED MARKOV RANDOM FIELD BASED ON HIERARCHICAL SEGMENTATION TREE WITH AUXILIARY LABELS

Abstract. In the remote sensing imagery, spectral and texture features are always complex due to different landscapes, which leads to misclassifications in the results of semantic segmentation. The object-based Markov random field provides an effective solution to this problem. However, the state-of-the-art object-based Markov random field still needs to be improved. In this paper, an object-based Markov Random Field model based on hierarchical segmentation tree with auxiliary labels is proposed. A remote sensing imagery is first segmented and the object-based hierarchical segmentation tree is built based on initial segmentation objects and merging criteria. And then, the object-based Markov random field with auxiliary label fields is established on the hierarchical tree structure. A probabilistic inference is applied to solve this model by iteratively updating label field and auxiliary label fields. In the experiment, this paper utilized a Worldview-3 image to evaluate the performance, and the results show the validity and the accuracy of the presented semantic segmentation approach.

Image segmentation using dense and sparse hierarchies of superpixels

We investigate the intersection between hierarchical and superpixel image segmentation. Two strategies are considered: (i) the classical region merging, that creates a dense hierarchy with a higher number of levels, and (ii) the recursive execution of some superpixel algorithm, which generates a sparse hierarchy with fewer levels. We show that, while dense methods can capture more intermediate or higher-level object information, sparse methods are considerably faster and usually with higher boundary adherence at finer levels. We first formalize the two strategies and present a sparse method, which is faster than its superpixel algorithm and with similar boundary adherence. We then propose a new dense method to be used as post-processing from the intermediate level, as obtained by our sparse method, upwards. This combination results in a unique strategy and the most effective hierarchical segmentation method among the compared state-of-the-art approaches, with efficiency comparable to the fastest superpixel algorithms.

Unveiling the Hierarchical Structure of Music by Multi-Resolution Community Detection

Human perception of musical structure is supposed to depend on the generation of hierarchies, which is inherently related to the actual organisation of sounds in music. Musical structures are indeed best retained by listeners when they form hierarchical patterns, with consequent implications on the appreciation of music and its performance. The automatic detection of musical structure in audio recordings is one of the most challenging problems in the field of music information retrieval, since even human experts tend to disagree on the structural decomposition of a piece of music. However, most of the current music segmentation algorithms in literature can only produce flat segmentations, meaning that they cannot segment music at different levels in order to reveal its hierarchical structure. We propose a novel methodology for the hierarchical analysis of music structure that is based on graph theory and multi-resolution community detection. This unsupervised method can perform both the tasks of boundary detection and structural grouping, without the need of particular constraints that would limit the resulting segmentation. To evaluate our approach, we designed an experiment that allowed us to compare its segmentation performance with that of the current state of the art algorithms for hierarchical segmentation. Our results indicate that the proposed methodology can achieve state of the art performance on a well-known benchmark dataset, thus providing a deeper analysis of musical structure.

Expanding e-government adoption in an emerging market economy: An exploratory segmentation based on socio-demographic and behavioral variables.

This exploratory study builds on previous research in the area of e-government implementation within the context of the emerging market economy of Lebanon. Because of ongoing advances in e-government, many governments intend to serve all citizens, including the marginalised, with available and efficient electronic services. Design of these e-services requires objective consumer segment adapted variables in order to facilitate targeted education on the personal and public value of e-government. This study contributes analysis toward the selection of socio-demographic and behavioural variables to identify segment clusters of e-service consumers. The results suggest targeted public consumer segmentation in order to better serve marginalised populations in emerging market economies. The study is based on an online survey of 262 Lebanese citizens. The hierarchical segmentation analysis revealed three distinct segments with key characteristics to improve access to e-services.

ROI-Based Compression Strategy of 3D MRI Brain Datasets for Wireless Communications

ObjectivesBecause of the large amount of medical imaging data, the transmission process becomes complicated in telemedicine applications. Thus, in order to adapt the data bit streams to the constraints related to the limitation of the bandwidths a reduction of the size of the data by compression of the images is essential. Despite the improvements in the field of compression, the transmission itself can also introduce errors. For this reason, it is important to develop an adequate strategy which will help reduce this volume of data without having to introduce some distortion and resist the errors introduced by the channel noise during transmission. Thus, in this paper, we propose a ROI-based coding strategy and unequal bit stream protection to meet this dual constraint. Material and methodsThe proposed ROI-based compression strategy with unequal bit stream protection is composed of three parts: the first one allows the extraction of the ROI region, the second one consists of a ROI-based coding and the third one allows an unequal protection of the ROI bit stream.First, the Regions Of Interest (ROI) are extracted by hierarchical segmentation of these regions according to a segmentation method based on the technique of Marker-based-watershed combined with the technique of active contours by level set. The resulting regions are selectively encoded by a 3D coder based on a shape adaptive discrete wavelet transform 3D-BISK, where the compression ratio of each region depends on its relevance in diagnosis. These obtained regions of interest are protected with an error-correcting code of Reed-Solomon type with a code rate that varies according to the relevance of the region by an unequal protection strategy (UEP). ResultsThe performance of the proposed compression scheme is evaluated in several ways. First, tests are performed to study the impact of errors on the different bit streams. In the first place, these tests are carried out in order to study the effect of the variation of the compression rates on the different bit streams. Secondly, different Reed Solomon error-correcting codes of different code rates are tested at different compression rates on a BSC channel. Finally, the performances of this coding strategy are compared with those of SPIHT 3D in the case of transmission on a BSC channel. ConclusionThe obtained results show that the proposed method is quite efficient in transmission time reduction. Therefore, our proposed scheme will reduce the volume of data without having to introduce some distortion and resist the errors introduced by the channel noise in the case of telemedicine.

A thumbnail-based hierarchical fuzzy clustering algorithm for SAR image segmentation

This paper proposes a novel algorithm for segmentation of synthetic aperture radar (SAR) image, our proposed algorithm (THFCM) is based on thumbnail representations and a hierarchical fuzzy C-means (FCM) approach. THFCM firstly divides the image into pixel groups to extract local feature, and the major pixels of each pixel group are selected to construct a thumbnail. FCM is then used to segment each thumbnail, and hierarchical segmentation is then performed on the overall image data, based on the results of thumbnail clustering. The thumbnail approach leverages local image information, helping to overcome speckle noise, while the hierarchical approach improves computational efficiency. Experiments on simulated and real SAR images suggest that THFCM outperforms several other state-of-the-art algorithms in terms of both segmentation accuracy and running time.

A rule based streaming video segmentation using minimum spanning tree algorithm

Video segmentation is to partition a video into volumetric portions. It is relevant to a wide assortment of vision applications, such as activity acknowledgment, scene arrangement, video outline, content-based video recovery, and 3D remaking. The main objective of this method is to extract significant entities from a video. This video segmentation is classified into offline and online video segmentation algorithms. Compared to online video segmentation, offline video segmentation requires huge memory space. Nevertheless, the main challenge for video segmentation is to segment the online streaming video. However, one of the efficient streaming video segmentation algorithms is hierarchical segmentation algorithm. In this paper, an efficient Segmentation Approach for Streaming Videos Using a Rule with the Minimum Spanning Tree Algorithm is presented for further reducing the runtime and memory consumption. In this approach, each frame from the streaming video is modeled as a weighted graph. From the weighted graph, the minimum weighted hierarchical tree is formed by applying Kruskal's minimum spanning tree algorithm. Then the hierarchical segmentation is done by removing the inconsistent edges based on a rule. Performance of our proposed approach is compared with the existing work in terms of Boundary Precision-Recall (BPR) and Volume Precision-Recall (VPR).

A General Framework for Depth Compression and Multi-Sensor Fusion in Asymmetric View-Plus-Depth 3D Representation

We present a general framework which can handle different processing stages of the three-dimensional (3D) scene representation referred to as “view-plus-depth” (V+Z). The main component of the framework is the relation between the depth map and the super-pixel segmentation of the color image. We propose a hierarchical super-pixel segmentation which keeps the same boundaries between hierarchical segmentation layers. Such segmentation allows for a corresponding depth segmentation, decimation and reconstruction with varying quality and is instrumental in tasks such as depth compression and 3D data fusion. For the latter we utilize a cross-modality reconstruction filter which is adaptive to the size of the refining super-pixel segments. We propose a novel depth encoding scheme, which includes specific arithmetic encoder and handles misalignment outliers. We demonstrate that our scheme is especially applicable for low bit-rate depth encoding and for fusing color and depth data, where the latter is noisy and with lower spatial resolution.

Expanding e-government adoption in an emerging market economy: an exploratory segmentation based on socio-demographic and behavioural variables

This exploratory study builds on previous research in the area of e-government implementation within the context of the emerging market economy of Lebanon. Because of ongoing advances in e-government, many governments intend to serve all citizens, including the marginalised, with available and efficient electronic services. Design of these e-services requires objective consumer segment adapted variables in order to facilitate targeted education on the personal and public value of e-government. This study contributes analysis toward the selection of socio-demographic and behavioural variables to identify segment clusters of e-service consumers. The results suggest targeted public consumer segmentation in order to better serve marginalised populations in emerging market economies. The study is based on an online survey of 262 Lebanese citizens. The hierarchical segmentation analysis revealed three distinct segments with key characteristics to improve access to e-services.