Hierarchical Image Research Articles

Multi-Faceted Hierarchical Image Segmentation Taxonomy (MFHIST)

An abundance of various segmentation techniques are available in the literature, that cater to wide range of image understanding applications. The paper proposes a unified way of systematic categorization of the research work on image segmentation called Multi-Faceted Hierarchical Image Segmentation Taxonomy (MFHIST), which consist of six facets presented in a hierarchical manner - scope, requirement, control, feature, image representation and approach specifications. Every scope is exemplified with research works from the literature for better understanding. The paper gives an illustration of populating MFHIST, to provide the reader a quick grasp of few important state-of-art image segmentation research works and their adaptations. As a case study, the illustrations display a limited version to uncover the journey of basic to modern adaptations in the areas region based segmentation approach, such as Markov Random Fields, Spectral Clustering, Active Contour Model, Mean Shift Clustering. The other segmentation approaches have not been considered here, owing to the enormous volume of works in the past four to five decades and limitation in articulating all of them using MFHIST. The performance analysis of the algorithms using quantitative metrics is not in the present scope and will be considered in future version of MFHIST.

A Parallel Algorithm of Image Mean Filtering Based on OpenCL

The image will be contaminated by noise during the imaging process, which severely degrades the image quality. It is necessary to filter the collected image. With the increasing amount of image data, the traditional single-processor or multiprocessor computing equipment has been unable to meet the requirements of real-time data processing. In this paper, the computational model of weighted mean filtering and the characteristics of high performance computer architecture are studied. An efficient hierarchical image weighted mean filtering parallel algorithm for Open Computing Language (OpenCL) is designed and implemented, which can fully express the parallelism of the computing model. The parallel algorithm takes full account of the characteristics of image discrete convolution computing and the multi-layer logic architecture of high performance computer, deeply excavates the parallelism of the computing platform and computing model, and realizes the efficient task mapping from computing model to computing resources. The model is implemented in parallel with the two levels of work-group and work-item. The experimental results show that compared with the serial algorithm based on CPU, the parallel algorithm based on Open Multi-Processing (OpenMP) and the parallel algorithm based on Compute Unified Device Architecture (CUDA), the parallel algorithm of weighted mean filtering achieves 20.88 times, 18.52 times and 1.26 times acceleration ratio on the NVIDIA GPU computing platform based on OpenCL architecture, respectively. It realizes better computing performance and runs on different Graphic Processing Unit (GPU) computing platforms, and has good portability and scalability.

Hierarchical Image Segmentation Based on Nonsymmetry and Anti-Packing Pattern Representation Model.

Image segmentation is the foundation of high-level image analysis and image understanding. How to effectively segment an image into regions that are "meaningful" to the human visual perception and ensure that the segmented regions are consistent at different resolutions is still a very challenging issue. Inspired by the idea of the Nonsymmetry and Anti-packing pattern representation Model in the Lab color space (NAMLab) and the "global-first" invariant perceptual theory, in this paper, we propose a novel framework for hierarchical image segmentation. Firstly, by defining the dissimilarity between two pixels in the Lab color space, we propose an NAMLab-based color image representation approach that is more in line with the human visual perception characteristics and can make the image pixels fast and effectively merge into the NAMLab blocks. Then, by defining the dissimilarity between two NAMLab-based regions and iteratively executing NAMLab-based merging algorithm of adjacent regions into larger ones to progressively generate a segmentation dendrogram, we propose a fast NAMLab-based algorithm for hierarchical image segmentation. Finally, the complexities of our proposed NAMLab-based algorithm for hierarchical image segmentation are analyzed in details. The experimental results presented in this paper show that our proposed algorithm when compared with the state-of-the-art algorithms not only can preserve more details of the object boundaries, but also it can better identify the foreground objects with similar color distributions. Also, our proposed algorithm can be executed much faster and takes up less memory and therefore it is a better algorithm for hierarchical image segmentation.

Multiscale Hierarchical Image Decomposition and Refinements: Qualitative and Quantitative Results

The multiscale hierarchical decomposition method (MHDM) proposed in [E. Tadmor, S. Nezzar, and L. Vese, Multiscale Model. Simul., 2 (2004), pp. 554--579] has been proven very appropriate for denois...

Object scale selection of hierarchical image segmentation with deep seeds

Object scale selection of hierarchical image segmentation with deep seeds

Using deep learning to investigate digital behavior in culinary tourism

Purpose The purpose of this study is to gather insights into digital consumer behaviour related to Chinese restaurents by examining visual contents on Tripadvisor platform. Design/methodology/approach Using the deep learning approach, this research assessed consumer-posted online content of dining experiences by implementing image analysis and clustering. Text mining using word cloud analysis revealed the most frequently repeated keywords. Findings First, 4,000 photos of nine Chinese restaurants posted on Tripadvisor’s website were analyzed using image recognition via Inception V3 and Google’s deep learning network; this revealed 12 hierarchical image clusters. Then, an open-questionnaire survey of 125 Chinese respondents investigated consumers’ information needs before visiting a restaurant and after purchasing behavior (motives to share). Practical implications This study contributes to culinary marketing development by introducing a new analysis methodology and demonstrating its application by exploring a wide range of keywords and visual images published on the internet. Originality/value This research extends and contributes to the literature regarding visual user-generated content in culinary tourism.

Pattern Recognition of Spiking Neural Networks Based on Visual Mechanism and Supervised Synaptic Learning.

Electrophysiological studies have shown that mammalian primary visual cortex are selective for the orientations of visual stimuli. Inspired by this mechanism, we propose a hierarchical spiking neural network (SNN) for image classification. Grayscale input images are fed through a feed-forward network consisting of orientation-selective neurons, which then projected to a layer of downstream classifier neurons through the spiking-based supervised tempotron learning rule. Based on the orientation-selective mechanism of the visual cortex and tempotron learning rule, the network can effectively classify images of the extensively studied MNIST database of handwritten digits, which achieves 96% classification accuracy based on only 2000 training samples (traditional training set is 60000). Compared with other classification methods, our model not only guarantees the biological plausibility and the accuracy of image classification but also significantly reduces the needed training samples. Considering the fact that the most commonly used deep learning neural networks need big data samples and high power consumption in image recognition, this brain-inspired computational neural network model based on the layer-by-layer hierarchical image processing mechanism of the visual cortex may provide a basis for the wide application of spiking neural networks in the field of intelligent computing.

An anatomically detailed and personalizable head injury model: Significance of brain and white matter tract morphological variability on strain

Finite element head (FE) models are important numerical tools to study head injuries and develop protection systems. The generation of anatomically accurate and subject-specific head models with conforming hexahedral meshes remains a significant challenge. The focus of this study is to present two developmental works: first, an anatomically detailed FE head model with conforming hexahedral meshes that has smooth interfaces between the brain and the cerebrospinal fluid, embedded with white matter (WM) fiber tracts; second, a morphing approach for subject-specific head model generation via a new hierarchical image registration pipeline integrating Demons and Dramms deformable registration algorithms. The performance of the head model is evaluated by comparing model predictions with experimental data of brain–skull relative motion, brain strain, and intracranial pressure. To demonstrate the applicability of the head model and the pipeline, six subject-specific head models of largely varying intracranial volume and shape are generated, incorporated with subject-specific WM fiber tracts. DICE similarity coefficients for cranial, brain mask, local brain regions, and lateral ventricles are calculated to evaluate personalization accuracy, demonstrating the efficiency of the pipeline in generating detailed subject-specific head models achieving satisfactory element quality without further mesh repairing. The six head models are then subjected to the same concussive loading to study the sensitivity of brain strain to inter-subject variability of the brain and WM fiber morphology. The simulation results show significant differences in maximum principal strain and axonal strain in local brain regions (one-way ANOVA test, p < 0.001), as well as their locations also vary among the subjects, demonstrating the need to further investigate the significance of subject-specific models. The techniques developed in this study may contribute to better evaluation of individual brain injury and the development of individualized head protection systems in the future. This study also contains general aspects the research community may find useful: on the use of experimental brain strain close to or at injury level for head model validation; the hierarchical image registration pipeline can be used to morph other head models, such as smoothed-voxel models.

ОЦІНЮВАННЯ ВАЖЛИВОСТІ ОБ’ЄКТІВ ПРОТИВНИКА В ХОДІ ПЛАНУВАННЯ РЕЙДОВИХ ДІЙ З ВИКОРИСТАННЯМ МЕТОДУ АНАЛІЗУ ІЄРАРХІЙ

The experience of modern armed conflicts and exercises shows the need for a detailed assessment of enemy targets, especially during the planning and conduct of raids. The article considers approaches to assessing the importance of enemy objects, which are based on the calculation of combat potentials and determining the coefficients of operational and tactical importance of enemy objects. The authors identify one of the main problems - the lack of a common understanding of the order of determining the importance (priority) of enemy objects, especially when planning raids.The purpose of this article is to consider the application of the method of hierarchy analysis to determine the coefficients of operational and tactical importance of enemy objects during the planning and conduct of raids. The authors disclose the general provisions and algorithm of decision-making using this method. To better understand the procedure for applying the method of analysis of hierarchies, an example of forming a hierarchical image of the problem, the procedure for forming matrices of pairwise comparisons depending on the level of hierarchy and their completion, mathematical processing of results, the procedure for checking expert judgments and the whole hierarchy.It is noted that this technique is part of a mathematical model of target distribution of various types of fire on enemy targets. The application of the technique will allow to determine the possibilities of the raid detachment to destroy (disable) enemy objects.

Robotized ultrasound imaging of the peripheral arteries – a phantom study

Abstract The first choice in diagnostic imaging for patients suffering from peripheral arterial disease (PAD) is 2D ultrasound (US). However, for a proper imaging process, a skilled and experienced sonographer is required. Additionally, it is a highly user-dependent operation. A robotized US system that autonomously scans the peripheral arteries has the potential to overcome these limitations. In this work, we extend a previously proposed system by a hierarchical image analysis pipeline based on convolutional neural networks (CNNs) in order to control the robot. The system was evaluated by checking its feasibility to keep the vessel lumen of a leg phantom within the US image while scanning along the artery. In 100% of the images acquired during the scan process the whole vessel lumen was visible. While defining an insensitivity margin of 2.74 mm, the mean absolute distance between vessel center and the horizontal image center line was 2.47 mm and 3.90 mm for an easy and complex scenario, respectively. In conclusion, this system presents the basis for fully automatized peripheral artery imaging in humans using a radiation-free approach.

Multi-scale region composition of hierarchical image segmentation

Hierarchical image segmentation is a prominent trend in the literature as a way to improve the segmentation quality. Generally, meaningful objects in an image are described by segments from different scales. Thus, one may spend extra effort on searching for the best representation of objects in the hierarchical segmentation result. In this paper, a novel algorithm is proposed to optimally select the segmentation scale, which leads to a composite segmentation as the output. To this end, the quality of regions from different scales of the hierarchical segmentation is evaluated. Then, a graphical model is constructed as a set of nodes. The weights among nodes are computed according to the segmentation quality of regions in multiple levels. In order to optimize the labeling of each node in the graph, the composition process is performed twice with two sampling intervals. Comprehensive experiments are conducted on different datasets for popular hierarchical image segmentation algorithms. The results show that the output of the proposed algorithm can improve the quality of hierarchical segmentation in a single scale at a low cost of computation.

Deep Learning Approach Applied to Prediction of Bone Age Based on Computed Tomography Orthopedic Image Processing

Objective: Deep learning and neural network models are new research directions in the field of machine learning and artificial intelligence. Deep learning has made breakthroughs in image recognition and speech recognition applications, and has also shown unique advantages in face recognition and information retrieval, and has been widely used. Methods: Thin-layer computed tomography (CT) scan and multiplanar reconstruction (MPR) and volume reconstruction (VR) techniques were used to perform CT thin-slice scan and volume of the bilateral clavicle sternum at 548 number of l5~25 years old. Reproduction (volume rendering, VR) and three-dimensional image recombination, measuring and calculating the longest diameter of the sternal end of the bilateral clavicle, the longest diameter of the metaphysis and its length ratio, the area of the epiphysis, the area of the metaphysis and its area ratio, etc. We establish a mathematical model of bone age inference, and then substitute 50 training samples into the mathematical model to test the accuracy of the model. Results: There was a statistically significant difference between the male and female sex ratios in the same age group (P &lt; 0.05). The established mathematical model shows that the developmental law of the sternal skeletal bone is highly correlated with the biological age. The accuracy of all models is 70.5% (±1.0 years old) and 82.5% (±1.5 years). Skeletal X-ray images show different gradation changes in black and white, with black-and-white contrast and hierarchical image features. Based on the advantages of deep learning in image recognition, we combine it with bone age assessment research to build a forensic bone age automation. Conclusions: This paper harnesses the basic concepts of deep learning and its network structure, and expounds the research progress of deep learning in image recognition in different research fields at home and abroad in recent years, as well as the advantages and application prospects of deep learning in bone age assessment.

Shallow Feature Based Dense Attention Network for Crowd Counting

While the performance of crowd counting via deep learning has been improved dramatically in the recent years, it remains an ingrained problem due to cluttered backgrounds and varying scales of people within an image. In this paper, we propose a Shallow feature based Dense Attention Network (SDANet) for crowd counting from still images, which diminishes the impact of backgrounds via involving a shallow feature based attention model, and meanwhile, captures multi-scale information via densely connecting hierarchical image features. Specifically, inspired by the observation that backgrounds and human crowds generally have noticeably different responses in shallow features, we decide to build our attention model upon shallow-feature maps, which results in accurate background-pixel detection. Moreover, considering that the most representative features of people across different scales can appear in different layers of a feature extraction network, to better keep them all, we propose to densely connect hierarchical image features of different layers and subsequently encode them for estimating crowd density. Experimental results on three benchmark datasets clearly demonstrate the superiority of SDANet when dealing with different scenarios. Particularly, on the challenging UCF_CC_50 dataset, our method outperforms other existing methods by a large margin, as is evident from a remarkable 11.9% Mean Absolute Error (MAE) drop of our SDANet.

Mapping Heterogeneous Urban Landscapes from the Fusion of Digital Surface Model and Unmanned Aerial Vehicle-Based Images Using Adaptive Multiscale Image Segmentation and Classification

Considering the high-level details in an ultrahigh-spatial-resolution (UHSR) unmanned aerial vehicle (UAV) dataset, detailed mapping of heterogeneous urban landscapes is extremely challenging because of the spectral similarity between classes. In this study, adaptive hierarchical image segmentation optimization, multilevel feature selection, and multiscale (MS) supervised machine learning (ML) models were integrated to accurately generate detailed maps for heterogeneous urban areas from the fusion of the UHSR orthomosaic and digital surface model (DSM). The integrated approach commenced through a preliminary MS image segmentation parameter selection, followed by the application of three supervised ML models, namely, random forest (RF), support vector machine (SVM), and decision tree (DT). These models were implemented at the optimal MS levels to identify preliminary information, such as the optimal segmentation level(s) and relevant features, for extracting 12 land use/land cover (LULC) urban classes from the fused datasets. Using the information obtained from the first phase of the analysis, detailed MS classification was iteratively conducted to improve the classification accuracy and derive the final urban LULC maps. Two UAV-based datasets were used to develop and assess the effectiveness of the proposed framework. The hierarchical classification of the pilot study area showed that the RF was superior with an overall accuracy (OA) of 94.40% and a kappa coefficient (K) of 0.938, followed by SVM (OA = 92.50% and K = 0.917) and DT (OA = 91.60% and K = 0.908). The classification results of the second dataset revealed that SVM was superior with an OA of 94.45% and K of 0.938, followed by RF (OA = 92.46% and K = 0.916) and DT (OA = 90.46% and K = 0.893). The proposed framework exhibited an excellent potential for the detailed mapping of heterogeneous urban landscapes from the fusion of UHSR orthophoto and DSM images using various ML models.

Deep-learning-based image segmentation integrated with optical microscopy for automatically searching for two-dimensional materials

Deep-learning algorithms enable precise image recognition based on high-dimensional hierarchical image features. Here, we report the development and implementation of a deep-learning-based image segmentation algorithm in an autonomous robotic system to search for two-dimensional (2D) materials. We trained the neural network based on Mask-RCNN on annotated optical microscope images of 2D materials (graphene, hBN, MoS2, and WTe2). The inference algorithm is run on a 1024 × 1024 px2 optical microscope images for 200 ms, enabling the real-time detection of 2D materials. The detection process is robust against changes in the microscopy conditions, such as illumination and color balance, which obviates the parameter-tuning process required for conventional rule-based detection algorithms. Integrating the algorithm with a motorized optical microscope enables the automated searching and cataloging of 2D materials. This development will allow researchers to utilize a large number of 2D materials simply by exfoliating and running the automated searching process. To facilitate research, we make the training codes, dataset, and model weights publicly available.

Learning to realign hierarchy for image segmentation

A hierarchical image segmentation is a set of image segmentations at different detail levels. However, objects (or even parts of the same object) may appear at different scales due to their size differences or to their distinct distances from the camera. One possible solution to cope with that is to realign the hierarchy such that every region containing an object (or its parts) is at the same level. In this work, we have explored the use of regression models to predict score values for regions belonging to a hierarchy of partitions, which are used to realign it. We have also proposed a new score calculation and a new assessment strategy considering all user-defined segmentations that exist in the ground-truth. Experimental results have pointed out that the use of new proposed score was able to improve final segmentation results.

Internet as a network or hierarchical structure: concept of network in postmodern philosophy and social science of the late XX – early XXI centuries

This article is dedicated to philosophical analysis of the phenomena and concepts of network and hierarchy. In philosophy of the postmodern theoreticians, hierarchical structures are subjected to criticism. Thus, one of the brightest images of non-hierarchical organization &amp;ndash; rhizome &amp;ndash; was suggested by G, Deleuze and F. Guattari in the context of their opposition to the traditional hierarchical image of a tree, which was attributed by the philosophers to the basis of Western culture and civilization. Criticism of hierarchical organization is associated with &amp;ldquo;dissolution&amp;rdquo; of a subject. The article analyzes the current state of computer networks and Internet for correspondence to the concept of &amp;ldquo;network&amp;rdquo;. The theoreticians of postmodernism associate the promotion of the concept of network with the postmodern reconsideration of a subject, since from now on it does not originate culture, but is originated and inscribed into one of the multiple cultural texts, and none of these texts does not have an advantage over others. The task of retaining integrity of the subject is no longer inherently valued, since any value of the subject simultaneously means establishing the hierarchy of events and occurrences around it. Therefore, the reverse action to integration becomes dissolution, associated with spread of the concept of network. The perception of Internet as a network in philosophy and social sciences is based on the desire to find confirmation to the theoretical model of network. But the Internet in many ways reproduces hierarchical structure within itself, which leaves open the question on the existence of nonhierarchical structures.

Simplifying TugGraph using zipping algorithms

Graphs are an invaluable modelling tool in many domains, but visualising large graphs in their entirety can be difficult. Hierarchical graph visualisation – recursively clustering a graph’s nodes to view it at a higher level of abstraction – has thus become popular. However, this can hide important information that a user needs to understand a graph’s topology, e.g. nodes’ neighbourhoods. TugGraph addressed this by ‘separating out’ a given node’s neighbours from their hierarchy ancestors to visualise them independently. Its original implementation was straightforward, but copied parts of the hierarchy, making it slow and memory-hungry. An optimised later version, which we refer to as FastTug, avoided this, but at a cost in clarity. Optimising TugGraph without sacrificing clarity is difficult because of the need to keep every hierarchy node connected, a common challenge for graph hierarchy editing algorithms. Recently, this problem has been addressed by ‘zipping’ algorithms, multi-level split/merge algorithms that preserve hierarchy node connectedness and can be built upon for higher-level editing. In this paper, we generalise the original unzipping algorithms to implement SimpleTug, a simple, modular version of TugGraph that is easy to understand and implement, and even faster and more memory-efficient than FastTug. We formally prove its equivalence to FastTug, and show how both can be parallelised. Using our millipede hierarchical image segmentation system, we show experimentally that both the serial and parallel versions of SimpleTug are around 25% faster than their FastTug counterparts, whilst using considerably less memory. Finally, we discuss the interesting theoretical connections between TugGraph and zipping, and suggest ideas for further work.

Shaping for PET image analysis

Component-trees constitute an efficient data structure for hierarchical image modeling. In particular they are relevant for processing and analyzing images where the structures of interest correspond either to local maxima or local minima of intensity. This is indeed the case of functional data in medical imaging. This motivates the use of component-tree-based approaches for analyzing Positron Emission Tomography (PET) images in the context of oncology. In this article, we present a simple, yet efficient, methodological framework for PET image analysis based on component-trees. More precisely, we show that the second-order paradigm of shaping, that broadly consists of computing the component-tree of a component-tree, provides a relevant way of generalizing the threshold-based strategies classically used by medical practitioners for handling PET images. In addition, it also allows to embed relevant priors regarding the sought cancer lesions.

A Method of Hierarchical Image Retrieval for Real-Time Photogrammetry Based on Multiple Features

In near real-time photogrammetry, the first step in processing each new added image is determining the most relevant image in pre-sequence unordered images quickly and exactly, which is pivotal for accurate image matching and 3D reconstruction. This paper presents a hierarchical image retrieval algorithm based on multiple features and details the choice for representation of multiple features which is critical to the improvement of accuracy of this algorithm. First, we represent global features using AlexNet-FC7(fully connected layers) or ResNet101-Pool5(pooling layers) and local features using SIFT (scale-invariant feature transform) in two parallel threads with support of GPU (Graphics Processing Unit). Next, we obtain candidates based on cosine similarities between global features of each pre-sequence image and new added image. Finally, we determine the most relevant image from those candidates according to feature matching results for each candidate and new added image. The experimental results confirm that the second step is rather fast and the third step is necessary to tackle the problem that global features cannot distinguish objects from the same class. The total time our algorithm takes is about 83.6ms for determining the most relevant image in 5063 pre-sequence unordered images of size $1024\times 768$ , which outperforms exhaustive pairwise matching, Bag of Words and multi-vocabulary trees. Accuracy of our algorithm also perform better than the state-of-the-art methods on three benchmark datasets. SIFT matching results obtained in the third step after eliminating mismatches with RANSAC (Random Sample Consensus) can also be used for high-precision incremental SFM (Structure from Motion) reconstruction.