Natural Image Segmentation Research Articles

Co-optimization of ethnic-pattern segmentation based on hierarchical patch matching

The segmentation of ethnic patterns is vital for digital analyses of ethnic cultures. Although existing image segmentation methods can properly segment natural images, they cannot preserve the main structure of the image. They also require many user interactions to segment ethnic patterns. This paper presents a co-optimization method that segments ethnic patterns using hierarchical patch matching. Exploiting the repetitive characteristics of ethnic patterns, the method first automatically detects all similar patterns using a global patch match. Second, the relative orientation between similar patterns is estimated by a local patch match, and an accurate dense correspondence is constructed by a constrained patch match. Finally, the pre-segmentation of ethnic patterns is co-optimized to preserve their main structures. Our method can segment all similar ethnic patterns into separate elements with dense correspondence. Besides reducing the user interaction and improving the segmentation accuracy, the proposed method improves the quality of ethnic pattern digital analysis such as vectorization. Experiments demonstrated the validity of our method.

Automated LULC map production using deep neural networks

ABSTRACTThis article presents an approach to automating the creation of land-use/land-cover classification (LULC) maps from satellite images using deep neural networks that were developed to perform semantic segmentation of natural images. This work is important since the production of accurate and timely LULC maps is becoming essential to government and private companies that rely on them for large-scale monitoring of land resource changes. In this work, deep neural networks are trained to classify each pixel of a satellite image into one of a number of LULC classes. The presented deep neural networks are all pre-trained using the ImageNet Large-Scale Visual Recognition Competition (ILSVRC) datasets and then fine-tuned using approximately 19,000 Landsat 5/7 satellite images of resolution taken of the Province of Manitoba in Canada. The result is an automated solution that can produce LULC maps images significantly faster than current semi-automated methods. The contributions of this article are the observation that deep neural networks developed for semantic segmentation can be used to automate the task of producing LULC maps; the use of these networks to produce LULC maps; and a comparison of several popular semantic segmentation architectures for solving the problem of automated LULC map production.

Woodblock image decomposition of Chinese new year paintings

Woodblock printed Chinese new year (WNY) painting has been a popular art form in Chinese folk culture. To make a WNY painting involves carving images on woodblocks and printing colors using woodblocks. Although thousands of WNY paintings were preserved, the ten-year national survey reveals that a great number of woodblocks were damaged or lost. In this paper, we study a novel problem of decomposing woodblock images from WNY paintings, which currently requires a tremendous amount of manual labor. We also find that the state-of-the-art methods of natural image segmentation generate poor results in our application. Instead of using sophisticated schemes, we develop a simple yet robust decomposition approach, which contains the extraction of line block image and the separation of color block images. The effectiveness of the proposed approach is validated through both quantitative evaluation and visual quality comparison with six state-of-the-art methods on multiple WNY paintings.

Prostate segmentation in MRI using a convolutional neural network architecture and training strategy based on statistical shape models.

Most of the existing convolutional neural network (CNN)-based medical image segmentation methods are based on methods that have originally been developed for segmentation of natural images. Therefore, they largely ignore the differences between the two domains, such as the smaller degree of variability in the shape and appearance of the target volume and the smaller amounts of training data in medical applications. We propose a CNN-based method for prostate segmentation in MRI that employs statistical shape models to address these issues. Our CNN predicts the location of the prostate center and the parameters of the shape model, which determine the position of prostate surface keypoints. To train such a large model for segmentation of 3D images using small data (1) we adopt a stage-wise training strategy by first training the network to predict the prostate center and subsequently adding modules for predicting the parameters of the shape model and prostate rotation, (2) we propose a data augmentation method whereby the training images and their prostate surface keypoints are deformed according to the displacements computed based on the shape model, and (3) we employ various regularization techniques. Our proposed method achieves a Dice score of 0.88, which is obtained by using both elastic-net and spectral dropout for regularization. Compared with a standard CNN-based method, our method shows significantly better segmentation performance on the prostate base and apex. Our experiments also show that data augmentation using the shape model significantly improves the segmentation results. Prior knowledge about the shape of the target organ can improve the performance of CNN-based segmentation methods, especially where image features are not sufficient for a precise segmentation. Statistical shape models can also be employed to synthesize additional training data that can ease the training of large CNNs.

Left Atrial Appendage Segmentation Using Fully Convolutional Neural Networks and Modified Three-Dimensional Conditional Random Fields.

Thrombosis has become a global disease threatening human health. The left atrial appendage (LAA) is a major source of thrombosis in patients with atrial fibrillation (AF). Positive correlation exists between LAA volume and AF risk. LAA morphology has been suggested to influence thromboembolic risk in AF patients and to help predict thromboembolic events in low-risk patient groups. Automatic segmentation of LAA can greatly help physicians diagnose AF. In consideration of the large anatomical variations of the LAA, we proposed a robust method for automatic LAA segmentation on computed tomographic angiography (CTA) data using fully convolutional neural networks with three-dimensional (3-D) conditional random fields (CRFs). After manual localization of ROI of LAA, we adopted the FCN in natural image segmentation and transferred their learned models by fine-tuning the networks to segment each 2-D LAA slice. Subsequently, we used a modified dense 3-D CRF that accounts for the 3-D spatial information and larger contextual information to refine the segmentations of all slices. Our method was evaluated on 150 sets of CTA data using five-fold cross validation. Compared with manual annotation, we obtained a mean dice overlap of and a mean volume overlap of with a computation time of less than 40 s per volume. Experimental results demonstrated the robustness of our method in dealing with large anatomical variations and computational efficiency for adoption in a daily clinical routine.).

Model-Based segmentation of image data using spatially constrained mixture models

Abstract In this paper, a novel Bayesian statistical approach is proposed to tackle the problem of natural image segmentation. The proposed approach is based on finite Dirichlet mixture models in which contextual proportions (i.e., the probabilities of class labels) are modeled with spatial smoothness constraints. The major merits of our approach are summarized as follows: Firstly, it exploits the Dirichlet mixture model which can obtain a better statistical performance than commonly used mixture models (such as the Gaussian mixture model), especially for proportional data (i.e, normalized histogram). Secondly, it explicitly models the mixing contextual proportions as probability vectors and simultaneously integrate spatial relationship between pixels into the Dirichlet mixture model, which results in a more robust framework for image segmentation. Finally, we develop a variational Bayes learning method to update the parameters in a closed-form expression. The effectiveness of the proposed approach is compared with other mixture modeling-based image segmentation approaches through extensive experiments that involve both simulated and natural color images.

ファジィ分離統合法によるカラー自然画像の大まかな領域分割とダーモスコピー画像への応用(ITS画像処理,映像メディア及び一般)

ファジィ分離統合法によるカラー自然画像の大まかな領域分割とダーモスコピー画像への応用(ITS画像処理,映像メディア及び一般)

Spatially variant mixture model for natural image segmentation

We tackle the problem of natural image segmentation by proposing a statistical approach that is based on spatially variant finite mixture models with generalized means. The contributions can be summarized as follows: first, the proposed spatially variant mixture model exploits beta-Liouville as basic distributions for describing the underlying data structure, which demonstrated better segmentation performance than commonly used distributions, such as Gaussian; second, the mixing proportions (i.e., the probabilities of class labels) in our model are modeled via the Dirichlet compound multinomial probability density, and the spatial smoothness is imposed by adopting the function of generalized mean over the mixture model as well as mixing proportions; and finally, a variational Bayes learning approach is developed to estimate model parameters and model complexity simultaneously with closed-form solutions. The robustness, accuracy, and effectiveness of the proposed model in image segmentation are demonstrated through experiments on both natural images and synthetic images degraded by noise compared with other state-of-the-art image segmentation methods.

Interactive image segmentation with a regression based ensemble learning paradigm

To achieve fine segmentation of complex natural images, people often resort to an interactive segmentation paradigm, since fully automatic methods often fail to obtain a result consistent with the ground truth. However, when the foreground and background share some similar areas in color, the fine segmentation result of conventional interactive methods usually relies on the increase of manual labels. This paper presents a novel interactive image segmentation method via a regression-based ensemble model with semi-supervised learning. The task is formulated as a non-linear problem integrating two complementary spline regressors and strengthening the robustness of each regressor via semi-supervised learning. First, two spline regressors with a complementary nature are constructed based on multivariate adaptive regression splines (MARS) and smooth thin plate spline regression (TPSR). Then, a regressor boosting method based on a clustering hypothesis and semi-supervised learning is proposed to assist the training of MARS and TPSR by using the region segmentation information contained in unlabeled pixels. Next, a support vector regression (SVR) based decision fusion model is adopted to integrate the results of MARS and TPSR. Finally, the GraphCut is introduced and combined with the SVR ensemble results to achieve image segmentation. Extensive experimental results on benchmark datasets of BSDS500 and Pascal VOC have demonstrated the effectiveness of our method, and the comparison with experiment results has validated that the proposed method is comparable with the state-of-the-art methods for interactive natural image segmentation.

An efficient level set model with self-similarity for texture segmentation

Textures widely exist in the natural scenes while traditional level set models generally use only intensity information to construct energy and ignore the inherent texture features. Thus these models have difficulty in segmenting texture images especially when the texture objects have similar intensity to the background. To solve this problem, we propose a new level set model for texture segmentation that considers the impact of local Gaussian distribution fitting (LGDF), local self-similarity (LSS) and a new numerical scheme on the evolving contour. The proposed method first introduces a texture energy term based on the local self-similarity texture descriptor to the LGDF model, and then the evolving contour could effectively snap to the textures boundary. Secondly, a lattice Boltzmann method (LBM) is deployed as a new numerical scheme to solve the level set equation, which can break the restriction of the Courant–Friedrichs–Lewy (CFL) condition that limits the time step of iterations in former numerical schemes. Moreover, GPU acceleration further improves the efficiency of the contour evolution. Experimental results show that our model can effectively handle the segmentation of synthetic and natural texture images with heavy noises, intensity inhomogeneity and messy background. At the same time, the proposed model has a relatively low complexity.

Unsupervised segmentation of natural images based on statistical modeling

A novel unsupervised scheme for natural image segmentation is proposed aiming to acquire perceptually consistent results. Firstly, comprehensive visual features besides raw color values are extracted, including spatial frequency, contrast sensitivity, color deviation, and so on. Secondly, high correlations among visual features are reduced via principal component analysis (PCA) and the raw image pixels are then converted to a collection of feature vectors in a multi-dimensional feature space. Thirdly, the Gaussian mixture model (GMM) is employed to approximate the class distribution of image pixels and an improved expectation maximization (EM) algorithm is introduced to estimate model parameters. Finally, segmentation results are obtained by grouping of pixels based on the mixture components. Experiments are conducted and the results demonstrate that, compared with existing techniques, the proposed scheme can acquire more perceptually consistent results.

Cogent Confabulation based Expert System for Segmentation and Classification of Natural Landscape Images

Ever since there has been an increase in the number of automatic wildfire monitoring and surveillance systems in the last few years, natural landscape images have been of great importa ...

Object segmentation by saliency-seeded and spatial-weighted region merging

In this paper, we present a region merging-based method for object segmentation in natural images. The method consists of three separate steps: (1) initial over-segmentation such that pixels in each region are as homogeneous as possible and therefore likely to be from the same object; (2) saliency-seeded interaction to provide proper prior input to guide the segmentation; (3) region merging by an introduced maximal spatially weighted similarity (MSWS) criterion. Saliency-seeded interaction can well reflect the human intention but does not require any manual user editing, which makes our method applicable to increasingly large-scale image databases. The MSWS criterion takes into account both the color similarity and spatial distance of the candidate regions for merging, which allows the region merging-based method to achieve better performance. Extensive experiments show that our method can reliably and automatically segment the objects from a great variety of natural images.

Multi-level thresholding with a decomposition-based multi-objective evolutionary algorithm for segmenting natural and medical images

The objective of multilevel thresholding is to segmenting a gray-level image into several distinct homogeneous regions. This paper presents an alternative approach for unsupervised segmentation of natural and medical images to improve the separation between objects in the framework of multi-objective optimization. In contrast to the existing single-objective optimization and entropy-based methods, a multi-objective framework is adopted by combining two objectives based on the Minimum Cross Entropy (MCE) and Rényi Entropy (RE). One of the most competitive Multi-Objective Evolutionary Algorithms (MOEAs) of current interest, called MOEA/D-DE (Decomposition based MOEA with Differential Evolution) is then applied to determine the set of Pareto optimal solutions for these two objectives. The threshold values for multi-level segmentations are obtained from the approximated Pareto Fronts (PFs) generated by MOEA/D-DE. The performance of MOEA/D-DE is also investigated extensively through comparison with other popular nature-inspired single-objective and multi-objective optimizers. Moreover, outcome of the proposed method is evaluated by comparing against the results of other well cited algorithms both qualitatively and quantitatively on test-suites comprising well-known natural and medical test images in order to showcase the efficiency of the proposed algorithm.

Robust natural image segmentation by using spatially constrained multivariate mixed Student’s t-distribution and TV flow edge

In this paper, a new method of spatially constrained energy function and total variation flow edge (TVFE) model is proposed for natural image segmentation. As the segmentation process of natural image is sensitive to corrupted noise, the Markov random field (MRF) based spatially constrained information is combined with Graph cut model. Secondly, a total variation flow edge is extracted by calculating the accumulated gradient difference from a nonlinear diffusion filter image. Thirdly, to improve the segmentation performance, the spatially constrained region term and the TVFE edge term are combined together. As the optimization solution of the proposed model is NP hard problem, and then an Expectation Maximum and Multilayer graph cut (EM2GC) algorithm is presented. Lastly, testing experiments are carried out on some synthetic images and real natural scene images, which demonstrate the superiority of our proposed method, such as the effectiveness, robustness, and high accuracy.

Image segmentation by adaptive nonconvex local and global subspace representation

We formulate image segmentation as subspace clustering of image feature vectors. We propose a subspace representation model using a nonconvex extension of trace Lasso and a nonconvex approximation of rank function to regularize the subspace representation. The proposed model can adaptively capture the local and the global structures of the subspace representation so that the subspace representation can reveal the real subspace structure of the data and obtains excellent clustering performance. Experimental results show that the proposed model is better than the previous models in clustering and natural image segmentation.

A cooperative honey bee mating algorithm and its application in multi-threshold image segmentation

Multilevel thresholding has been one of the most popular image segmentation techniques; however, most of these techniques are time-consuming. This paper proposes a cooperative honey bee mating-based algorithm for natural scenery image segmentation using multilevel thresholding (CHBMA) to save computation time while conquering the curse of dimensionality. The characteristics of natural scenery images are random, imprecise, complicated, and noisy. The locations of interest points in them are not regular. Our proposed algorithm, which is based on honey bee mating algorithms (HBMA) and cooperative learning, considerably enhances the search capability of the algorithm. In the algorithm, we adopt a new population initialization strategy to make the search more efficient, and this strategy works in accordance with the characteristics of multilevel thresholding in an image; the thresholding is arranged from a low gray level to a high gray level. Extensive experiments have demonstrated that our proposed algorithm can deliver more effective and efficient results than state-of-the-art population-based thresholding methods. Thus, our algorithm can be applied in complex image processing, such as automatic target recognition.

Interactive natural image segmentation with regression based clustering algorithm

Interactive natural image segmentation with regression based clustering algorithm

Unsupervised Texture Segmentation of Natural Scene Images Using Region-based Markov Random Field

In analyzing natural scene images, texture plays an important role because such images are full of various textures. Although texture is crucial information in analyzing natural scene images, the texture segmentation problem is still hard to solve since the texture often exhibit non-uniform statistical characteristics. Although there are several supervised approaches that partition an image according to pre-defined semantic categories, the ever-changing appearances in the natural images make such schemes intractable. To overcome this limitation, we propose a novel unsupervised texture segmentation method for natural images by using the Region-based Markov Random Field (RMRF) model which enforces the spatial coherence between neighbor regions. We introduce the concept of pivot regions which plays a decisive role to incorporate local data interaction. By forcing pivot regions to adhere to initial labels, we make the Markov Random Field evolve fast and precisely. The proposed algorithm based on the pivot regions and the MRF for encapsulating spatial dependencies between neighborhoods yields high performance for the unsupervised segmentation of natural scene images. Quantitative and qualitative evaluations prove that the proposed method achieves comparable results with other algorithms.

Medical and Natural Image Segmentation Algorithm using M-F based Optimization Model and Modified Fuzzy Clustering: A Novel Approach

In this paper, we propose and present a novel algorithm for medical image segmentation (MIS). By analyzing the current state-of-the-art related algorithms, we introduce the multi-band active contour model based limit function to make the multilayer segmentation available. With the development of image segmentation technology, the development of medical image segmentation technology also got very big, because there is no find common, accepted effect ideal is suitable for medical image segmentation method, almost existing each kind of segmentation method has application in the field of medical image segmentation. Furtherly, with the optimized aims of being robust to the noise and avoiding the bad effluence on the result, we adopt the kernel method and new initialization curve. This model suffers from low noise robustness, and model algorithm is difficult to achieve. Integrated segmentation technology refers to two or more technology is used, combined with their own advantages, so they can on the accuracy or efficiency to achieve better performance than when using a single. A new penalty term is introduced to improve numerical stability and the step length is increased to improve efficiency. As far as the robustness and effectiveness are concerned, our method is better than the existing medical image segmentation algorithms. Experimental analysis verifies the success of our method.