Mumford-Shah Model Research Articles

A Bayesian Mumford–Shah Model for Radiography Image Segmentation

This paper investigates the segmentation of radiographic images using a level set method based on a Bayesian Mumford–Shah model. The objective is to separate regions in an image that have very close arithmetic means, where a model based on the statistical mean is not effective. Experimental results show that the proposed model can successfully separate such regions, in both synthetic images and real radiography images.

Image Segmentation via Mean Curvature Regularized Mumford-Shah Model and Thresholding

Due to the limitations in imaging devices and subject-induced susceptibility effect, general image segmentation is still an open problem. Typical challenges include image noise, intensity inhomogeneity and various image modalities. In this paper, we propose to use a two-step strategy. Specifically, we first utilize a mean curvature regularized Mumford-Shah model to recover an intermediate image with enhanced saliency, and then the segmentation is obtained by a thresholding procedure. For images with intensity inhomogeneity, a bias-corrected fuzzy K-means method is used to correct the bias field before K-means thresholding. The proposed model can be minimized efficiently using the augmented Lagrangian algorithm. Experimental results and comparison analysis demonstrate that the proposed framework is not only able to preserve the geometry of object shapes, especially object corners, but it is also more accurate than state-of-the-art methods.

Weighted variational model for selective image segmentation with application to medical images

Selective image segmentation is an important topic in medical imaging and real applications. In this paper, we propose a weighted variational selective image segmentation model which contains two steps. The first stage is to obtain a smooth approximation related to Mumford-Shah model to the target region in the input image. Using weighted function, the approximation provides a larger value for the target region and smaller values for other regions. In the second stage, we make use of this approximation and perform a thresholding procedure to obtain the object of interest. The approximation can be obtained by the alternating direction method of multipliers and the convergence analysis of the method can be established. Experimental results for medical image selective segmentation are given to demonstrate the usefulness of the proposed method. We also do some comparisons and show that the performance of the proposed method is more competitive than other testing methods.

Classification of small structures in piecewise-constant Mumford–Shah model

Mumford–Shah model is a very popular variational model in image restoration and image classification. As a simplification, piecewise constant Mumford–Shah model is very useful and has been extensively studied in recent two decades. An interesting topic on Mumford–Shah model is how to choose the weight parameters for implementation. This paper aims at discussing and analyzing the relation between choosing weight parameters and removing/preserving small structures, including noise, for piecewise-constant Mumford–Shah model. The main contributions are: (1) provided a necessary condition on the weight parameter of regularity term for removing a small structure from background. It is proved that whether or not a small structure could be removed from the background in the piecewise-constant Mumford–Shah model depends on two aspects: the ratio of the area to the perimeter for the smaller structure and the intensities of other classes; (2) provided a decision-making strategy on the class that a small structure will be classified to if it does not belong to the background; (3) developed a balanced Mumford–Shah model with which the scale measurements (weights for fidelity terms) can be chosen based on prior knowledge or users’ purposes.

Fast Segmentation From Blurred Data in 3D Fluorescence Microscopy.

We develop a fast algorithm for segmenting 3D images from linear measurements based on the Potts model (or piecewise constant Mumford-Shah model). To that end, we first derive suitable space discretizations of the 3D Potts model, which are capable of dealing with 3D images defined on non-cubic grids. Our discretization allows us to utilize a specific splitting approach, which results in decoupled subproblems of moderate size. The crucial point in the 3D setup is that the number of independent subproblems is so large that we can reasonably exploit the parallel processing capabilities of the graphics processing units (GPUs). Our GPU implementation is up to 18 times faster than the sequential CPU version. This allows to process even large volumes in acceptable runtimes. As a further contribution, we extend the algorithm in order to deal with non-negativity constraints. We demonstrate the efficiency of our method for combined image deconvolution and segmentation on simulated data and on real 3D wide field fluorescence microscopy data.

A Three-Stage Approach for Segmenting Degraded Color Images: Smoothing, Lifting and Thresholding (SLaT)

In this paper, we propose a SLaT (Smoothing, Lifting and Thresholding) method with three stages for multiphase segmentation of color images corrupted by different degradations: noise, information loss, and blur. At the first stage, a convex variant of the Mumford-Shah model is applied to each channel to obtain a smooth image. We show that the model has unique solution under the different degradations. In order to properly handle the color information, the second stage is dimension lifting where we consider a new vector-valued image composed of the restored image and its transform in the secondary color space with additional information. This ensures that even if the first color space has highly correlated channels, we can still have enough information to give good segmentation results. In the last stage, we apply multichannel thresholding to the combined vector-valued image to find the segmentation. The number of phases is only required in the last stage, so users can choose or change it all without the need of solving the previous stages again. Experiments demonstrate that our SLaT method gives excellent results in terms of segmentation quality and CPU time in comparison with other state-of-the-art segmentation methods.

A general framework of piecewise-polynomial Mumford–Shah model for image segmentation

ABSTRACTA new general framework of piecewise-polynomial Mumford–Shah model is proposed. In terms of the fidelity term, we use piecewise polynomials to approximate the inner and outer regions of the contour of the objective image. For more accurate approximation of the image, the proposed model has no constraint on the regularization term for polynomials. Moreover, we apply the anisotropic control to drive the initial contour to the desirable position. The proposed model generalizes the well-known Chan–Vese model and improves Vese's model, which is almost the simplest framework to apply piecewise polynomials to approximate the original Mumford–Shah model. Instead of solving the Euler–Lagrange equation by evolution implementation, we utilize the split Bregman iteration, which is shown to be a fast algorithm. Experimental results demonstrate that the proposed model has more desirable performance in terms of segmentation accuracy, efficiency and robustness, compared with several other variational models in addressing some challenging segmentation scenarios.

A Mumford–Shah-type approach to simultaneous reconstruction and segmentation for emission tomography problems with Poisson statistics

Abstract We propose a variational model to simultaneous reconstruction and segmentation in emission tomography. As in the original Mumford–Shah model [27] we use the contour length as penalty term to preserve edge information whereas a different data fidelity term is used to measure the information discrimination between the computed tomography data of the reconstructed object and the observed (or simulated) data. As data fidelity term we use the Kullback–Leibler divergence which originates from the Poisson distribution present in emission tomography. In this paper we focus on piecewise constant reconstructions which is a reasonable assumption in medical imaging. The segmenting contour as well as the corresponding reconstructions are found as minimizers of a Mumford–Shah-type functional over the space of piecewise constant functions. The numerical scheme is implemented by evolving the level-set surface according to the shape derivative of the functional. The method is validated for simulated data with different levels of noise.

On the Gamma-limit of joint image segmentation and registration functionals based on phase fields

A classical task in image processing is the following: Given two images, identify the structures inside (for instance detect all image edges or all homogeneous regions; this is called segmentation) and find a deformation which maps the structures in one image onto the corresponding ones in the other image (called registration). In medical imaging, for instance, one might segment the organs in two patient images and then identify corresponding organs in both images for automated comparison purposes. The image segmentation is classically performed variationally using the Mumford–Shah model, and the obtained structures are then mapped onto each other by minimizing a registration energy in which the deformation is regularized via elasticity. Experimentally it often seems more robust to perform segmentation and registration simultaneously so that both can benefit from each other. The question to be examined here is how phase field approximations of the Mumford–Shah model behave if used for the joint segmentation and registration problem.We mathematically analyze corresponding generic phase field models and reveal interesting phenomena that rule out some of the models. These phenomena are characteristic of coupling phase fields with deformations and thus are interesting in their own right. In essence, region-based segmentation and registration problems can well be approximated using phase fields, while edge-based approaches typically suffer from different types of vanishing or newly appearing edges. We conjecture how the introduction of a different scaling could remedy those shortcomings.

Generalizing Mumford-Shah Model for Multiphase Piecewise Smooth Image Segmentation.

This paper concerns multiphase piecewise smooth image segmentation with intensity inhomogeneities. Traditional methods based on the Mumford-Shah (MS) model require solving complicated diffusion equations evolving in irregular subdomains, leading to significant difficulties in efficient and accurate segmentation, especially in multiphase scenarios. In this paper, we propose a general framework to modify the MS model by using smoothing operators that can avoid the complicated implementation and inaccurate segmentation of traditional approaches. A detailed analysis connecting the smoothing operators and the diffusion equations is given to justify the modification. In addition, we present an efficient algorithm based on the direct augmented Lagrangian method, which requires fewer parameters than the commonly used augmented Lagrangian method. Typically, the smoothing operator in the general model is chosen to be Gaussian kernel, the bilateral kernel, and the directional diffusion kernel, respectively. Ample numerical results are provided to demonstrate the efficiency and accuracy of the modified model and the proposed minimization algorithm through various comparisons with existing approaches.

Sea-surface temperature chart enhancement in frontal zones

Infrared and microwave satellite images used for sea-surface temperature (SST) retrieval often have distortions such as noise and blurring of thermal front lines that decrease the quality of SST charts. In order to solve this problem, it is proposed to use an approach based on the Mumford–Shah model that approximates an image with a piecewise smooth function. In order to combine the advantages of the proposed approach and conventional methods for noise filtering and image restoration it is proposed to divide images into flat and frontal zones and process them separately. The SST quality is enhanced by the use of edge-preserving noise filtering and restoration algorithms. The latter use the features of radiometers and different stages of the SST construction procedure to improve their accuracy. The images obtained using the MTSAT/VISSR, METEOR-M/MSU-MR, and AQUA/AMSR-E radiometers are used for testing the developed approach.

An efficient segmentation method based on dynamic graph merging

A novel energy functional based on the Mumford–Shah model is established for performing automatic image segmentation. And in order to optimize the global model using graph-based methods, we develop a localized formula. Then, we propose a merging predicate for determining whether an edge connecting two neighboring pixels or regions merge. The dynamic graph merging (DGM) method is applied based on this merging predicate. That is, those edges with large energy merge and the edges with low energy are remained, such that the energy functional is minimized. Compared with other graph-based segmentation methods, our algorithm based on DGM has an important characteristic which is its ability to produce good segmentation on some complex texture images. Another characteristic is that this segmentation algorithm can avoid the “shrinking bias” problem. We also apply DGM to interactive image segmentation and find the results to be encouraging too.

A posteriori error control for the binary Mumford-Shah model

The binary Mumford-Shah model is a widespread tool for image segmentation and can be considered as a basic model in shape optimization with a broad range of applications in computer vision, ranging from basic segmentation and labeling to object reconstruction. This paper presents robust a posteriori error estimates for a natural error quantity, namely the area of the non properly segmented region. To this end, a suitable strictly convex and non-constrained relaxation of the originally non-convex functional is investigated and Repin’s functional approach for a posteriori error estimation is used to control the numerical error for the relaxed problem in the L-norm. In combination with a suitable cut out argument, a fully practical estimate for the area mismatch is derived. This estimate is incorporated in an adaptive meshing strategy. Two different adaptive primal-dual finite element schemes, and the most frequently used finite difference discretization are investigated and compared. Numerical experiments show qualitative and quantitative properties of the estimates and demonstrate their usefulness in practical applications.

An anisotropic Mumford–Shah model

A variational model is introduced for the segmentation problem of thin structures, like tubes or thin plates, in an image. The energy is based on the Mumford–Shah model with a surfacic term perturbed by a Finsler metric. The formulation in the special space of functions with bounded variations is given and, in order to get an energy more adapted for numerics, a result of Γ-convergence is proved.

Simultaneous reconstruction and segmentation with the Mumford-Shah functional for electron tomography.

Electron micrography (EM) is an important method for determining the three-dimensional (3D) structure of macromolecular complexes and biological specimens. But there are several limitations such as poor signal-to-noise, limitation on range of tilt angles and sub-region subject to electron exposure, unintentional movements of the specimen, with EM systems that make the reconstruction procedure a severely ill-posed problem. A different choice of reconstruction method may lead to different results and create different additional artifacts in reconstructed images. In this paper, we combined the artifacts reduction strategy and the iterative reconstruction algorithm using a Mumford-Shah model. With the combined method, one can not only regularize the ill-posedness and provide segmentation simultaneously but also smooth additional artifacts due to the limited data. We applied the method to both simulated data from the Shepp-Logan phantom and cryo-specimen tomography. The results demonstrate the performance of the method in reducing the noise and artifacts while preserving and enhancing the edges in the reconstructed image.

Two-Stage Image Segmentation Scheme Based on Inexact Alternating Direction Method

AbstractImage segmentation is a fundamental problem in both image processing and computer vision with numerous applications. In this paper, we propose a two-stage image segmentation scheme based on inexact alternating direction method. Specifically, we first solve the convex variant of the Mumford-Shah model to get the smooth solution, the segmentation are then obtained by apply the K-means clustering method to the solution. Some numerical comparisons are arranged to show the effectiveness of our proposed schemes by segmenting many kinds of images such as artificial images, natural images, and brain MRI images.

AMSR-E sea surface temperature (SST) charts enhancement

Microwave satellite images used for retrieving sea surface temperatures often have such distortions as noise and blurring of the thermal fronts. An image processing approach based on the Mumford-Shah model of optimal image approximation is considered for the solution to this problem. We divide images into flat areas and frontal zones, and then process these areas separately. Image fragmentation is based on automatic detection of the thermal front lines. SST enhancement in frontal zones is achieved by using image deconvolution methods. It has been shown that SST errors in high gradient areas reach 1–3 °C. The proposed approach can decrease this discrepancy.

SU-F-T-335: Piecewise Uniform Dose Prescription and Optimization Based On PET/CT Images

Purpose: In intensity modulated radiation therapy (IMRT), the tumor target volume is given a uniform dose prescription, which does not consider the heterogeneous characteristics of tumor such as hypoxia, clonogen density, radiosensitivity, tumor proliferation rate and so on. Our goal is to develop a nonuniform target dose prescription method which can spare organs at risk (OARs) better and does not decrease the tumor control probability (TCP). Methods: We propose a piecewise uniform dose prescription (PUDP) based on PET/CT images of tumor. First, we propose to delineate biological target volumes (BTV) and sub-biological target volumes (sub-BTVs) by our Hierarchical Mumford-Shah Vector Model based on PET/CT images of tumor. Then, in order to spare OARs better, we make the BTV mean dose minimized while restrict the TCP to a constant. So, we can get a general formula for determining an optimal dose prescription based on a linearquadratic model (LQ). However, this dose prescription is high heterogeneous, it is very difficult to deliver by IMRT. Therefore we propose to use the equivalent uniform dose (EUD) in each sub-BTV as its final dose prescription, which makes a PUDP for the BTV. Results: We have evaluated the IMRT planning of a patient with nasopharyngeal carcinoma respectively using PUDP and UDP. The results show that the highest and mean doses inside brain stem are 48.425Gy and 19.151Gy respectively when the PUDP is used for IMRT planning, while they are 52.975Gy and 20.0776Gy respectively when the UDP is used. Both of the resulting TCPs(0.9245, 0.9674) are higher than the theoretical TCP(0.8739), when 70Gy is delivered to the BTV. Conclusion: Comparing with the UDP, the PUDP can spare the OARs better while the resulting TCP by PUDP is not significantly lower than by UDP. This work was supported in part by National Natural Science Foundation of China undergrant no.61271382 and by the foundation for construction of scientific project platform forthe cancer hospital of Hunan province.

TU‐H‐CAMPUS‐IeP3‐01: Simultaneous PET Restoration and PET/CT Co‐Segmentation Using a Variational Method

Purpose:PET images are usually blurred due to the finite spatial resolution, while CT images suffer from low contrast. Segment a tumor from either a single PET or CT image is thus challenging. To make full use of the complementary information between PET and CT, we propose a novel variational method for simultaneous PET image restoration and PET/CT images co‐segmentation.Methods:The proposed model was constructed based on the Γ‐convergence approximation of Mumford‐Shah (MS) segmentation model for PET/CT co‐segmentation. Moreover, a PET de‐blur process was integrated into the MS model to improve the segmentation accuracy. An interaction edge constraint term over the two modalities were specially designed to share the complementary information. The energy functional was iteratively optimized using an alternate minimization (AM) algorithm. The performance of the proposed method was validated on ten lung cancer cases and five esophageal cancer cases. The ground truth were manually delineated by an experienced radiation oncologist using the complementary visual features of PET and CT. The segmentation accuracy was evaluated by Dice similarity index (DSI) and volume error (VE).Results:The proposed method achieved an expected restoration result for PET image and satisfactory segmentation results for both PET and CT images. For lung cancer dataset, the average DSI (0.72) increased by 0.17 and 0.40 than single PET and CT segmentation. For esophageal cancer dataset, the average DSI (0.85) increased by 0.07 and 0.43 than single PET and CT segmentation.Conclusion:The proposed method took full advantage of the complementary information from PET and CT images.This work was supported in part by the National Cancer Institute Grants R01CA172638. Shan Tan and Laquan Li were supported in part by the National Natural Science Foundation of China, under Grant Nos. 60971112 and 61375018.

New second order Mumford–Shah model based on Γ-convergence approximation for image processing

In this paper, a second order variational model named the Mumford–Shah total generalized variation (MSTGV) is proposed for simultaneously image denoising and segmentation, which combines the original Γ-convergence approximated Mumford–Shah model with the second order total generalized variation (TGV). For image denoising, the proposed MSTGV can eliminate both the staircase artefact associated with the first order total variation and the edge blurring effect associated with the quadratic H1 regularization or the second order bounded Hessian regularization. For image segmentation, the MSTGV can obtain clear and continuous boundaries of objects in the image. To improve computational efficiency, the implementation of the MSTGV does not directly solve its high order nonlinear partial differential equations and instead exploits the efficient split Bregman algorithm. The algorithm benefits from the fast Fourier transform, analytical generalized soft thresholding equation, and Gauss–Seidel iteration. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the proposed model.