Mumford Shah Research Articles

Assembling a Learnable Mumford–Shah Type Model with Multigrid Technique for Image Segmentation

Assembling a Learnable Mumford–Shah Type Model with Multigrid Technique for Image Segmentation

Segmentation in Measure Spaces

We consider an abstract concept of perimeter measure space as a very general framework in which one can properly consider two of the most well-studied variational models in image processing: the Rudin–Osher–Fatemi model for image denoising (ROF) and the Mumford–Shah model for image segmentation (MS). We show the linkage between the ROF model and the two phases piecewise constant case of MS in perimeter measure spaces. We show applications of our results to nonlocal image segmentation, via discrete weighted graphs, and to multiclass classification on high dimensional spaces.

Semi-supervised learning towards automated segmentation of PET images with limited annotations: application to lymphoma patients.

Manual segmentation poses a time-consuming challenge for disease quantification, therapy evaluation, treatment planning, and outcome prediction. Convolutional neural networks (CNNs) hold promise in accurately identifying tumor locations and boundaries in PET scans. However, a major hurdle is the extensive amount of supervised and annotated data necessary for training. To overcome this limitation, this study explores semi-supervised approaches utilizing unlabeled data, specifically focusing on PET images of diffuse large B-cell lymphoma (DLBCL) and primary mediastinal large B-cell lymphoma (PMBCL) obtained from two centers. We considered 2-[18F]FDG PET images of 292 patients PMBCL (n = 104) and DLBCL (n = 188) (n = 232 for training and validation, and n = 60 for external testing). We harnessed classical wisdom embedded in traditional segmentation methods, such as the fuzzy clustering loss function (FCM), to tailor the training strategy for a 3D U-Net model, incorporating both supervised and unsupervised learning approaches. Various supervision levels were explored, including fully supervised methods with labeled FCM and unified focal/Dice loss, unsupervised methods with robust FCM (RFCM) and Mumford-Shah (MS) loss, and semi-supervised methods combining FCM with supervised Dice loss (MS + Dice) or labeled FCM (RFCM + FCM). The unified loss function yielded higher Dice scores (0.73 ± 0.11; 95% CI 0.67-0.8) than Dice loss (p value < 0.01). Among the semi-supervised approaches, RFCM + αFCM (α = 0.3) showed the best performance, with Dice score of 0.68 ± 0.10 (95% CI 0.45-0.77), outperforming MS + αDice for any supervision level (any α) (p < 0.01). Another semi-supervised approach with MS + αDice (α = 0.2) achieved Dice score of 0.59 ± 0.09 (95% CI 0.44-0.76) surpassing other supervision levels (p < 0.01). Given the time-consuming nature of manual delineations and the inconsistencies they may introduce, semi-supervised approaches hold promise for automating medical imaging segmentation workflows.

Image Segmentation Using Bayesian Inference for Convex Variant Mumford–Shah Variational Model

Image Segmentation Using Bayesian Inference for Convex Variant Mumford–Shah Variational Model

Existence, uniqueness and L2t(H2x) ∩ L∞t(H1x) ∩ H1t(L2x) regularity of the gradient flow of the Ambrosio–Tortorelli functional

We consider the gradient flow of the Ambrosio–Tortorelli functional, proving existence, uniqueness and L2t(H2x) ∩ L∞t(H1x) ∩ H1t(L2x) regularity of the solution in dimension 2. Such functional is an approximation in the sense of Γ-convergence of the Mumford–Shah functional often used in problems of image segmentation and fracture mechanics. The strategy of the proof essentially follows the one of []Feng and Prohl, M2AN Math. Model. Numer. Anal. 38 (2004) 291–320] but the crucial estimate is attained employing a different technique, and in the end it allows to prove better estimates than the ones obtained in [Feng and Prohl, M2AN Math. Model. Numer. Anal. 38 (2004) 291–320]. In particular we prove that if U ⊂ ℝ2 is a bounded C2 domain, the initial data (u0, z0) ∈ [H1(U)]2 with 0 ≤ z0 ≤ 1, then for every T &gt; 0 there exists a unique gradient flow (u(t), z(t)) of the Ambrosio–Tortorelli functional such that (u, z) ∈ [L2(0, T; H2(U)) ∩ L∞(0,T;H1(U)) ∩H1(0,T;L2(U)]2 while previously such regularity was known only for short times.

Multilevel thresholding image segmentation algorithm based on Mumford–Shah model

Abstract Image segmentation is one of the important tasks of computer vision and computer image processing, and the purpose of image segmentation is to achieve the extraction and recognition of the target image region. The classical Mumford–Shah (MSh) image segmentation model is used to achieve the segmentation of images. With the goal to get the best segmentation effect on images by minimizing the MSh energy generalization function, a level set strategy is developed, and a model with global information infinite curve evolution is utilized. However, considering the low efficiency of this model for processing level set curves and the general quality of image segmentation. A multi-layer threshold search scheme is proposed to achieve rapid convergence of the target image level set curve. The experimental results showed that the multi-level thresholding image segmentation algorithm based on the MSh model can significantly improve the segmentation effect of images and reduce the segmentation time. The suggested MSK method outperforms the MPO algorithm, SSA algorithm, and EMO algorithm in the picture segmentation convergence time test, respectively, in terms of runtime efficiency by 356, 289, and 71%. Additionally, it performs superbly in both threshold searches and picture quality tests. The research topic has significant reference value for the study of contemporary computer vision imaging technologies.

A fractional-order image segmentation model with application to low-contrast and piecewise smooth images

In this paper, we propose a two-stage image segmentation model based on structure tensor and fractional-order regularization. In the first stage, we use the fractional-order regularization to approximate the Hausdorff measure of the Mumford-Shah (MS) model. The existence and uniqueness of the solution is proved and the alternating direction implicit (ADI) scheme is used to find the solution of the modified MS model. In the second stage, a thresholding is used to induce the segmentation of the target. The superior performances of the proposed model are demonstrated by some comparative experimental results with several state-of-art methods.

A note on the one-dimensional critical points of the Ambrosio–Tortorelli functional

This note addresses the question of convergence of critical points of the Ambrosio–Tortorelli functional in the one-dimensional case under pure Dirichlet boundary conditions. An asymptotic analysis argument shows the convergence to two possible limits points: either a globally affine function or a step function with a single jump at the middle point of the space interval, which are both critical points of the one-dimensional Mumford–Shah functional under a Dirichlet boundary condition. As a byproduct, non minimizing critical points of the Ambrosio–Tortorelli functional satisfying the energy convergence assumption as in (Babadjian, Millot and Rodiac (2022)) are proved to exist.

Higher order Ambrosio–Tortorelli scheme with non-negative spatially dependent parameters

Abstract The Ambrosio–Tortorelli approximation scheme with weighted underlying metric is investigated. It is shown that it Γ-converges to a Mumford–Shah image segmentation functional depending on the weight ω ⁢ d ⁢ x {\omega\,dx} , where ω is a special function of bounded variation, and on its values at the jumps.

Hyperparameter selection for Discrete Mumford–Shah

This work focuses on a parameter-free joint piecewise smooth image denoising and contour detection. Formulated as the minimization of a discrete Mumford-Shah functional and estimated via a theoretically grounded alternating minimization scheme, the bottleneck of such a variational approach lies in the need to fine-tune their hyperparameters, while not having access to ground truth data. To that aim, a Stein-like strategy providing optimal hyperparameters is designed, based on the minimization of an unbiased estimate of the quadratic risk. Efficient and automated minimization of the estimate of the risk crucially relies on an unbiased estimate of the gradient of the risk with respect to hyperparameters. Its practical implementation is performed using a forward differentiation of the alternating scheme minimizing the Mumford-Shah functional, requiring exact differentiation of the proximity operators involved. Intensive numerical experiments are performed on synthetic images with different geometry and noise levels, assessing the accuracy and the robustness of the proposed procedure. The resulting parameter-free piecewise-smooth estimation and contour detection procedure, not requiring prior image processing expertise nor annotated data, can then be applied to real-world images.

FAS-UNet: A Novel FAS-Driven UNet to Learn Variational Image Segmentation

Solving variational image segmentation problems with hidden physics is often expensive and requires different algorithms and manually tuned model parameters. The deep learning methods based on the UNet structure have obtained outstanding performances in many different medical image segmentation tasks, but designing such networks requires many parameters and training data, which are not always available for practical problems. In this paper, inspired by the traditional multiphase convexity Mumford–Shah variational model and full approximation scheme (FAS) solving the nonlinear systems, we propose a novel variational-model-informed network (FAS-UNet), which exploits the model and algorithm priors to extract the multiscale features. The proposed model-informed network integrates image data and mathematical models and implements them through learning a few convolution kernels. Based on the variational theory and FAS algorithm, we first design a feature extraction sub-network (FAS-Solution module) to solve the model-driven nonlinear systems, where a skip-connection is employed to fuse the multiscale features. Secondly, we further design a convolutional block to fuse the extracted features from the previous stage, resulting in the final segmentation possibility. Experimental results on three different medical image segmentation tasks show that the proposed FAS-UNet is very competitive with other state-of-the-art methods in the qualitative, quantitative, and model complexity evaluations. Moreover, it may also be possible to train specialized network architectures that automatically satisfy some of the mathematical and physical laws in other image problems for better accuracy, faster training, and improved generalization.

Higher integrability of the gradient for the thermal insulation problem

We prove the higher integrability of the gradient for local minimizers of the thermal insulation problem: an analogue of De Giorgi’s conjecture for the Mumford–Shah functional. We deduce that the singular part of the free boundary has Hausdorff dimension strictly less than $n - 1$.

Feature-preserving Mumford–Shah mesh processing via nonsmooth nonconvex regularization

Motivated by the success in image processing, the Mumford–Shah functional has attracted extensive attentions in geometry processing. Existing methods, mainly focusing on discretizations on the triangulated mesh, either over-smooth sharp features or are sensitive to noises or outliers. In this paper, we first introduce a nonsmooth nonconvex Mumford–Shah model for a feature-preserving filtering of face normal field to ameliorate the staircasing artifacts that appear in the original Mumford–Shah total variation (MSTV) and develop an alternating minimization scheme based on alternating direction method of multipliers to realize the proposed model. After restoring the face normal field, vertex updating is then employed by incorporating the oriented normal constraints and discontinuities to achieve a detail-preserving reconstruction of mesh geometry. Extensive experimental results demonstrate the effectiveness of the above shape optimization routine for various geometry processing applications such as mesh denoising, mesh inpainting and mesh segmentation.

Low-Cost Probabilistic 3D Denoising with Applications for Ultra-Low-Radiation Computed Tomography.

We propose a pipeline for synthetic generation of personalized Computer Tomography (CT) images, with a radiation exposure evaluation and a lifetime attributable risk (LAR) assessment. We perform a patient-specific performance evaluation for a broad range of denoising algorithms (including the most popular deep learning denoising approaches, wavelets-based methods, methods based on Mumford–Shah denoising, etc.), focusing both on accessing the capability to reduce the patient-specific CT-induced LAR and on computational cost scalability. We introduce a parallel Probabilistic Mumford–Shah denoising model (PMS) and show that it markedly-outperforms the compared common denoising methods in denoising quality and cost scaling. In particular, we show that it allows an approximately 22-fold robust patient-specific LAR reduction for infants and a 10-fold LAR reduction for adults. Using a normal laptop, the proposed algorithm for PMS allows cheap and robust (with a multiscale structural similarity index >90%) denoising of very large 2D videos and 3D images (with over voxels) that are subject to ultra-strong noise (Gaussian and non-Gaussian) for signal-to-noise ratios far below 1.0. The code is provided for open access.

Mumford–Shah regularization in electrical impedance tomography with complete electrode model

In electrical impedance tomography (EIT), we aim to solve the conductivity within a target body through electrical measurements made on the surface of the target. This inverse conductivity problem is severely ill-posed, especially in real applications with only partial boundary data available. Thus regularization has to be introduced. Conventionally regularization promoting smooth features is used, however, the Mumford–Shah (M–S) regularizer familiar for image segmentation is more appropriate for targets consisting of several distinct objects or materials. It is, however, numerically challenging. We show theoretically through Γ-convergence that a modification of the Ambrosio–Tortorelli approximation of the M–S regularizer is applicable to EIT, in particular the complete electrode model of boundary measurements. With numerical and experimental studies, we confirm that this functional works in practice and produces higher quality results than typical regularizations employed in EIT when the conductivity of the target consists of distinct smoothly-varying regions.

A Distribution-Dependent Mumford–Shah Model for Unsupervised Hyperspectral Image Segmentation

Hyperspectral images provide a rich representation of the underlying spectrum for each pixel, allowing for a pixel-wise classification/segmentation into different classes. As the acquisition of labeled training data is very time-consuming, unsupervised methods become crucial in hyperspectral image analysis. The spectral variability and noise in hyperspectral data make this task very challenging and define special requirements for such methods. Here, we present a novel unsupervised hyperspectral segmentation framework. It starts with a denoising and dimensionality reduction step by the well-established Minimum Noise Fraction (MNF) transform. Then, the Mumford-Shah (MS) segmentation functional is applied to segment the data. We equipped the MS functional with a novel robust distribution-dependent indicator function designed to handle the characteristic challenges of hyperspectral data. To optimize our objective function with respect to the parameters for which no closed form solution is available, we propose an efficient fixed point iteration scheme. Numerical experiments on four public benchmark datasets show that our method produces competitive results, which outperform three state-of-the-art methods substantially on three of these datasets.

K-Means image segmentation using Mumford–Shah model

k-Means (KM) is well known for its ease of implementation as a clustering technique. It has been applied for color quantization in RGB, YUV, hyperspectral image, Lab, and other spaces, but this leads to fragmented segments as the pixels are clustered only in the color space without considering connectivity. The problem has been attacked by adding connectivity constraints, or using joint color and spatial features (r, g, b, x, y), which prevent fragmented and nonconvex segments. However, it does not take into account the complexity of the shape itself. The Mumford–Shah model has been earlier used to overcome this problem but with slow and complex mathematical optimization algorithms. We integrate Mumford–Shah model directly into KM context and construct a fast and simple implementation of the algorithm. The proposed approach uses standard KM algorithm with distance function derived from Mumford–Shah model so that it optimizes both the content and the shape of the segments jointly. We demonstrate by experiments that the proposed algorithm provides better results than comparative methods when compared using various error evaluation criteria. The algorithm is applied on 100 images in the Weizmann dataset and two remote sensing images.

On a Variational and Convex Model of the Blake-Zisserman Type for Segmentation of Low-Contrast and Piecewise Smooth Images.

This paper proposes a new variational model for segmentation of low-contrast and piecewise smooth images. The model is motivated by the two-stage image segmentation work of Cai–Chan–Zeng (2013) for the Mumford–Shah model. To deal with low-contrast images more effectively, especially in treating higher-order discontinuities, we follow the idea of the Blake–Zisserman model instead of the Mumford–Shah. Two practical ideas are introduced here: first, a convex relaxation idea is used to derive an implementable formulation, and second, a game reformulation is proposed to reduce the strong dependence of coupling parameters. The proposed model is then analysed for existence and further solved by an ADMM solver. Numerical experiments can show that the new model outperforms the current state-of-the-art models for some challenging and low-contrast images.

Unsupervised Object Segmentation Based on Bi-Partitioning Image Model Integrated with Classification

The development of convolutional neural networks for deep learning has significantly contributed to image classification and segmentation areas. For high performance in supervised image segmentation, we need many ground-truth data. However, high costs are required to make these data, so unsupervised manners are actively being studied. The Mumford–Shah and Chan–Vese models are well-known unsupervised image segmentation models. However, the Mumford–Shah model and the Chan–Vese model cannot separate the foreground and background of the image because they are based on pixel intensities. In this paper, we propose a weakly supervised model for image segmentation based on the segmentation models (Mumford–Shah model and Chan–Vese model) and classification. The segmentation model (i.e., Mumford–Shah model or Chan–Vese model) is to find a base image mask for classification, and the classification network uses the mask from the segmentation models. With the classifcation network, the output mask of the segmentation model changes in the direction of increasing the performance of the classification network. In addition, the mask can distinguish the foreground and background of images naturally. Our experiment shows that our segmentation model, integrated with a classifier, can segment the input image to the foreground and the background only with the image’s class label, which is the image-level label.

Uncertainty Quantification in Image Segmentation Using the Ambrosio\u2013Tortorelli Approximation of the Mumford\u2013Shah Energy

The quantification of uncertainties in image segmentation based on the Mumford–Shah model is studied. The aim is to address the error propagation of noise and other error types in the original image to the restoration result and especially the reconstructed edges (sharp image contrasts). Analytically, we rely on the Ambrosio–Tortorelli approximation and discuss the existence of measurable selections of its solutions as well as sampling-based methods and the limitations of other popular methods. Numerical examples illustrate the theoretical findings.