Merriman-Bence-Osher Research Articles

An MBO method for modularity optimisation based on total variation and signless total variation

Abstract In network science, one of the significant and challenging subjects is the detection of communities. Modularity [1] is a measure of community structure that compares connectivity in the network with the expected connectivity in a graph sampled from a random null model. Its optimisation is a common approach to tackle the community detection problem. We present a new method for modularity maximisation, which is based on the observation that modularity can be expressed in terms of total variation on the graph and signless total variation on the null model. The resulting algorithm is of Merriman–Bence–Osher (MBO) type. Different from earlier methods of this type, the new method can easily accommodate different choices of the null model. Besides theoretical investigations of the method, we include in this paper numerical comparisons with other community detection methods, among which the MBO-type methods of Hu et al. [2] and Boyd et al. [3], and the Leiden algorithm [4].

Graph-Based Bidirectional Transformer Decision Threshold Adjustment Algorithm for Class-Imbalanced Molecular Data

Data sets with imbalanced class sizes, where one class size is much smaller than that of others, occur exceedingly often in many applications, including those with biological foundations, such as disease diagnosis and drug discovery. Therefore, it is extremely important to be able to identify data elements of classes of various sizes, as a failure to do so can result in heavy costs. Nonetheless, many data classification procedures do not perform well on imbalanced data sets as they often fail to detect elements belonging to underrepresented classes. In this work, we propose the BTDT-MBO algorithm, incorporating Merriman–Bence–Osher (MBO) approaches and a bidirectional transformer, as well as distance correlation and decision threshold adjustments, for data classification tasks on highly imbalanced molecular data sets, where the sizes of the classes vary greatly. The proposed technique not only integrates adjustments in the classification threshold for the MBO algorithm in order to help deal with the class imbalance, but also uses a bidirectional transformer procedure based on an attention mechanism for self-supervised learning. In addition, the model implements distance correlation as a weight function for the similarity graph-based framework on which the adjusted MBO algorithm operates. The proposed method is validated using six molecular data sets and compared to other related techniques. The computational experiments show that the proposed technique is superior to competing approaches even in the case of a high class imbalance ratio.

Integrating transformer and autoencoder techniques with spectral graph algorithms for the prediction of scarcely labeled molecular data

In molecular and biological sciences, experiments are expensive, time-consuming, and often subject to ethical constraints. Consequently, one often faces the challenging task of predicting desirable properties from small data sets or scarcely-labeled data sets. Although transfer learning can be advantageous, it requires the existence of a related large data set. This work introduces three graph-based models incorporating Merriman–Bence–Osher (MBO) techniques to tackle this challenge. Specifically, graph-based modifications of the MBO scheme are integrated with state-of-the-art techniques, including a home-made transformer and an autoencoder, in order to deal with scarcely-labeled data sets. In addition, a consensus technique is detailed. The proposed models are validated using five benchmark data sets. We also provide a thorough comparison to other competing methods, such as support vector machines, random forests, and gradient boosting decision trees, which are known for their good performance on small data sets. The performances of various methods are analyzed using residue-similarity (R-S) scores and R-S indices. Extensive computational experiments and theoretical analysis show that the new models perform very well even when as little as 1% of the data set is used as labeled data.

Multiscale Laplacian Learning.

Machine learning has greatly influenced many fields, including science. However, despite of the tremendous accomplishments of machine learning, one of the key limitations of most existing machine learning approaches is their reliance on large labeled sets, and thus, data with limited labeled samples remains a challenge. Moreover, the performance of machine learning methods often severely hindered in case of diverse data, usually associated with smaller data sets or data associated with areas of study where the size of the data sets is constrained by high experimental cost and/or ethics. These challenges call for innovative strategies for dealing with these types of data. In this work, the aforementioned challenges are addressed by integrating graph-based frameworks, semi-supervised techniques, multiscale structures, and modified and adapted optimization procedures. This results in two innovative multiscale Laplacian learning (MLL) approaches for machine learning tasks, such as data classification, and for tackling data with limited samples, diverse data, and small data sets. The first approach, multikernel manifold learning (MML), integrates manifold learning with multikernel information and incorporates a warped kernel regularizer using multiscale graph Laplacians. The second approach, the multiscale MBO (MMBO) method, introduces multiscale Laplacians to the modification of the famous classical Merriman-Bence-Osher (MBO) scheme, and makes use of fast solvers. We demonstrate the performance of our algorithms experimentally on a variety of benchmark data sets, and compare them favorably to the state-of-art approaches.

PTN: A Poisson Transfer Network for Semi-supervised Few-shot Learning

The predicament in semi-supervised few-shot learning (SSFSL) is to maximize the value of the extra unlabeled data to boost the few-shot learner. In this paper, we propose a Poisson Transfer Network (PTN) to mine the unlabeled information for SSFSL from two aspects. First, the Poisson Merriman–Bence–Osher (MBO) model builds a bridge for the communications between labeled and unlabeled examples. This model serves as a more stable and informative classifier than traditional graph-based SSFSL methods in the message-passing process of the labels. Second, the extra unlabeled samples are employed to transfer the knowledge from base classes to novel classes through contrastive learning. Specifically, we force the augmented positive pairs close while push the negative ones distant. Our contrastive transfer scheme implicitly learns the novel-class embeddings to alleviate the over-fitting problem on the few labeled data. Thus, we can mitigate the degeneration of embedding generality in novel classes. Extensive experiments indicate that PTN outperforms the state-of-the-art few-shot and SSFSL models on miniImageNet and tieredImageNet benchmark datasets.

Mass-conserving diffusion-based dynamics on graphs

An emerging technique in image segmentation, semi-supervised learning and general classification problems concerns the use of phase-separating flows defined on finite graphs. This technique was pioneered in Bertozzi and Flenner (2012, Multiscale Modeling and Simulation10(3), 1090–1118), which used the Allen–Cahn flow on a graph, and was then extended in Merkurjev et al. (2013, SIAM J. Imaging Sci.6(4), 1903–1930) using instead the Merriman–Bence–Osher (MBO) scheme on a graph. In previous work by the authors, Budd and Van Gennip (2020, SIAM J. Math. Anal.52(5), 4101–4139), we gave a theoretical justification for this use of the MBO scheme in place of Allen–Cahn flow, showing that the MBO scheme is a special case of a ‘semi-discrete’ numerical scheme for Allen–Cahn flow. In this paper, we extend this earlier work, showing that this link via the semi-discrete scheme is robust to passing to the mass-conserving case. Inspired by Rubinstein and Sternberg (1992, IMA J. Appl. Math.48, 249–264), we define a mass-conserving Allen–Cahn equation on a graph. Then, with the help of the tools of convex optimisation, we show that our earlier machinery can be applied to derive the mass-conserving MBO scheme on a graph as a special case of a semi-discrete scheme for mass-conserving Allen–Cahn. We give a theoretical analysis of this flow and scheme, proving various desired properties like existence and uniqueness of the flow and convergence of the scheme, and also show that the semi-discrete scheme yields a choice function for solutions to the mass-conserving MBO scheme.

Classification and image processing with a semi‐discrete scheme for fidelity forced Allen–Cahn on graphs

AbstractThis paper introduces a semi‐discrete implicit Euler (SDIE) scheme for the Allen‐Cahn equation (ACE) with fidelity forcing on graphs. The continuous‐in‐time version of this differential equation was pioneered by Bertozzi and Flenner in 2012 as a method for graph classification problems, such as semi‐supervised learning and image segmentation. In 2013, Merkurjev et. al. used a Merriman‐Bence‐Osher (MBO) scheme with fidelity forcing instead, as heuristically it was expected to give similar results to the ACE. The current paper rigorously establishes the graph MBO scheme with fidelity forcing as a special case of an SDIE scheme for the graph ACE with fidelity forcing. This connection requires the use of the double‐obstacle potential in the ACE, as was already demonstrated by Budd and Van Gennip in 2020 in the context of ACE without a fidelity forcing term. We also prove that solutions of the SDIE scheme converge to solutions of the graph ACE with fidelity forcing as the discrete time step converges to zero. In the second part of the paper we develop the SDIE scheme as a classification algorithm. We also introduce some innovations into the algorithms for the SDIE and MBO schemes. For large graphs, we use a QR decomposition method to compute an eigendecomposition from a Nyström extension, which outperforms the method used by, for example, Bertozzi and Flenner in 2012, in accuracy, stability, and speed. Moreover, we replace the Euler discretization for the scheme's diffusion step by a computation based on the Strang formula for matrix exponentials. We apply this algorithm to a number of image segmentation problems, and compare the performance with that of the graph MBO scheme with fidelity forcing. We find that while the general SDIE scheme does not perform better than the MBO special case at this task, our other innovations lead to a significantly better segmentation than that from previous literature. We also empirically quantify the uncertainty that this segmentation inherits from the randomness in the Nyström extension.

Blind Hyperspectral Unmixing Based on Graph Total Variation Regularization

Remote sensing data from hyperspectral cameras suffer from limited spatial resolution, in which a single pixel of a hyperspectral image may contain information from several materials in the field of view. Blind hyperspectral image unmixing is the process of identifying the pure spectra of individual materials (i.e., endmembers) and their proportions (i.e., abundances) at each pixel. In this article, we propose a novel blind hyperspectral unmixing model based on the graph total variation (gTV) regularization, which can be solved efficiently by the alternating direction method of multipliers (ADMM). To further alleviate the computational cost, we apply the Nyström method to approximate a fully connected graph by a small subset of sampled points. Furthermore, we adopt the Merriman-Bence-Osher (MBO) scheme to solve the gTV-involved subproblem in ADMM by decomposing a gray-scale image into a bitwise form. A variety of numerical experiments on synthetic and real hyperspectral images are conducted, showcasing the potential of the proposed method in terms of identification accuracy and computational efficiency.

An MBO scheme for clustering and semi-supervised clustering of signed networks

We introduce a principled method for the signed clustering problem, where the goal is to partition a weighted undirected graph whose edge weights take both positive and negative values, such that edges within the same cluster are mostly positive, while edges spanning across clusters are mostly negative. Our method relies on a graph-based diffuse interface model formulation utilizing the Ginzburg-Landau functional, based on an adaptation of the classic numerical Merriman-Bence-Osher (MBO) scheme for minimizing such graph-based functionals. The proposed object ive function aims to minimize the total weight of inter-cluster positively-weighted edges, while maximizing the total weight of the inter-cluster negatively-weighted edges. Our method scales to large sparse networks, and can be easily adjusted to incorporate labelled data information, as is often the case in the context of semisupervised learning. We tested our method on a number of both synthetic stochastic block models and real-world data sets (including financial correlation matrices), and obtained promising results that compare favourably against a number of state-of-the-art approaches from the recent literature.

A maximal energy pointset configuration problem

We consider the extremal pointset configuration problem of maximizing a kernel-based energy subject to the geometric constraints that the points are contained in a fixed set, the pairwise distances are bounded below, and that every closed ball of fixed radius contains at least one point. We also formulate an extremal density problem, whose solution provides an upper bound for the pointset configuration problem in the limit as the number of points tends to infinity. Existence of solutions to both problems is established and the relationship between the parameters in the two problems is studied. Several examples are studied in detail, including the density problem for the d-dimensional ball and sphere, where the solution can be computed exactly using rearrangement inequalities. We develop a computational method for the density problem that is very similar to the Merriman-Bence-Osher (MBO) diffusion-generated method. The method is proven to be increasing for all non-stationary iterations and is applied to study more examples.

Graph Merriman--Bence--Osher as a SemiDiscrete Implicit Euler Scheme for Graph Allen--Cahn Flow

In recent years there has been an emerging interest in PDE-like flows defined on finite graphs, with applications in clustering and image segmentation. In particular for image segmentation and semi-supervised learning Bertozzi and Flenner (2012) developed an algorithm based on the Allen-Cahn gradient flow of a graph Ginzburg-Landau functional, and Merkurjev, Kosti\'c and Bertozzi (2013) devised a variant algorithm based instead on graph Merriman-Bence-Osher (MBO) dynamics. This work offers rigorous justification for this use of the MBO scheme in place of Allen-Cahn flow. First, we choose the double-obstacle potential for the Ginzburg-Landau functional, and derive well-posedness and regularity results for the resulting graph Allen-Cahn flow. Next, we exhibit a "semi-discrete" time-discretisation scheme for Allen-Cahn flow of which the MBO scheme is a special case. We investigate the long-time behaviour of this scheme, and prove its convergence to the Allen-Cahn trajectory as the time-step vanishes. Finally, following a question raised by Van Gennip, Guillen, Osting and Bertozzi (2014), we exhibit results towards proving a link between double-obstacle Allen-Cahn flow and mean curvature flow on graphs. We show some promising $\Gamma$-convergence results, and translate to the graph setting two comparison principles used by Chen and Elliott (1994) to prove the analogous link in the continuum.

A Max-Cut approximation using a graph based MBO scheme

The Max-Cut problem is a well known combinatorial optimization problem. In this paper we describe a fast approximation method. Given a graph $ G $, we want to find a cut whose size is maximal among all possible cuts. A cut is a partition of the vertex set of $ G $ into two disjoint subsets. For an unweighted graph, the size of the cut is the number of edges that have one vertex on either side of the partition; we also consider a weighted version of the problem where each edge contributes a nonnegative weight to the cut.We introduce the signless Ginzburg–Landau functional and prove that this functional $ \Gamma $-converges to a Max-Cut objective functional. We approximately minimize this functional using a graph based signless Merriman–Bence–Osher (MBO) scheme, which uses a signless Laplacian. We derive a Lyapunov functional for the iterations of our signless MBO scheme. We show experimentally that on some classes of graphs the resulting algorithm produces more accurate maximum cut approximations than the current state-of-the-art approximation algorithm. One of our methods of minimizing the functional results in an algorithm with a time complexity of $ \mathcal{O}(|E|) $, where $ |E| $ is the total number of edges on $ G $.

Analysis of Diffusion Generated Motion for Mean Curvature Flow in Codimension Two: A Gradient-Flow Approach

The Merriman–Bence–Osher (MBO) scheme, also known as diffusion generated motion or thresholding, is an efficient numerical algorithm for computing mean curvature flow (MCF). It is fairly well understood in the case of hypersurfaces. This paper establishes the first convergence proof of the scheme in codimension two. We concentrate on the case of the curvature motion of a filament (curve) in $${{\mathbb{R}}^3}$$ . Our proof is based on a new generalization of the minimizing movements interpretation for hypersurfaces (Esedoglu–Otto ’15) by means of an energy that approximates the Dirichlet energy of the state function. As long as a smooth MCF exists, we establish uniform energy estimates for the approximations away from the smooth solution and prove convergence towards this MCF. The current result that holds in codimension two relies in a very crucial manner on a new sharp monotonicity formula for the thresholding energy. This is an improvement of an earlier approximate version.

Unsupervised Classification in Hyperspectral Imagery With Nonlocal Total Variation and Primal-Dual Hybrid Gradient Algorithm

In this paper, a graph-based nonlocal total variation method (NLTV) is proposed for unsupervised classification of hyperspectral images (HSI). The variational problem is solved by the primal-dual hybrid gradient (PDHG) algorithm. By squaring the labeling function and using a stable simplex clustering routine, an unsupervised clustering method with random initialization can be implemented. The effectiveness of this proposed algorithm is illustrated on both synthetic and real-world HSI, and numerical results show that the proposed algorithm outperforms other standard unsupervised clustering methods such as spherical K-means, nonnegative matrix factorization (NMF), and the graph-based Merriman-Bence-Osher (MBO) scheme.

Topology optimization of multi-material negative Poisson’s ratio metamaterials using a reconciled level set method

Metamaterials are defined as a family of rationally designed artificial materials which can provide extraordinary effective properties compared with their nature counterparts. This paper proposes a level set based method for topology optimization of both single and multiple-material Negative Poisson’s Ratio (NPR) metamaterials. For multi-material topology optimization, the conventional level set method is advanced with a new approach exploiting the reconciled level set (RLS) method. The proposed method simplifies the multi-material topology optimization by evolving each individual material with a single level set function and reconciling the result level set field with the Merriman–Bence–Osher (MBO) operator. The NPR metamaterial design problem is recast as a variational problem, where the effective elastic properties of the spatially periodic microstructure are formulated as the strain energy functionals under uniform displacement boundary conditions. The adjoint variable method is utilized to derive the shape sensitivities by combining the general linear elastic equation with a weak imposition of Dirichlet boundary conditions. The design velocity field is constructed using the steepest descent method and integrated with the level set method. Both single and multiple-material mechanical metamaterials are achieved in 2D and 3D with different Poisson’s ratios and volumes. Benchmark designs are fabricated with multi-material 3D printing at high resolution. The effective auxetic properties of the achieved designs are verified through finite element simulations and characterized using experimental tests as well.

Global Binary Optimization on Graphs for Classification of High-Dimensional Data

This work develops a global minimization framework for segmentation of high-dimensional data into two classes. It combines recent convex optimization methods from imaging with recent graph- based variational models for data segmentation. Two convex splitting algorithms are proposed, where graph-based PDE techniques are used to solve some of the subproblems. It is shown that global minimizers can be guaranteed for semi-supervised segmentation with two regions. If constraints on the volume of the regions are incorporated, global minimizers cannot be guaranteed, but can often be obtained in practice and otherwise be closely approximated. Experiments on benchmark data sets show that our models produce segmentation results that are comparable with or outperform the state-of-the-art algorithms. In particular, we perform a thorough comparison to recent MBO (Merriman---Bence---Osher, AMS-Selected Lectures in Mathematics Series: Computational Crystal Growers Workshop, 1992) and phase field methods, and show the advantage of the algorithms proposed in this paper.

Multiclass Data Segmentation Using Diffuse Interface Methods on Graphs.

We present two graph-based algorithms for multiclass segmentation of high-dimensional data on graphs. The algorithms use a diffuse interface model based on the Ginzburg-Landau functional, related to total variation and graph cuts. A multiclass extension is introduced using the Gibbs simplex, with the functional's double-well potential modified to handle the multiclass case. The first algorithm minimizes the functional using a convex splitting numerical scheme. The second algorithm uses a graph adaptation of the classical numerical Merriman-Bence-Osher (MBO) scheme, which alternates between diffusion and thresholding. We demonstrate the performance of both algorithms experimentally on synthetic data, image labeling, and several benchmark data sets such as MNIST, COIL and WebKB. We also make use of fast numerical solvers for finding the eigenvectors and eigenvalues of the graph Laplacian, and take advantage of the sparsity of the matrix. Experiments indicate that the results are competitive with or better than the current state-of-the-art in multiclass graph-based segmentation algorithms for high-dimensional data.

Mean Curvature, Threshold Dynamics, and Phase Field Theory on Finite Graphs

In the continuum, close connections exist between mean curvature flow, the Allen-Cahn (AC) partial differential equation, and the Merriman-Bence-Osher (MBO) threshold dynamics scheme. Graph analogues of these processes have recently seen a rise in popularity as relaxations of NP-complete combinatorial problems, which demands deeper theoretical underpinnings of the graph processes. The aim of this paper is to introduce these graph processes in the light of their continuum counterparts, provide some background, prove the first results connecting them, illustrate these processes with examples and identify open questions for future study. We derive a graph curvature from the graph cut function, the natural graph counterpart of total variation (perimeter). This derivation and the resulting curvature definition differ from those in earlier literature, where the continuum mean curvature is simply discretized, and bears many similarities to the continuum nonlocal curvature or nonlocal means formulation. This new graph curvature is not only relevant for graph MBO dynamics, but also appears in the variational formulation of a discrete time graph mean curvature flow. We prove estimates showing that the dynamics are trivial for both MBO and AC evolutions if the parameters (the time-step and diffuse interface scale, respectively) are sufficiently small (a phenomenon known as "freezing" or "pinning") and also that the dynamics for MBO are nontrivial if the time step is large enough. These bounds are in terms of graph quantities such as the spectrum of the graph Laplacian and the graph curvature. Adapting a Lyapunov functional for the continuum MBO scheme to graphs, we prove that the graph MBO scheme converges to a stationary state in a finite number of iterations. Variations on this scheme have recently become popular in the literature as ways to minimize (continuum) nonlocal total variation.

A Method Based on Total Variation for Network Modularity Optimization Using the MBO Scheme

The study of network structure is pervasive in sociology, biology, computer science, and many other disciplines. One of the most important areas of network science is the algorithmic detection of cohesive groups of nodes called “communities.” One popular approach to finding communities is to maximize a quality function known as modularity to achieve some sort of optimal clustering of nodes. In this paper, we interpret the modularity function from a novel perspective: we reformulate modularity optimization as a minimization problem of an energy functional that consists of a total variation term and an $\ell_2$ balance term. By employing numerical techniques from image processing and $\ell_1$ compressive sensing---such as convex splitting and the Merriman--Bence--Osher (MBO) scheme---we develop a variational algorithm for the minimization problem. We present our computational results using both synthetic benchmark networks and real data.

An MBO Scheme on Graphs for Classification and Image Processing

In this paper we present a computationally efficient algorithm utilizing a fully or seminonlocal graph Laplacian for solving a wide range of learning problems in binary data classification and image processing. In their recent work [Multiscale Model. Simul., 10 (2012), pp. 1090--1118], Bertozzi and Flenner introduced a graph-based diffuse interface model utilizing the Ginzburg--Landau functional for solving problems in data classification. Here, we propose an adaptation of the classic numerical Merriman--Bence--Osher (MBO) scheme for minimizing graph-based diffuse interface functionals, like those originally proposed by Bertozzi and Flenner. We also make use of fast numerical solvers for finding eigenvalues and eigenvectors of the graph Laplacian. Various computational examples are presented to demonstrate the performance of our algorithm, which is successful on images with texture and repetitive structure due to its nonlocal nature. The results show that our method is multiple times more efficient than o...