A hierarchical Bayesian perspective on majorization-minimization for non-convex sparse regression: application to M/EEG source imaging

Yousra Bekhti,Felix Lucka,Alexandre Gramfort,Joseph Salmon

doi:10.1088/1361-6420/aac9b3

Abstract

Majorization-minimization (MM) is a standard iterative optimization technique which consists in minimizing a sequence of convex surrogate functionals. MM approaches have been particularly successful to tackle inverse problems and statistical machine learning problems where the regularization term is a sparsity-promoting concave function. However, due to non-convexity, the solution found by MM depends on its initialization. Uniform initialization is the most natural and often employed strategy as it boils down to penalizing all coefficients equally in the first MM iteration. Yet, this arbitrary choice can lead to unsatisfactory results in severely under-determined inverse problems such as source imaging with magneto- and electro-encephalography (M/EEG). The framework of hierarchical Bayesian modeling (HBM) is an alternative approach to encode sparsity. This work shows that for certain hierarchical models, a simple alternating scheme to compute fully Bayesian maximum a posteriori (MAP) estimates leads to the exact same sequence of updates as a standard MM strategy (see the adaptive lasso). With this parallel outlined, we show how to improve upon these MM techniques by probing the multimodal posterior density using Markov Chain Monte-Carlo (MCMC) techniques. Firstly, we show that these samples can provide well-informed initializations that help MM schemes to reach better local minima. Secondly, we demonstrate how it can reveal the different modes of the posterior distribution in order to explore and quantify the inherent uncertainty and ambiguity of such ill-posed inference procedure. In the context of M/EEG, each mode corresponds to a plausible configuration of neural sources, which is crucial for data interpretation, especially in clinical contexts. Results on both simulations and real datasets show how the number or the type of sensors affect the uncertainties on the estimates.

Highlights

Over the last two decades, sparsity has emerged as a key concept to solve inverse problems such as tomographic image reconstruction, deconvolution or inpainting, and to regularize high dimensional regression problems in the field of machine learning
We describe a hierarchical Bayesian modeling (HBM) inference strategy based upon an Markov chain Monte-Carlo (MCMC) sampling and show on simulated and exper imental magneto- and electro-encephalography (M/EEG) datasets how these stochastic MCMC-based techniques can help to improve upon deterministic approaches, and help to reveal multiple plausible solutions to the inverse problem
We examine the benefits of our re-interpretation of the MM algorithm described in section 2.1 as a specific way to compute a full-maximum a posteriori (MAP) estimate for a specific HBM as described in sections 2.2 and 2.3

Summary

Introduction

Over the last two decades, sparsity has emerged as a key concept to solve inverse problems such as tomographic image reconstruction, deconvolution or inpainting, and to regularize high dimensional regression problems in the field of machine learning. The first route, embraced by the optimization community and frequentist statisticians, is to promote sparsity using convex optimization theory. This line of work has led to mature theoretical guarantees (Foucart and Rauhut 2013) when using regularization functions based on 1-norm and other convex variants (Tibshirani 1996). It has been popularized in the signal processing community under the name of compressed sensing (Candès and Wakin 2008) when combined with incoherent measurements. There are some limitations of sparsity-promoting convex penalties based on the 1-norm. Convex regularizations lead to a systematic underestimation bias in the amplitude of the coefficients (Osher et al 2006, Chartrand 2007, Candès et al 2008, Saab et al 2008, Chzhen et al 2017)

Methods

Results

Conclusion