Glassy Nature of the Hard Phase in Inference Problems

Fabrizio Antenucci,Pierfrancesco Urbani,Silvio Franz,Lenka Zdeborová

doi:10.1103/physrevx.9.011020

Abstract

A new analysis of ``hard phase'' inference problems reveals glasslike behavior. Accounting for this insight does not improve algorithm performance, bolstering the notion that such problems cannot be solved in a practical amount of time.

Highlights

Inference problems are ubiquitous in many scientific areas involving data
When it comes to the reconstruction of the signal, our analysis leads us to the remarkable conclusion that, in contrast to constraint-satisfaction and optimization problems, in inference problems, taking into account the glassiness of the hard phase does not improve upon the performance of the simplest approximate message-passing (AMP) algorithm
We evaluate their number as a function of their internal free energy to conclude that this glassiness extends to a range of the noise parameter Δ even larger than the extent of the hard phase

Summary

INTRODUCTION

Inference problems are ubiquitous in many scientific areas involving data. They can be summarized as follows: A signal is measured or observed in some way, and the inference task is to reconstruct the signal from the set of observations. We pose the problem whether, in inference tasks, the reconstruction of the signal becomes easier when one uses algorithms in which the glassiness is correctly taken into account We investigate this strategy thoroughly in the present work. We confirm that the hard phase is glassy in the sense that it consists of an exponential number of local optima at higher free energy than the equilibrium one When it comes to the reconstruction of the signal, our analysis leads us to the remarkable conclusion that, in contrast to constraint-satisfaction and optimization problems, in inference problems, taking into account the glassiness of the hard phase does not improve upon the performance of the simplest AMP algorithm. Thanks to the output universality result of Refs. [23,27], the results presented in this paper hold for a model where the observations Yij ∈ f0; 1g correspond to the adjacency matrix of an unweighted graph with Fisher information corresponding to the inverse of the variance Δ

BAYESIAN INFERENCE AND APPROXIMATE MESSAGE PASSING

SUMMARY OF MAIN ALGORITHMIC RESULT

THE REPLICA APPROACH TO THE POSTERIOR MEASURE

Glassy phase and complexity

CONCLUSION