There is a significant ongoing effort in realizing quantum annealing with different physical platforms. The challenge is to achieve a fully programmable quantum device featuring coherent adiabatic quantum dynamics. Here we show that combining the well-developed quantum simulation toolbox for Rydberg atoms with the recently proposed Lechner–Hauke–Zoller (LHZ) architecture allows one to build a prototype for a coherent adiabatic quantum computer with all-to-all Ising interactions and, therefore, a platform for quantum annealing. In LHZ an infinite-range spin-glass is mapped onto the low energy subspace of a spin-1/2 lattice gauge model with quasi-local four-body parity constraints. This spin model can be emulated in a natural way with Rubidium and Caesium atoms in a bipartite optical lattice involving laser-dressed Rydberg–Rydberg interactions, which are several orders of magnitude larger than the relevant decoherence rates. This makes the exploration of coherent quantum enhanced optimization protocols accessible with state-of-the-art atomic physics experiments.