When light propagates through a complex medium, such as a multimode optical fiber (MMF), the spatial information it carries is scrambled. In this work we experimentally demonstrate an all-optical strategy to unscramble this light again. We first create a digital model capturing the way light has been scattered, and then use this model to inverse-design and build a complementary optical system–which we call an optical inverter–that reverses this scattering process. Our implementation of this concept is based on multi-plane light conversion, and can also be understood as a physical matrix pre-conditioner, or a linear diffractive optical neural network. We present three design strategies allowing different aspects of device performance to be prioritized. We experimentally demonstrate a prototype optical inverter capable of simultaneously unscrambling up to 30 spatial modes that have propagated through a 1 m long MMF, and show how this promises near instantaneous incoherent imaging, without the need for any beam scanning or computational processing. We also demonstrate the reconfigurable nature of this prototype, allowing it to adapt and deliver a new optical transformation if the MMF it is matched to changes configuration. Our work represents a step towards a new way to see through scattering media. Beyond imaging, this concept may also have applications to the fields of optical communications, optical computing and quantum photonics.