We develop a model-independent approach to Lagrangian perturbation theory for the large scale structure of the universe. We focus on the displacement field for dark matter particles, and derive its most general structure without assuming a specific form for the equations of motion, but implementing a set of general requirements based on symmetry principles and consistency with the perturbative approach. We present explicit results up to sixth order, and provide an algorithmic procedure for arbitrarily higher orders. The resulting displacement field is expressed as an expansion in operators built up from the linear density field, with time-dependent coefficients that can be obtained, in a specific model, by solving ordinary differential equations. The derived structure is general enough to cover a wide spectrum of models beyond ΛCDM, including modified gravity scenarios of the Horndeski type and models with multiple dark matter species. This work is a first step towards a complete model-independent Lagrangian forward model, to be employed in cosmological analyses with power spectrum and bispectrum, other summary statistics, and field-level inference.