In this paper, we formulate a generalised photon Boltzmann hierarchy that allows us to model the evolution and creation of spectral distortion anisotropies caused by energy release in the early Universe. We directly build on our first paper in this series, extending the thermalisation Green's function treatment to the anisotropic case. We show that the problem can be described with the common Boltzmann hierarchy for the photon field extended by new spectral parameters — a step that reduces the complexity of the calculation by at least two orders of magnitude. Our formalism describes the effects of i) Doppler and potential driving, ii) spectral evolution by Compton scattering, iii) perturbed thermalisation and iv) anisotropic heating on the distortion anisotropies. We highlight some of the main physical properties of the equations and also outline the steps for computing CMB power spectra including distortion anisotropies. Limitations and extensions of the formulation are also briefly discussed. The novel Boltzmann hierarchy given here is the basis for a series of companion papers studying how distortion anisotropies evolve in the perturbed Universe and which physical processes could be constrained using future CMB imaging techniques.