In this paper, we study the mathematical imaging problem of optical diffraction tomography (ODT) for the scenario of a microscopic rigid particle rotating in a trap created, for instance, by acoustic or optical forces. Under the influence of the inhomogeneous forces the particle carries out a time-dependent smooth, but irregular motion described by a set of affine transformations. The rotation of the particle enables one to record optical images from a wide range of angles, which largely eliminates the ‘missing cone problem’ in optics. This advantage, however, comes at the price that the rotation axis in this scenario is not fixed, but continuously undergoes some variations, and that the rotation angles are not equally spaced, which is in contrast to standard tomographic reconstruction assumptions. In the present work, we assume that the time-dependent motion parameters are known, and that the particle’s scattering potential is compatible with making the first order Born or Rytov approximation. We prove a Fourier diffraction theorem and derive novel backpropagation formulae for the reconstruction of the scattering potential, which depends on the refractive index distribution inside the object, taking its complicated motion into account. This provides the basis for solving the ODT problem with an efficient non-uniform discrete Fourier transform.