Context. Models of active galactic nuclei (AGN) suggest that their circumnuclear media are complex with clumps and filaments, while recent observations hint towards polar extended structures of gas and dust, as opposed to the classical torus paradigm. The X-ray band could form an interesting observational window to study these circumnuclear media in great detail. Aims. We want to extend the radiative transfer code SKIRT with the X-ray processes that govern the broad-band X-ray spectra of obscured AGN, to study the structure of AGN circumnuclear media in full 3D, based on their reflected X-ray emission. Methods. We extended the SKIRT code with Compton scattering on free electrons, photo-absorption and fluorescence by cold atomic gas, scattering on bound electrons, and extinction by dust. This includes a novel treatment of extreme-forward scattering by dust, and a detailed description of anomalous Rayleigh scattering. To verify our X-ray implementation, we performed the first dedicated benchmark of X-ray torus models, comparing five X-ray radiative transfer codes. Results. The resulting radiative transfer code covers the X-ray to millimetre wavelength range self-consistently, has all the features of the established SKIRT framework, is publicly available, and is fully optimised to operate in arbitrary 3D geometries. In the X-ray regime, we find an excellent agreement with the simulation results of the MYTORUS and REFLEX codes, which validates our X-ray implementation. We find some discrepancies with other codes, which illustrates the complexity of X-ray radiative transfer and motivates the need for a robust framework that can handle non-linear 3D radiative transfer effects. We illustrate the 3D nature of the code by producing synthetic X-ray images and spectra of clumpy torus models. Conclusions. SKIRT forms a powerful new tool to model circumnuclear media in full 3D, and make predictions for the X-ray band in addition to the dust-dominated infrared-to-UV wavelength range. The new X-ray functionalities of the SKIRT code allow for uncomplicated access to a broad suite of 3D X-ray models for AGN that can easily be tested and modified. This will be particularly useful with the advent of X-ray microcalorimeter observations in the near future.