We study the weak mixing of photons and relativistic axionlike particles (axions) in plasmas with background magnetic fields, $\mathbf{B}$. We show that, to leading order in the axion-photon coupling, the conversion probability, ${P}_{\ensuremath{\gamma}\ensuremath{\rightarrow}a}$, is given by the one-dimensional power spectrum of the magnetic field components perpendicular to the particle trajectory. Equivalently, we express ${P}_{\ensuremath{\gamma}\ensuremath{\rightarrow}a}$ as the Fourier transform of the magnetic field autocorrelation function, and establish a dictionary between properties of the real-space magnetic field and the energy-dependent conversion probability. For axions more massive than the plasma frequency, (${m}_{a}>{\ensuremath{\omega}}_{\mathrm{pl}}$), we use this formalism to analytically solve the problem of perturbative axion-photon mixing in a general magnetic field. In the general case where ${\ensuremath{\omega}}_{\mathrm{pl}}/{m}_{a}$ varies arbitrarily along the trajectory, we show that a naive application of the standard formalism for ``resonant'' conversion can give highly inaccurate results, and that a careful calculation generically gives nonresonant contributions at least as large as the resonant contribution. Furthermore, we demonstrate how techniques based on the Fast Fourier Transform provide a new, highly efficient numerical method for calculating axion-photon mixing. We briefly discuss magnetic field modeling in galaxy clusters in the light of our results and argue, in particular, that a recently proposed ``regular'' model used for studying axion-photon mixing (specifically applied to the Perseus cluster) is inconsistent with observations. Our formalism suggests new methods to search for imprints of axions, and will be important for spectrographs with percent level sensitivity, which includes existing X-ray observations by Chandra as well as the upcoming Athena mission.