Mikusiński’s operational calculus is a method for interpreting and solving fractional differential equations, formally similar to Laplace transforms but more rigorously justified. This formalism was established for Riemann–Liouville and Caputo fractional calculi in the 1990s, and more recently for other types of fractional calculus. In the general setting of fractional calculus with respect to functions, the authors recently extended Mikusiński’s operational calculus to Riemann–Liouville type derivatives, but the case of Caputo type derivatives with respect to functions remains open. Here, we establish all the function spaces, formalisms, and identities required to build a version of Mikusiński’s operational calculus which covers Caputo derivatives with respect to functions. In the process, we gain a deeper understanding of some of the structures involved in applying Mikusiński’s operational calculus to fractional calculus, such as the existence of a group isomorphic to R. The mathematical structure established here is used to solve fractional differential equations using Caputo derivatives with respect to functions, the solutions being written using multivariate Mittag-Leffler functions, in agreement with the results found in other recent work.