We give formulae for the cumulants of complex Wishart (LUE) and inverse Wishart matrices (inverse LUE). Their large- N expansions are generating functions of double (strictly and weakly) monotone Hurwitz numbers which count constrained factorisations in the symmetric group. The two expansions can be compared and combined with a duality relation proved in [F. D. Cunden, F. Mezzadri, N. O’Connell, and N. J. Simm, Moments of random matrices and hypergeometric orthogonal polynomials, Comm. Math. Phys. 369 (2019), no. 3, 1091–1145] to obtain: i) a combinatorial proof of the reflection formula between moments of LUE and inverse LUE at genus zero and, ii) a new functional relation between the generating functions of monotone and strictly monotone Hurwitz numbers. The main result resolves the integrality conjecture formulated in [F. D. Cunden, F. Mezzadri, N. J. Simm, and P. Vivo, Correlators for the Wigner–Smith time-delay matrix of chaotic cavities, J. Phys. A 49 (2016), no. 18, 18LT01, 20 pp] on the time-delay cumulants in quantum chaotic transport. The precise combinatorial description of the cumulants given here may cast new light on the concordance between random matrix and semiclassical theories.