The lifetimes of doubly charmed hadrons are analyzed within the framework of the heavy quark expansion (HQE). Lifetime differences arise from spectator effects such as $W$-exchange and Pauli interference. The $\Xi_{cc}^{++}$ baryon is longest-lived in the doubly charmed baryon system owing to the destructive Pauli interference absent in the $\Xi_{cc}^+$ and $\Omega_{cc}^+$. In the presence of dimension-7 contributions, its lifetime is reduced from $\sim5.2\times 10^{-13}s$ to $\sim3.0\times 10^{-13}s$. The $\Xi_{cc}^{+}$ baryon has the shortest lifetime of order $0.45\times 10^{-13}s$ due to a large contribution from the $W$-exchange box diagram. It is difficult to make a precise quantitative statement on the lifetime of $\Omega_{cc}^+$. Contrary to $\Xi_{cc}$ baryons, $\tau(\Omega_{cc}^+)$ becomes longer in the presence of dimension-7 effects and the Pauli interference $\Gamma^{\rm int}_+$ even becomes negative. This implies that the subleading corrections are too large to justify the validity of the HQE. Demanding the rate $\Gamma^{\rm int}_+$ to be positive for a sensible HQE, we conjecture that the $\Omega_c^0$ lifetime lies in the range of $(0.75\sim 1.80)\times 10^{-13}s$. The lifetime hierarchy pattern is $\tau(\Xi_{cc}^{++})>\tau(\Omega_{cc}^+)>\tau(\Xi_{cc}^+)$ and the lifetime ratio $\tau(\Xi_{cc}^{++})/\tau(\Xi_{cc}^+)$ is predicted to be of order 6.7.