The conventional non-coherent differential detection of continuous phase modulations (CPM) is quite robust to channel impairments such as phase and Doppler shifts. Its implementation is on top of that simple. It consists in multiplying the received baseband signal by its conjugate version delayed by one symbol period. However it suffers from a signal-to-noise ratio gap compared to the optimum coherent detection. In this letter, we improve the error rate performance of the conventional differential detection by using a delay higher than one symbol period. We derive the trellis description as well as the branch and cumulative metrics that take into account a delay of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> symbol periods. We then determine an optimized delay <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K_{\mathrm {opt}}$ </tex-math></inline-formula> based on the minimum Euclidean distance between two differential signals for some popular CPM formats. The optimized values are confirmed by error rate simulations.