We consider, in Minkowski spacetime, higher-order Maxwell Lagrangians with terms quadratic in the derivatives of the field strength tensor, and study their degrees of freedom. Using a 3+1 decomposition of these Lagrangians, we extract the kinetic matrix for the components of the electric field, corresponding to second time derivatives of the gauge field. If the kinetic matrix is invertible, the theory admits five degrees of freedom, namely the usual two polarisations of a photon plus three extra degrees of freedom which are shown to be Ostrogradski ghosts. We also classify the cases where the kinetic matrix is non-invertible and, using analogous simple models, we argue that, even though the degeneracy conditions reduce the number of degrees of freedom, it does not seem possible to fully eliminate all potential Ostrogradski ghosts.