One of the primary aims of upcoming spaceborne gravitational wave detectors is to measure radiation in the mHz range from extreme-mass-ratio inspirals. Such a detection would place strong constraints on hypothetical departures from a Kerr description for astrophysically stable black holes. The Kerr geometry, which is unique in general relativity, admits a higher-order symmetry in the form of a Carter constant, which implies that the equations of motion describing test particle motion in a Kerr background are Liouville-integrable. In this article, we investigate whether the Carter symmetry itself is discernible from a generic deformation of the Kerr metric in the gravitational waveforms for such inspirals. We build on previous studies by constructing a new metric which respects current observational constraints, describes a black hole, and contains two non-Kerr parameters, one of which controls the presence or absence of the Carter symmetry, thereby controlling the existence of chaotic orbits, and another which serves as a generic deformation parameter. We find that these two parameters introduce fundamentally distinct features into the orbital dynamics, and evince themselves in the gravitational waveforms through a significant dephasing. Although only explored in the quadrupole approximation, this, together with a Fisher metric analysis, suggests that gravitational wave data analysis may be able to test, in addition to the governing theory of gravity, the underlying symmetries of spacetime.