Glass-forming colloids consisting of soft core-shell particles were investigated experimentally under medium and large amplitude oscillatory shear (MAOS and LAOS) using Fourier transform rheology to decompose the stress signal into a series of higher harmonics. The anharmonicity of the stress response under MAOS and LAOS is quantified by the intensity of the third harmonic normalized to the fundamental (I3/1=I3/I1) and within the intrinsic nonlinearity framework of the Q-parameter (Q0=limγ0→0(I3/1/γ02)). Furthermore, the results of the strain amplitude dependence were compared to the literature showing the mechanical anharmonic behavior of the core-shell system being close to the behavior of ultrasoft systems. In the glassy state, I3/1 shows an unusual scaling of I3/1∝γ04 at low frequencies, similar to amorphous polymeric materials when they undergo plastic deformation. For investigating the frequency dependence of the anharmonicity in a specially designed binary mixture to test for critical behavior close to the glass transition as predicted by mode coupling theory (MCT) and extend the measurements to the glassy state, we used the frequency sweep MAOS methodology. Using this time-efficient method, the frequency dependence of a wide range of volume fractions and frequencies was investigated, finding the anharmonicity parameter Q0 to be maximal in the region of the α-relaxation for colloidal liquids. The colloidal glasses do not exhibit a maximum in Q0, but an increase in Q0 with decreasing frequency over the investigated region, as the α-relaxation slows down significantly in colloidal glasses. Predictions from MCT from the literature show agreement with the experimentally determined scaling laws.