In this paper, we investigate the problem of impulsive noise cancellation in systems that use jointly overcomplete expansion representations, namely the cyclic geometrically uniform (CGU) frame and sigma-delta (SD) quantization. We first describe the existing analogy between the CGU frames and Reed–Solomn (RS) codes and how it can be exploited in order to reformulate the impulsive noise cancellation problem. As the impulsive noise does by nature represent a sparse signal, it fits the compressed sensing (CS) framework. This is why we investigate how to remodel our impulsive noise cancellation in a CGU-/SD-based system as a CS problem. Then, we study the use and limitations of some existing CS recovery techniques, namely Smoothed L0 (SL0), orthogonal matching pursuit (OMP), compressive sampling matching pursuit (CoSaMP), and Bayes-based approach (BBA). In order to overcome such limitations, we propose how to improve such techniques, and we show that their related improved versions, namely Improved SL0 (I-SL0), I-OMP and I-CoSaMP, and I-BBA efficiently, cancel the impulsive noise of CGU/SD systems and do lead to a quasi-optimal performance. Simulation results are given to support our claims.