Automatic modulation recognition (AMR) serves as a crucial component in domains such as cognitive radio and electromagnetic countermeasures, acting as a significant prerequisite for the efficient signal processing of receivers. Deep neural networks (DNNs), despite their effectiveness, are known to be vulnerable to adversarial attacks. This vulnerability has inspired the introduction of subtle interference to wireless communication signals—interference so minuscule that it is difficult for the human eye to discern. Such interference can mislead eavesdroppers into erroneous modulation pattern recognition when using DNNs, thereby camouflaging communication signal modulation patterns. Nonetheless, the majority of current camouflage methods used for electromagnetic signal modulation recognition rely on a global perturbation of the signal. They fail to consider the local agility of signal disturbance and the concealment requirements for bait signals that are intercepted by the interceptor. This paper presents a generator framework designed to produce perturbations with sparse properties. Furthermore, we introduce a method to reduce spectral loss, which minimizes the spectral difference between adversarial perturbation and the original signal. This method makes perturbation more challenging to monitor, thereby deceiving enemy electromagnetic signal modulation recognition systems. The experimental results validated that the proposed method significantly outperformed existing methods in terms of generation time. Moreover, it can generate adversarial signals characterized by high deceivability and transferability even under extremely sparse conditions.