Given the popularity of audio and video applications, compressed audio has become an important carrier of covert communication on the Internet. Many novel compressed audio steganography schemes have emerged that offer good hiding capability and aural concealment. In this paper, a universal steganalysis scheme called MultiSpecNet is proposed to detect steganography based on multiple embedding domains (advanced audio coding (AAC) and MPEG-1 Audio Layer III (MP3)), which are currently the two most popular compressed audio standards. The basic idea is that modification of either domain by a steganography scheme will change the time-frequency relationship of the audio signal after decoding. The proposed approach adopts the spectrogram as the input feature to extract richer information. DeepResNet is used to learn the distinguishing feature representations, and multiscale spectrograms are used to enrich the feature diversity. The experimental results show that the proposed scheme is effective at detecting different steganography schemes based on the AAC and MP3 embedding domains. The detection accuracy of the proposed scheme is higher than that achieved by other state-of-the-art schemes. Using spectrograms as the input, DeepResNet achieves better performance than schemes using quantized modified discrete cosine transform (MDCT) coefficients and mel-spectrogram, although the quantized MDCT coefficient is the parameter modified by the steganography schemes directly and mel-spectrogram is very popular and effective for general audio signal analysis. To the best of our knowledge, this work is the first audio steganalysis scheme that can detect multiple steganography schemes in both the MP3 and AAC embedding domains. The method proposed in this paper can be extended to audio steganalysis for other codecs or for audio forensics purposes.