According to the situation that traditional speech authentication algorithms don’t be appropriated for present speech communication, we proposed a speech authentication algorithm of perceptual hashing based on Immittance Spectral Pairs. It can satisfy the requirement of the efficiency and the robustness for speech authentication. Firstly, the speech signal pre-processing, for framing, adding window, obtained for each speech frame immittance spectral Pairs parameters, constitute an immittance spectral Pairs parameter matrix. Then process cepstral mean and variance normalization for immittance spectral Pairs parameter matrix, cepstral mean and variance normalization can effectively improve the robustness of the Gaussian white noise. And parameter matrix for non-negative matrix factorization. Finally, quantifying the formed weight matrix and getting perceptual hashing sequences.Experiments show that the proposed algorithm has good robustness for content preserving operations, and it doesn’t reduce the efficiency while meeting robustness, it can satisfy the real-time requirement of speech communication.