Session initiation protocol (SIP), a widely used signal protocol for controlling multimedia communication sessions, is under numerous attacks when performing the authentication steps between the user and server. So secure authentication schemes are needed to be presented for SIP. Recently, Arshad et al. advanced novel schemes for SIP using elliptic curve cryptography (ECC) and claimed their schemes can resist various attacks. However, Lu et al. found that Arshad et al.’s scheme cannot resist trace and key-compromise impersonation attacks; hence, it cannot provide proper mutual authentication. Meanwhile, an enhanced scheme was advanced by Lu et al. and they stated that their scheme can stand up to possible known attacks. Nevertheless, in this paper, we conclude that Arshad and Nikooghadam’s scheme is insecure against impersonation attack and Lu et al.’s scheme is still vulnerable to impersonation attack. To overcome these weaknesses of their schemes, we present a novel anonymous ECC-based scheme for SIP. Security analysis and performance analysis show that our proposed scheme can resist various known attacks and efficient in the meantime.