The session initiation protocol (SIP) has been receiving a lot of attention to provide security in the Voice over IP (VoIP) in Internet and mobility management. Recently, Yeh et al. proposed a smart card-based authentication scheme for SIP using elliptic curve cryptography (ECC). They claimed that their scheme is secure against known security attacks. However, in this paper, we indicate that Yeh et al.'s scheme is vulnerable to off-line password guessing attack, user impersonation attack and server impersonation attack, in the case that the smart card is stolen and the information stored in the smart card is disclosed. As a remedy, we also propose an improved smart card-based authentication scheme which not only conquers the security weaknesses of the related schemes but also provides a reduction in computational cost. The proposed scheme also provides the user anonymity and untraceability, and allows a user to change his/her password without informing the remote server. To show the security of our protocol, we prove its security the random oracle model.