In the last decade, E-voting has received great attention due to its advantages in efficiency and accuracy. Fan et al. presented a novel E-voting system named HSE-Voting by utilizing homomorphic signcryption. The HSE-Voting system was claimed to gain a provable security goal under the standard proof. In this paper, we illustrate that their scheme may suffer from some potential security issues. On the one hand, the voting information could be recovered by the authentication center (AC). On the other hand, any malicious voter could disrupt the voting system undetected by locally modifying his ballot. In order to increase the resilience of the voting system to risks, an improvement of the HSE-Voting system is developed. Our improved system fixes the above security weaknesses but increases the computation cost on the AC side by a small amount. In addition, the proposed scheme satisfies voter anonymity, ballot privacy, and verifiability of election results.