Recently, Jiang proposed a semi-quantum private comparison (SQPC) protocol without the use of a quantum measurement device (Quantum Inf Process 19:182, 2020). However, this paper highlights a security loophole during the transmission phase of the protocol in which a malicious participant can perform a double CNOT attack to steal partial information of the other participant’s private secret without being detected. With the same assumption of the semi-quantum environment as Jiang’s protocol, this paper presents an improved protocol by adopting a strategy of pre-sharing session keys to overcome the double CNOT attack. Moreover, a security analysis of the collective attack is also provided for Jiang’s and the improved SQPC protocols.