The Internet of connected Vehicles (IOV) can collect, process, compute and release the information of intelligent transportation systems. IOV is an integrated service system that can support the applications for automatic driving, intelligent transport and information services. As the number of incidents on IOV has been on the rise in the past few years, IOV security is becoming increasingly important in the IOV architecture. One of the most notable risks of IOV faces is intelligent terminal security. The vehicle's intelligent terminal can be used to launch for further attacks on the on-board operating system to penetrate into the internal network of connected vehicle, and consequently threaten the safety of the vehicle. Thus, it is of paramount importance that we protect the security of the intelligent terminal. We propose two intelligent terminal based privacy-preserving multi-modal implicit authentication protocols to protect the security of the intelligent terminal in IOV. The proposed protocols use the password and the vehicle owner's behavior features as the authentication factors to protect the security of the intelligent terminal. Since the vehicle owner's behavior features are sensitive and the privacy information of the user must be protected, we also consider the privacy protection of the behavior features. Our protocols do not reveal any information about the vehicle owner's behavior features to the authentication server and the adversary except the ciphertext size of the feature vector. We analyze the security of our proposed protocol and compare them with other related protocols in terms of computation and communications costs. Our results demonstrate that our proposed protocols yield better security and efficiency.