The current face scanning era can quickly and conveniently attain identity authentication, but face images imply sensitive information simultaneously. Under such context, we introduce a novel cancelable face recognition methodology by using quaternion transform based convolutional network. Firstly, face images in different modalities (e.g., RGB and depth or near-infrared) are encoded into full quaternion matrix for synchronous processing. Based on the designed multiresolution quaternion singular value decomposition, we can obtain pyramid representation. Then they are transformed through random projection for making the process noninvertible. Even if the feature template is compromised, a new one can be generated. Subsequently, a three-stream convolutional network is developed to learn features, where predefined filters are stemmed from quaternion two-dimensional discrete cosine transform basis. Extensive experiments on the TIII-D, NVIE and CASIA datasets have demonstrated that the proposed method obtains competitive performance, also satisfies redistributable and irreversible.