Rotation Invariant Convolutional Neural Network Based on Orientation Pooling and Covariance Pooling

Abstract

Convolutional neural networks (CNNs) have shown promising performance in a variety of visual tasks. However, by using convolution operations, CNNs have limited ability to handle orientation changes of input images. To alleviate this problem, data augmentation is typically used. However, this solution brings additional computational and storage costs and has no sufficient theoretical guarantees on rotation invariance. Alternative solutions have been proposed which only take the first-order features into consideration, without utilizing the higher-order information for representation learning. In this paper, we propose an end-to-end rotation invariant CNN (RICNN) based on orientation pooling and covariance pooling to classify rotated images. Specifically, we learn deep rotated filters to extract rotation invariant feature maps by using two types of orientation pooling (OP), including max OP and average OP. Furthermore, we employ covariance pooling to extract rotation invariant hierarchical second-order features. Experiments on two datasets demonstrate the effectiveness of RICNN for rotation invariant image classification.