We propose two rotation-invariant approaches for color object classification and recognition using multi-channel Zernike moments (MZMs)-based bag-of-visual-words (BoVWs) and deep convolutional neural networks (DCNN). The first approach, referred to as MZMs-BoVWs, derives the BoVWs descriptors from a color image and its sub-images after forming its hierarchical sub-divisions and computing the MZMs from each of the sub-images to determine similarity among sub-images to facilitate the construction of BoVWs. The second approach, called DCNN-MZMs, derives the MZMs descriptors from the feature maps of a convolutional layer of a DCNN architecture designed in this paper. A fusion of the descriptors, MZMs-BoVWs and CNN-MZMs, along with the high-performing color histograms (CH) descriptor, i.e., CH+MZMs-BoVWs+CNN-MZMs, is also proposed. The process of the object classification is performed using SVM and its multiple-kernel learning approach. Experimental results show that the proposed approaches outperform the existing state-of-the-art approaches under image-rotation and under limited data for training the CNNs.