Image texture description and analysis technology are the basis of many practical applications in pattern recognition. This paper presents a novel and simple, yet powerful method, namely multiple channels local binary pattern (MCLBP), which is the natural extension and development of local binary pattern (LBP) algorithm for color texture representation and classification. MCLBP combines single-channel texture characteristics with multi-channel color information, which reflects the correlations and dependency among different channels. Furthermore, we decompose local color differences into color-difference signs and color-difference magnitudes and MCLBP is extended to MCLBP+M. Then, the resulted image descriptor is a histogram representation, which fuses rich features including color difference sign and color difference magnitude. Comprehensive experiments conducted on five benchmark databases, including Outex, KTH-TIPS, CUReT, STex and KTH-TIPS2-b clearly demonstrate that our proposed method outperforms most of the existed color texture features in terms of classification accuracy. Particularly, our method achieves the best classification performance in CUReT and STex databases.