This paper proposes an effective scale invariant texture representation based on frequency decomposition and gradient orientation. First, the image intensities are decomposed into different orientations by using wedge filters in the frequency domain, and the N-nary coding method is adopted for the vector quantization. Second, the scale invariant gradient orientation is generated by selecting the most stable value of the gradient orientation with different Gaussian scales. Finally, the 2D joint distribution of the two types of local descriptors is used as the representation. The performance was evaluated on texture classification using a nearest neighbor classifier. Simple but not ordinary, our method achieves state of the art classification performance on the KTH-TIPS dataset under the traditional experimental design. Moreover, the main experiments were conducted on the KTH-TIPS and KTH-TIPS2-b datasets with the experimental designs of scale invariance validation. Compared with the methods of basic image features (BIFs) and local energy pattern (LEP), the proposed representation achieves superior performance with much lower dimension of representation.