Diffusion tensor imaging (DTI) is a promising technique for characterizing organic microstructural information by mapping a spatial point as a 3 × 3 symmetric matrix (called diffusion tensor). Analyzing DTI images directly is challenging since each element of the DTI image is a 3-order matrix rather than a scalar. It is meaningful to extract the scalar-valued feature for DTI image processing and analysis. In this paper, we propose an intrinsic anisotropic feature of DTI images using global geometric attributes of diffusion tensors on the Riemannian manifold. To measure the intrinsic similarity between diffusion tensors as symmetric matrices, we convert them into symmetric positive definite (SPD) matrices and use the normalized geodesic distance in the tensor manifold to define the visible intrinsic feature. Moreover, we extend the geodesic distance to make sure our feature can measure the intrinsic properties of diffusion tensors, and the neighborhood attributes and even the global information of a DTI image are utilized to define the proposed feature. To measure the anisotropy of DTI images, we define the average diffusion sphere at each point to measure the isotropic attributes and use the difference between the diffusion tensor and its average diffusion tensor to describe the anisotropic properties. It is experimentally demonstrated that the proposed feature performs adequately in characterizing various attributes of DTI images, including but not limited anisotropy, intrinsic geometric property, and average diffusion degree, etc. Compared with the most popular features, our anisotropy feature can represent DTI image properties more robustly and effectively since the contrast of various details in the feature is improved. Moreover, the results of fiber tracking defined by the anisotropy feature indicate that our feature performs better on representing substantial anisotropic attributes of DTI images and is potentially used in DTI image processing and analysis.