Texture image segmentation is a challenging problem in image processing field due to wide variability of characterizing textures and a lack of proper contour information. In this paper, an effective presentation is found with which different textures can be represented with some corresponding distributions generated by feature values of local inverse difference moment (LIDM). By the analysis of local statistical information in gray level co-occurrence matrix (GLCM), we found that similar textures can be characterized with similar distributions. In this way, an interactive segmentation method is presented to achieve the segmentation of texture image based on GLCM with an optimizing model. Our scheme can be narrated separately as follows, firstly, a proper Gaussian kernel is selected to discriminate two classes of textures by analyzing LIDM feature distributions, which are obtained from two different local regions marked manually in the initial texture image. Secondly, the LIDM feature map can be constructed by computing LIDM feature values of image patches with the proper Gaussian kernel, and the center of these image patches may traverse the whole image domain. Finally, the texture image segmentation is implemented based on an improved optimizing model with local binary fitting and local extremum regularizing. In order to validate the performance of our proposed method, two kinds of experiments about discriminative feature map construction and texture image segmentation are carried out to demonstrate its well performance, and more experiments on real texture images are also conducted.