Various surface mucosal pit patterns, as recognized by endoscopists, correlate with the histologic features of colorectal cancers. We investigated whether magnified endoscopy images of these pit patterns could be analyzed quantitatively and thus facilitate computer-aided diagnosis of colorectal lesions. We applied both texture analysis and scale-invariant feature transform (SIFT) descriptors and discriminant analysis to magnified endoscopy images of 165 neoplastic colorectal lesions (pit patterns: type III(L)/IV, n = 44; type V(I)-mildly irregular, n = 36; type V(I)-severely irregular, n = 45; type V(N), n = 40) [histologic findings: tubular adenoma (TA), n = 56; carcinoma with intramucosal or even scant submucosal invasion (M/SM-s), n = 52, carcinoma with massive submucosal invasion (SM-m), n = 57]. We analyzed differences in pit pattern values and corresponding histologic values to determine whether the values were diagnostically meaningful. Gray-level difference matrix (GLDM) inverse difference moment and spatial gray-level dependence matrix (SGLDM) local homogeneity values differed significantly between type III(L)/IV and type V(N) pit patterns. Values differed significantly for each analyzed feature between type III(L)/IV and type V(I)-severely irregular patterns and were high but descending for type III(L)/IV, type V(I)-mildly irregular, and type V(I)-severely irregular pit patterns (in that order). Similarly, texture analysis yielded high but descending values for TA, M/SM-s, and SM-m (in that order). Furthermore, SIFT descriptors and discriminant analysis yielded differences that were superior to those obtained by texture analyses. Computer analysis of magnified endoscopy images for the diagnosis of colorectal lesions appears feasible. We anticipate further developments in the computer-aided diagnosis of pit patterns on magnified endoscopy images.