The partition algorithm as a digital image processing technique is significant to many applications, such as data encryption, image denoising, and 3-D reconstruction. In order to achieve well partition that can availably reduce the distortion phenomenon, a novel approach named image adaptive triangular partition (IATP) is proposed, which considers the grayscale distribution of the image and removes the shared edges between the adjacent triangles in the partition mesh. The least-squares method is used to fit the sampled position-associated gray value of the image to determine whether further partition should be performed, that is, if the sum of squared residuals is bigger than the preselected control value, the current area will be divided into four separated sub-triangles by using the self-similar method, and then preparing the next fitting on each of them in recursion; otherwise, the terminal operation is reached. When the recursive partition of the image is done, the triangular partition mesh with the quaternary notations is obtained. The experimental results demonstrate that the performance of the IATP algorithm proposed in this paper is better than the existing state-of-the-art nonuniform partitions, and it solves the redundant coding problem and reduces the image quality losses. In addition, two applications–image steganography and information encryption–are selected to verify that the proposed algorithm has good feasibility and robustness.