Most image encryption algorithms encrypt the whole image, but only part of the data is important in the image. In this paper, we propose a multidimensional chaotic image encryption algorithm based on the region of interest (ROI). The histogram of oriented gradients (HOG) feature extraction and support vector machine (SVM) are used to separate the region of interest from the whole image. Then, the region of interest pixels is messed up by using the improved Henon sequence, Joseph sequence and the region of interest pixels are diffused by using the unified chaotic sequence to hide the sensitive information in the image, so as to achieve the purpose of private protection. Furthermore, the improved logistic sequence is used to hide the edge information of the target image to achieve the tradeoff between the secrecy of information and the complexity of encryption. A series of analyses are carried out including key space analysis, key sensitivity analysis, statistical analysis, information entropy analysis, analysis of the fixed-point obscuration analysis, quality analysis and image decryption for our encryption algorithm. Through experiments and comparisons, the proposed algorithm has good performance in encrypting image and coping with various invasions. The image encryption algorithm based on ROI has a good performance of security, moreover through the main encryption of ROI can effectively shorten the encryption time, so as to achieve the compromise of security and computational complexity.