To address the issue of inaccurate transmission estimation in areas with sudden depth changes in hazy images, the image dehazing algorithm with Heterogeneity Constrained Color Ellipsoid Prior (HC-CEP) is proposed. Firstly, a local threshold optimization quadtree search method is designed, in order to solve the problem of inaccurate global atmospheric light estimation and avoid the overall darkening of the image caused by white objects.Then, different regions are selected by a local block heterogeneity window to construct diversity prior vectors to estimate the initial transmission. Finally, by constructing neighborhood heterogeneity weighted constraints on each pixel of the initial transmission and improving the high-order differential filtering of the Scharr operator, the optimization of the transmission is achieved, eliminating halo artifacts in the depth mutation area of the image. Both qualitative and quantitative experimental results show that the proposed algorithm comprehensively considers the heterogeneity of local pixel characteristics, and the dehazing image obtained has better color fidelity and edge detail preservation ability, which can effectively eliminate halo artifacts on the edges of hazy images.The proposed method has better dehazing performance and has a 8% improvement in terms of the FADE metric compared to state-of-the-art dehazing methods.