Segmenting outdoor images in the presence of haze, fog or smog (which fades the colors and diminishes the contrast of the observed objects) has been a challenging task in image processing with several important applications. In this paper, we propose a new fractional-order variational model that will be able to de-haze and segment a given image simultaneously. The proposed method incorporates the atmospheric veil estimation based on the dark channel prior (DCP). This transmission map can reduce significantly the edge artifacts and enhance estimation precision in the resulting image. The transmission map is then changed over to the high-quality depth map, with which the new fractional-order variational model can be framed to look for the haze free segmenting image for both grey and color outdoor images. An explicit gradient descent scheme is employed to find efficiently the minimizer of the proposed energy functional. Experimental tests on real world scenes show that the proposed method can jointly de-haze and segment hazy or foggy images effectively and efficiently.