Image segmentation is an important task in the analysis of dermoscopy images since the extraction of skin lesion borders provides important cues for accurate diagnosis. In recent years, gradient vector flow based algorithms have demonstrated their merits in image segmentation. However, due to the compromise of internal and external energy forces within the partial differential equation these methods commonly lead to under- or over-segmentation problems. In this paper, we introduce a new mean shift based gradient vector flow (GVF) algorithm that drives the internal/external energies towards the correct direction. The proposed segmentation method incorporates a mean shift operation within the standard GVF cost function. Theoretical analysis proves that the proposed algorithm converges rapidly, while experimental results on a large set of diverse dermoscopy images demonstrate that the presented method accurately determines skin lesion borders in dermoscopy images.