A vision-based finger motion tracking approach is presented to gather quantitative data about breast palpation for cancer detection, such as finger positions, search pattern and coverage area, and this approach is used to develop a prototype palpation training system. Special color markers are used as features of interest because in breast palpation the background of the image is the breast itself which is similar to the fingers in color. This color similarity can hinder the ability or efficiency of other feature extraction methods if real-time performance is required. To simplify the feature extraction process, color space transform is utilized instead of directly using the original RGB values of the image. Although the clinical environment will be well illuminated, normalization of color attributes is applied to compensate for minor changes in illumination. A neighbor search is employed to ensure real-time performance, and a three-finger pattern topology is checked for the extracted features to avoid any possible false features. After detecting the features in the images, 3-D positions of the color marked fingers are calculated using the stereo vision principle. Experimental results with the prototype training system are given to show the performance and effectiveness of the proposed approach. This approach is expected to significantly improve the training quality of breast palpation, thus increasing the detection rate and accuracy of breast cancer.