Breast thermography as a clinical diagnostic procedure is used to measure the heat coming from the body that provides information based on heat patterns that are strongly indicative of breast abnormality. In this study, an attempt is made to segment hottest regions for detecting region of interest in the infrared breast thermal images using particle swarm optimisation and further with k-means cluster segmentation. Thermography images having different pathologies such as fibrocystic, ductal carcinoma, inflammatory cancer and angiogenesis. Segmentation of breast hot region is performed after removing non-breast region by multiplying original image and ground truth mask. Left and right breast regions are separated by cropping. Segmentation of images based on particle swarm optimisation for determining the threshold level and further k-means cluster is used to estimate the class prototypes indicated by the dense groupings. The differences between cancerous and non-cancerous cases are identified from segmented images using fractal measures.