From the perspectives of health monitoring and life extension of structural materials, this paper addresses the problem of early detection of fatigue cracks in metallic materials (e.g., polycrystalline alloys). To this end, optical images have been collected from an ensemble of test specimens to construct computationally efficient models of crack evolution; these images are segmented into two major categories. The first category comprises images of (structurally) healthy specimens, while the second category contains images of specimens with cracks, including those in early stages of crack evolution. Based on this information, algorithms for early detection of crack formation are formulated in the setting of image classification, where the bag-of-words (BoW) technique has been used to develop models of the sensed images from a microscope, resulting in computationally efficient crack detection algorithms. To evaluate the performance of these crack detection algorithms, experiments have been conducted on a special-purpose fatigue testing apparatus, equipped with a computer-controlled and computer-instrumented confocal microscope system. The results of experimentation with multiple test specimens show excellent crack detection capabilities when the proposed BoW-based feature extraction is combined with quadratic support vector machine (QSVM) for pattern classification. Comparative evaluation with other classification tools establishes superiority of the proposed BoW/QSVM technique.