Machine vision has been widely deployed in many industrial applications. However, the accuracy and maturity level for the inspection of through-hole technology (THT) solder joints have yet to reach its ultimate goal. In this paper, we have presented a detailed explanation on a set of novel algorithms that can be effectively used to implement an automatic vision system that is capable of classifying the quality of THT solder joints very precisely. This vision system can be easily integrated into a soldering robotic system that performs automatic soldering on THT solder joints. The proposed vision system is capable of controlling the entire robotic system while providing automatic rework capability on the defective solder joints with much less user interaction. The experimental results indicate that the proposed system has an improved recognition rate and good resilience to noise and uneven illumination distribution.