Vision inspection as an important automation technology in manufacturing industry undertakes many appearance quality related tasks, such as product positioning, recognition, and measurement. Intelligent manufacturing increases the demand for vision methods in production scenes. However, the diversity and increment of the targets, and the feature robustness and scheme migration have restricted further implementation of this technology. To address this problem, this study establishes a robust inspection system, including type insensitive target ontology rough localization, component-based model reverse incremental recognition, and geometric attribute extraction, based on the joint strategy of lightweight neural network recommendation and knowledge guidance. Domain common sense and artificial knowledge are integrated into the whole method, of which the reasonable structural design alleviates the limitation of reliance on artificial intelligence or traditional image processing alone. Taking the connector in the electronic and electrical industry as an example, the proposed method demonstrates successful model recognition and component position extraction of 48 types (including two new models) of connectors, and significant adaptivity to the target posture, imaging environment, and feature diversity.