Integrating deep learning and rule-based systems into a smart devices decision support system for visual inspection in production

Abstract

Due to the increasing implementation of cyber physical systems and the internet of things in industry, there is a trend towards flexible, multi-variant production with a very high degree of automation to strengthen the resilience of the company. Each variant is subject to high quality requirements that must be checked to ensure quality. A common means for this is quality assurance through visual inspections, which are carried out manually. Visual inspections are time-consuming yet prone to wrong decisions, since they are subjective, inconsistent, and susceptible to uncertainties. The quality of the inspection relies heavily on the experience of the personnel. This work addresses this issue through the concept and design of a system for objective decision making in visual inspection by integrating Deep Learning (DL) models with a Belief Rule Based Expert System (BRBES) inside a smart devices application. Smart devices like tablets and smartphones serve to generate information by recording and evaluating image material of the components being inspected. Based on this data, DL models are trained and used to classify defects on new image material to automate part of the inspection process. Furthermore, smart devices serve to provide context-dependent decision recommendations in the visual inspection process, which were calculated by the BRBES with the inclusion of uncertainties. The knowledge base of the BRBES is fed by the expert knowledge of experienced visual inspectors using knowledge elicitation techniques. In this way, the system can enable optimized and objectified visual inspection based on the data-driven and knowledge-driven approaches used. This paper outlines the concept of integrating DL and BRBES into a smart devices decision support system.