Deep multimodal learning for manufacturing problem solving

Abstract

Case-based reasoning systems prove to be an effective and efficient support for manufacturing problem solving in several industrial domains. However, focusing on case retrieval, their predictive power is limited to date. To account for the semi-structured data taxonomy in in this domain consisting of structured categorical features and text-based problem descriptions, a multimodal approach is necessary for the design of a predictive model. In the presented approach, we employ a multimodal neural network to learn combined representations of structured categorical features along with historic problem descriptions. This model predicts respective problem impacts regarding time, quality and cost. The incorporation of an LSTM-based input branch for text features into a feed-forward neural network architecture processing categorical features boosts prediction performance significantly. It outperforms comparable state of the art deep learning models and simple classification models. Deployed into production, our approach mitigates incorrect problem assessment and failure propagation into subsequent process phases.