A novel deep ordinal classification approach for aesthetic quality control classification

César Hervás-Martínez,Pedro Antonio Gutiérrez,Luca Romeo,Víctor Manuel Vargas,Riccardo Rosati,Emanuele Frontoni

doi:10.1007/s00521-022-07050-6

Abstract

Nowadays, decision support systems (DSSs) are widely used in several application domains, from industrial to healthcare and medicine fields. Concerning the industrial scenario, we propose a DSS oriented to the aesthetic quality control (AQC) task, which has quickly established itself as one of the most crucial challenges of Industry 4.0. Taking into account the increasing amount of data in this domain, the application of machine learning (ML) and deep learning (DL) techniques offers great opportunities to automatize the overall AQC process. State-of-the-art is mainly oriented to approach this problem with a nominal DL classification method which does not exploit the ordinal structure of the AQC task, thus not penalizing the error among distant AQC classes (which is a relevant aspect for the real use case). The paper introduces a DL ordinal methodology for the AQC classification. Differently from other deep ordinal methods, we combined the standard categorical cross-entropy with the cumulative link model and we imposed the ordinal constraint via the thresholds and slope parameters. Experimental results were performed for solving an AQC task on a novel image dataset originated from a specific company’s demand (i.e., aesthetic assessment of wooden stocks). We demonstrated how the proposed methodology is able to reduce misclassification errors (up to 0.937 quadratic weight kappa loss) among distant classes while overcoming other state-of-the-art deep ordinal models and reducing the bias factor related to the item geometry. The proposed DL approach was integrated as the main core of a DSS supported by Internet of Things (IoT) architecture that can support the human operator by reducing up to 90% the time needed for the qualitative analysis carried out manually in this specific domain.

Highlights

Learning to classify patterns from labeled examples and predicting discrete classes in a new unseen set is the main goal of supervised machine learning (ML)
State-of-the-art is mainly oriented to approach this problem with a nominal deep learning (DL) classification method which does not exploit the ordinal structure of the aesthetic quality control (AQC) task, not penalizing the error among distant AQC classes
It is worth noting that for formulation A the cumulative link models (CLMs) parameters were learned in the training set, while for formulations B and C the slope was tuned in a separate validation set and kept fixed during the training stage

Summary

Introduction

Learning to classify patterns from labeled examples and predicting discrete classes in a new unseen set is the main goal of supervised machine learning (ML). The application of ML and DL techniques offers great opportunities to automatize the overall QC process [11] These methodologies for QC have been employed in several industrial areas, but state-of-the-art is mainly oriented to present ad hoc rather than vanilla ML solutions capable of dealing with challenges of this domain, namely the intrinsic variability of the annotation procedure and the Neural Computing and Applications (2022) 34:11625–11639 difficulty to generalize across different sets. The aesthetic quality control (AQC) task is a non-metric QC task where the aesthetic aspect of the material is not measurable and is based on expert observation In this domain, the classes of the target variable often exhibit a natural ordering. This outcome is in line with the industrial demands in order to provide a DL-based DSS for AQC that is as aligned as possible with the human operator (human agency and oversight [12])

Objectives

Methods

Results

Conclusion