Analysis of the trade-offs between parametric and non-parametric classifiers for classification of faults in pneumatic actuator. Case: DAMADICS

Abstract

This paper disseminates an extra dimension of substantial analysis demonstrating the trade-offs between the performance of Parametric (P) and Non-Parametric (NP) classification algorithms when applied to classify faults occurring in pneumatic actuators. Owing to the criticality of the actuator failures, classifying faults accurately may lead to robust fault tolerant models. In most cases, when applying machine learning, the choice of existing classifier algorithms for an application is random. This work, addresses the issue and quantitatively supports the selection of appropriate algorithm for non-parametric datasets. For the case study, popular parametric classification algorithms namely: Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Perceptron (PER) and non-parametric algorithms namely: Multi-Layer Perceptron (MLP), k Nearest Neighbor (kNN), Support Vector Machine (SVM), Decision Tree (DT) and Random Forest (RF) are implemented over a non-parametric, imbalanced synthetic dataset of a benchmark actuator process. Upon using parametric classifiers, severe adultery in results is witnessed which misleads the interpretation towards the accuracy of the model. Experimentally, about 20% improvement in accuracy is obtained on using non-parametric classifiers over the parametric ones. The robustness of the models is evaluated by inducing label noise varying between 5% to 20%. Triptych analysis is applied to discuss the interpretability of each machine learning model. The trade-offs in choice and performance of algorithms and the evaluating metrics for each estimator are analyzed both quantitatively and qualitatively. For a more cogent reasoning through validation, the results obtained for the synthetic dataset are compared against the industrial dataset of the pneumatic actuator of the sugar refinery, Development and Application of Methods for Actuator Diagnosis in Industrial Control Systems (DAMADICS). The efficiency of non-parametric classifiers for the pneumatic actuator dataset is well proved.