Fault Detection on Big Data: A Novel Algorithm for Clustering Big Data to Detect and Diagnose Faults

Abstract

With computer technology improving exponentially, data will grow incomprehensibly in size, complexity, and noise. However, latent within the data, valuable signals are hidden that, if discovered, can offer abundant information, such as fault detection. Traditionally, principal component analysis has been used to perform fault detection in large, multivariate systems. However, these methods often struggle to find the true origin, as they are susceptible to contribution smearing. In this work, a chemical plant system was analyzed and a novel cluster and detect method for fault detection utilizing machine-learning clustering algorithms was created in aim to improve fault detection time and diagnosis. Plant data containing complex variables were simulated, clustered into groups through a unique algorithm based upon correlations, and analyzed through principal component analysis as individual groups. This approach often resulted in quicker identification and more accurate diagnosis than the traditional principal component analysis method.