Using Principal Component Analysis to Find Correlations and Patterns at Diamond Light Source

Abstract

Principal component analysis is a powerful data analysis tool, capableofreducinglargecomplexdatasetscontaining many variables. Examination of the principal components set allows the user to spot underlying trends and patterns that might otherwise be masked in a very large volume of data, or hidden in noise. Diamond Light Source archives many gigabytes of machine data every day, far more than any one human could effectively search through for correlations. Presented in this paper are some of the results from running principal component analysis on years of archived data in order to find underlying correlations that may otherwise have gone unnoticed. The advantages and limitations of the technique are discussed.