Denoising and Stability using Independent Component Analysis in High Dimensions – Visual Inspection Still Required

Abstract

Independent Component Analysis (ICA) has emerged as a useful method for separation of components, such as in removing noise from data. We examine one of the challenges of ICA - instability, particularly in high dimensions, when the independent components vary, each time when ICA is performed. This may be due to various causes including the stochastic nature of the algorithm and the additive noise. The objective of this study is to examine denoising and stability issues of ICA in high dimensions and make a comparative evaluation of select approaches. We take a challenging electrocardiogram dataset which is a high-dimensional time series of multiple sensors. We experiment with a mix of approaches and methods – for resampling, clustering, ICA algorithms and dimensionality. We check the internal validity using the Icasso stability index, the Amari separation performance index and the Minimum Distance (MD) index. The first key contribution of this work is that it finds counter-evidence to the claim that resampling (bootstrapping) tackles the question of stability. The second contribution is that it finds evidence of an important limitation of the Minimum Distance index when dealing with high dimensional data – the index may become highly concentrated and may remain sub-optimal at all dimensions. Selectively removing noise components by visual inspection can improve the Amari, the Icasso or the MD index values. Some automated tools exist, but in high dimensions, visual inspection of the individual components is still required for effective denoising – data driven methods are not good enough.