Comparison of statistical tests in effective connectivity analysis of ECoG data

Abstract

BackgroundThe effects of statistical testing on the results of multivariate autoregressive (MVAR)-based effective connectivity analysis have not been adequately investigated, and it is still unclear which statistical test can provide the most accurate results. New methodsUsing simulated and real electrocorticographic (ECoG) data, we investigated the performance of three nonparametric statistical tests – Monte Carlo permutation, bootstrap resampling, and surrogate data method in MVAR-based effective connectivity analysis. Receiver operating characteristic (ROC) analysis and area under the ROC curve (AUC) were used to assess the performance of each statistical test method. In addition, we found optimal p-values for each method based on ROC analysis. Finally, we investigated the application of statistical tests on partial directed coherence analysis of ECoG data collected in a patient with epilepsy. ResultsThe bootstrap statistical test performed more accurately than other methods. The surrogate method slightly outperformed the Monte Carlo permutation method. Optimal p-values of statistical tests depended on signal-to-noise ratio (SNR) of data, and its value increased by reducing SNR of data. By considering the typical SNR range of electrophysiological data, we recommended an optimal p-value range for the application of each statistical test method. Comparison with existing methodsLimited studies have investigated the performance of statistical tests for MVAR-based effective connectivity analysis. For the first time, we have investigated the effects of baseline connections on the various performances of statistical tests. ConclusionsWe recommend utilizing the bootstrap statistical test with p-value between 0.05 and 0.1 for effective connectivity analysis of ECoG data.