Classification of ECG using ensemble of residual CNNs with or without attention mechanism

Abstract

Objective. This paper introduces a winning solution (team ISIBrno-AIMT) to the official round of PhysioNet Challenge 2021. The main goal of the challenge was a classification of ECG recordings into 26 multi-label pathological classes with a variable number of leads (e.g. 12, 6, 4, 3, 2). The main objective of this study is to verify whether the multi-head-attention mechanism influences the model performance. Approach. We introduced an ECG classification method based on the ResNet architecture with a multi-head attention mechanism for the official round of the challenge. However, empirical findings collected during model development suggested that the multi-head attention layer might not significantly impact the final classification performance. For this reason, during the follow-up round, we removed a multi-head attention layer to test the influence on model performance. Like the official round, the model is optimized using a mixture of loss functions, i.e. binary cross-entropy, custom challenge score loss function, and custom sparsity loss function. Probability thresholds for each classification class are estimated using the evolutionary optimization method. The final architecture consists of three submodels forming a majority voting classification ensemble. Main results. The modified model without the multi-head attention layer increased the overall challenge score to 0.59 compared to the 0.58 from the official round. Significance. Our findings from the follow-up submission support the fact that the multi-head attention layer in the proposed architecture does not significantly affect the classification performance.