Atrial fibrillation (AF) is a most common arrhythmia with high morbidity and mortality. However, the conventional detection of AF is time-consuming and laborious because it is mainly completed by physician's visual inspection of electrocardiogram (ECG). Thus, it is essential to build the intelligent computer-aided diagnosis system strategy for AF detection. In this work, we present a novel intelligent approach based on the multi-scale convolution kernel (MCK) and Squeeze-and-Excitation network (SENet) for AF detection. The model not only is able to overcome the limitations that exist in the single-scale convolution kernel of traditional convolution neural network (CNN), but also explicitly establish the inter-dependences between the extracted feature channels and screen out the critical ECG features for AF signals recognition, thus improving the model performance. The results demonstrate that the proposed model achieves noticeable performance improvements with the accuracy of 98.3% and 97.5% using a subject-independent validation scheme on the two public databases. Besides, the corresponding ablation experiments show the effectiveness of the proposed MCK strategy. To our knowledge, this is the first time to redesign the convolution kernel in traditional CNN for AF detection, while showing its great potential as an auxiliary tool to help physicians.