Nuclide identification technology plays a critical role in various fields such as nuclear energy, medicine, environmental monitoring, and defense. However, traditional nuclide identification algorithms face challenges, such as high computational complexity and long response time, when handling large-scale data. The adaptability and universality of these algorithms for different domains and tasks are also confronted with certain challenges. In recent years, numerous nuclide identification methods based on convolutional neural networks have been proposed, to some extent, addressing the issues present in traditional nuclide identification algorithms. Yet, in the context of unmanned nuclear emergencies, nuclear counterterrorism, and post-nuclear contamination processing, there is a widespread need for rapid nuclide identification methods suitable for deployment on edge computing devices. Therefore, this paper combines the advantages of self-attention mechanisms and convolutional neural networks to design a novel network architecture which can meet the aforementioned application requirements. We simulated and measured energy spectrum data for radioactive sources, including 241Am, 57Co, 60Co, 99mTc, 131I, 133Ba, 137Cs, 226Ra, 232Th, and 40K, as the training dataset. In experiments with a training set containing only individual radioactive source energy spectra, the model achieves 99.49% accuracy, with an average inference time of 0.14 ms per sample. It has 0.27 million parameters and 1.55 billion FLOPs. When the training set includes both individual and mixed radioactive source energy spectra, the model achieves 99.84% accuracy, with an average inference time of 0.25 ms per sample. It has 0.83 million parameters and 1.58 billion FLOPs.