In the field of image recognition, machine learning technologies, especially deep learning, have been rapidly advancing alongside the advances of hardware such as GPUs. In image recognition, in general, large numbers of labeled images to be identified are input to a neural network, and repeatedly learning the images enables the neural network to identify objects with high accuracy. A new profiling side-channel attack method, the deep learning side-channel attack (DL-SCA), utilizes the neural network’s high identifying ability to unveil a cryptographic module’s secret key from side-channel information. In DL-SCAs, the neural network is trained with power waveforms captured from a target cryptographic module, and the trained network extracts the leaky part that depends on the secret. However, at this stage, the main target of investigation has been software implementation, and studies regarding hardware implementation, such as ASIC, are somewhat lacking. In this paper, we first depict deep learning techniques, profiling side-channel attacks, and leak models to clarify the relation between secret and side channels. Next, we investigate the use of DL-SCA against hardware implementations of AES and discuss the problem derived from the Hamming distance model and ShiftRow operation of AES. To solve the problem, we propose a new network training method called “mixed model dataset based on round-round XORed value.” We prove that our proposal solves the problem and gives the attack capability to neural networks. We also compare the attack performance and characteristics of DL-SCA to conventional analysis methods such as correlation power analysis and conventional template attack. In our experiment, a dedicated ASIC chip for side-channel analysis is utilized and the chip is also equipped with a side-channel countermeasure AES. We show how DL-SCA can recover secret keys against the side-channel countermeasure circuit. Our results demonstrate that DL-SCA can be a more powerful option against side-channel countermeasure implementations than conventional SCAs.