The atomic decomposition (AD) algorithm for Power Quality Disturbance (PQD) signals can obtain sparser and physically clearer results than the conventional fixed basis decomposition method. However, the conventional atomic decomposition (CAD) algorithm for PQD signals suffers from excessive computational resource consumption and low accuracy of sub-dictionaries selection. In this paper, the CAD algorithm for PQD signals is optimized by introducing a Convolutional Neural Networks based (CNN) sub-dictionary predictor inspired by the PQD signal classification technique. In the optimized algorithm, by using a sub-dictionary predictor to select sub-dictionaries directly, the range of each atomic search is significantly reduced. In addition, the undesirable effects of the intelligent optimization algorithm are mitigated, which match the non-global optimal results in search of sub-dictionaries. Besides, the goal of reducing computation resources is achieved, and the accuracy of sub-dictionaries selection is improved. Finally, the CAD algorithm and optimized CNN-based atomic decomposition (CNN-AD) algorithm are compared and analyzed on the synthetic dataset and the measured dataset from IEEE 1159.2 Working Group on power quality disturbances. Consequently, it is verified that the CNN-AD can reduce the requirement of computation resources and improve the accuracy of sub-dictionaries selection in the decomposition for PQD signals.