Moisture damage is a prevalent problem for hot-mix asphalt (HMA) pavements all over the world and the use of moisture-resistant pavement materials is critical for ensuring durable pavements. There is thus a need for the development of an accurate method of identification of moisture-susceptible mixes during laboratory mix design. The objective of this study was to develop a system of identification of poor- and good-performance HMA mixes based on artificial intelligence. The work involved stiffness and strength testing and imaging of pre- and post-conditioned mix samples that were compacted from plant-produced loose mixes with known field performance. A deep convolutional neural network (CNN) was applied to classify the moisture damage potential of the mixes based on images. As the number of samples was small, transfer learning using a standard CNN architecture (Inception V3) was used, which was pre-trained on a large-scale object identification task. The predictions from the resulting model were 88% accurate, which is higher than the accuracy of statistical analyses of the results of mechanical tests and black pixel analysis. Implementation of the proposed method in laboratory mix design can help engineers in screening poor mixes quickly and with high accuracy.