Overhead transmission line insulators are non-conductive materials that separate conductors from grounded transmission towers. Once in operation, they frequently experience environmental pollution and electrical or mechanical stress. Since adverse operational conditions can lead to insulation failure, regular inspections are essential to prevent power outages. To this end, this paper proposes a novel technique based on deep convolutional neural networks (CNNs) to classify high-voltage insulator surface conditions based on their image. Successful applications of CNNs in computer vision have led to several pretrained architectures for image classification. To use these pretrained models, a practitioner typically fine-tunes and selects one final model via a model selection stage and discards all other models. In contrast with many existing studies that use such a “winner-takes-all” approach, here, we identify the best subset of seven popular pretrained CNN architectures that are combined by soft voting to form an ensemble classifier. From a machine learning (ML) perspective, this focus is warranted because the convolutional base of each pretrained architecture operates as a feature extractor and an ensemble of them works as a combination of various feature extraction rules. Our numerical experiments demonstrate the advantage of the identified ensemble model to individual pretrained architectures.