Damage estimation is part of daily operation of power utilities, often requiring a manual process of crew deployment and damage report to quantify and locate damages. Advancement in unmanned aerial vehicles (UAVs) as well as real-time communication and learning technologies could be harnessed towards efficient and accurate automation of this process. This paper develops a model to automate the process of estimating and localizing damages in power distribution poles, which utilizes the images taken by UAVs transferred in real-time to an intelligent damage classification and estimation (IDCE) unit. The IDCE unit integrates four convolutional neural networks to learn the states of poles from images, extract the image characteristics, and train an automated intelligent tool to replace manual fault location and damage estimation. The proposed model first determines the type of pole damages, including falling and burning, and then estimates the percentage of damage in each type. The IDCE unit also localizes damages in the poles by locating possible burning or arcing parts. A data set of 1615 images is utilized to train, validate and test the proposed model, which demonstrates high accuracy of the model in classifying and estimating damages in distribution poles.