By leveraging neural networks, deep distance metric learning has yielded impressive results in computer vision applications. However, the existing approaches mostly focus a single deep distance metric based on pairs or triplets of samples. It is difficult for them to handle heterogeneous data and avoid overfitting. This study proposes a boosting-based learning method of multiple deep distance metrics, which generates the final distance metric through iterative training of multiple weak distance metrics. Firstly, the distance of sample pairs was mapped by a convolution neural network (CNN) and evaluated by a piecewise linear function. Secondly, the evaluation function was added as a weak learner to the boosting algorithm to generate a strong learner. Each weak learner targets the difficult samples different from the samples of previous learners. Next, an alternating optimization method was employed to train the network and loss function. Finally, the effectiveness of our method was demonstrated in contrast to state of the arts on retrieving the images from the CUB-200-2011, Cars-196, and Stanford Online Products (SOP) datasets.