ABSTRACT Due to the diversity of debris morphologies and their states in the images, automatic wear debris analysis is still a challenge. Instance segmentation is an advanced technique of deep learning used for computer vision tasks. It can detect and delineate each distinct object of interest appearing in an image. An instance segmentation model using a modified mask branch with larger feature map and dilated convolution is proposed and applied to the ferrography images, so that five typical types of wear debris corresponding to different wear conditions can be located, segmented and recognized, even if they are small, blurred or overlapped. The mean average accuracy on the test set is about 88% when the intersection over union threshold is 0.5. This method realizes an end-to-end process of wear debris identification, which can eliminate the error caused by the intermediate steps. And it obtains accurate quantitative debris information, i.e., the type, quantity and size of debris, which provides a solution for intelligent wear debris analysis and automatic wear detection.