Abstract Automatic defect detection is of great significance in steel manufacturing industry. In this paper, we present a new detection algorithm for defects of scale-covered steel billets. Features of surface images such as brightness and textures are non-uniform because the billet surface is partially covered with scale. In order to discriminate between defects and pseudo defects due to scales and to improve the accuracy of defect detection, a new detection method based on morphological top-hat filter is proposed. Experimental results conducted on billets surface image from actual steel production line show that the proposed algorithm is capable of detecting defects on billet surface.