Gear defect detection is a crucial component in power automation systems. Methods based on deep learning have exhibited excellent performance in detecting gears. However, the effect of defect detection for tilted gears is not as good. This is due to the traditional horizontal bounding box annotation method, which inevitably produces many overlapping areas in the annotated bounding box when used in tilted gear targets, and a large part of the areas do not belong to defects. In order to address the issue of low precision in the detection of defects in tilted gears based on deep learning, a dataset preprocessing scheme has been proposed. Initially, the images and annotation files of the training set and validation set are automatically rotated. Subsequently, the rotated data are utilized to train the defect detection model. Finally, the test set is subjected to the same image rotation method as the first step, resulting in the generation of high‐precision defect detection results, which are then input into the defect detection model. In order to facilitate comparison with the ground truth, the JSON file obtained from the detection result is rotated in reverse and the result is mapped to the original test set before rotation. In order to verify the effectiveness of the dataset optimization method proposed in this article, the same configuration file is used to train and evaluate the gear defect detection model with the dataset before and after optimization, respectively. The three detection models with the highest comprehensive indicators were employed to assess the dataset before and after optimization, with the average detection precision mAP serving as a quantitative comparison indicator. The findings demonstrate that the data optimization method proposed in this article has markedly enhanced mAP in tilted gear defect detection. Upon testing the proposed method on the PP‐YOLOE + model, mAP (0.5) exhibited an increase of 18.5%, while mAP (0.5 : 0.95) demonstrated a 29.1% improvement. When the method was tested on the RT‐DETR model, mAP (0.5) exhibited an increase of 24.7%, while mAP (0.5 : 0.95) demonstrated an 18.4% improvement. When tested on the few‐short model, mAP (0.5) increased by 23.3% and mAP (0.5 : 0.95) increased by 19.8%. This validates the effectiveness of the dataset processing method proposed in this article in the detection of tilted gear defects.