Balanced Sample Assignment and Objective for Single-Model Multi-Class 3D Object Detection

Abstract

Accurately detecting multi-class objects in a single pass is critical but challenging for real-world autonomous driving scenarios. Several single-class anchor-based methods have recently achieved the state-of-the-art performance in the car category, but when extending to multi-class detection tasks, their performance on small objects ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., pedestrians and cyclists) is limited. We find that the core problem that causes this phenomenon lies in the unbalanced sample quality and the classification objective. To address this problem, we proposed a single-model multi-class 3D object detector with balanced sample assignment and objective, named BSAODet. Specifically, the quality-balanced sample assignment (QBSA) is introduced to dynamically collect stable high-quality samples for each class according to the predicted sample performance and geometric constraints. In conjunction with the QBSA, the class-balanced classification objective (CBCO) performs instance-wise label normalization and weighting on positive samples, preventing the model from biasing toward objects with more samples. Extensive experiments on the popular KITTI dataset, the latest large-scale ONCE dataset, and the challenging Waymo Open Dataset show that our method steadily improves the performance of current state-of-the-art detectors by 2–7 mAP in pedestrians and cyclists while maintaining competitiveness in cars. Moreover, our best model achieves 66.31 mAP on three classes, outperforming all published LiDAR-only detectors on the KITTI benchmark.