Abstract

Deep neural network pruning is effective in enabling high-performance perception models to be deployed on autonomous driving platforms with limited computation and memory resources. With their rapid development, state-of-the-art autonomous driving perception (ADP) models have advocated the use of multimodal sensors to extract diverse feature categories. However, existing pruning studies in the ADP area focus on single-modal models and neglect multimodal models. Compared with conventional pruning, multimodal pruning presents a new type of redundancy, namely modal-wise redundancy that is caused by multimodal branches extracting similar perception information. When a specific type of information is extracted by more than one modal branch, although this extraction is deemed essential from an individual single-modal standpoint, it becomes redundant when viewed from a modal-wise perspective. Therefore, modal branches must be handled cooperatively to eliminate modal-wise redundancy while concurrently preserving the original perception accuracy. Building on this, we propose CrossPrune, a modal cooperative pruning framework designed for camera–LiDAR fused perception in autonomous driving. The primary objective of CrossPrune is to effectively eliminate redundancy and achieve nondestructive pruning for multimodal ADP models. This was accomplished by approaching the problem as a multi-objective optimization task, encompassing both weight pruning and the restriction of feature distortions caused by pruning. Experiments conducted on the nuScenes and KITTI datasets demonstrated that CrossPrune attained superior pruning ratios while minimizing accuracy loss, surpassing the performance of the baselines. The key results indicated that the proposed CrossPrune achieved relative improvements of 9.6% in mAP and 11.5% in NDS under 89.8% pruning sparsity.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.