Energy-Constrained Deep Neural Network Compression for Depth Estimation

Abstract

Many applications, such as autonomous driving, robotics, etc., require accurately estimating depth in real time. Currently, deep learning is the most popular approach to stereo depth estimation. Some of these models have to operate in highly energy-constrained environments, while they are usually computationally intensive, containing massive parameter sets ranging from thousands to millions. This makes them hard to perform on low-power devices with limited storage in practice. To overcome this shortcoming, we model the training process of a deep neural network (DNN) for depth estimation under a given energy constraint as a constrained optimization problem and solve it through a proposed projected adaptive cubic quasi-Newton method (termed ProjACQN). Moreover, the trained model is also deployed on a GPU and an embedded device to evaluate its performance. Experiments show that the stage four results of ProjACQN on the KITTI-2012 and KITTI-2015 datasets under a 70% energy budget achieve (1) 0.13% and 0.61%, respectively, lower three-pixel error than the state-of-the-art ProjAdam when put on a single RTX 3090Ti; (2) 4.82% and 7.58%, respectively, lower three-pixel error than the pruning method Lottery-Ticket; (3) 5.80% and 0.12%, respectively, lower three-pixel error than ProjAdam on the embedded device Nvidia Jetson AGX Xavier. These results show that our method can reduce the energy consumption of depth estimation DNNs while maintaining their accuracy.