A Hardware-adaptive Deep Feature Matching Pipeline for Real-time 3D Reconstruction

Abstract

This paper presents a hardware-adaptive feature modeling framework to automatically generate and optimize deep neural networks to support real-time feature extraction and matching on a given hardware platform. This framework consists of a deep feature extraction and matching pipeline and a neural architecture search scheme, with which deep neural networks can be automatically generated and optimized according to given hardware to achieve reliable real-time feature matching. Built on our feature matching approach, we also developed a real-time 3D scene reconstruction pipeline that could run adaptively on hardware with different computational performance. We designed experiments to validate the proposed matching and reconstruction pipelines on hardware platforms with different performance. The results demonstrated our algorithm’s effectiveness on both matching and reconstruction tasks.