A fast scene geometric modeling approach for digital twins combining neural rendering and model retrieval

Abstract

ABSTRACT Digital twin technology is a crucial driving technology for the realization of Industry 4.0, which enables simulation, analysis, and prediction by constructing geometric scenes that correspond to physical scenes. Current automation in scene modeling is limited, relying heavily on costly and inefficient manual operations. This hinders digital twin technology progress. This paper proposes a framework for a fast scene geometric modeling method that combines neural rendering and model retrieval for digital twins. Neural rendering techniques first train the collected image data, and the point cloud data of physical entities are rendered. Then based on the semantic mapping algorithm between the point cloud data of physical entities and the multi-view image data of 3D CAD models proposed in this study, the corresponding geometric models are retrieved from the geometric asset library by inputting the point cloud data, and all the retrieved geometric models are embedded in the geometric scene to complete the scene geometric modeling. Finally, in a case study of the digital twin-based scene construction for decommissioned lithium battery dismantling, the effectiveness of this method is demonstrated, which can improve the speed and reduce the cost of scene geometric modeling.