LiDAR-based odometry and mapping is used in many robotic applications to retrieve the robot's position in an unknown environment and allows for autonomous operation in GPS-denied (e.g., indoor) environments. With a 3D LiDAR sensor, highly accurate localization becomes possible, which enables high quality 3D reconstruction of the environment. In this letter we extend the well-known LOAM framework by leveraging prior knowledge about a reference object in the environment to further improve the localization accuracy. This requires a known 3D model of the reference object and its known position in a global coordinate frame. Instead of only relying on the point features in the mapping module of LOAM, we also include mesh features extracted from the 3D triangular mesh of the reference object in the optimization problem. For fast correspondence computation of mesh features, we use the Axis-Aligned-Bounding-Box-Tree (AABB) structure. Essentially, our approach not only makes use of the previously built map for absolute localization in the environment, but also takes the relative position to the reference object into account, effectively reducing long-term drift. To validate the proposed concept, we generated datasets using the Gazebo simulation environment in exemplary visual inspection scenarios of an airplane inside a hangar and the Eiffel Tower. An actuated 3D LiDAR sensor is mounted via a 1-DoF gimbal on a UAV capturing 360° scans. We benchmark our approach against the state-of-the-art open-source LOAM framework. The results show that the proposed joint optimization using both point and mesh features yields a significant reduction in Absolute Pose Error (APE) and therefore improves the map and 3D reconstruction quality during long-term operations.