3D Semantic Model Research Articles

RetSeg3D: Retention-based 3D semantic segmentation for autonomous driving

LiDAR semantic segmentation is one of the crucial tasks for scene understanding in autonomous driving. Recent trends suggest that voxel- or fusion-based methods obtain improved performance. However, the fusion-based methods are computationally expensive. On the other hand, the voxel-based methods uniformly employ local operators (e.g., 3D SparseConv) without considering the varying-density property of LiDAR point clouds, which result in inferior performance, specifically on far away sparse points due to limited receptive field. To tackle this issue, we propose novel retention block to capture long-range dependencies, maintain the receptive field of far away sparse points and design RetSeg3D, a retention-based 3D semantic segmentation model for autonomous driving. Instead of vanilla attention mechanism to model long-range dependencies, inspired by RetNet, we design cubic window multi-scale retentive self-attention (CW-MSRetSA) module with bidirectional and 3D explicit decay mechanism to introduce 3D spatial distance related prior information into the model to improve not only the receptive field but also the model capacity. Our novel retention block maintains the receptive field which significantly improve the performance of far away sparse points. We conduct extensive experiments and analysis on three large-scale datasets: SemanticKITTI, nuScenes and Waymo. Our method not only outperforms existing methods on far away sparse points but also on close and medium distance points and efficiently runs in real time at 52.1 FPS on a RTX 4090 GPU.

A Specialized Pipeline for Efficient and Reliable 3D Semantic Model Reconstruction of Buildings from Indoor Point Clouds.

Recent advances in laser scanning systems have enabled the acquisition of 3D point cloud representations of scenes, revolutionizing the fields of Architecture, Engineering, and Construction (AEC). This paper presents a novel pipeline for the automatic generation of 3D semantic models of multi-level buildings from indoor point clouds. The architectural components are extracted hierarchically. After segmenting the point clouds into potential building floors, a wall detection process is performed on each floor segment. Then, room, ground, and ceiling extraction are conducted using the walls 2D constellation obtained from the projection of the walls onto the ground plan. The identification of the openings in the walls is performed using a deep learning-based classifier that separates doors and windows from non-consistent holes. Based on the geometric and semantic information from previously detected elements, the final model is generated in IFC format. The effectiveness and reliability of the proposed pipeline are demonstrated through extensive experiments and visual inspections. The results reveal high precision and recall values in the extraction of architectural elements, ensuring the fidelity of the generated models. In addition, the pipeline's efficiency and accuracy offer valuable contributions to future advancements in point cloud processing.

The Concept of Levels of Detail for 3D Niche Models in CityGML

Abstract. Buddhist niches in grottoes can be represented in three-dimensional (3D) for their detailed geometries on surfaces by using triangular meshes generated from point clouds. However, not all applications require 3D models with high geometric detail. The mesh models of niches have drawbacks such as large data volumes, lack of semantic information, and absence of spatial relationships between structural components and members within niches. Those limitations make mesh models suitable only for visualization and challenging to use directly in tasks like spatial analysis, simulation experiments, mechanical analysis, and disease investigation. To address this problem, this study defines four Levels of Detail (LoDs) for Buddhist niches, ranging from LoD0 to LoD3, drawing on the concept of LoDs for urban buildings in CityGML 3.0. As the LoD level increases, 3D models contain more detailed geometries, including spatial points, bounding boxes, niche structural components, and component members. Those 3D models at different LoDs can represent niches with varying degrees of abstraction, making them suitable for different applications and guiding the production of standardized 3D semantic models. To validate the feasibility of the LoDs definition for Buddha niches, this paper reconstructs 3D models of niches at different LoDs based on high-precision mesh models. Finally, a comparison is made between the original mesh model and models at different LoDs in terms of data size and potential application scenarios.

Leveraging Visual Language Model and Generative Diffusion Model for Zero-Shot SAR Target Recognition

Simulated data play an important role in SAR target recognition, particularly under zero-shot learning (ZSL) conditions caused by the lack of training samples. The traditional SAR simulation method is based on manually constructing target 3D models for electromagnetic simulation, which is costly and limited by the target’s prior knowledge base. Also, the unavoidable discrepancy between simulated SAR and measured SAR makes the traditional simulation method more limited for target recognition. This paper proposes an innovative SAR simulation method based on a visual language model and generative diffusion model by extracting target semantic information from optical remote sensing images and transforming it into a 3D model for SAR simulation to address the challenge of SAR target recognition under ZSL conditions. Additionally, to reduce the domain shift between the simulated domain and the measured domain, we propose a domain adaptation method based on dynamic weight domain loss and classification loss. The effectiveness of semantic information-based 3D models has been validated on the MSTAR dataset and the feasibility of the proposed framework has been validated on the self-built civilian vehicle dataset. The experimental results demonstrate that the first proposed SAR simulation method based on a visual language model and generative diffusion model can effectively improve target recognition performance under ZSL conditions.

Attention-guided LiDAR segmentation and odometry using image-to-point cloud saliency transfer

LiDAR odometry estimation and 3D semantic segmentation are crucial for autonomous driving, which has achieved remarkable advances recently. However, these tasks are challenging due to the imbalance of points in different semantic categories for 3D semantic segmentation and the influence of dynamic objects for LiDAR odometry estimation, which increases the importance of using representative/salient landmarks as reference points for robust feature learning. To address these challenges, we propose a saliency-guided approach that leverages attention information to improve the performance of LiDAR odometry estimation and semantic segmentation models. Unlike in the image domain, only a few studies have addressed point cloud saliency information due to the lack of annotated training data. To alleviate this, we first present a universal framework to transfer saliency distribution knowledge from color images to point clouds, and use this to construct a pseudo-saliency dataset (i.e. FordSaliency) for point clouds. Then, we adopt point cloud based backbones to learn saliency distribution from pseudo-saliency labels, which is followed by our proposed SalLiDAR module. SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance. Finally, we introduce SalLONet, a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation. Our extensive experiments on benchmark datasets demonstrate that the proposed SalLiDAR and SalLONet models achieve state-of-the-art performance against existing methods, highlighting the effectiveness of image-to-LiDAR saliency knowledge transfer. Source code will be available at https://github.com/nevrez/SalLONet

CityGML Grotto ADE for modelling niches in 3D with semantic information

The regions of East Asia, as well as South Asia and the Middle East are rich in cultural heritage of grottoes where a large number of Buddhist niches exist. Three-dimensional (3D) semantic modelling enriches 3D geometric models with an understanding of the historical and cultural value of cultural heritage, facilitating interoperability and analysis beyond mere visualization. 3D models with semantic information act as essential digital infrastructure for heritage management, knowledge dissemination and simulation analysis in cultural heritage. However, due to the lack of standardized data model for the grotto domain, it is difficult to exchange information, share knowledge and the advancement of spatial analysis and simulation. In order to fill the above-mentioned gap, this study develops a data model for niches as a CityGML Application Domain Extension (ADE) based on the CityGML 3.0 standard. In this ADE, niche components and their attributes are defined on two semantic levels, i.e., niche structural component and niche component member. A famous niche in China belonging to the World Heritage was selected as a case study to demonstrate the integration of geometries, semantics and attributes, illustrating that the extended ADE module complies with the CityGML 3.0 standard. This study provides novel insights into the 3D semantic modelling of niches as well as expands the applications scope of CityGML standard within the cultural heritage sector.

Knowledge graph representation method for semantic 3D modeling of Chinese grottoes

The integration of 3D geometric models with semantic information significantly improves the applicability and comprehensibility of cultural heritage. The semantic 3D modeling of Chinese grottoes poses challenges for individuals without expertise in cultural heritage due to gaps in domain knowledge and discrepancies in understanding. However, the existing domain ontology and knowledge graph provide an insufficient representation of the knowledge of Chinese grottoes. To overcome these obstacles, we propose a knowledge graph representation method to provide explicit knowledge for participants at different stages of semantic 3D modeling of Chinese grottoes, which includes schema layer construction and data layer construction. On the schema layer, we design a domain ontology named ChgOnto (Chinese Grottoes Ontology) that consists of four high-level concept classes: spatial object, informational object, digital device, and temporal object. Among the classes in the ChgOnto, the components (e.g., cliff wall, cave roof, cliff wall footing), elements (e.g., primary Buddha statue, pedestal, decoration), the properties (e.g., length, width, depth) of caves and niches in Chinese grottoes as well as the spatial relationships between them are all precisely defined. ChgOnto also reuse the classes from the renowned CIDOC CRM ontology in the cultural heritage field and GeoSPARQL in the geospatial domain, facilitating integration between the two subjects. Considering the schema layer as the conceptual data model, the data layer extracts knowledge from unstructured text through natural language processing tools to instantiate the abstract classes and fill the properties of the schema layer. Finally, the knowledge required for semantic 3D modeling of Chinese grottoes is expressed in the data layer by a knowledge graph in a fixed expression form. Dazu Rock Carvings, a World Heritage site in China, is selected as a case study to validate the practicality and effectiveness of the proposed method. The results reveal that our method offers a robust knowledge-sharing platform for the semantic 3D modeling of Chinese grottoes and demonstrates excellent scalability. The method proposed in this paper can also serve as an informative reference for other types of cultural heritage.

Light mixed-supervised segmentation for 3D medical image data.

Accurate 3D semantic segmentation models are essential for many clinical applications. To train a model for 3D segmentation, voxel-level annotation is necessary, which is expensive to obtain due to laborious work and privacy protection. To accurately annotate 3D medical data, such as MRI, a common practice is to annotate the volumetric data in a slice-by-slice contouring way along principalaxes. In order to reduce the annotation effort in slices, weakly supervised learning with a bounding box (Bbox) was proposed to leverage the discriminating information via a tightness prior assumption. Nevertheless, this method requests accurate and tight Bboxes, which will significantly drop the performance when tightness is not held, that is when a relaxed Bbox is applied. Therefore, there is a need to train a stable model based on relaxed Bboxannotation. This paper presents a mixed-supervised training strategy to reduce the annotation effort for 3D segmentation tasks. In the proposed approach, a fully annotated contour is only required for a single slice of the volume. In contrast, the rest of the slices with targets are annotated with relaxed Bboxes. This mixed-supervised method adopts fully supervised learning, relaxed Bbox prior, and contrastive learning during the training, which ensures the network exploits the discriminative information of the training volumes properly. The proposed method was evaluated on two public 3D medical imaging datasets (MRI prostate dataset and Vestibular Schwannoma [VS] dataset). The proposed method obtained a high segmentation Dice score of 85.3% on an MRI prostate dataset and 83.3% on a VS dataset with relaxed Bbox annotation, which are close to a fully supervised model. Moreover, with the same relaxed Bbox annotations, the proposed method outperforms the state-of-the-art methods. More importantly, the model performance is stable when the accuracy of Bbox annotationvaries. The presented study proposes a method based on a mixed-supervised learning method in 3D medical imaging. The benefit will be stable segmentation of the target in 3D images with low accurate annotation requirement, which leads to easier model training on large-scaledatasets.

Fine-Grained 3D Modeling and Semantic Mapping of Coral Reefs Using Photogrammetric Computer Vision and Machine Learning.

Corals play a crucial role as the primary habitat-building organisms within reef ecosystems, forming expansive structures that extend over vast distances, akin to the way tall buildings define a city's skyline. However, coral reefs are vulnerable to damage and destruction due to their inherent fragility and exposure to various threats, including the impacts of climate change. Similar to successful city management, the utilization of advanced underwater videography, photogrammetric computer vision, and machine learning can facilitate precise 3D modeling and the semantic mapping of coral reefs, aiding in their careful management and conservation to ensure their survival. This study focuses on generating detailed 3D mesh models, digital surface models, and orthomosaics of coral habitats by utilizing underwater coral images and control points. Furthermore, an innovative multi-modal deep neural network is designed to perform the pixel-wise semantic segmentation of orthomosaics, enabling the projection of resulting semantic maps onto a 3D space. Notably, this study achieves a significant milestone by accomplishing semantic fine-grained 3D modeling and rugosity evaluation of coral reefs with millimeter-level accuracy, providing a potent means to understand coral reef variations under climate change with high spatial and temporal resolution.

Automatic generation of as-is BEM models of buildings

This paper shows a method with which to create a proprietary as-is 3D semantic thermal model of a building and transform it into standard BEM formats. Geometric and relational information regarding the essential structural elements of the building (i.e., zone, ceiling, floor, wall, door, etc.) is, together with their corresponding as-is temperatures, first extracted from thermal points clouds of the scene and then adapted to the standard geometric BEM requirements. Non-geometric properties concerning definitions and properties of envelopes, windows and doors are also defined as essential information. After all this information has been collected, a BEM file is eventually generated in order to be used in energy simulation software. One of the specific features of this work is the generation of as-is BIM formats of the interior of a building that are later adapted for energy simulation applications in EnergyPlus and TRNSYS software. An ASCII file containing the temperatures of the main structural elements of the building is also created. The method has been tested on real thermal point clouds of the ground floor of a building, yielding promising results.

Improved 3D Semantic Segmentation Model Based on RGB Image and LiDAR Point Cloud Fusion for Automantic Driving

Improved 3D Semantic Segmentation Model Based on RGB Image and LiDAR Point Cloud Fusion for Automantic Driving

Efficient 3D Scene Semantic Segmentation via Active Learning on Rendered 2D Images.

Inspired by Active Learning and 2D-3D semantic fusion, we proposed a novel framework for 3D scene semantic segmentation based on rendered 2D images, which could efficiently achieve semantic segmentation of any large-scale 3D scene with only a few 2D image annotations. In our framework, we first render perspective images at certain positions in the 3D scene. Then we continuously fine-tune a pre-trained network for image semantic segmentation and project all dense predictions to the 3D model for fusion. In each iteration, we evaluate the 3D semantic model and re-render images in several representative areas where the 3D segmentation is not stable and send them to the network for training after annotation. Through this iterative process of rendering-segmentation-fusion, it can effectively generate difficult-to-segment image samples in the scene, while avoiding complex 3D annotations, so as to achieve label-efficient 3D scene segmentation. Experiments on three large-scale indoor and outdoor 3D datasets demonstrate the effectiveness of the proposed method compared with other state-of-the-art.

A methodology for using dynamic visualizations to enhance citizens engagement

Many cities are developing semantic 3D models as digital representation of their environment in order to assess the outcomes of the implementation of new mobility services before their real-life piloting. This innovative process serves as an accelerator key for decision makers. At the same time, cities develop user-centric approaches setting citizen engagement and co-creation in the core of their strategies towards a more integrated and sustainable mobility system. The purpose of this paper is to present digital twins as a tool for vulnerable citizen engagement, improving efficiency of co-creation in mobility planning. So far, they reinforce evidence-based approaches for policy making. The present study presents a methodology of how digital twins are integrated in engagement processes by providing dynamic impacts’ representation of the citizens’ decisions on their neighborhoods.

Retrofitting Potential of Building envelopes Based on Semantic Surface Models Derived From Point Clouds

To meet the climate goals of the Paris agreement, the focus on energy efficiency needs to be shifted to increase the retrofitting rate of the existing building stock. Due to the lack of usable information on the existing building stock, reasoning about the retrofitting potential in early design stages is difficult. Therefore, deconstructing and building new is often regarded as the more reliable and economical option. Digital methods are missing or not robust enough to capture and reconstruct digital models of existing buildings efficiently and automatically derive reliable decision-support about whether demolition and new construction or retrofitting of existing buildings is more suitable. This paper proposes a robust, automated method for calculating existing buildings' life cycle assessments (LCA) using point clouds as input data. The main focus lies in bridging the gap between point clouds and importing semantic 3D models for LCA calculation. Therefore, the automation steps include a geometric transformation from point cloud to 3D surface model, followed by a semantic classification of the surfaces to thermal layers and their materials by assuming the surface elements by building age class.

Dense context distillation network for semantic parsing of oblique UAV images

Semantic segmentation of oblique unmanned aerial vehicle (UAV) images serves as a foundation for many modern urban applications, such as road scene monitoring and semantic 3D modeling. However, objects in UAV images can vary intensely in size and undergo severe perspective distortion because of the oblique viewing style. Existing general segmentation models designed for ground and remote sensing images rarely considered these challenges specific to UAV images. Therefore, they have large difficulties in learning discriminative representation for simultaneously reasoning the extremely large and small objects in UAV images. In this paper, we propose a dense context distillation network (DCDNet) to learn distortion-robust feature representation for semantic segmentation of UAV images. The basic DCDNet is deployed as an dual-branch encoder–decoder architecture. To accomplish the goal of dense context distillation, DCDNet is first equipped with several cross-scale context selectors at different encoding stages to densely and selectively gather the useful context from low- to high-level dual-scale feature maps. A joint supervision is then applied to reinforce the learning of shallower features for distilling more low-level contexts that are vital to the reasoning of small or thin structures. A multi-scale feature aggregator is incorporated to adaptively fuse the long-range context during decoding, which absorbs the complementary merits of the dense context captured from feature maps of different levels. With the dense context distillation, DCDNet is more capable of offering the differently scaled objects with the required context for better learning and prediction. Extensive experiments on the challenging UAVid dataset demonstrate that our DCDNet can well adapt to the oblique UAV images, achieving a state-of-the-art segmentation performance with a mIoU score of 72.38%. • A DCDNet is proposed for semantic parsing of UAV images. • DCDNet offers enough context for reasoning the large and small road scene objects. • DCDNet achieves state-of-the-art segmentation performance.

Multiscale unsupervised domain adaptation for automatic pancreas segmentation in CT volumes using adversarial learning.

Computer-aided automatic pancreas segmentation is essential for early diagnosis and treatment of pancreatic diseases. However, the annotation of pancreas images requires professional doctors and considerable expenditure. Due to imaging differences among various institution population, scanning devices, imaging protocols, and so on, significant degradation in the performance of model inference results is prone to occur when models trained with domain-specific (usually institution-specific) datasets are directly applied to new (other centers/institutions) domain data. In this paper, we propose a novel unsupervised domain adaptation method based on adversarial learning to address pancreas segmentation challenges with the lack of annotations and domain shift interference. A 3D semantic segmentation model with attention module and residual module is designed as the backbone pancreas segmentation model. In both segmentation model and domain adaptation discriminator network, a multiscale progressively weighted structure is introduced to acquire different field of views. Features of labeled data and unlabeled data are fed in pairs into the proposed multiscale discriminator to learn domain-specific characteristics. Then the unlabeled data features with pseudodomain label are fed to the discriminator to acquire domain-ambiguous information. With this adversarial learning strategy, the performance of the segmentation network is enhanced to segment unseen unlabeleddata. Experiments were conducted on two public annotated datasets as source datasets, respectively, and one private dataset as target dataset, where annotations were not used for the training process but only for evaluation. The 3D segmentation model achieves comparative performance with state-of-the-art pancreas segmentation methods on source domain. After implementing our domain adaptation architecture, the average dice similarity coefficient (DSC) of the segmentation model trained on the NIH-TCIA source dataset increases from 58.79% to 72.73% on the local hospital dataset, while the performance of the target domain segmentation model transferred from the medical segmentation decathlon (MSD) source dataset rises from 62.34% to 71.17%. Correlations of features across data domains are utilized to train the pancreas segmentation model on unlabeled data domain, improving the generalization of the model. Our results demonstrate that the proposed method enables the segmentation model to make meaningful segmentation for unseen data of the training set. In the future, the proposed method has the potential to apply segmentation model trained on public dataset to clinical unannotated CT images from local hospital, effectively assisting radiologists in clinicalpractice.

Seamless Navigation, 3D Reconstruction, Thermographic and Semantic Mapping for Building Inspection.

We present a workflow for seamless real-time navigation and 3D thermal mapping in combined indoor and outdoor environments in a global reference frame. The automated workflow and partly real-time capabilities are of special interest for inspection tasks and also for other time-critical applications. We use a hand-held integrated positioning system (IPS), which is a real-time capable visual-aided inertial navigation technology, and augment it with an additional passive thermal infrared camera and global referencing capabilities. The global reference is realized through surveyed optical markers (AprilTags). Due to the sensor data’s fusion of the stereo camera and the thermal images, the resulting georeferenced 3D point cloud is enriched with thermal intensity values. A challenging calibration approach is used to geometrically calibrate and pixel-co-register the trifocal camera system. By fusing the terrestrial dataset with additional geographic information from an unmanned aerial vehicle, we gain a complete building hull point cloud and automatically reconstruct a semantic 3D model. A single-family house with surroundings in the village of Morschenich near the city of Jülich (German federal state North Rhine-Westphalia) was used as a test site to demonstrate our workflow. The presented work is a step towards automated building information modeling.

2D TO 3D LABEL PROPAGATION FOR THE SEMANTIC SEGMENTATION OF HERITAGE BUILDING POINT CLOUDS

Abstract. During the last decade, the use of semantic models of 3D buildings and structures kept growing, fostered in particular by the spread of Building Information Models (BIMs), becoming quite popular in several civil engineering and geomatics applications. Nevertheless, semantic model production usually requires quite a lot of human interaction, which may result in quite long and annoying procedures for human operators. The production of 3D semantic models of buildings often takes advantage of already available 3D reconstructions of the considered objects. Given the ever increasing resolution of 3D reconstructions, obtained thanks to the recently developed laser scanners and photogrammetric software, the availability of tools for supporting the automatic or semi-automatic generation of semantic models represents a key step for easing and speeding up the process of semantic model production. In particular, the correct semantic interpretation of the different parts of a 3D point cloud, can be seen as the basic step for the production of a BIM model. The most frequently used methods for point cloud semantic segmentation can be separated in two categories: those directly segmenting the point clouds and those based on the ancillary semantic segmentation of images representing the object of interest, then transferring back the segmentation results to the point cloud. This work focuses on the latter method, considering more specifically the application of heritage building semantic segmentation. To be more specific, this paper investigates the semantic segmentation performance on a set of four heritage buildings, obtained first applying deep-learning based image semantic segmentation and then propagating back the semantic information to the point cloud by means of a voting strategy. The obtained results are quite encouraging, motivating future investigations on improvements of this strategy, in particular when including more buildings in the considered dataset.

Cross-view SLAM solver: Global pose estimation of monocular ground-level video frames for 3D reconstruction using a reference 3D model from satellite images

Accurate pose estimation of monocular ground-level images with respect to a satellite/aerial photogrammetric dataset is an extremely challenging task. Existing solutions often perform an offline post-registration on 3D results from both sources, which however, suffers from non-rigid geometric distortions of the 3D monocular reconstruction and the lack of overlaps between the air to ground content. This paper provides an online solution that performs accurate pose estimation of the ground images with respect to a 3D model derived from satellite images, followed by a dense 3D reconstruction. Our solution takes advantage of the simultaneous localization and mapping (SLAM) paradigm to dynamically incorporate reference observations from the satellite 3D model during the incremental pose estimation, called a cross-view SLAM solver, which leverages both ground-to-satellite error and image-level reprojection errors at the frame level to yield image poses that are well-registered to the satellite 3D model for facade point cloud reconstruction. This process also has the advantage of correcting non-rigid distortions and trajectory drifts that are often presented in monocular SLAM systems. In addition, our solution leverages both the geometric and semantic information from the satellite model and ground images to perform a per-frame correction for frame-level pose initialization, in which a novel scheme called pose buffer is introduced to initialize the pose of each keyframe through robust visual hull alignment of ground objects. The proposed approach has been experimented using four trajectories of monocular videos collections (around 7,000 frames per trajectory on average) and a 3D semantic model from multi-view satellite images to estimate the poses of the video frames and yield point clouds consistent with the satellite 3D models, evaluated by using LiDAR ground-truth. Both qualitative and quantitative experiments demonstrate that our solution yields accurate, drift-free poses and point clouds consistent with the satellite data and visually much more pleasing 3D models with facade information. Compared to the LiDAR ground-truth, the derived 3D models with ground-level images have achieved a mean absolute error of 1.78 m (improved from 3.15 m achieved using SLAM without utilizing satellite 3D models) (A testing program will be made available through https://github.com/GDAOSU/Cross-View-SLAM).

GeoRec: Geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes

Semantic indoor 3D modeling with multi-task deep neural networks is an efficient and low-cost way for reconstructing an indoor scene with geometrically complete room structure and semantic 3D individuals. Challenged by the complexity and clutter of indoor scenarios, the semantic reconstruction quality of current methods is still limited by the insufficient exploration and learning of 3D geometry information. To this end, this paper proposes an end-to-end multi-task neural network for geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes (termed as GeoRec). In the proposed GeoRec, we build a geometry extractor that can effectively learn geometry-enhanced feature representation from depth data, to improve the estimation accuracy of layout, camera pose and 3D object bounding boxes. We also introduce a novel object mesh generator that strengthens the reconstruction robustness of GeoRec to indoor occlusion with geometry-enhanced implicit shape embedding. With the parsed scene semantics and geometries, the proposed GeoRec reconstructs an indoor scene by placing reconstructed object mesh models with 3D object detection results in the estimated layout cuboid. Extensive experiments conducted on two benchmark datasets show that the proposed GeoRec yields outstanding performance with 5.19×10-3 mean chamfer distance error for object reconstruction on the challenging Pix3D dataset, 70.45% mAP for 3D object detection and 77.1% 3D mIoU for layout estimation on the commonly-used SUN RGB-D dataset. Especially, the mesh reconstruction sub-network of GeoRec trained on Pix3D can be directly transferred to SUN RGB-D without any fine-tuning, manifesting a high generalization ability.