Scene Segmentation Research Articles

A Novel Method for Fast Generation of 3D Objects from Multiple Depth Sensors

Abstract Scanning real 3D objects face many technical challenges. Stationary solutions allow for accurate scanning. However, they usually require special and expensive equipment. Competitive mobile solutions (handheld scanners, LiDARs on vehicles, etc.) do not allow for an accurate and fast mapping of the surface of the scanned object. The article proposes an end-to-end automated solution that enables the use of widely available mobile and stationary scanners. The related system generates a full 3D model of the object based on multiple depth sensors. For this purpose, the scanned object is marked with markers. Markers type and positions are automatically detected and mapped to a template mesh. The reference template is automatically selected for the scanned object, which is then transformed according to the data from the scanners with non-rigid transformation. The solution allows for the fast scanning of complex and varied size objects, constituting a set of training data for segmentation and classification systems of 3D scenes. The main advantage of the proposed solution is its efficiency, which enables real-time scanning and the ability to generate a mesh with a regular structure. It is critical for training data for machine learning algorithms. The source code is available at https://github.com/SATOffice/improved_scanner3D.

One step surgical scene restoration for robot assisted minimally invasive surgery

Minimally invasive surgery (MIS) offers several advantages to patients including minimum blood loss and quick recovery time. However, lack of tactile or haptic feedback and poor visualization of the surgical site often result in some unintentional tissue damage. Visualization aspects further limits the collection of imaged frame contextual details, therefore the utility of computational methods such as tracking of tissue and tools, scene segmentation, and depth estimation are of paramount interest. Here, we discuss an online preprocessing framework that overcomes routinely encountered visualization challenges associated with the MIS. We resolve three pivotal surgical scene reconstruction tasks in a single step; namely, (i) denoise, (ii) deblur, and (iii) color correction. Our proposed method provides a latent clean and sharp image in the standard RGB color space from its noisy, blurred, and raw inputs in a single preprocessing step (end-to-end in one step). The proposed approach is compared against current state-of-the-art methods that perform each of the image restoration tasks separately. Results from knee arthroscopy show that our method outperforms existing solutions in tackling high-level vision tasks at a significantly reduced computation time.

Lane Line Detection and Object Scene Segmentation Using Otsu Thresholding and the Fast Hough Transform for Intelligent Vehicles in Complex Road Conditions

An Otsu-threshold- and Canny-edge-detection-based fast Hough transform (FHT) approach to lane detection was proposed to improve the accuracy of lane detection for autonomous vehicle driving. During the last two decades, autonomous vehicles have become very popular, and it is constructive to avoid traffic accidents due to human mistakes. The new generation needs automatic vehicle intelligence. One of the essential functions of a cutting-edge automobile system is lane detection. This study recommended the idea of lane detection through improved (extended) Canny edge detection using a fast Hough transform. The Gaussian blur filter was used to smooth out the image and reduce noise, which could help to improve the edge detection accuracy. An edge detection operator known as the Sobel operator calculated the gradient of the image intensity to identify edges in an image using a convolutional kernel. These techniques were applied in the initial lane detection module to enhance the characteristics of the road lanes, making it easier to detect them in the image. The Hough transform was then used to identify the routes based on the mathematical relationship between the lanes and the vehicle. It did this by converting the image into a polar coordinate system and looking for lines within a specific range of contrasting points. This allowed the algorithm to distinguish between the lanes and other features in the image. After this, the Hough transform was used for lane detection, making it possible to distinguish between left and right lane marking detection extraction; the region of interest (ROI) must be extracted for traditional approaches to work effectively and easily. The proposed methodology was tested on several image sequences. The least-squares fitting in this region was then used to track the lane. The proposed system demonstrated high lane detection in experiments, demonstrating that the identification method performed well regarding reasoning speed and identification accuracy, which considered both accuracy and real-time processing and could satisfy the requirements of lane recognition for lightweight automatic driving systems.

A Robust Sphere Detection in a Realsense Point Cloud by USING Z-Score and RANSAC

Three-dimensional vision cameras, such as RGB-D, use 3D point cloud to represent scenes. File formats as XYZ and PLY are commonly used to store 3D point information as raw data, this information does not contain further details, such as metadata or segmentation, for the different objects in the scene. Moreover, objects in the scene can be recognized in a posterior process and can be used for other purposes, such as camera calibration or scene segmentation. We are proposing a method to recognize a basketball in the scene using its known dimensions to fit a sphere formula. In the proposed cost function we search for three different points in the scene using RANSAC (Random Sample Consensus). Furthermore, taking into account the fixed basketball size, our method differentiates the sphere geometry from other objects in the scene, making our method robust in complex scenes. In a posterior step, the sphere center is fitted using z-score values eliminating outliers from the sphere. Results show our methodology converges in finding the basketball in the scene and the center precision improves using z-score, the proposed method obtains a significant improvement by reducing outliers in scenes with noise from 1.75 to 8.3 times when using RANSAC alone. Experiments show our method has advantages when comparing with novel deep learning method.

Multi‐directional feature refinement network for real‐time semantic segmentation in urban street scenes

AbstractEfficient and accurate semantic segmentation is crucial for autonomous driving scene parsing. Capturing detailed information and semantic information efficiently through two‐branch networks has been widely utilised in real‐time semantic segmentation. This study proposes a network named MRFNet based on two‐branch strategy to solve the problem of accuracy and speed of segmentation in urban scenes. Many real‐time networks do not comprehensively consider contextual information from sub‐regions in different directions and at different scales. To handle this problem, a Multi‐directional Feature Refinement Module (MFRM) which has three sub‐paths to capture information at different scales and directions is proposed. And MFRM reduces computation by using strip pooling and dilated convolution operations. In particular, the authors propose a Feature Cross‐guide Aggregation Module to aggregate detailed information and contextual information through the mutual guidance of detailed information and semantic information. This module guides the extraction of feature maps in a more precise direction. Experiments on Cityscapes and CamVid datasets demonstrate the effectiveness of our method by achieving a balance between accuracy and inference speed. Specially, on single 1080Ti GPU, our method yields 78.9% mean intersection over union (mIoU) and 77.4% mIoU at speed of 144.5 frames per second (FPS) and 120.8 FPS on Cityscapes and CamVid datasets respectively.

Arthroscopic scene segmentation using multispectral reconstructed frames and deep learning

BackgroundKnee arthroscopy is one of the most complex minimally invasive surgeries, and it is routinely performed to treat a range of ailments and injuries to the knee joint. Its complex ergonomic design imposes visualization and navigation constraints, consequently leading to unintended tissue damage and a steep learning curve before surgeons gain proficiency. The lack of robust visual texture and landmark frame features further limits the success of image-guided approaches to knee arthroscopy Feature- and texture-less tissue structures of knee anatomy, lighting conditions, noise, blur, debris, lack of accurate ground-truth label, tissue degeneration, and injury make semantic segmentation an extremely challenging task. To address this complex research problem, this study reported the utility of reconstructed surface reflectance as a viable piece of information that could be used with cutting-edge deep learning technique to achieve highly accurate segmented scenes. MethodsWe proposed an intraoperative, two-tier deep learning method that makes full use of tissue reflectance information present within an RGB frame to segment texture-less images into multiple tissue types from knee arthroscopy video frames. This study included several cadaver knees experiments at the Medical and Engineering Research Facility, located within the Prince Charles Hospital campus, Brisbane Queensland. Data were collected from a total of five cadaver knees, three were males and one from a female. The age range of the donors was 56–93 years. Aging-related tissue degeneration and some anterior cruciate ligament injury were observed in most cadaver knees. An arthroscopic image dataset was created and subsequently labeled by clinical experts. This study also included validation of a prototype stereo arthroscope, along with conventional arthroscope, to attain larger field of view and stereo vision. We reconstructed surface reflectance from camera responses that exhibited distinct spatial features at different wavelengths ranging from 380 to 730 nm in the RGB spectrum. Toward the aim to segment texture-less tissue types, this data was used within a two-stage deep learning model. ResultsThe accuracy of the network was measured using dice coefficient score. The average segmentation accuracy for the tissue-type articular cruciate ligament (ACL) was 0.6625, for the tissue-type bone was 0.84, and for the tissue-type meniscus was 0.565. For the analysis, we excluded extremely poor quality of frames. Here, a frame is considered extremely poor quality when more than 50% of any tissue structures are over- or underexposed due to nonuniform light exposure. Additionally, when only high quality of frames was considered during the training and validation stage, the average bone segmentation accuracy improved to 0.92 and the average ACL segmentation accuracy reached 0.73. These two tissue types, namely, femur bone and ACL, have a high importance in arthroscopy for tissue tracking. Comparatively, the previous work based on RGB data achieved a much lower average accuracy for femur, tibia, ACL, and meniscus of 0.78, 0.50, 0.41, and 0.43 using U-Net and 0.79, 0.50, 0.51, and 0.48 using U-Net++. From this analysis, it is clear that our multispectral method outperforms the previously proposed methods and delivers a much better solution in achieving automatic arthroscopic scene segmentation. ConclusionThe method was based on the deep learning model and requires a reconstructed surface reflectance. It could provide tissue awareness in an intraoperative manner that has a high potential to improve surgical precisions. It could be applied to other minimally invasive surgeries as an online segmentation tool for training, aiding, and guiding the surgeons as well as image-guided surgeries.

A transformed-feature-space data augmentation method for defect segmentation

Data augmentation is widely used in convolutional neural network (CNN) models to improve the performance of downstream tasks. The images generated by traditional data augmentation methods are usually random and can struggle to overcome the range limitation of the sampled data distribution, which can result in meaningless augmented data and poor diversity. As a result, a novel data augmentation algorithm is proposed to generate images that are outside the scope of the sampled data distribution along the feature direction with diversity weights. First, an image is mapped to the latent feature space based on the encoder. Then, the range loss function can restrict the latent variables within a hypersphere of radius k. The feature directions and diversity weights can be found based on the entropy variation in the latent feature space. During the CNN model’s training process, probabilities are constructed based on diversity weights to select the feature direction. Then, the latent variable of the input image is transformed in the feature direction to the region beyond the hypersphere. Finally, the transformed latent variable is mapped to the pixel space based on the decoder to generate images outside the sampled data space to improve diversity. Experimentally, the proposed method considerably improves the performance of defect segmentation in different industrial scenes.

Bi-Unet: A Dual Stream Network for Real-Time Highway Surface Segmentation

Highway surface segmentation consists of extracting road surface at pixel-level from the surveillance camera view. Since the intelligent traffic event detection task does not require the detection of off-road scene, the segmentation of highway surface is of great demand. However, it is challenging to accurately extract road surface in real time scenarios. To cope with the above issues, Bi-Unet, a dual stream lightweight network is proposed. Firstly, the dual stream structure enhances segmentation performance on the narrow remote end of the highway and preserves the detailed border information. Then, to perform real-time segmentation, a novel lightweight module (LSM) is introduced to lighten the model and provide higher segmentation accuracy. Moreover, to ensure road segmentation for complex scenes, a Road Attention Network (RAN) module is proposed. Lastly, due to the lack of a suitable benchmark dataset serve for the highway segmentation problem, a new large and high-quality segmentation dataset named Highway-Surface-Free (HSF) is proposed in this paper, which is collected from the perspective of highway surveillance cameras under all-day and all-weather conditions. Compared with the state of arts, the extensive experimental results verify that our proposed Bi-Unet achieves the best overall performance on our proposed HSF dataset.

Spatial-Temporal Aggregated Shuffle Attention for Video Instance Segmentation of Traffic Scene

Aiming at the problem of spatial focus regions distribution dispersion and dislocation in feature pyramid networks and insufficient feature dependency acquisition in both spatial and channel dimensions, this paper proposes a spatial-temporal aggregated shuffle attention for video instance segmentation (STASA-VIS). First, an mixed subsampling (MS) module to embed activating features from the low-level target area of feature pyramid into the high-level is designed, so as to aggregate spatial information on target area. Taking advantage of the coherent information in video frames, STASA-VIS uses the first ones of every 5 video frames as the key-frames and then propagates the keyframe feature maps of the pyramid layers forward in the time domain, and fuses with the non-keyframe mixed subsampled features to achieve time-domain consistent feature aggregation. Finally, STASA-VIS embeds shuffle attention in the backbone to capture the pixel-level pairwise relationship and dimensional dependencies among the channels and reduce the computation. Experimental results show that the segmentation accuracy of STASA-VIS reaches 41.2%, and the test speed reaches 34FPS, which is better than the state-of-the-art one stage video instance segmentation (VIS) methods in accuracy and achieves real-time segmentation.

ILabel: Revealing Objects in Neural Fields

A neural field trained with self-supervision to efficiently represent the geometry and colour of a 3D scene tends to automatically decompose it into coherent and accurate object-like regions, which can be revealed with sparse labelling interactions to produce a 3D semantic scene segmentation. Our real-time iLabel system takes input from a hand-held RGB-D camera, requires zero prior training data, and works in an ‘open set’ manner, with semantic classes defined on the fly by the user. iLabel's underlying model is a simple multilayer perceptron (MLP), trained from scratch to learn a neural representation of a single 3D scene. The model is updated continually and visualised in real-time, allowing the user to focus interactions to achieve extremely efficient semantic segmentation. A room-scale scene can be accurately labelled into 10+ semantic categories with around 100 clicks, taking less than 5 minutes. Quantitative labelling accuracy scales powerfully with the number of clicks, and rapidly surpasses standard pre-trained semantic segmentation methods. We also demonstrate a hierarchical labelling variant of iLabel and a ‘hands-free’ mode where the user only needs to supply label names for automatically-generated locations.

PointGLR: Unsupervised Structural Representation Learning of 3D Point Clouds.

This work explores the use of global and local structures of 3D point clouds as a free and powerful supervision signal for representation learning. Local and global patterns of a 3D object are closely related. Although each part of an object is incomplete, the underlying attributes about the object are shared among all parts, which makes reasoning about the whole object from a single part possible. We hypothesize that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects. Based on this hypothesis, we propose a new framework to learn point cloud representations by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape. Moreover, we extend the unsupervised structural representation learning method to more complex 3D scenes. By introducing structural proxies as the intermediate-level representations between local and global ones, we propose a hierarchical reasoning scheme among local parts, structural proxies, and the overall point cloud to learn powerful 3D representations in an unsupervised manner. Extensive experimental results demonstrate that the unsupervised representations can be very competitive alternatives of supervised representations in discriminative power, and exhibit better performance in generalization ability and robustness. Our method establishes the new state-of-the-art of unsupervised/few-shot 3D object classification and part segmentation. We also show our method can serve as a simple yet effective regime for model pre-training on 3D scene segmentation and detection tasks. We expect our observations to offer a new perspective on learning better representations from data structures instead of human annotations for point cloud understanding.

Error-Correcting Mean-Teacher: Corrections instead of consistency-targets applied to semi-supervised medical image segmentation

Semantic segmentation is an essential task in medical imaging research. Many powerful deep-learning-based approaches can be employed for this problem, but they are dependent on the availability of an expansive labeled dataset. In this work, we augment such supervised segmentation models to be suitable for learning from unlabeled data. Our semi-supervised approach, termed Error-Correcting Mean-Teacher, uses an exponential moving average model like the original Mean Teacher but introduces our new paradigm of error correction. The original segmentation network is augmented to handle this secondary correction task. Both tasks build upon the core feature extraction layers of the model. For the correction task, features detected in the input image are fused with features detected in the predicted segmentation and further processed with task-specific decoder layers. The combination of image and segmentation features allows the model to correct present mistakes in the given input pair. The correction task is trained jointly on the labeled data. On unlabeled data, the exponential moving average of the original network corrects the student’s prediction. The combined outputs of the students’ prediction with the teachers’ correction form the basis for the semi-supervised update. We evaluate our method with the 2017 and 2018 Robotic Scene Segmentation data, the ISIC 2017 and the BraTS 2020 Challenges, a proprietary Endoscopic Submucosal Dissection dataset, Cityscapes, and Pascal VOC 2012. Additionally, we analyze the impact of the individual components and examine the behavior when the amount of labeled data varies, with experiments performed on two distinct segmentation architectures. Our method shows improvements in terms of the mean Intersection over Union over the supervised baseline and competing methods. Code is available at https://github.com/CloneRob/ECMT.

Intraocular glare affects lightness illusions

Lightness Illusions (Contrast, Assimilation, and Natural Scenes with Edges and Gradients) show that Lightness appearances do not correlate with the light sent from the scene to the eye. Illusions modify “the-rest-of-the-scene” to make two identical-luminance Gray segments appear different from each other. Scene segments have two properties in human vision: apparent Lightness, and apparent Uniformity. Models of vision have two scene-dependent processes that spatially transform scene luminances. The first is optical veiling glare that modifies the sharpness of the edges, and replaces uniform scene segments with low-slope gradients. The second scene-dependent transformation is neural spatial processing. This means that this spatial transformation has many tasks to perform in generating appearances. They include: making edges appear sharp; making gradients in scene segments appear uniform; and compensating for glare’s many local redistributions of light. In short, neural spatial processing does an excellent job of ignoring glare’s distortions of scene luminance. In fact it over compensates glare in a way that generates appearances reported in Contrast Illusions, B&W Mondrians, and Checkershadow Illusions.

Correlation between neural responses and human perception in figure-ground segregation.

Segmentation of a natural scene into objects (figures) and background (ground) is one of crucial functions for object recognition and scene understanding. Recent studies have investigated neural mechanisms underlying figure-ground (FG) segregation and reported neural modulation to FG in the intermediate-level visual area, V4, of macaque monkeys (FG neurons). However, whether FG neurons contribute to the perception of FG segregation has not been clarified. To examine the contribution of FG neurons, we examined the correlations between perceptual consistency (PC), which quantified perceptual ambiguity in FG determination, and the reliability of neural signals in response to FG. First, we evaluated PCs for the images that were used in the previous neural recording in V4; specifically, we measured how consistently FG can be determined across trials and participants for each stimulus. The PCs were widely distributed, so that we identified the ambiguity in FG segregation for each stimulus. Next, we analyzed the correlation between the PCs and the reliability of neural modulation to FG. We found that the stimuli with higher PCs evoked more consistent and greater modulation in the responses of single neurons than those with lower PCs. Since perception is expected to show a greater correlation with responses of neural population compared to those of single neurons, we examined the correlation between the PCs and the consistency of the population responses in FG determination. Stimuli with higher PCs evoked higher population consistency than those with lower PCs. Finally, we analyzed the correlation between the PCs and neural latencies in FG modulation. We found that the stimuli with higher PCs showed shorter reaction times in FG perception and evoked shorter modulation latencies in FG neurons. These results indicate that the responses of FG neurons recorded from macaque monkeys show significant correlations with human FG perception, suggesting that V4 neurons with FG-dependent responses contribute to the perception of FG segregation.

Fast instruments and tissues segmentation of micro-neurosurgical scene using high correlative non-local network

Surgical scene segmentation provides critical information for guidance in micro-neurosurgery. Segmentation of instruments and critical tissues contributes further to robot assisted surgery and surgical evaluation. However, due to the lack of relevant scene segmentation dataset, scale variation and local similarity, micro-neurosurgical segmentation faces many challenges. To address these issues, a high correlative non-local network (HCNNet), is proposed to aggregate multi-scale feature by optimized non-local mechanism. HCNNet adopts two-branch design to generate features of different scale efficiently, while the two branches share common weights in shallow layers. Several short-term dense concatenate (STDC) modules are combined as the backbone to capture both semantic and spatial information. Besides, a high correlative non-local module (HCNM) is designed to guide the upsampling process of the high-level feature by modeling global context generated from the low-level feature. It filters out confused pixels of different classes in the non-local correlation map. Meanwhile, a large segmentation dataset named NeuroSeg is constructed, which contains 15 types of instruments and 3 types of tissues that appear in meningioma resection surgery. The proposed HCNNet achieves the state-of-the-art performance on NeuroSeg, it reaches an inference speed of 54.85 FPS with the highest accuracy of 59.62% mIoU, 74.7% Dice, 70.55% mAcc and 87.12% aAcc.

Fast and Effective Scene Segmentation Method for Luminance Adjustment of Multi-Exposure Image

Multi-exposure image fusion is an important task for high dynamic range imaging. The performance of a fusion method is highly dependent on the quality of the input multi-exposure image. However, it is often difficult to obtain a multi-exposure image thoroughly covering the dynamic range of a scene. In such a situation, some areas are unclear in all the images contained by the multi-exposure image, leading to a low-visibility fusion result. To overcome this problem, Kinoshita and Kiya proposed scene segmentation-based luminance adjustment (SSLA). In SSLA, the scene is segmented into regions and the luminance of each region is adjusted so that the region has mid-level brightness. A fusion result with better visibility can be obtained by fusing the luminance adjusted multi-exposure image. Kinoshita and Kiya proposed two scene segmentation approaches for SSLA. One of them is fast but sometimes fails to segment dark areas and bright areas into different regions. The other can generate a better scene segmentation result but it requires a time-consuming iterative process. In this paper, we propose a new scene segmentation method for SSLA. The proposed scene segmentation method uses information obtained from the nonlinear luminance distribution and area occupancy of the segmented regions. A fast implementation for the proposed scene segmentation method is also described. Experimental results show that the proposed scene segmentation method is fast and stably generates good scene segmentation results.

MPCNet: Improved MeshSegNet Based on Position Encoding and Channel Attention

In the process of orthodontic treatment, it is a very important step to accurately segment each tooth and jaw model with computer assistance. The use of deep learning technology methods for tooth segmentation can not only save a lot of manual interaction and time cost but also improve the treatment effect. 3D tooth segmentation is a hot topic of interest for international related scholars, and some end-to-end tooth segmentation methods based on dental mesh scanning models have been emerging in recent years. Due to the limited variety of existing models, they are not well suited for different 3D segmentation scenarios, and the feature extraction capability and segmentation effect of these models still need to be improved. In this paper, we propose a novel end-to-end tooth segmentation method, MPCNet, which adds multi-scale mesh density information to the input layer, uses position encoding and channel attention mechanism to improve MeshSegNet, and uses graph-cut post-processing to perform 3D tooth segmentation in real scenes. The effectiveness of MPCNet is evaluated on a real 3D scanned tooth segmentation dataset, which significantly outperforms the current mainstream segmentation methods.

Mesh-Based DGCNN: Semantic Segmentation of Textured 3-D Urban Scenes

Textured 3D mesh is one of the final user products in photogrammetry and remote sensing. However, research on the semantic segmentation of complex urban scenes represented by textured 3D meshes is in its infancy. We present a mesh-based dynamic graph CNN (DGCNN) for the semantic segmentation of textured 3D meshes. To represent each mesh facet, composite input feature vectors are constructed by concatenating the face-inherent features, i.e., XYZ coordinates of the center of gravity (CoG), texture values, and normal vectors. A texture fusion module is embedded into the proposed mesh-based DGCNN to generate high-level semantic features of the high-resolution texture information, which is useful for semantic segmentation. We achieve competitive accuracies when the proposed method is applied to the SUM mesh datasets. The overall accuracy (OA), Kappa coefficient (Kap), mean precision (mP), mean recall (mR), mean F1 score (mF1), and mean intersection over union (mIoU) are 93.3%, 88.7%, 79.6%, 83.0%, 80.7%, and 69.6%, respectively. In particular, the OA, mean class accuracy (mAcc), mIoU, and mF1 increase by 0.3%, 12.4%, 3.4%, and 6.9%, respectively, compared to the state-of-the-art method.

An adaptive wavelet shrinkage based accumulative frame differencing model for motion segmentation

Motion segmentation in real-world scenes is a fundamental component in computer vision. There exists a variety of motion recognition algorithms, each with varying degrees of accuracy and computational complexity. The most widely used techniques, in the case of static cameras, are those based on frame difference. Those methods have a significant weakness when it comes to detect slow moving objects. Therefore, we introduce in this paper a novel approach that aims to improve motion segmentation by proposing an accumulative wavelet based frame differencing technique. Moreover, in the proposed approach we exploit a combination of several techniques to efficiently enhance the quality of motion segmentation results. The approach's performance on real-world video sequences shows that comparing frames using the 2D wavelet transform increases motion segmentation quality.

Efficient 3D Scene Semantic Segmentation via Active Learning on Rendered 2D Images.

Inspired by Active Learning and 2D-3D semantic fusion, we proposed a novel framework for 3D scene semantic segmentation based on rendered 2D images, which could efficiently achieve semantic segmentation of any large-scale 3D scene with only a few 2D image annotations. In our framework, we first render perspective images at certain positions in the 3D scene. Then we continuously fine-tune a pre-trained network for image semantic segmentation and project all dense predictions to the 3D model for fusion. In each iteration, we evaluate the 3D semantic model and re-render images in several representative areas where the 3D segmentation is not stable and send them to the network for training after annotation. Through this iterative process of rendering-segmentation-fusion, it can effectively generate difficult-to-segment image samples in the scene, while avoiding complex 3D annotations, so as to achieve label-efficient 3D scene segmentation. Experiments on three large-scale indoor and outdoor 3D datasets demonstrate the effectiveness of the proposed method compared with other state-of-the-art.