Abstract
Abstract. The goal of this paper is to use transfer learning for semi supervised semantic segmentation in 2D images: given a pretrained deep convolutional network (DCNN), our aim is to adapt it to a new camera-sensor system by enforcing predictions to be consistent for the same object in space. This is enabled by projecting 3D object points into multi-view 2D images. Since every 3D object point is usually mapped to a number of 2D images, each of which undergoes a pixelwise classification using the pretrained DCNN, we obtain a number of predictions (labels) for the same object point. This makes it possible to detect and correct outlier predictions. Ultimately, we retrain the DCNN on the corrected dataset in order to adapt the network to the new input data. We demonstrate the effectiveness of our approach on a mobile mapping dataset containing over 10’000 images and more than 1 billion 3D points. Moreover, we manually annotated a subset of the mobile mapping images and show that we were able to rise the mean intersection over union (mIoU) by approximately 10% with Deeplabv3+, using our approach.
Highlights
The problem of overfitting in deep neural networks is the norm rather than the exception when they are trained on small datasets
Retraining of a deep convolutional network (DCNN) in the new domain with the corrected data set, resulting in a ∼ 10% increase in mean intersection over union (mIoU)
We show that we are able to correct outliers based on 3D point features and the list of class predicitons by training a neural network on the annotated subset
Summary
The problem of overfitting in deep neural networks is the norm rather than the exception when they are trained on small datasets. Much effort is put into reusing knowledge or adapting pretrained networks to new problems in order to avoid high costs for labeling data and increase the performance of the models. In computer vision, this is often accomplished by training a DCNN on a publicly available dataset and fine-tuning it to the target dataset to solve a similar task. It is desirable to use only few or even no annotations in the target dataset In those semior unsupervised cases one often has to make assumptions about the nature of the data in order to solve these problems
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
