Mobile Mapping Images Research Articles

Tree species classification on images from airborne mobile mapping using ML.NET

ABSTRACT Deep learning is a powerful tool for automating the process of recognizing and classifying objects in images. In this study, we used ML.NET, a popular open-source machine learning framework, to develop a model for identifying tree species in images obtained from airborne mobile mapping. These high-resolution images can be used to create detailed maps of the landscape. They can also be analyzed and processed to extract information about visual features, including tree species recognition. The deep learning model was trained using ML.NET to classify two tree species based on the combination of airborne mobile mapping images. Our approach yielded impressive results, with a maximum classification accuracy of 93.9%. This demonstrates the effectiveness of combining imagery sources with deep learning tools in ML.NET for efficient and accurate tree species classification. This study highlights the potential of the ML.NET framework for automating object classification and can provide valuable insights and information for forestry management and conservation efforts. The primary objective of this research was to evaluate the effectiveness of an approach for identifying tree species through a model generated using a combination of ortho and oblique images captured by a mobile mapping system.

IMPROVING DEEP LEARNING BASED SEMANTIC SEGMENTATION WITH MULTI VIEW OUTLIER CORRECTION

Abstract. The goal of this paper is to use transfer learning for semi supervised semantic segmentation in 2D images: given a pretrained deep convolutional network (DCNN), our aim is to adapt it to a new camera-sensor system by enforcing predictions to be consistent for the same object in space. This is enabled by projecting 3D object points into multi-view 2D images. Since every 3D object point is usually mapped to a number of 2D images, each of which undergoes a pixelwise classification using the pretrained DCNN, we obtain a number of predictions (labels) for the same object point. This makes it possible to detect and correct outlier predictions. Ultimately, we retrain the DCNN on the corrected dataset in order to adapt the network to the new input data. We demonstrate the effectiveness of our approach on a mobile mapping dataset containing over 10’000 images and more than 1 billion 3D points. Moreover, we manually annotated a subset of the mobile mapping images and show that we were able to rise the mean intersection over union (mIoU) by approximately 10% with Deeplabv3+, using our approach.

CORRECTION OF MOBILE MAPPING TRAJECTORIES IN GNSS-DENIED ENVIRONMENTS USING AERIAL NADIR AND AERIAL OBLIQUE IMAGES

Abstract. Mobile mapping enables highly accurate as well as high-resolution image data capture at low cost and high speed. As a terrestrial acquisition technique predominately employed in urban, and thus built-up areas, non-line-of-sight and multipath effects challenge its absolute positioning capabilities provided by GNSS. In conjunction with IMU drift, the platform’s trajectory has an unknown accuracy, which influences the quality of the data product. By employing a highly accurate co-registration technique for identifying tie correspondences between mobile mapping images and aerial nadir as well as aerial oblique images, reliable ground control can be introduced into an adjustment solution. We exemplify the performance of our registration results by showcasing adjusted mobile mapping trajectories in four different test areas, each with about 100 consecutive recording locations (approx. 500 m length) in the city centre of Rotterdam, The Netherlands. The mobile mapping data has been adjusted in different configurations, i.e. with nadir or oblique aerial correspondences only and if possible in conjunction. To compare the horizontal as well as the vertical accuracy before and after the respective adjustments, more than 30 ground control points were surveyed for these experiments. In general, the aim of our technique is not only to correct mobile mapping trajectories in an automated fashion but also to verify their accuracy without the need to acquire ground control points. In most of our test cases, the overall accuracy of the mobile mapping image positions in the trajectory could be improved. Depending on the test area, an RMSE in 3D between 15 and 21 cm and an RMSE in 2D between 11 and 18 cm is achievable.

Co‐registration of panoramic mobile mapping images and oblique aerial images

AbstractMobile mapping relies on satellite‐based positioning, which suffers from line‐of‐sight and multipath issues. As an alternative, this paper presents a fully automatic approach for the co‐registration of mobile mapping and oblique aerial images to introduce highly accurate and reliable ground control for mobile mapping data adjustment. An oblique view of a scene introduces similarities as well as challenges regarding co‐registration with mobile mapping images, which is supported by mutual planes in both datasets. Façade planes from a sparse point cloud are used as projection surfaces for the mobile mapping and aerial datasets, overcoming large perspective differences between them to simplify the registration. The performance of the procedure indicates an inlier rate of around 80%.

Active use of panoramic mobile mapping systems for as built surveying and heritage documentation

ABSTRACTMobile mapping systems (MMS) are widely used technology nowadays for spatial data collection of large scale projects like for city and highway mapping. The systems are mainly equipped with laser scanning sensors and/or imaging sensors mounted on a moving vehicle during the scene capture. Imaging sensors are normally cameras which either capture perspective or panoramic images covering the whole horizon of the vehicle. The orientation of the captured panoramic images is accurate to centimeters’ level because of the precise positioning and navigation systems equipped with these mapping systems. However, the positioning accuracy of mobile mapping systems can be degraded in city centers or urban canyons because of the satellite signal disturbances.In this article, we discuss the following objectives: (1) the possibility to use the mobile mapping images for cultural heritage documentation and as built surveying and how accurate the mapping can be; (2) the concept of using the mobile mapping images as a tool of georeferencing the crowdsource images; and (3) the efficiency of using the multi-temporal mobile mapping images for occluded free cultural heritage facade orthophotos. The mobile mapping systems of CycloMedia with two panoramic products of Cyclorama images (12 MP) and HD Cycloramas (100 MP) are used for the experimental tests in this research article.

3D change detection at street level using mobile laser scanning point clouds and terrestrial images

Automatic change detection and geo-database updating in the urban environment are difficult tasks. There has been much research on detecting changes with satellite and aerial images, but studies have rarely been performed at the street level, which is complex in its 3D geometry. Contemporary geo-databases include 3D street-level objects, which demand frequent data updating. Terrestrial images provides rich texture information for change detection, but the change detection with terrestrial images from different epochs sometimes faces problems with illumination changes, perspective distortions and unreliable 3D geometry caused by the lack of performance of automatic image matchers, while mobile laser scanning (MLS) data acquired from different epochs provides accurate 3D geometry for change detection, but is very expensive for periodical acquisition. This paper proposes a new method for change detection at street level by using combination of MLS point clouds and terrestrial images: the accurate but expensive MLS data acquired from an early epoch serves as the reference, and terrestrial images or photogrammetric images captured from an image-based mobile mapping system (MMS) at a later epoch are used to detect the geometrical changes between different epochs. The method will automatically mark the possible changes in each view, which provides a cost-efficient method for frequent data updating. The methodology is divided into several steps. In the first step, the point clouds are recorded by the MLS system and processed, with data cleaned and classified by semi-automatic means. In the second step, terrestrial images or mobile mapping images at a later epoch are taken and registered to the point cloud, and then point clouds are projected on each image by a weighted window based z-buffering method for view dependent 2D triangulation. In the next step, stereo pairs of the terrestrial images are rectified and re-projected between each other to check the geometrical consistency between point clouds and stereo images. Finally, an over-segmentation based graph cut optimization is carried out, taking into account the color, depth and class information to compute the changed area in the image space. The proposed method is invariant to light changes, robust to small co-registration errors between images and point clouds, and can be applied straightforwardly to 3D polyhedral models. This method can be used for 3D street data updating, city infrastructure management and damage monitoring in complex urban scenes.

Image-Based Rendering of LOD1 3D City Models for traffic-augmented Immersive Street-view Navigation

Abstract. It may be argued that urban areas may now be modeled with sufficient details for realistic fly-through over the cities at a reasonable price point. Modeling cities at the street level for immersive street-view navigation is however still a very expensive (or even impossible) operation if one tries to match the level of detail acquired by street-view mobile mapping imagery. This paper proposes to leverage the richness of these street-view images with the common availability of nation-wide LOD1 3D city models, using an image-based rendering technique : projective multi-texturing. Such a coarse 3D city model may be used as a lightweight scene proxy of approximate coarse geometry. The images neighboring the interpolated viewpoint are projected onto this scene proxy using their estimated poses and calibrations and blended together according to their relative distance. This enables an immersive navigation within the image dataset that is perfectly equal to – and thus as rich as – original images when viewed from their viewpoint location, and which degrades gracefully in between viewpoint locations. Beyond proving the applicability of this preprocessing-free computer graphics technique to mobile mapping images and LOD1 3D city models, our contributions are three-fold. Firstly, image distortion is corrected online in the GPU, preventing an extra image resampling step. Secondly, externally-computed binary masks may be used to discard pixels corresponding to moving objects. Thirdly, we propose a shadowmap-inspired technique that prevents, at marginal cost, the projective texturing of surfaces beyond the first, as seen from the projected image viewpoint location. Finally, an augmented visualization application is introduced to showcase the proposed immersive navigation: images are unpopulated from vehicles using externally-computed binary masks and repopulated using a 3D visualization of a 2D traffic simulation.

Automated processing of mobile mapping image sequences

Automated approaches to image sequence processing using mobile mapping imagery have been under investigation in the Department of Geomatics Engineering at The University of Calgary. This paper presents an overview of several methods developed for the VISAT™ mobile mapping system at The University of Calgary. Following a brief overview of mobile mapping technology, an analysis of mobile mapping image sequences from the viewpoint of visual motion theory is provided. Particular attention is paid to image and object domain constraints that can be exploited in the processing of mobile mapping image sequences. Several key methods to automated processing of mobile mapping image sequences are then described. These methods can be grouped into two categories, namely, information extraction and image-based trajectory determination (bridging).