Multi View Research Articles

Structure-from-Motion on shallow reefs and beaches: potential and limitations of consumer-grade drones to reconstruct topography and bathymetry

Reconstructing the topography of shallow underwater environments using Structure-from-Motion—Multi View Stereo (SfM-MVS) techniques applied to aerial imagery from Unmanned Aerial Vehicles (UAVs) is challenging, as it involves nonlinear distortions caused by water refraction. This study presents an experiment with aerial photographs collected with a consumer-grade UAV on the shallow-water reef of Fuvahmulah, the Maldives. Under conditions of rising tide, we surveyed the same portion of the reef in ten successive flights. For each flight, we used SfM-MVS to reconstruct the Digital Elevation Model (DEM) of the reef and used the flight at low tide (where the reef is almost entirely dry) to compare the performance of DEM reconstruction under increasing water levels. Our results show that differences with the reference DEM increase with increasing depth, but are substantially larger if no underwater ground control points are taken into account in the processing. Correcting our imagery with algorithms that account for refraction did not improve the overall accuracy of reconstruction. We conclude that reconstructing shallow-water reefs (less than 1 m depth) with consumer-grade UAVs and SfM-MVS is possible, but its precision is limited and strongly correlated with water depth. In our case, the best results are achieved when ground control points were placed underwater and no refraction correction is used.

3D Foot Reconstruction Based on Mobile Phone Photographing

Foot measurement is necessary for personalized customization. Nowadays, people usually obtain their foot size by using a ruler or foot scanner. However, there are some disadvantages to this, namely, large measurement error and variance when using rulers, and high price and poor convenience when using a foot scanner. To tackle these problems, we obtain foot parameters by 3D foot reconstruction based on mobile phone photography. Firstly, foot images are taken by a mobile phone. Secondly, the SFM (Structure-from-Motion) algorithm is used to acquire the corresponding parameters and then to calculate the camera position to construct the sparse model. Thirdly, the PMVS (Patch-based Multi View System) is adopted to build a dense model. Finally, the Meshlab is used to process and measure the foot model. The result shows that the experimental error of the 3D foot reconstruction method is around 1 mm, which is tolerable for applications such as shoe tree customization. The experiment proves that the method can construct the 3D foot model efficiently and easily. This technology has broad application prospects in the fields of shoe size recommendation, high-end customized shoes and medical correction.

A Multi-View SVM Approach for Seizure Detection from Single Channel EEG Signals

Seizures are the part of the epilepsy that occurs in central nervous system which leads to abnormal brain activity. Electroencephalogram (EEG) signal recordings are mostly used in epileptic seizure detection process. Detection of seizures is a crucial part for further treatment of patients. This paper proposes a multi-view SVM model for seizure detection using the single channel EEG signals. In this experiment, two views of the EEG data have been extracted, (1) the time domain features using Independent Component Analysis (ICA) and (2) power spectral densities are obtained in the frequency domain. Extracted features have been fed to multi-view SVM classification model. In this study, a single channel EEG dataset is used for seizure detection. Performance estimation parameters namely Accuracy, Sensitivity, Specificity, F1-score, and AUC value have been estimated for evaluating the proposed model. The model classified seizure and non-seizure over the sets A vs E and B vs E with an accuracy greater than 99% using k-fold cross validation. The classification accuracy obtained by multi-view SVM is better by 1–4% than single view SVM using the same features. Furthermore, the proposed model is also compared with existing single view SVM models. It is observed that the multi view SVM model performed significantly better, compare to a single view SVM model over the same features.

3D Reconstruction of Coastal Cliffs from Fixed-Wing and Multi-Rotor UAS: Impact of SfM-MVS Processing Parameters, Image Redundancy and Acquisition Geometry

Monitoring the dynamics of coastal cliffs is fundamental for the safety of communities, buildings, utilities, and infrastructures located near the coastline. Structure-from-Motion and Multi View Stereo (SfM-MVS) photogrammetry based on Unmanned Aerial Systems (UAS) is a flexible and cost-effective surveying technique for generating a dense 3D point cloud of the whole cliff face (from bottom to top), with high spatial and temporal resolution. In this paper, in order to generate a reproducible, reliable, precise, accurate, and dense point cloud of the cliff face, a comprehensive analysis of the SfM-MVS processing parameters, image redundancy and acquisition geometry was performed. Using two different UAS, a fixed-wing and a multi-rotor, two flight missions were executed with the aim of reconstructing the geometry of an almost vertical cliff located at the central Portuguese coast. The results indicated that optimizing the processing parameters of Agisoft Metashape can improve the 3D accuracy of the point cloud up to 2 cm. Regarding the image acquisition geometry, the high off-nadir (90°) dataset taken by the multi-rotor generated a denser and more accurate point cloud, with lesser data gaps, than that generated by the low off-nadir dataset (3°) taken by the fixed wing. Yet, it was found that reducing properly the high overlap of the image dataset acquired by the multi-rotor drone permits to get an optimal image dataset, allowing to speed up the processing time without compromising the accuracy and density of the generated point cloud. The analysis and results presented in this paper improve the knowledge required for the 3D reconstruction of coastal cliffs by UAS, providing new insights into the technical aspects needed for optimizing the monitoring surveys.

Solution to Model Consistency in Equipment Support Simulation

In order to solve the problem of multi view inconsistency in equipment support simulation, a semantic oriented MCM method based on ontology model is proposed. Firstly, the rules of extracting the basic elements of the model are given. Secondly, the algorithm description of ontology mapping is given. Finally, each meta concept in the domain is described by some model description method to form the corresponding description model.

3D reconstruction using Structure From Motion (SFM) algorithm and Multi View Stereo (MVS) based on computer vision

The development of the Information and Computer Technology (ICT) sector, three-dimensional (3D) technology is also growing rapidly. Currently, the need to visualize 3D objects is widely used in animation and graphic applications, architecture, education, cultural recognition and Virtual Reality. 3D modeling of historic buildings has become a concern in recent years. 3D reconstruction is an attempt to document reconstruction or restoration if the building is destroyed. By using the 3D model reconstruction using Structure from Motion (SFM) and Multi View Stereo (MVS) algorithm based on Computer Vision, it is hoped that the results of this 3D modeling can be utilized as an effort to preserve 3D objects in the Penataran Temple cultural heritage area. This research was conducted by taking as many as 61 images of objects in the Blitar Penataran Temple area. The photos obtained were reconstructed into a 3D model using the Structure From Motion algorithm in the meshroom. This research a trial of the original image with a compressed image for reconstruction is used to compare the 3D reconstruction process from the two input data. From 61 images processed using the Structure Form Motion algorithm, 33 poses of camera pose and 3D points were improved, both original and compressed images. The number of iterations compresses 1.4% less than the original image and takes 43.53% faster than the original image.

Non-negative sparse Laplacian regularized latent multi-view subspace clustering

Recently, in the area of artificial intelligence and machine learning, subspace clustering of multi-view data is a research hotspot. The goal is to divide data samples from different sources into different groups. We proposed a new subspace clustering method for multi-view data which termed as Non-negative Sparse Laplacian regularized Latent Multi-view Subspace Clustering (NSL2MSC) in this paper. The method proposed in this paper learns the latent space representation of multi view data samples, and performs the data reconstruction on the latent space. The algorithm can cluster data in the latent representation space and use the relationship of different views. However, the traditional representation-based method does not consider the non-linear geometry inside the data, and may lose the local and similar information between the data in the learning process. By using the graph regularization method, we can not only capture the global low dimensional structural features of data, but also fully capture the nonlinear geometric structure information of data. The experimental results show that the proposed method is effective and its performance is better than most of the existing alternatives.

REKONTRUKSI MODEL 3D DARI BANYAK GAMBAR MENGGUNAKAN ALGORITMA STRUCTURE FROM MOTION (SFM) DAN MULTI VIEW STEREO (MVS) BERBASIS COMPUTER VISION

With the development of the field of Information and Computer Technology (ICT), three-dimensional technology (3D) is also growing rapidly. Currently, the need to visualize 3D objects is widely used in animation and graphics applications, architecture, education, cultural recognition and virtual reality. 3D modeling of historical buildings has become a concern in recent years. 3D reconstruction is a documentation effort for reconstruction or restoration if the building is destroyed. By using a 3D model reconstruction approach based on multiple images using the Structure From Motion (SFM) and Multi View Stereo (MVS) algorithm, it is hoped that the 3D modeling results can be used as an effort to preserve 3D objects in the cultural heritage area of Penataran Temple. This research was conducted by taking an object in the form of photos as many as 61 pictures in the area of ​​the Blitar Penataran Temple. The resulting photos are reconstructed into a 3D model using the Structure From Motion algorithm in the meshroom. In this study, a test was carried out on the original image with the compressed image for reconstruction to be compared to the 3D reconstruction process from the two input data. From 61 images processed using the Structure Form Motion algorithm, 33 camera pose and 3D point data were obtained, both original and compressed images. For the number of iterations the compressed image is 1.4% less than the original image and takes 43.53% faster than the original image.

UAV Photogrammetry and 3D Modelling of Complex Architecture for Maintenance Purposes: the Case Study of the Masonry Bridge on the Sele River, Italy

The aim of this paper is to identify a suitable methodology to realize, in an easy and quick way, 3D models of complex structures. To achieve this aim, the first step is to build the 3D model of the scene under investigation using photogrammetric modelling. This task was carried out by the use of algorithms based on Structure from Motion (SfM) - Multi View Stereo (MVS) approaches and using camera-generated images supplied in the UAS (Unmanned Aerial system). Once built the 3D point cloud of the structure under investigation, the geometry of each element was reconstructed with 3D profile reconstruction using Rhinoceros software and a few plug-ins developed in this software. Indeed, this paper shows, through a case study of a masonry bridge of special architectural and historic value built in the middle of 1800s and located in the south of Italy, the potential of the method developed in order to manage a maintenance or restoration project.

A Rapid Method of the Rock Mass Surface Reconstruction for Surface Deformation Detection at Close Range.

Characterizing the surface deformation during the inter-survey period could assist in understanding rock mass progressive failure processes. Moreover, 3D reconstruction of rock mass surface is a crucial step in surface deformation detection. This study presents a method to reconstruct the rock mass surface at close range in a fast way using the improved structure from motion—multi view stereo (SfM) algorithm for surface deformation detection. To adapt the unique feature of rock mass surface, the AKAZE algorithm with the best performance in rock mass feature detection is introduced to improve SfM. The surface reconstructing procedure mainly consists of image acquisition, feature point detection, sparse reconstruction, and dense reconstruction. Hereafter, the proposed method was verified by three experiments. Experiment 1 showed that this method effectively reconstructed the rock mass model. Experiment 2 proved the advanced accuracy of the improved SfM compared with the traditional one in reconstructing the rock mass surface. Eventually, in Experiment 3, the surface deformation of rock mass was quantified through reconstructing images before and after the disturbance. All results have shown that the proposed method could provide reliable information in rock mass surface reconstruction and deformation detection.

On the use of ensemble method for multi view textual data

ABSTRACT Nowadays, trends detection is an important task on social media to determine trends that are being discussed the most on a social platform. One of the main challenges of this task is the processing of unstructured textual data which has different representations. Therefore, multi view text clustering presents a useful solution for trends detection by integrating various representations called ‘views’ to provide a complementary description of the same content. In this context, we propose a new ensemble method for multi-view text clustering that exploits different representations of text in order to produce more accurate and high quality clustering. Extensive experiments on real-world text datasets were conducted to demonstrate its superiority by comparing with the existing methods. An application of the proposed method in trends detection from twitter is also illustrated.

RESEARCH ON URBAN CONSTRUCTION LAND CHANGE DETECTION METHOD BASED ON DENSE DSM AND TDOM OF AERIAL IMAGES

Abstract. The scale of urban construction land is an important factor of the process of urbanization. This paper presented an urban construction land change detection method combining dense DSM difference and TDOM tree extraction of traditional aerial photos: the pixel-level DSM and TDOM for two period are obtained by Semi-Global Matching (SGM) and Multi View Stereo (MVS), respectively. After DSM pre-processing including noise filter and hole filling, the difference of DSM is calculated. The segmentation and tree extraction of TDOM are proposed to reduce the change errors caused by the crown influence of different seasons. Based on this method, 2 experiments were carried out. One was for the urban construction land located at 5th ring road south in Beijing with ADS80 aerial images obtained in 2016 and 2017, and the other was for the building demolition and construction of Dongcheng and Xicheng District of Beijing with UltraCam, the frame camera and RCD30 between 2015 and 2017. Through the experiments, it was concluded that the result of the method could identify all the changes both in the plane and in the elevation, and the edges of the change patches were clear and regular, which could assist checking the manual change extraction results. The minimum area of the change patches could be 5 m2 with the DSM resolution of 0.2 m, which was of great significance for the urban construction change detection including the factory, residential area and also the urban regional greening.

The Performance of multi view scope for tracheal intubation of pediatric patients (randomized controlled trial)

The Performance of multi view scope for tracheal intubation of pediatric patients (randomized controlled trial)

Towards Benthic Habitat 3D Mapping Using Machine Learning Algorithms and Structures from Motion Photogrammetry

The accurate classification and 3D mapping of benthic habitats in coastal ecosystems are vital for developing management strategies for these valuable shallow water environments. However, both automatic and semiautomatic approaches for deriving ecologically significant information from a towed video camera system are quite limited. In the current study, we demonstrate a semiautomated framework for high-resolution benthic habitat classification and 3D mapping using Structure from Motion and Multi View Stereo (SfM-MVS) algorithms and automated machine learning classifiers. The semiautomatic classification of benthic habitats was performed using several attributes extracted automatically from labeled examples by a human annotator using raw towed video camera image data. The Bagging of Features (BOF), Hue Saturation Value (HSV), and Gray Level Co-occurrence Matrix (GLCM) methods were used to extract these attributes from 3000 images. Three machine learning classifiers (k-nearest neighbor (k-NN), support vector machine (SVM), and bagging (BAG)) were trained by using these attributes, and their outputs were assembled by the fuzzy majority voting (FMV) algorithm. The correctly classified benthic habitat images were then geo-referenced using a differential global positioning system (DGPS). Finally, SfM-MVS techniques used the resulting classified geo-referenced images to produce high spatial resolution digital terrain models and orthophoto mosaics for each category. The framework was tested for the identification and 3D mapping of seven habitats in a portion of the Shiraho area in Japan. These seven habitats were corals (Acropora and Porites), blue corals (H. coerulea), brown algae, blue algae, soft sand, hard sediments (pebble, cobble, and boulders), and seagrass. Using the FMV algorithm, we achieved an overall accuracy of 93.5% in the semiautomatic classification of the seven habitats.

An Adaptive Multiview Active Learning Approach for Spectral–Spatial Classification of Hyperspectral Images

Combining spectral and spatial features in hyperspectral image classification is a common practice due to the improvements in classification accuracy that can be obtained by extracting information from neighboring pixels. However, the resulting high dimensionality of the input data and the typically limited number of labeled samples are two key challenges that affect the overall performance of supervised classification methods. To alleviate these two issues, we propose an adaptive multiview (MV)-based active learning (AL) approach that is different from the existing MV AL methods in two main ways: 1) to improve the view sufficiency, a spectral–spatial view generation approach is proposed by incorporating spatial features derived from the segmentation maps into each view and 2) to increase the diversity across views, a dynamic view is generated at each AL iteration by selecting important features from the predefined views. The performance of each view is further improved by applying the proposed AL algorithm in conjunction with an ensemble approach as back-end classifier, a scenario less explored in the remote sensing community than single classifier-based AL methodologies. The proposed approach is applied to three widely analyzed hyperspectral data sets [i.e., Kennedy Space Center (KSC), Indian Pine, and University of Houston (UH)], and the results demonstrate the efficacy of the proposed method compared with other state-of-the-art AL classification methods.

STUDY ON TIME EMERGENCE OF DOMINANT DESIGN OF INKJET PRINTER AND NC MACHINE BY USING F-TERM IN JAPANESE PATENT

This research is aimed at identifying the trend of technology and time emergence of the dominant design of inkjet printer and NC machine by using F-term in Japanese Patent. F-term was employed instead of International Patent Classification (IPC) because of the multi view point of the theme and it provides the certain technical area. The firm‘s product entering time to the market was evaluated to the S curve of the product lifecycle and technology life cycle was also investigated.

3D MODELING OF ROMAN BRIDGE BY THE INTEGRATION OF TERRESTRIAL AND UAV PHOTOGRAMMETRIC SURVEY FOR STRUCTURAL ANALYSIS PURPOSE

Abstract. The 3D survey of the masonry bridges is rather complex and may require the integration of terrestrial and aerial surveys. In fact, in order to perform the survey of the superstructure of a bridge, the photogrammetric survey may be realized by the use UAV (Unmanned Aerial Vehicle) since it allows acquiring all the details of the upper part of the bridge. Indeed, especially the extreme lateral parts of the bridge cannot be performed through a traditional terrestrial survey. In addition, the UAV photogrammetry allows speeding up and optimizing work. As regards the terrestrial survey, instead, it has the advantage of being able to use highly performing optics and cameras (high resolution of the sensor, stabilized optics, etc.). However, these types of sensors are quite heavy and, in general, could not be mounted on an aerial platform on commercial and low-cost drones. In any case, beyond the payload of common commercial drones at low cost, there is also the problem that the digital cameras mounted on the UAV platforms cannot observe the arches of the bridge from the bottom upwards by low-cost tools. Therefore, a simple way to build a unique 3D model consists in the integration of terrestrial and aerial surveys; in this way, using a suitable procedure, sensor and software based on the Structure from Motion (SfM) and Multi View Stereo (MVS) approaches, it was possible to build detailed, precise and textured three-dimensional models of structures, such as the buildings, historical artefact, bridges etc.In this paper, a case study concerning the construction of a three-dimensional model according a photogrammetric method of a Roman bridge is discussed. In particular, the bridge taken under investigation, belong to the Cultural Heritage, is called “San Cono” and is located in south of Italy. After built the 3D model of masonry bridge (3D point cloud and 3D mesh), it was identified a special procedure that allowed to obtain the model of the bridge suitable for a structural analysis within of dedicate software.

THE COMBINATION OF TERRESTRIAL LIDAR AND UAV PHOTOGRAMMETRY FOR INTERACTIVE ARCHITECTURAL HERITAGE VISUALIZATION USING UNITY 3D GAME ENGINE

Abstract. The digital 3D documentation of architectural heritage using advanced 3D measurement technologies such as UAV photogrammetry and terrestrial LiDAR (TLS) becomes a potential and efficient method since it can produce 3D pointclouds in detail and high density of pointclouds levels. However, TLS is unable to scan the roof part of tall building, whereas UAV photogrammetry achieves high density of pointclouds at that area. In order to make a complete 3D pointclouds of heritage building, we merged and integrated the TLS and UAV pointclouds data by using Iterative Closest Point (ICP) algorithms into one reference system. In this study, we collected two architectural heritage building in Yogyakarta, Indonesia, i.e., "Vredeburg Fort Museum (VFM)" and "Kotagede Great Mosque (KGM)", the oldest mosque in Yogyakarta. For the data acquisition, we used Faro Focus X330 and GLS 2000 Laser Scanner. We produced three-dimensional point clouds from UAV imagery by using Structure from Motion and Multi View Stereo (SfM-MVS) technique through Photoscan software. In order to merging and integrating both of pointclouds data, Maptek I-Site Studio 6.1 with Educational License was used. Those data were successfully registered, and according to the registration report, we had observed 20.60 mm of RMS error. The 3D models and their textures in outdoor and indoor side were processed using Autodesk software. Modelling was carried out on the structure of building’s façade base on simple geometric primitive as planes, straight lines, circles, spheres and cylinder. For interactive visualization, a modern and widely accessible game engine technology (Unity3D) was used. The result was an interactive displaying 3D model of an architectural heritage building in LOD3 level with spatial function for measuring the size and dimension, as well as the area of object. Finally, we created the online version of interactive 3D viewer utilizing WebGL API and Mapbox Unity SDK.

Depth-map completion for large indoor scene reconstruction

Traditional Multi View Stereo (MVS) algorithms are often difficult to deal with large-scale indoor scene reconstruction, due to the photo-consistency measurement errors in weak textured regions, which are commonly exist in indoor scenes. To solve this limitation, in this paper we proposed a point cloud completion strategy that combines learning-based depth-map completion and geometry-based consistency filtering to fill large-area missing in depth-maps. The proposed method takes nonuniform and noisy MVS depth-map as input, and completes each depth-map individually. In the completion process, we first complete depth-maps using learning based method, and then filter each depth-map using depth consistency validation with its neighboring depth-maps. This depth-map completion and geometric filtering steps are performed iteratively until the number of depth points is converged. Experiments on large-scale indoor scenes and benchmark MVS datasets demonstrate the effectiveness of the proposed methods.

A Survey on Automated Human Action Recognition Using Multi view Feature

A Survey on Automated Human Action Recognition Using Multi view Feature