Image Mosaic Research Articles

Quantifying misidentification rates of Europa's microfeatures in low-resolution Galileo imaging

Europa is a compelling target because of its unique geology and potential for habitability. Efforts to map the small-scale global geology of the moon have been hampered by the availability of suitable imaging (≤ 230 m/pix), which covers only 15% of the surface. New data on the abundance of microfeatures (features ≤100 km2 in area) on a global scale and their locations would help put better constraints on the geophysics of the icy world. We mapped microfeatures in low-resolution images of the E15RegMap01 region of Europa and compared our low-resolution dataset to a validated dataset collected using regional mosaic imaging. We evaluated how many features were missed in low-resolution mapping, the feature types to which they belonged, how often misclassifications were made, and how many false-positive features there were in the low-resolution dataset. We found that microchaos, hybrids, and spots were found at relatively high completeness rates when compared to the regional mosaic dataset, but domes and especially pits were almost always missed. Microchaos, hybrids, and spots also had the highest rates of consistent classification between the low-resolution and regional mosaic datasets. Finally, 36% of our total potential feature count from the low-resolution dataset were revealed to be false positives, mostly caused by ridges and shadows that were mistaken for features in low-resolution imaging. These findings quantitatively estimate the likely errors in existing and future global geologic maps that note the presence of microfeatures, offer remedies to minimize these errors, and provide guidance for future Europa mappers.

High frame rate video mosaicking microendoscope to image large regions of intact tissue with subcellular resolution.

High-resolution microendoscopy (HRME) is a low-cost strategy to acquire images of intact tissue with subcellular resolution at frame rates ranging from 11 to 18 fps. Current HRME imaging strategies are limited by the small microendoscope field of view (∼0.5 mm2); multiple images must be acquired and reliably registered to assess large regions of clinical interest. Image mosaics have been assembled from co-registered frames of video acquired as a microendoscope is slowly moved across the tissue surface, but the slow frame rate of previous HRME systems made this approach impractical for acquiring quality mosaicked images from large regions of interest. Here, we present a novel video mosaicking microendoscope incorporating a high frame rate CMOS sensor and optical probe holder to enable high-speed, high quality interrogation of large tissue regions of interest. Microendoscopy videos acquired at >90 fps are assembled into an image mosaic. We assessed registration accuracy and image sharpness across the mosaic for images acquired with a handheld probe over a range of translational speeds. This high frame rate video mosaicking microendoscope enables in vivo probe translation at >15 millimeters per second while preserving high image quality and accurate mosaicking, increasing the size of the region of interest that can be interrogated at high resolution from 0.5 mm2 to >30 mm2. Real-time deployment of this high-frame rate system is demonstrated in vivo and source code made publicly available.

Image stitching based on angle-consistent warping

Many warping methods for image stitching have been proposed to construct panoramic image mosaics free of artifacts. Existing methods heavily rely on coordinate correspondences between keypoints in stitching, which may not provide adequate constraints for alignment. In this paper, we discover and employ a new constraint — angle correspondences to address the above problem. The angle of a feature point represents the local directional structure of the point, which is an extension to its position and customarily ignored in image stitching. We propose to jointly consider the coordinates as well as the angles in keypoint correspondences. Such a strategy helps to generate a correct warping in the overlapping regions of the stitched image. In addition, we propose a novel constraint — mesh angle preservation to prevent undesired distortion in non-overlapping areas. Experiments in several challenging cases demonstrate that our method yields more accurate results with significantly less artifacts in comparison with state-of-the-art methods.

SEAMLESS MOSAIC USING REMOVAL SUNGLINT TO SUPPORT A LARGE-SCALE MAPPING ACCELERATION

Mosaics of remote sensing images to support the acceleration of large-scale mapping are one of the steps in the data preparation process for dissemination to users, where generally users need seamless, mosaic images, especially on the land area. To produce a seamless mosaic image on the land area, it is sometimes constrained by the data that contains sunglints due to the direction of the recording that is opposite to the direction to the sun which causes the mosaic results to look not uniform in color on the land area. In this study, mosaics were carried out in the Pacitan area using Pleiades satellite data. From the existing problems, this study aims to compare the results of the mosaic image by removing sunglint compared to mosaic without removing sunglint. The results of this study indicate that the mosaic image by
 removing the sunglint produces a more seamless mosaic than the mosaic without removing the sunglint.

Measuring loblolly pine crowns with drone imagery through deep learning

In modeling forest stand growth and yield, crown width, a measure for stand density, is among the parameters that allows for estimating stand timber volumes. However, accurately measuring tree crown size in the field, in particular for mature trees, is challenging. This study demonstrated a novel method of applying machine learning algorithms to aerial imagery acquired by an unmanned aerial vehicle (UAV) to identify tree crowns and their widths in two loblolly pine plantations in eastern Texas, USA. An ortho mosaic image derived from UAV-captured aerial photos was acquired for each plantation (a young stand before canopy closure, a mature stand with a closed canopy). For each site, the images were split into two subsets: one for training and one for validation purposes. Three widely used object detection methods in deep learning, the Faster region-based convolutional neural network (Faster R-CNN), You Only Look Once version 3 (YOLOv3), and single shot detection (SSD), were applied to the training data, respectively. Each was used to train the model for performing crown recognition and crown extraction. Each model output was evaluated using an independent test data set. All three models were successful in detecting tree crowns with an accuracy greater than 93%, except the Faster R-CNN model that failed on the mature site. On the young site, the SSD model performed the best for crown extraction with a coefficient of determination (R2) of 0.92, followed by Faster R-CNN (0.88) and YOLOv3 (0.62). As to the mature site, the SSD model achieved a R2 as high as 0.94, follow by YOLOv3 (0.69). These deep leaning algorithms, in particular the SSD model, proved to be successfully in identifying tree crowns and estimating crown widths with satisfactory accuracy. For the purpose of forest inventory on loblolly pine plantations, using UAV-captured imagery paired with the SSD object detention application is a cost-effective alternative to traditional ground measurement.

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

Ghosts are a common phenomenon widely present in unmanned aerial vehicle (UAV) remote sensing image stitching that seriously affect the naturalness of stitching results. In order to effectively remove ghosts and produce visually natural stitching results, we propose a novel image stitching method that can identify and eliminate ghosts through multi-component collaboration without object distortion, segmentation or repetition. Specifically, our main contributions are as follows: first, we propose a ghost identification component to locate a potential ghost in the stitching area; and detect significantly moving objects in the two stitched images. In particular, due to the characteristics of UAV shooting, the objects in UAV remote sensing images are small and the image quality is poor. We propose a mesh-based image difference comparison method to identify ghosts; and use an object tracking algorithm to accurately correspond to each ghost pair. Second, we design an image information source selection strategy to generate the ghost replacement region, which can replace the located ghost and avoid object distortion, segmentation and repetition. Third, we find that the process of ghost elimination can produce natural mosaic images by eliminating the ghost caused by initial blending with selected image information source. We validate the proposed method on VIVID data set and compare our method with Homo, ELA, SPW and APAP using the peak signal to noise ratio (PSNR) evaluation indicator.

Seamless 3D Image Mapping and Mosaicing of Valles Marineris on Mars Using Orbital HRSC Stereo and Panchromatic Images

A seamless mosaic has been constructed including a 3D terrain model at 50 m grid-spacing and a corresponding terrain-corrected orthoimage at 12.5 m using a novel approach applied to ESA Mars Express High Resolution Stereo Camera orbital (HRSC) images of Mars. This method consists of blending and harmonising 3D models and normalising reflectance to a global albedo map. Eleven HRSC image sets were processed to Digital Terrain Models (DTM) based on an opensource stereo photogrammetric package called CASP-GO and merged with 71 published DTMs from the HRSC team. In order to achieve high quality and complete DTM coverage, a new method was developed to combine data derived from different stereo matching approaches to achieve a uniform outcome. This new approach was developed for high-accuracy data fusion of different DTMs at dissimilar grid-spacing and provenance which employs joint 3D and image co-registration, and B-spline fitting against the global Mars Orbiter Laser Altimeter (MOLA) standard reference. Each HRSC strip is normalised against a global albedo map to ensure that the very different lighting conditions could be corrected and resulting in a tiled set of seamless mosaics. The final 3D terrain model is compared against the MOLA height reference and the results shown of this intercomparison both in altitude and planum. Visualisation and access mechanisms to the final open access products are described.

Image Mosaic in Visual Detection of Wellbore

Based on the research on the visual detection system, this paper analyzes the characteristics of wellbore video (image), proposes the key techniques of image processing, and designs the methods of image mosaic. The experiment results show that when the image correction, the perspective transform, the image registration, and image fusion are implemented, a panoramic image of wellbore can be obtained. This study provides a new perspective for the visual detection and the technology reserves for further quantitative evaluation of visual detection.

Illumination Correction of Multi-Time RGB Images Obtained with an Unmanned Aerial Vehicle

The goal of this research was to study and optimize multi-temporal RGB images obtained by a UAV (unmanned aerial vehicle). A digital camera onboard the UAV allows obtaining data with a high temporal and spatial resolution of ground objects. In the case considered by us, the object of study is agricultural fields, for which, based on numerous images covering the agricultural field, image mosaics (orthomosaics) are constructed. The acquisition time for each orthomosaic takes at least several hours, which imposes a change in the illuminance of each image, when considered separately. Orthomosaics obtained in different periods of the year (several months) will also differ from each other in terms of illuminance. For a comparative analysis of different parts of the field (orthomosaic), obtained in the same time interval or comparison of areas for different periods of time, their alignment by illumination is required. Currently, the majority of alignment approaches rely rather on colour (RGB) methods, which cannot guarantee finding efficient solutions, especially when it is necessary to obtain a quantitative result. In the paper, a new method is proposed that takes into account the change in illuminance during the acquisition of each image. The general formulation of the problem of light correction of RGB images in terms of assessing the colour vegetation index Greenness is considered. The results of processing real measurements are presented.

Transmission line image mosaic based on improved SIFT-LATCH

In the transmission line inspection, in order to obtain wide-view and large-scale images, it is necessary to stitch images taken by drones. This article proposes an improved SIFT-LATCH-based UAV aerial transmission line image stitching algorithm. First, the image is processed in parallel through multi-threading. The SIFT algorithm is used to extract the feature points in the image, and the LATCH descriptor is used to describe the feature points. The sampling consensus algorithm is used to purify the feature points, and finally the translational homography matrix is used to solve the problem of black lines in the stitched image. Experiments show that the algorithm proposed in the article has the advantages of good real-time and high accuracy in splicing UAV aerial pictures of transmission lines.

Workflow for the Generation of Expert-Derived Training and Validation Data: A View to Global Scale Habitat Mapping

Our ability to completely and repeatedly map natural environments at a global scale have increased significantly over the past decade. These advances are from delivery of a range of on-line global satellite image archives and global-scale processing capabilities, along with improved spatial and temporal resolution satellite imagery. The ability to accurately train and validate these global scale-mapping programs from what we will call “reference data sets” is challenging due to a lack of coordinated financial and personnel resourcing, and standardized methods to collate reference datasets at global spatial extents. Here, we present an expert-driven approach for generating training and validation data on a global scale, with the view to mapping the world’s coral reefs. Global reefs were first stratified into approximate biogeographic regions, then per region reference data sets were compiled that include existing point data or maps at various levels of accuracy. These reference data sets were compiled from new field surveys, literature review of published surveys, and from individually sourced contributions from the coral reef monitoring and management agencies. Reference data were overlaid on high spatial resolution satellite image mosaics (3.7 m × 3.7 m pixels; Planet Dove) for each region. Additionally, thirty to forty satellite image tiles; 20 km × 20 km) were selected for which reference data and/or expert knowledge was available and which covered a representative range of habitats. The satellite image tiles were segmented into interpretable groups of pixels which were manually labeled with a mapping category via expert interpretation. The labeled segments were used to generate points to train the mapping models, and to validate or assess accuracy. The workflow for desktop reference data creation that we present expands and up-scales traditional approaches of expert-driven interpretation for both manual habitat mapping and map training/validation. We apply the reference data creation methods in the context of global coral reef mapping, though our approach is broadly applicable to any environment. Transparent processes for training and validation are critical for usability as big data provide more opportunities for managers and scientists to use global mapping products for science and conservation of vulnerable and rapidly changing ecosystems.

Deep learning‐based classification and instance segmentation of leakage‐area and scaling images of shield tunnel linings

This paper presents an approach for the integrated process of classification and instance segmentation of leakage-area and scaling images from shield tunnel linings. For this purpose, the previously established dataset of leakage-area images by the authors is enlarged by means of adding scaling ones. Afterwards, data augmentation is implemented to enrich the database in the classification dataset, and the augmented classification dataset contains 5776 images. The instance segmentation dataset is subsequently enlarged through original images without any data augmentation, including 1496 images. Then a residual net with 101 layers (i.e., ResNet-101) is applied to the classification dataset to obtain a model that can identify leakage-area and scaling images from those of shield tunnel linings. The ResNet-101 classification model achieves an accuracy of 93.37% in terms of testing classification dataset. Moreover, a mask region-based convolutional neural network (Mask R-CNN) is utilized to perform instance segmentation of leakage areas and scaling in the images classified by the ResNet-101 model. The segmentation results of the Mask R-CNN model show 96.1% and 95.6% average precision (AP) with intersection over union (IoU) of 0.5 for bounding box and mask predication, respectively. By using the proposed approach, the leakage-area and scaling defects can be automatically classified and quantified with an overall accuracy of 89.3%, which is quite promising compared to the inherent uncertainty in geotechnical engineering. Image mosaicing is finally applied to provide inspectors with the intuitive observation of the location and distribution information of defects on the tunnel lining.

Research and Implementation of Digital Microscope Image Acquisition System Based on Embedded Linux

<p style="text-align: justify;"><span style="font-family: "times new roman";">With the rapid development of multimedia and Internet in today's society, embedded Linux system also involves many places, and the application of digital microscope is also reused in many industrial manufacturing areas. The main purpose of this paper is to complete the research and implementation of digital microscope image acquisition system based on Embedded Linux, including image acquisition card, logic function, hardware circuit design and image acquisition system. In this paper, the CMOS digital camera is used to connect the CMOS digital camera with the computer through the transmission network, so as to collect the data. In the image preprocessing, the adaptive method based on mean and variance is implemented to correct the gray level of the collected image to avoid high error. In the image registration, the registration algorithm is used to smooth the mosaic place. Then, Sobel algorithm is used to carry out the color transition step by step to eliminate the gap in the image and realize the microscopic image mosaic. The results are as follows: the overall design scheme of the system is designed; the circuit board of digital microscope image acquisition and processing unit based on Embedded Linux is realized; image mosaic is realized by using image preprocessing, registration algorithm and fusion technology, In order to improve the quality of digital microscope image acquisition; through the feasibility test of the system, it is concluded that the performance of the embedded Linux digital microscope image acquisition system proposed in this paper is good, which is worthy of wide application.</span>

Adaptive Multi-View Image Mosaic Method for Conveyor Belt Surface Fault Online Detection

In order to improve the accuracy and real-time of image mosaic, realize the multi-view conveyor belt surface fault online detection, and solve the problem of longitudinal tear of conveyor belt, we in this paper propose an adaptive multi-view image mosaic (AMIM) method based on the combination of grayscale and feature. Firstly, the overlapping region of two adjacent images is preliminarily estimated by establishing the overlapping region estimation model, and then the grayscale-based method is used to register the overlapping region. Secondly, the image of interest (IOI) detection algorithm is used to divide the IOI and the non-IOI. Thirdly, only for the IOI, the feature-based partition and block registration method is used to register the images more accurately, the overlapping region is adaptively segmented, the speeded up robust features (SURF) algorithm is used to extract the feature points, and the random sample consensus (RANSAC) algorithm is used to achieve accurate registration. Finally, the improved weighted smoothing algorithm is used to fuse the two adjacent images. The experimental results showed that the registration rate reached 97.67%, and the average time of stitching was less than 500 ms. This method is accurate and fast, and is suitable for conveyor belt surface fault online detection.

Panoramic Stereo Imaging of a Bionic Compound-Eye Based on Binocular Vision.

With the rapid development of the virtual reality industry, one of the bottlenecks is the scarcity of video resources. How to capture high-definition panoramic video with depth information and real-time stereo display has become a key technical problem to be solved. In this paper, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed. Combined with the real-time processing algorithm of multi detector mosaic panoramic stereo imaging image, a panoramic stereo real-time imaging system is developed. Firstly, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed, and the space coordinate calibration platform of ultra-high precision panoramic camera based on theodolite angle compensation function is constructed. The projection matrix of adjacent cameras is obtained by solving the imaging principle of binocular stereo vision. Then, a real-time registration algorithm of multi-detector mosaic image and Lucas-Kanade optical flow method based on image segmentation are proposed to realize stereo matching and depth information estimation of panoramic imaging, and the estimation results are analyzed effectively. Experimental results show that the stereo matching time of panoramic imaging is 30 ms, the registration accuracy is 0.1 pixel, the edge information of depth map is clearer, and it can meet the imaging requirements of different lighting conditions.

Seamless maps of major elements of the Moon: Results from high-resolution geostationary satellite

Major elements such as Fe, Ti, Mg, Al, Ca and Si play very important roles in understanding the origin and evolution of the Moon. Previous maps of these major elements derived from orbital data are based on mosaic images or low-resolution gamma-ray data. The hue variations and gaps among orbital boundaries in the mosaic images are not conducive to geological studies. This paper aims to produce seamless and homogenous distribution maps of major elements using the single-exposure image of the whole lunar disk obtained by China’s high-resolution geostationary satellite, Gaofen-4, with a spatial resolution of ∼ 500 m. The elemental contents of soil samples returned by Apollo and Luna missions are regarded as ground truth, and are correlated with the reflectance of the sampling sites extracted from Gaofen-4 data. The final distribution maps of these major oxides are generated with the statistical regression model. With these products, the average contents and proportions of the major elements for maria and highlands were estimated and compared. The results showed that SiO2 and TiO2 have the highest and lowest fractions in mare and highland areas, respectively. Moreover, the relative concentrations of these elements could serve as indicators of geologic processes, e.g., the obviously asymmetric distributions of Al2O3, CaO and SiO2 around Tycho crater may suggest that Tycho crater was formed by an oblique impact from the southwest direction.

Multi-FAN: multi-spectral mosaic super-resolution via multi-scale feature aggregation network

This paper introduces a novel method to super-resolve multi-spectral images captured by modern real-time single-shot mosaic image sensors, also known as multi-spectral cameras. Our contribution is twofold. Firstly, we super-resolve multi-spectral images from mosaic images rather than image cubes, which helps to take into account the spatial offset of each wavelength. Secondly, we introduce an external multi-scale feature aggregation network (Multi-FAN) which concatenates the feature maps with different levels of semantic information throughout a super-resolution (SR) network. A cascade of convolutional layers then implicitly selects the most valuable feature maps to generate a mosaic image. This mosaic image is then merged with the mosaic image generated by the SR network to produce a quantitatively superior image. We apply our Multi-FAN to RCAN (residual channel attention network), which is the state-of-the-art SR algorithm. We show that Multi-FAN improves both quantitative results (PSNR and SSIM) and inference time.

Historical air photo missions in the Maritimes during the early 1920s: coverage, thematic scope, and utility 100 years later

The historical and technological developments of powered flight and aerial photography have early connections in the Maritimes. Following the Great War (1914-18), a series of pioneering survey missions were initiated by the Canada Air Board in the civilian domain. From a science perspective, the air photos offer a unique opportunity for the detection of environmental change at an unusual centennial time scale. The missions of the early 1920s initially relied on military surplus seaplanes and innovative camera equipment that yielded several thousand high-resolution vertical air photos. This paper is focussed on the scope and outcome of the first experiments carried out over Nova Scotia and New Brunswick between 1921 and 1925, prior to more systematic use for topographic mapping during the remainder of that decade. The research is based on archival records and partial reconstruction of the digitized air photos into image mosaics. Photo interpretation and comparison with recent high-resolution satellite imagery offer insights concerning land use and land cover changes, coastal dynamics, and transformation of urban, rural and industrial landscapes. Experience todate with these early air photos and mosaics of the Maritimes holds promise for examining similar aerial survey missions in other parts of Canada.

Uso e ocupação do solo do município de novo progresso no Estado do Pará-Brasil

O objetivo foi analisar o processo de uso e ocupação do solo do município de Novo Progresso no Estado do Pará, interligando-o com as atividades de maior importância econômica desenvolvidas nesta região. Utilizou-se o shapefile de limite do município de Novo Progresso na plataforma online Google Earth Engine (GEE), que disponibilizou um mosaico de imagens orbitais, do satélite Landsat-8/OLI-TIRS, referentes ao ano de 2019. O processo de classificação foi feito a partir do Code Editor do GEE, utilizando um Índice espectral de vegetação para auxiliar a classificação (Normalized Difference Vegetation Index – NDVI). Foi utilizado o Software QGis 3.10.6 para elaborar os mapas de localização do município e o de classificação de uso e cobertura do solo. Os dados foram tabulados em planilhas para determinar as taxas de crescimento do período analisado. Para realizar a avaliação da confiabilidade da classificação foi utilizado o método de Exatidão Global e o Índice Kappa. Foi possível identificar que no ano de 2019, houve a incidência de 3.064.396,65 ha (80,3%) de floresta densa, uma área de 496.104,07 ha (13,0%) com solo exposto, 248.052,03 ha (6,5%) de floresta secundária, e apenas 7.632,37 ha (0,2%) com predominância de hidrografia, totalizando uma área de 3.816.185,13 ha. As áreas que encontram-se com o solo exposto não estão diretamente relacionadas com o crescimento populacional, mas sim a forma como é estabelecido o uso do solo, com base nas principais atividades desenvolvidas na região considerando que a lógica produtiva ocorre de forma desordenada, não respeitando os critérios de desenvolvimento sustentável.

Development of spontaneous vegetation on reclaimed land in Singapore measured by NDVI.

Population and economic growth in Asia has led to increased urbanisation. Urbanisation has many detrimental impacts on ecosystems, especially when expansion is unplanned. Singapore is a city-state that has grown rapidly since independence, both in population and land area. However, Singapore aims to develop as a 'City in Nature', and urban greenery is integral to the landscape. While clearing some areas of forest for urban sprawl, Singapore has also reclaimed land from the sea to expand its coastline. Reclaimed land is usually designated for future urban development, but must first be left for many years to stabilise. During the period of stabilisation, pioneer plant species establish, growing into novel forest communities. The rate of this spontaneous vegetation development has not been quantified. This study tracks the temporal trends of normalized difference vegetation index (NDVI), as a proxy of vegetation maturity, on reclaimed land sensed using LANDSAT images. Google Earth Engine was used to mosaic cloud-free annual LANDSAT images of Singapore from 1988 to 2015. Singapore's median NDVI increased by 0.15 from 0.47 to 0.62 over the study period, while its land area grew by 71 km2. Five reclaimed sites with spontaneous vegetation development showed variable vegetation covers, ranging from 6% to 43% vegetated cover in 2015. On average, spontaneous vegetation takes 16.9 years to develop to a maturity of 0.7 NDVI, but this development is not linear and follows a quadratic trajectory. Patches of spontaneous vegetation on isolated reclaimed lands are unlikely to remain forever since they are in areas slated for future development. In the years that these patches exist, they have potential to increase urban greenery, support biodiversity, and provide a host of ecosystem services. With this knowledge on spontaneous vegetation development trajectories, urban planners can harness the resource when planning future developments.