Image Mosaic Research Articles

Two-Step Approach toward Alignment of Spatiotemporal Wide-Area Unmanned Aerial Vehicle Imageries

Recently, analysis and decision-making based on spatiotemporal unmanned aerial vehicle (UAV) high-resolution imagery are gaining significant attention in smart agriculture. Constructing a spatiotemporal dataset requires multiple UAV image mosaics taken at different times. Because the weather or a UAV flight trajectory is subject to change when the images are taken, the mosaics are typically unaligned. This paper proposes a two-step approach, composed of global and local alignments, for spatiotemporal alignment of two wide-area UAV mosaics of high resolution. The first step, global alignment, finds a projection matrix that initially maps keypoints in the source mosaic onto matched counterparts in the target mosaic. The next step, local alignment, refines the result of the global alignment. The proposed method splits input mosaics into patches and applies individual transformations to each patch to enhance the remaining local misalignments at patch level. Such independent local alignments may result in new artifacts at patch boundaries. The proposed method uses a simple yet effective technique to suppress those artifacts without harming the benefit of the local alignment. Extensive experiments validate the proposed method by using several datasets for highland fields and plains in South Korea. Compared with a recent work, the proposed method improves the accuracy of alignment by up to 13.21% over the datasets.

Sparse representation-based demosaicking method for joint chromatic and polarimetric imagery

Division of focal plane (DoFP) polarization camera has become mainstream among various polarimetric imaging systems whose sensor is equipped with Bayer filter and directional polarization filter, so joint chromatic and polarimetric image demosaicking is essential. However, since 15 or 14 pixels are missed out of 16 pixels in color polarization mosaic images, it’s challenging to adopt a method to reconstruct full-resolution images well. In this paper, we propose a demosaicking model based on sparse coding. The model introduces RGB-polarization channels correlation, adaptive sub-dictionaries and non-local self-similarity restrictions, and combines them into an optimal problem to solve. The experimental results, including reconstructing full-resolution synthetic images and camera captured real scenes, demonstrate that our proposed method outperforms the current state-of-the-art method in terms of quantitative measures and visual quality.

Learning Domain-Adaptive Landmark Detection-Based Self-Supervised Video Synchronization for Remote Sensing Panorama

The synchronization of videos is an essential pre-processing step for multi-view reconstruction such as the image mosaic by UAV remote sensing; it is often solved with hardware solutions in motion capture studios. However, traditional synchronization setups rely on manual interventions or software solutions and only fit for a particular domain of motions. In this paper, we propose a self-supervised video synchronization algorithm that attains high accuracy in diverse scenarios without cumbersome manual intervention. At the core is a motion-based video synchronization algorithm that infers temporal offsets from the trajectories of moving objects in the videos. It is complemented by a self-supervised scene decomposition algorithm that detects common parts and their motion tracks in two or more videos, without requiring any manual positional supervision. We evaluate our approach on three different datasets, including the motion of humans, animals, and simulated objects, and use it to build the view panorama of the remote sensing field. All experiments demonstrate that the proposed location-based synchronization is more effective compared to the state-of-the-art methods, and our self-supervised inference approaches the accuracy of supervised solutions, while being much easier to adapt to a new target domain.

Fuzzy correlation based algorithm for UAV image mosaic construction

Fuzzy correlation based algorithm for UAV image mosaic construction

DoLP and AoP Synthesis from division of focal plane polarimeters using CycleGAN

Division of focal plane (DoFP) polarimeter brings significant advantages for capturing polarization information, and its unique structure enables temporal synchronization of polarization intensity measurements across scenes. While DoFP can obtain polarization information in all four directions simultaneously, it also reduces the spatial resolution of the image to 1/4 of the camera resolution. In order to solve low resolution and the cumulative error caused by the formula calculation. Using CycleGAN to construct a raw mosaic image-to-polarization properties images model, which differs from general interpolation methods and other existing deep learning methods with paired datasets and takes microgrid images as input and outputs the degree of linear polarization (DoLP) and angle of polarization (AoP) directly by learning the mapping function between unpaired the mosaic images and the polarization properties. In addition, Gradient Magnitude Similarity Deviation (GMSD) metric is applied to preserve satisfactory reconstructed image edge contour information and reduce IFOV artifacts and aliasing. The experimental results demonstrate that the quantitative performance evaluation metrics scores of our method are all higher than other methods and the reconstructed DoLP and AoP images are comparable to the ground truth imagery in terms of visual quality. The end-to-end network structure trained on unpaired datasets solves the severe limitations of paired datasets training, is more suitable for DoLP and AoP reconstructing.

Research on Image Mosaic Method Based on Fracture Edge Contour of Bone Tag

Damaged edges of bone tag images contain external factors such as impurities and damage, which affect the stitching process and lead to repair errors. Therefore, this paper proposes a stitching method based on image edge position feature matching. The objective is to improve the accuracy of image stitching by matching feature points based on the position of the image edge pixel so as to solve the accurate stitching of broken edge contour. In the first step of this method, the image containing the broken edge is preprocessed by edge detection, and the location of the broken edge pixel is proposed. Secondly, the feature descriptors were calculated to extract the shape and texture information of the feature points on the fracture edge. Finally, the feature points are optimized by minimum correction and image mosaic is carried out. In terms of image stitching, pre-registration is performed by finding the feature descriptors that are most similar to the edge of the optimum fracture surface profile. The matching operator is added to the overlapping region to obtain the corrected image, and the panoramic image mosaic of the image fracture surface is performed. The experimental results show that feature descriptor matching can ensure the integrity of the fracture, improve the matching accuracy, optimize the uneven deformation of the fracture, ensure the quality of image stitching, and reduce the degree of image distortion.

Data Quality Management Strategy To Improve Remote Sensing Data Quality: A Case Study On National Remote Sensing Data Bank

National Remote Sensing Data Bank (Bank Data Penginderaan Jauh Nasional; BDPJN) is the operational implementer in the field of remote sensing to meet the needs of national data on remote sensing images. Remote sensing images are essential in the Indonesian government for various needs, such as in regional (spatial planning, city boundaries), disaster (forest fires, floods, landslides, earthquakes, volcanic eruptions), plantation, natural resources, and environment sectors. Quality management of BDPJN’s satellite images becomes challenging due to the increasing number of products owned and users annually. For this reason, a data quality management strategy is needed to guarantee and improve the quality of BDPJN data. To develop such a strategy, an assessment of the maturity of BDPJN’s data quality man- agement was conducted from the aspect of data processing by implementing Loshin’s Data Quality Management Maturity Model (DQM3) to find out the characteristics that were lacking. The results were then mapped based on Data Quality Management (DQM) activities in DAMA-DMBOK as a recommendation for data quality management strategies. This study applies quantitative research where data collection was done by distributing questionnaires to 24 respondents who are data stewards of medium, high, and very high-resolution mosaic images. Based on the assessment, BDPJN is in the maturity level of Defined to Managed. The recommendations are 21 DQM activities that can be carried out to improve BDPJN data quality.

Dynamic Mosaic algorithm for data augmentation.

Convolutional Neural Networks (CNNs) have achieved remarkable results in the computer vision field. However, the newly proposed network architecture has deeper network layers and more parameters, which is more prone to overfitting, resulting in reduced recognition accuracy of the CNNs. To improve the recognition accuracy of the model of image recognition used in CNNs and overcome the problem of overfitting, this paper proposes an improved data augmentation approach based on mosaic algorithm, named Dynamic Mosaic algorithm, to solve the problem of the information waste caused by the gray background in mosaic images. This algorithm improves the original mosaic algorithm by adding a dynamic adjustment step that reduces the proportion of gray background in the mosaic image by dynamically increasing the number of spliced images. Moreover, to relieve the problem of network overfitting, also a Multi-Type Data Augmentation (MTDA) strategy, based on the Dynamic Mosaic algorithm, is introduced. The strategy divides the training samples into four parts, and each part uses different data augmentation operations to improve the information variance between the training samples, thereby preventing the network from overfitting. To evaluate the effectiveness of the Dynamic Mosaic algorithm and the MTDA strategy, we conducted a series of experiments on the Pascal VOC dataset and compared it with other state-of-the-art algorithms. The experimental results show that the Dynamic Mosaic algorithm and MTDA strategy can effectively improve the recognition accuracy of the model, and the recognition accuracy is better than other advanced algorithms.

Progressive Keypoint Detection with Dense Siamese Network for SAR Image Registration

SAR image registration plays a key role in the applications such as change detection, image mosaic, which is one of the most challenging tasks in recent years due to speckle noise, geometric distortion and nonlinear radiation differences between SAR images. In the feature-based SAR image registration methods, the repeatability of keypoints and the effectiveness of feature descriptors could directly affect the registration accuracy. In this paper, we propose a stable feature intersection-based (FI) keypoint detector, which contains three progressive detectors, i.e., phase coherency (PC) detector, horizontal/vertical and major/minor diagonal oriented gradient detectors, and local coefficient of variation (LCoV) detector. Our proposed keypoint detector can not only effectively extract keypoints with high repeatability, but also greatly reduce the number of false keypoints, thus reducing the computational cost of feature description and matching. Then we propose the Cross Stage Partial Siamese Network (CSPSNet) to rapidly extract feature descriptors containing both deep and shallow features, which can be used to obtain more correct matching point pairs than traditional man-made shallow descriptors. In addition, we design a Hard Example Mining Loss (HEMLoss) to minimize the matching distance between the matching descriptor and adaptive selected non-matching descriptors. Experimental results on different pairs of SAR images demonstrate that our proposed method achieves better performance than other state-of-the-art methods.

Crimean Mosaic: Spatial Analysis of the Structure and Dynamics of the Regional Museum Network

Referring to the data of the author’s database of Crimean museums, this article offers a spatial analysis of the structure and dynamics of the museum network. The official digital resources (websites) of the Ministry of Culture of the Russian Federation, the Museums of Russia portal, the website of the Private and Folk Museums of Russia, and the websites of municipalities are used as the main sources of information. The database information makes it possible to reconstruct the images of the region, taking into account their geographic reference. The museum landscape of the region forms a polyphonic historical and cultural image of Crimea, which is divided into five clusters (zones): literary and historical (Southern coast of Crimea); military-historical (Sevastopol); historical and local history (steppe Crimea); historical and archaeological (Kerch Peninsula); historical and ethnographic (mountainous Crimea). Clusters differ in the structure of the museum network and the historical dynamics of its formation. The author analyses the features of the formation of each museum cluster and the images of the past it broadcasts. The image of the war is most intensively present in the museums of Sevastopol and Balaklava as the places of the deployment of the Black Sea Fleet. Literary Crimea is concentrated in cities and urban-type settlements of the southern coast of Crimea, where many famous writers rested, were treated, and created their works. Images of the Soviet past are disseminated by local history museums of the steppe Crimea. The ancient civilization and medieval Crimea are represented in the Kerch cluster and partly by the historical and archaeological museums of the South Coast and the mountainous Crimea. And, finally, the national (Crimean-Tatar) image of Crimea is preserved in the museums of Bakhchisaray and Simferopol. Such a complex mosaic of Crimean images is the result of the influence of external and internal factors — the richness and diversity of the historical and cultural heritage of the region formed over the centuries and the zigzags of the state cultural policy. The change of statehood over the twentieth century (imperial, Soviet, post-Soviet — Ukrainian and Russian — periods) has left its mark on the museum space of Crimea.

Retracted: Image Mosaic Algorithm-Based Analysis of Pathological Characteristics of Gastric Polyp Patients Using Computed Tomography Images.

[This retracts the article DOI: 10.1155/2021/6086106.].

Efficient Global Color, Luminance, and Contrast Consistency Optimization for Multiple Remote Sensing Images

Light and color uniformity is essential for the production of high-quality remote-sensing image mosaics. Existing color correction methods mainly use flexible models to express the color differences between multiple images and impose specific constraints (e.g., image gradient or contrast constraints) to preserve image texture information as much as possible. Due to these constraints, it is usually difficult to correct for the differences in texture between images during image processing. We propose a method that can optimize the luminance, contrast, and color difference of remote-sensing images. In the YCbCr color space, this method processes the chrominance and luminance channels of the image. This is conducive to reducing the influence of the different channels. In the luminance channel, the block-based Wallis transform method is used to optimize the luminance and contrast of the image. In the chromaticity channel, to optimize the color differences, a spline curve is used as a model; the color differences are formulated as a cost function and solved using convex quadratic programming. Moreover, considering the efficiency of our method, we use a graphics processing unit to make the algorithm parallel. The proposed method has been tested on several challenging datasets that cover different topographic regions. In terms of visuals and quality indicators, it shows better results than state-of-the-art approaches.

深度学习和APAP模型结合的大视差图像拼接算法

深度学习和APAP模型结合的大视差图像拼接算法

Unmanned Aerial Vehicle (UAV) in Precision Agriculture to Identify the Crop Water Shortage by Using Multi-Spectral Sensor

Accurate diagnosis of crop water shortage and scientific irrigation decisions are crucial. A new method of multi-spectral imaging remote sensing image extraction of tea canopy temperature is proposed, and an automatic processing system of remote sensing thermal imaging images is established. In this short communication, A UAV (Unmanned Aerial Vehicle) with multi-spectral sensors used to capture the images. The research results show that the system can efficiently mosaic images without image gap, and ensure that the soil background is wholly eliminated. This research gives us new methods to set an intelligent method for precision agriculture, which greatly improves the level of agricultural intelligence.

Detection of the Land Surface Temperature Changes in Ma’an Governorate using Remote Sensing Data during the Period (1990-2018)

This paper investigates multi-temporal land surface temperature (LST) for large ungauged areas of Ma’an Governorate, Jordan, based on changes in normalized difference vegetation index (NDVI) using remotely sensed data. Five mosaic images were taken for both the 1990 and 2018 years by Landsat 5 (TM) and Landsat 8 (OLI) (path/row, 174/36- 174/37). These were used as the basic data source, where most of Ma’an Governorate has no meteorological stations. The five-mosaic images for both Landsat 5 and Landsat 8-OLI were taken in September. LST and NDVI maps have been generated to determine the changes in LTS during the monitoring period. The results showed that the minimum value of LST increased by 4°C, and the mean surface temperature increased nearly by 2°C between 1990 and 2018. The average LST has been rising at a rate of 0.071°C/y.

A Parallax Image Mosaic Method for Low Altitude Aerial Photography with Artifact and Distortion Suppression.

In this paper, we propose an aerial images stitching method based on an as-projective-as-possible (APAP) algorithm, aiming at the problem artifacts, distortions, or stitching failure due to fewer feature points for multispectral aerial image with certain parallax. Our method incorporates accelerated nonlinear diffusion algorithm (AKAZE) into APAP algorithm. First, we use the fast and stable AKAZE to extract the feature points of aerial images, and then, based on the registration model of the APAP algorithm, we add line protection constraints, global similarity constraints, and local similarity constraints to protect the image structure information, to produce a panorama. Experimental results on several datasets demonstrate that proposed method is effective when dealing with multispectral aerial images. Our method can suppress artifacts, distortions, and reduce incomplete splicing. Compared with state-of-the-art image stitching methods, including APAP and adaptive as-natural-as-possible image stitching (AANAP), and two of the most popular UAV image stitching tools, Pix4D and OpenDroneMap (ODM), our method achieves them both quantitatively and qualitatively.

An Upscaling–Downscaling Optimal Seamline Detection Algorithm for Very Large Remote Sensing Image Mosaicking

For the mosaicking of multiple remote sensing images, obtaining the optimal stitching line in the overlapping region is a key step in creating a seamless mosaic image. However, for very large remote sensing images, the computation of finding seamlines involves a huge amount of image pixels. To handle this issue, we propose a stepwise strategy to obtain pixel-level optimal stitching lines for large remote sensing images via an upscaling–downscaling image sampling procedure. First, the resolution of the image is reduced and the graph cut algorithm is applied to find an energy-optimal seamline in the reduced image. Then, a stripe along the preliminary seamline is identified from the overlap area to remove the other inefficient nodes. Finally, the graph cut algorithm is applied nested within the identified stripe to seek the pixel-level optimal seamline of the original image. Compared to the existing algorithms, the proposed method produces fewer spectral differences between stitching lines and less-crossed features in the experiments. For a wide range of remote sensing images involving large data, the new method uses less than 10 percent of the time needed by the SLIC+ graph cut method.

Calligraphy and Painting Identification 3D-CNN Model Based on Hyperspectral Image MNF Dimensionality Reduction.

As a kind of cultural art, calligraphy and painting are not only an important part of traditional culture but also has important value of art collection and trade. The existence of forgeries has seriously affected the fair trade, protection, and inheritance of calligraphy and painting. There is an urgent need for the efficient, accurate, and intelligent technical identification method. By combining the advantages of material attribute recognition and imaging detection of hyperspectral imaging technology with the powerful feature expression ability and classification ability of the convolutional neural network, it can greatly improve the comprehension efficiency of calligraphy and painting identification; meanwhile, in order to reduce the redundancy and the amount of parameters in the method of directly using the hyperspectral image, an objective convex dimensionality reduction method should be used for compressing the original hyperspectral image before deep learning. Based on this, we propose a kind of deep learning method to classify author and authenticity based on the multichannel images obtained by minimum noise fraction (MNF) dimensionality reduction to calligraphy and painting hyperspectral data, and its core is the 2D-CNN or 3D-CNN model with the basic network of "4 convolution layers + 4 pooling layers + 2 full-link layers." The experimental results show that the identification accuracy of the 2D-CNN calligraphy and painting identification with MNF pseudocolor image mosaic as input and the 2D-CNN calligraphy and painting identification with multichannel MNF dimensionality reduced images direct as input have high accuracy, while the 3D-CNN calligraphy and painting identification with multichannel MNF dimensionality reduced images direct as input not only maintains excellent identification accuracy but also has better learning convergence (step number) and stability compared with the 2D-CNN model. Especially, the 3D-CNN identification accuracy of calligraphy and painting's author and authenticity on the test set can reach 93.2% and 95.2%, respectively.

A snapshot near-infrared hyperspectral demosaicing method with convolutional neural networks in low illumination environment

Snapshot near-infrared hyperspectral imaging technology based on multispectral color filter array has been widely utilized in quantitative and visualized analysis. Hyperspectral demosaicing method is a key part to increase spatial resolution for the snapshot hyperspectral imaging. However, the existing demosaicing methods will be disturbed by noise in case of the low illumination. In this paper, a snapshot hyperspectral demosaicing method is proposed to reconstruct raw mosaic images into full-resolution hyperspectral images. The demosaicing process is integrated with Convolutional Neural Network (CNN) to extract features automatically, guaranteeing a good performance and strong robustness without parameter adjustment. We experimentally validate the merits of the method by 5 × 5 raw mosaic images. The Peak Signal-to-Noise Ratio (PSNR) and Spectral Angle Mapper (SAM) values of reconstructed hyperspectral images can be improved by the proposed method, which has been successfully used in commercial hyperspectral cameras.

A Novel Fingerprint Encryption Based on Image and Feature Mosaic

A Novel Fingerprint Encryption Based on Image and Feature Mosaic