Image Mosaic Research Articles

Research on the analysis method of digital media art communication based on 3D image recognition

3D image has important applications in reverse engineering, pattern recognition, artificial intelligence and other fields. 3D image processing technology has become a research hotspot in the field of 3D image, and the dissemination of digital media technology has gradually become a key application content. Therefore, this paper mainly studies the key technologies such as 3D image mosaic, surface reconstruction and model simplification, studies the 2D image filtering, and analyzes the transmission of 3D technology and digital media technology. It can be seen from an example that digital media communication has a good application effect in 3D technology.

Optical biopsy mapping on endoscopic image mosaics with a marker-free probe

Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope.

Application of Image Mosaic Technology in Tai Chi Animation Creation.

Panorama can reflect the image seen at any angle of view at a certain point of view. How to improve the quality of panorama stitching and use it as a data foundation in the “smart tourism” system has become a research hotspot in recent years. Image stitching means to use the overlapping area between the images to be stitched for registration and fusion to generate a new image with a wider viewing angle. This article takes the production of “Tai Chi” animation as an example to apply image stitching technology to the production of realistic 3D model textures to simplify the production of animation textures. A handheld camera is used to collect images in a certain overlapping area. After cylindrical projection, the Harris algorithm based on scale space is adopted to detect image feature points, the two-way normalized cross-correlation algorithm matches the feature points, and the algorithm to extract the threshold T iteratively removes mismatches. The transformation parameter model is quickly estimated through the improved RANSAC algorithm, and the spliced image is projected and transformed. The Szeliski grayscale fusion method directly calculates the grayscale average of the matching points to fuse the image, and finally, the best stitching method is used to eliminate the ghosting at the image mosaic. Data experiments based on Matlab show that the proposed image splicing technology has the advantages of high efficiency and clear spliced images and a more satisfactory panoramic image visual effect can be achieved.

Research on image stitching method based on improved ORB and stitching line calculation

Image matching is one of the hot issues in recent years. Image matching determines the effect of image mosaic to a large extent. Therefore, a stable, accurate, and fast image matching algorithm is very important for the subsequent processing of image information. Most of the traditional image matching methods have the problems of high-error matching rate and difficulty in to eliminating false matching. To resolve the shortcomings listed here, an improved algorithm was proposed in which a combined measure of Hamming distance. Similarity measurement with Tanimoto similarity measurement was adopted to dispose binary feature vectors. In addition, as an important step in the process of image mosaics, the calculation of the best mosaics can eliminate the ghosting and ghosting that may appear in the mosaics. However, the traditional dynamic planning method is easy to pass through the obstacles and form obvious stitching traces when searching for stitching lines in images with large obstacles. In view of the aforementioned content, we propose a fast and robust method to search the image difference matrix and structure difference matrix of the overlapping area of the input image according to the search strategy to determine the pixel coordinates of the splicing line. Finally, from an optimal splicing line, the experimental results show that our algorithm performs well for eliminating error matching points. Compared with traditional algorithms, matching accuracy is improved by 25%. In terms of calculating the optimal splicing line, the method proposed is faster than the traditional method and has good robustness.

Subpixel-Scale Topography Retrieval of Mars Using Single-Image DTM Estimation and Super-Resolution Restoration

We propose using coupled deep learning based super-resolution restoration (SRR) and single-image digital terrain model (DTM) estimation (SDE) methods to produce subpixel-scale topography from single-view ESA Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) and NASA Mars Reconnaissance Orbiter High Resolution Imaging Science Experiment (HiRISE) images. We present qualitative and quantitative assessments of the resultant 2 m/pixel CaSSIS SRR DTM mosaic over the ESA and Roscosmos Rosalind Franklin ExoMars rover’s (RFEXM22) planned landing site at Oxia Planum. Quantitative evaluation shows SRR improves the effective resolution of the resultant CaSSIS DTM by a factor of 4 or more, while achieving a fairly good height accuracy measured by root mean squared error (1.876 m) and structural similarity (0.607), compared to the ultra-high-resolution HiRISE SRR DTMs at 12.5 cm/pixel. We make available, along with this paper, the resultant CaSSIS SRR image and SRR DTM mosaics, as well as HiRISE full-strip SRR images and SRR DTMs, to support landing site characterisation and future rover engineering for the RFEXM22.

Generation of Large-Scale Orthophoto Mosaics Using MEX HRSC Images for the Candidate Landing Regions of China’s First Mars Mission

Planetary mapping products play a significant role in landing site selection, surface operations, and scientific investigations. This paper describes the techniques and processes used to create geometrically controlled image mosaics for the candidate landing regions of China’s first Mars exploration mission (i.e., Tianwen-1) using Mars Express (MEX) High-Resolution Stereo Camera (HRSC) images. In order to deal with the extremely complex situations due to large data volumes (hundreds of images), various imaging conditions (e.g., 5–80° incidence angles), and long time spans (>10 years), we developed corresponding methods and in-house software for the photogrammetric processing of MEX HRSC images. According to the characteristics of planetary images and multi-line pushbroom imaging, we optimized the algorithms of establishing control network, eliminating blunders, setting weights and processing abnormal images. The generated MEX HRSC image mosaics with a resolution of 12.5 m delivered a high relative accuracy (<1 pixel). Compared with the existing Mars global image mosaics, the generated image mosaics exhibit significantly better spatial resolution. The techniques and methods that we developed to solve the complicated photogrammetric processing problems can also be applied to produce global or large-scale Mars mapping products using existing and new returned orbital images.

Radiometric Principle-Based Radiometric Normalization Method for SAR Images Mosaic

Radiometric normalization minimizes the radiometric inconsistencies between images in synthetic aperture radar (SAR) images mosaic. However, the radiometric principle of SAR have not been fully considered by the existing methods. To this issue, a radiometric principle-based radiometric normalization method is proposed. First, image areas with consistent coverage areas in object space and approximate rough ground surface between images are extracted as adjustment candidates based on imaging principle. Then, considering the radiometric characteristic of SAR image, all images are taken as a whole to solve the radiometric adjustment model, which transfers radiometric normalization into the least square optimization. Finally, a global quantization strategy is used to ensure radiometric consistency during orthorectification and mosaicking. The experimental results of Gaofen-3 SAR images demonstrate that the proposed method has the best performance, and can effectively and stably eliminate the radiometric differences between images.

Extraction of Point-of-Interest in Multispectral Images for Face Recognition

Security systems in companies, airports, enter-prises, etc. face numerous challenges. Among the major ones there is objects or face recognition. The problem with the robustness of recognition systems that usually affects color images nowadays can be addressed by multispectral image acquisition in the near infrared range with cameras equipped with new high performance sensors able to take images in dark or uncontrolled environments with much more accuracy. Multispectral CMOS (Complementary Metal Oxide Semi-conductor) sensors in a single shot record several wavelengths that are isolated and allow very specific analyses. They are equipped with new acquisition methods and provide observations that are more accurate. The current generation of these imaging sensors involve scientific and technical interest because they provide much more information than those that operate in visible range; precise nature and spatio-temporal evolution of the areas need to be analyzed. In this study, multispectral images acquired by camera equipped with a hybrid sensor operating in near infrared has been used. This camera is built in the ImViA laboratory of the University of Bourgogne as part of the European project EXIST (EXtended Image Sensing Technologies). The process involved in image acquisition, image mosaicing and image demosaicing by using mosaic filters. After acquisition process the interest points be extract in these bands of images in order to know how information is shared out all over the bands. The results were satisfactory because information is spread all over the images bands and the algorithms used also have detected many interest points. Based on the results, a large database can be set up for a face recognition system building.

RTSfM: Real-Time Structure From Motion for Mosaicing and DSM Mapping of Sequential Aerial Images With Low Overlap

Inspired by simultaneous localization and mapping (SLAM) style workflow, this article presented an online sequential structure from motion (SfM) solution for high-frequency video and large baseline high-resolution aerial images with high efficiency and novel precision. First, as traditional SLAM systems are not good in processing low overlap images, based on our novel hierarchical feature matching paradigm with multihomography and BoW, we proposed a robust tracking method where the relative pose and its scale are estimated separately followed by a joint optimization by considering both perspective-n-point (PnP) and epipolar constraints. Second, to further optimize the camera poses for the sparse map and dense pointcloud reconstruction, we provided a graph-based optimization with reprojection and GPS constraints, which make the camera trajectory and map georeferenced. We also incrementally generated the dense point cloud in real time from keyframes after local mapping optimization. Finally, we use a publicly available aerial image dataset with sequences of different environments, to evaluate the effectiveness of the proposed method, meanwhile, the robust performance of our solution is demonstrated with applications of high-quality aerial images mosaic and digital surface model (DSM) reconstruction in real time. Compared with the state-of-the-art SLAM and traditional SfM methods, the presented system can output large-scale high-quality ortho-mosaic and DSM in real time with the low computational cost.

Multimodal Imaging of Laser Speckle Contrast Imaging Combined With Mosaic Filter-Based Hyperspectral Imaging for Precise Surgical Guidance.

To enable a real-time surgical guidance system that simultaneously monitors blood vessel perfusion, oxygen saturation, thrombosis, and tissue recovery by combining multiple optical imaging techniques into a single system: visible imaging, mosaic filter-based snapshot hyperspectral imaging (HSI), and laser speckle contrast imaging (LSCI). The multimodal optical imaging system was demonstrated by clamping blood vessels in the small intestines of rats to create areas of restricted blood flow. Subsequent tissue damage and regeneration were monitored during procedures. Using LSCI, vessel perfusion was measured, revealing the biological activity and survival of organ tissues. Blood oxygen saturation was monitored using HSI in the near-infrared region. Principal component analysis was used over the spectral dimension to identify an HSI wavelength combination optimized for hemodynamic biomarker visualization. HSI and LSCI were complimentary, identifying thrombus generation and tissue recovery, which was not possible in either modalityalone. By analyzing multimodal tissue information from visible imaging, LSCI perfusion imaging, and HSI, a recovery prognosis could be determined based on the blood supply to the organ. The unique combination of the complementary imaging techniques into a single surgical microscope holds promise for improving the real-time determination of blood supply and tissue prognosis during surgery. Precise real-time monitoring for vascular anomalies promises to reduce the risk of organ damage in precise surgical operations such as tissue resection and transplantation. In addition, the convergence of label-free imaging technologies removes delays associated with the injection and diffusion of vascular monitoring dyes.

Remote Sensing Image Mosaic Based on Distribution Measure and Saliency Information

Remote Sensing Image Mosaic Based on Distribution Measure and Saliency Information

Image mosaicing based on BRISK and improved RANSAC algorithm

Image mosaicing based on BRISK and improved RANSAC algorithm

Moon Imaging Technique and Experiments Based on Sanya Incoherent Scatter Radar

This article introduces the experiment design for Moon imaging based on Sanya incoherent scatter radar (SYISR) and algorithm research in data processing. The peak power of SYISR is 2 MW. The transmitted frequency used for Moon imaging experiments is 430 MHz. We conducted Moon imaging experiments using two types of waveforms, 13-bit Barker code, and linear frequency modulation (LFM) chirp. Considering both resolution and signal-to-noise ratio (SNR), the use of an LFM chirp with a bandwidth of 0.3 MHz and a pulsewidth of 2 ms can give higher SNR and resolution for Moon imaging using SYISR. Several key techniques were applied in the experiment design and data processing: 1) for the reliability of the imaging algorithm, the range-Doppler imaging algorithm commonly used in synthetic aperture imaging was applied; 2) to avoid the sidelobe effect of the 13-bit Barker code matched filter, a sidelobe-free filter was used; and 3) to mitigate the problem of “north–south ambiguity,” mosaic imaging of the Doppler northern and southern hemispheres of the nearside of the Moon was adopted. Two types of imaging results are obtained: mosaic images of the northern and southern hemispheres of the Moon and local regional images. The results demonstrate the feasibility and reliability of Moon imaging based on SYISR, which enables potential further lunar geology investigations in the future.

Динамическое картографирование нарушенных земель с использованием данных дистанционного зондирования

Due to the increase in the areas of disturbed lands, the relevance of developing methods and methods for obtaining and analyzing spatial data in order to make decisions on rational nature management is increasing every year. Monitoring of natural and anthropogenic systems is largely related to the collection, analysis and visualization of dynamic processes, so the technologies for compiling of dynamic maps are at the peak of relevance. A number of factors necessitate the using of dynamic geoimages. Firstly, these images are an inseparable combination of spatial-temporal links on the certain areas. Secondly, it is the possibility of a full-fledged analysis of spatial changes taking into account time. Thirdly, it is the forecasting of natural and socio-economic factors and phenomena. In addition, dynamic mapping opens up opportunities for multimedia data visualization, which increases the observer’s perception of geoimages by several times with the focus on specific objects. Remote sensing data is one of the main sources for compiling and updating thematic dynamic maps. This article demonstrates the development of the method for creating working layers used in geographic information systems (GIS) for compiling dynamic maps using remote sensing data. The authors note a distinctive feature of the methodology: it is aimed at a wide range of users who do not fully have the skills and abilities to work with remote sensing data. These are managers of any level, whose direct work is not related to the compiling of geo-images, but whose competence is to make managerial decisions. Another advantage of the described methods is its implementation in an open source GIS (QGIS), as well as its application not only for single images, but also for a mosaic image. The article presents a description of the entire path from image processing to the creation of visual images. Disturbed lands of the Zabaykalsky Krai of the Russian Federation were chosen as a special example of working polygons. These territories have a large number of environmental problems, causing an increase in the areas of disturbed lands: open pits, an increase in the number of mining and processing factories, degradation of agricultural and forest lands due to anthropogenic activities and erosion processes, active seismic processes, mudflows and avalanche hazard.

Multilevel Feature Extraction Using Wavelet Attention for Deep Joint Demosaicking and Denoising

Demosaicking and denoising are critical for deciding on digital camera performance. However, conventional joint demosaicking and denoising methods use traditional hand-crafted filters for preprocessing and image restoration. Therefore, it is susceptible to noise and can produce many artifacts for images with numerous edges. This paper proposes an end-to-end multi-level wavelet attention convolutional neural network (CNN) that improves image restoration performance by reducing false color artifacts, over-smoothing, and blurring during demosaicing and denoising. In detail, this paper proposes a CNN-based preprocessing method that learns the mosaic image&#x2019;s inter-color correlation, improving the edge&#x2019;s color restoration performance. Furthermore, this paper presents a multi-level feature extraction convolutional block using the Haar wavelet-based discrete wavelet transform (DWT) to remove noise from images and restore colors better at the same time. Therefore, it preserves the information of input image and feature maps, learns the correlation between global and local features, improves image restoration performance, and suppresses phenomena such as over-smoothing that tend to occur in DWT-based denoising. The proposed method is an end-to-end network structure with CNN-based preprocessing methods. In experimental results, the proposed method improves PSNR by 3.67 dB and 0.35 dB on average compared with the traditional methods and the CNN-based methods, respectively, for the dataset with many high-frequency components.

Impact of scanner variability on lymph node segmentation in computational pathology

Computer-aided diagnostics in histopathology are based on the digitization of glass slides. However, heterogeneity between the images generated by different slide scanners can unfavorably affect the performance of computational algorithms. Here, we evaluate the impact of scanner variability on lymph node segmentation due to its clinical importance in colorectal cancer diagnosis. 100 slides containing 276 lymph nodes were digitized using 4 different slide scanners, and 50 of the lymph nodes containing metastatic cancer cells. These 400 scans were subsequently annotated by 2 experienced pathologists to precisely label lymph node boundary. Three different segmentation methods were then applied and compared: Hematoxylin-channel-based thresholding (HCT), Hematoxylin-based active contours (HAC), and a convolution neural network (U-Net). Evaluation of U-Net trained from both a single scanner and an ensemble of all scanners was completed. Mosaic images based on representative tiles from a scanner were used as a reference image to normalize the new data from different test scanners to evaluate the performance of a pre-trained model. Fine-tuning was carried out by using weights of a model trained on one scanner to initialize model weights for other scanners. To evaluate the domain generalization, domain adversarial learning and stain mix-up augmentation were also implemented. Results show that fine-tuning and domain adversarial learning decreased the impact of scanner variability and greatly improved segmentation across scanners. Overall, U-Net with stain mix-up (Matthews correlation coefficient (MCC) = 0.87), domain adversarial learning (MCC = 0.86), and HAC (MCC = 0.87) were shown to outperform HCT (MCC = 0.81) for segmentation of lymph nodes when compared against the ground truth. The findings of this study should be considered for future algorithms applied in diagnostic routines.

A Multi-Stage Approach Combining Very High-Resolution Satellite Image, GIS Database and Post-Classification Modification Rules for Habitat Mapping in Hong Kong

Identification and mapping of various habitats with sufficient spatial details are essential to support environmental planning and management. Considering the complexity of diverse habitat types in a heterogeneous landscape, a context-dependent mapping framework is expected to be superior to traditional classification techniques. With the aim to produce a territory-wide habitat map in Hong Kong, a three-stage mapping procedure was developed to identify 21 habitats by combining very-high-resolution satellite images, geographic information system (GIS) layers and knowledge-based modification rules. In stage 1, several classification methods were tested to produce initial results with 11 classes from a WorldView-2/3 image mosaic using a combination of spectral, textural, topographic and geometric variables. In stage 2, modification rules were applied to refine the classification results based on contextual properties and ancillary data layers. Evaluation of the classified maps showed that the highest overall accuracy was obtained from pixel-based random forest classification (84.0%) and the implementation of modification rules led to an average 8.8% increase in the accuracy. In stage 3, the classification scheme was expanded to all 21 habitats through the adoption of additional rules. The resulting habitat map achieved &gt;80% accuracy for most of the evaluated classes and &gt;70% accuracy for the mixed habitats when validated using field-collected points. The proposed mapping framework was able to utilize different information sources in a systematic and controllable workflow. While transitional mixed habitats were mapped using class membership probabilities and a soft classification method, the identification of other habitats benefited from the hybrid use of remote-sensing classification and ancillary data. Adaptive implementation of classification procedures, development of appropriate rules and combination with spatial data are recommended when producing an integrated and accurate map.

Water tank and swimming pool detection based on remote sensing and deep learning: Relationship with socioeconomic level and applications in dengue control.

Studies have shown that areas with lower socioeconomic standings are often more vulnerable to dengue and similar deadly diseases that can be spread through mosquitoes. This study aims to detect water tanks installed on rooftops and swimming pools in digital images to identify and classify areas based on the socioeconomic index, in order to assist public health programs in the control of diseases linked to the Aedes aegypti mosquito. This study covers four regions of Campinas, São Paulo, characterized by different socioeconomic contexts. With mosaics of images obtained by a 12.1 MP Canon PowerShot S100 (5.2 mm focal length) carried by unmanned aerial vehicles, we developed deep learning algorithms in the scope of computer vision for the detection of water tanks and swimming pools. An object detection model, which was initially created for areas of Belo Horizonte, Minas Gerais, was enhanced using the transfer learning technique, and allowed us to detect objects in Campinas with fewer samples and more efficiency. With the detection of objects in digital images, the proportions of objects per square kilometer for each region studied were estimated by adopting a Chi-square distribution model. Thus, we found that regions with low socioeconomic status had more exposed water tanks, while regions with high socioeconomic levels had more exposed pools. Using deep learning approaches, we created a useful tool for Ae. aegypti control programs to utilize and direct disease prevention efforts. Therefore, we concluded that it is possible to detect objects directly related to the socioeconomic level of a given region from digital images, which encourages the practicality of this approach for studies aimed towards public health.

Low concentration Phloxine B staining for high chemical contrast, nonlinear microscope mosaic imaging of skin alterations in pseudoxanthoma elasticum.

Pseudoxanthoma elasticum (PXE) is an autosomal recessive metabolic disorder characterized by ectopic mineralization of soft connective tissue. Histopathology findings include fragmented, mineralized elastic fibers and calcium deposits in the mid-dermis. Nonlinear microscopy (NLM) can be used for visualization of these histopathological alterations of the mid-dermis in PXE-affected skin sections. Upon introducing a normalized 3D color vector representation of emission spectra of three of the main tissue components (collagen, elastin and calcification) we found that due to their broad, overlapping emission spectra, spectral separation of emission from elastin and calcification is practically impossible in fresh-frozen or unstained, deparaffinized PXE sections. However, we found that the application of a low concentration Phloxine B staining after the deparaffinization process creates an imaging contrast for these two tissue components, which enables spectral decomposition of their fluorescence images. The obtained concentration maps for calcium deposits can be well suited for the determination of illness severity by quantitative analysis.

Improving the Usability of Galileo and Voyager Images of Jupiter's Moon Europa.

NASA's Voyager 1, Voyager 2, and Galileo spacecraft acquired hundreds of images of Jupiter's moon Europa. These images provide the only moderate‐ to high‐resolution views of the moon's surface and are therefore a critical resource for scientific analysis and future mission planning. Unfortunately, uncertain knowledge of the spacecraft's position and pointing during image acquisition resulted in significant errors in the location of the images on the surface. The result is that adjacent images are poorly aligned, with some images displaced by more than 100 km from their correct location. These errors severely degrade the usability of the Voyager and Galileo imaging data sets. To improve the usability of these data sets, we used the U.S. Geological Survey Integrated Software for Imagers and Spectrometers to build a nearly global image tie‐point network with more than 50,000 tie points and 135,000 image measurements on 481 Galileo and 221 Voyager images. A global least‐squares bundle adjustment of our final Europa tie‐point network calculated latitude, longitude, and radius values for each point by minimizing residuals globally, and resulted in root mean square (RMS) uncertainties of 246.6 m, 307.0 m, and 70.5 m in latitude, longitude, and radius, respectively. The total RMS uncertainty was 0.32 pixels. This work enables direct use of nearly the entire Galileo and Voyager image data sets for Europa. We are providing the community with updated NASA Navigation and Ancillary Information Facility Spacecraft, Planet, Instrument, C‐matrix (pointing), and Events kernels, mosaics of Galileo images acquired during each observation sequence, and individual processed and projected level 2 images.