Image Mosaic Research Articles

Reply to Comments by Kale and Pande on: “Machine-learning based re-classification of the stratigraphy of the Rajahmundry Traps, India”

Hasandağ is a double-peaked long-dormant volcano, where associated volcanic risk is poorly known and needs to be assessed. Weak fumaroles and water vapor emissions have been observed at several vents, on the western flank of the Greater Hasandağ summit, between 3000 and 3100 m (a.s.l.). The highest temperature and CO2 values measured in those vents were ~ 70 °C and > 100,000 ppm respectively. Multisource and wide-period thermal remote sensing analyses were performed to investigate the thermal state and to test the possibilities of remote thermal monitoring of the volcano.MODIS daily surface temperature data were analyzed for a period of 18 years. We used a statistical approach using the Seasonal Decomposition of Time-series procedure to decompose the temperature data into the trend, seasonal, and remainder components. The trend component was used to analyze temporal thermal anomalies in comparison to the decomposed meteorological temperature data. 56 nighttime ASTER TIR imagery acquired between 2001 and 2018 had been used to derive and analyze the Surface temperature, Surface temperature anomaly and Relative radiative heat flux. High-resolution ground-based and aerial thermal mosaic images were built to investigate the thermal radiation over the summit fumarole zone and its structural dependence.A baseline reference value for thermal monitoring of the fumarole zone was achieved using remotely sensed data and time-series decomposition of long-term TIR data. The resolution of the thermal imagery positively affects the mapping and monitoring capability. A permanent ground-based thermal observation station should be established for direct and remote thermal monitoring of the fumarole zone.

High throughput screening system for engineered cardiac tissues.

Introduction: Three dimensional engineered cardiac tissues (3D ECTs) have become indispensable as in vitro models to assess drug cardiotoxicity, a leading cause of failure in pharmaceutical development. A current bottleneck is the relatively low throughput of assays that measure spontaneous contractile forces exerted by millimeter-scale ECTs typically recorded through precise optical measurement of deflection of the polymer scaffolds that support them. The required resolution and speed limit the field of view to at most a few ECTs at a time using conventional imaging. Methods: To balance the inherent tradeoff among imaging resolution, field of view and speed, an innovative mosaic imaging system was designed, built, and validated to sense contractile force of 3D ECTs seeded on a 96-well plate. Results: The system performance was validated through real-time, parallel contractile force monitoring for up to 3 weeks. Pilot drug testing was conducted using isoproterenol. Discussion: The described tool increases contractile force sensing throughput to 96 samples per measurement; significantly reduces cost, time and labor needed for preclinical cardiotoxicity assay using 3D ECT. More broadly, our mosaicking approach is a general way to scale up image-based screening in multi-well formats.

A Machine Learning Approach for Mapping Chlorophyll Fluorescence at Inland Wetlands

Wetlands are a critical component of the landscape for climate mitigation, adaptation, biodiversity, and human health and prosperity. Keeping an eye on wetland vegetation is crucial due to it playing a major role in the planet’s carbon cycle and ecosystem management. By measuring the chlorophyll fluorescence (ChF) emitted by plants, we can get a precise understanding of the current state and photosynthetic activity. In this study, we applied the Extreme Gradient Boost (XGBoost) algorithm to map ChF in the Biebrza Valley, which has a unique ecosystem in Europe for peatlands, as well as highly diversified flora and fauna. Our results revealed the advantages of using a set of classifiers derived from EO Sentinel-2 (S-2) satellite image mosaics to accurately map the spatio-temporal distribution of ChF in a terrestrial landscape. The validation proved that the XGBoost algorithm is quite accurate in estimating ChF with a good determination of 0.71 and least bias of 0.012. The precision of chlorophyll fluorescence measurements is reliant upon determining the optimal S-2 satellite overpass time, which is influenced by the developmental stage of the plants at various points during the growing season. Finally, the model performance results indicated that biophysical factors are characterized by greenness- and leaf-pigment-related spectral indices. However, utilizing vegetation indices based on extended periods of remote sensing data that better capture land phenology features can improve the accuracy of mapping chlorophyll fluorescence.

Structure Cursor Fixing and Splicing Based on Cooperative Robot

In the field of machine vision, surface structured light image stitching can generate high-resolution images with large field of view, which can meet the needs of enterprise production. This paper introduces a 3D image Mosaic method based on robot arm and fringe projection technology. In this method, a surface structured light vision sensor installed at the end of the manipulator is used to scan and reconstruct the workpiece surface. The sensor consists of a DLP projector and a 3D camera. A set of sinusoidal structured light is projected at the measurement position, which is synchronously collected by the camera. The collected image is de-phased by using the principle of four-step phase shift and multi-frequency heterodyne to obtain 3D information. The coarse positioning algorithm of the manipulator is used to splice part of the acquired 3D information, and finally the complete 3D point cloud data on the surface of the workpiece to be measured is obtained. The experimental results show that the method can effectively and quickly obtain the 3D surface information of the car model, with strong stability and high accuracy.

The LOFAR LBA Sky Survey

Context. The Low Frequency Array (LOFAR) is the only existing radio interferometer able to observe at ultra-low frequencies (<100 MHz) with high resolution (<15") and high sensitivity (<1 mJy beam−1). To exploit these capabilities, the LOFAR Surveys Key Science Project is using the LOFAR Low Band Antenna (LBA) to carry out a sensitive wide-area survey at 41–66 MHz named the LOFAR LBA Sky Survey (LoLSS). Aims. LoLSS is covering the whole northern sky above declination 24° with a resolution of 15" and a sensitivity of 1–2 mJy beam−1 (1σ) depending on declination, field properties, and observing conditions. Here we present the first data release, including a discussion of the calibration strategy and the properties of the released images and catalogues. Methods. A fully automated pipeline was used to reduce the 95 fields included in this data release. The data reduction procedures developed for this project have a general application and are currently being used to process almost all LOFAR LBA interferometric observations. Compared to the preliminary release, direction-dependent errors have been derived and corrected for during the calibration process. This results in a typical sensitivity of 1.55 mJy beam−1, which is four times better than for the preliminary release, at the target resolution of 15". Results. The first data release of the LOFAR LBA Sky Survey covers 650 deg2 in the HETDEX spring field. The resultant data products released to the community include mosaic images (I and V Stokes) of the region, and a catalogue of 42 463 detected sources and related Gaussian components used to describe the sources' morphologies. Separate catalogues for the six in-band frequencies of 44, 48, 52, 56, 60, and 64 MHz are also released. Conclusions. The first data release of LoLSS shows that, despite the influences of the ionosphere and radio frequency interference, LOFAR can conduct large-scale surveys in the frequency window 42-66 MHz with unprecedentedly high sensitivity and resolution. The data can be used to derive unique information on the low-frequency spectral properties of many thousands of sources with a wide range of applications in extragalactic and galactic astronomy.

Large-scale time-lapse scanning electron microscopy image mosaic using a smooth stitching strategy.

Due to the trade-off between the field of view and resolution of various microscopes, obtaining a wide-view panoramic image through high-resolution image tiles is frequently encountered and demanded in numerous applications. Here, we propose an automatic image mosaic strategy for sequential 2D time-lapse scanning electron microscopy (SEM) images. This method can accurately compute pairwise translations among serial image tiles with indeterminate overlapping areas. The detection and matching of feature points are limited by geographical coordinates, thus avoiding accidental mismatching. Moreover, the nonlinear deformation of the mosaic part is also taken into account. A smooth stitching field is utilized to gradually transform the perspective transformation in overlapping regions into the linear transformation in non-overlapping regions. Experimental results demonstrate that better image stitching accuracy can be achieved compared with some other image mosaic algorithms. Such a method has potential applications in high-resolution large-area analysis using serial microscopy images. RESEARCH HIGHLIGHTS: An automatic image mosaic strategy for processing sequential scanning electron microscopy images is proposed. A smooth stitching field is applied in the image mosaic. Improved stitching accuracy is achieved compared with other conventional mosaic methods.

Image Mosaicing

Image mosaicing is a technique for combining many images of the same scene to generate a larger, seamless panoramic image. Image mosaicing can be traced back to the days of manually piecing images. Image-mosaicing techniques have been created to automate the process since the introduction of computer technology. The method consists of five phases, including feature matching and estimation of the homography matrix. Image mosaicing is commonly used in virtual tourism, video games, and photo editing. Despite the fact that the image panoramic feature is accessible in digital cameras, there is still potential for improvement, and more research in this sector is required.

Extended Depth of Field Imaging for Mosaic Hyperspectral Images.

Hyperspectral imaging (HSI) is an emerging modality for digital pathology. The purpose of this study is to develop an extended depth of field (EDOF) method for mosaic hyperspectral images acquired with a snapshot camera. EDOF is a technique for ensuring that an image is in focus at all points. A stack of mosaicked hyperspectral images of hematoxylin and eosin (H&E)-stained histologic slides were acquired at different positions along the z-axis and used to output a hyperspectral histologic image that was in-focus at every point. Three different methods were compared to achieve a fully focused image. We compared conventional patch-based methods to our proposed growth-based and band-based methods. The Brenner function was used to quantitatively measure the focus quality of each image measured. The results show that both of our proposed methods performed better qualitatively and quantitatively than the patch-based method, with the band-based method performing the best, as it leveraged dividing pixels into their proper wavelengths in addition to spatially, giving the algorithm better contrast to measure. In terms of speed, the band-based method was the fastest, followed by the patch-based method, with the growth-based method being the slowest. Our proposed extended depth of field hyperspectral imaging methods can have immediate applications in digital pathology, especially whole slide imaging, and other microscopic imaging.

OPTIMIZATION ASSISTED AUTOREGRESSIVE METHOD WITH DEEP CONVOLUTIONAL NEURAL NETWORK-BASED ENTROPY FILTER FOR IMAGE DEMOSAICING

The natural scenes are captured by digital cameras that adopt a single charged-coupled device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) sensor with a Color Filter Array (CFA). To reconstruct a full color image from the mosaiced CFA image, the missing components of color are recovered by a technique called color demosaicing. This research work introduces Deep Convolution Neural Network (DCNN)-based optimization assisted autoregressive method for image demosaicing. Here, the proposed optimization algorithm, termed Adaptive Autoregressive Water Wave Optimization algorithm (Adaptive Autoregressive-WWO), that is formed by combining Conditional autoregressive value at risk (CAViaR) model and Water Wave Optimization algorithm (WWO). Fusion process is carried out for residual images to generate the final demosaiced image based on entropy measure. Here, the output generated from DCNN is the first residual image. The LPA-ICI filter utilizes the optimization algorithm that generated second order polynomial coefficients to produce the second residual output image. Moreover, this proposed method is evaluated for its performance using metrics, such as Peak signal-to-noise ratio (PSNR) and Second Derivative like Measurement (SDME) and attained the highest PSNR values of 40.379dB and highest SDME values of 50.675dB.

High-throughput single-molecule localization microscopy: Potential clinical applications.

High-throughput single-molecule localization microscopy: Potential clinical applications.

Calibrating X-Ray Binary Luminosity Functions via Optical Reconnaissance. II. The High-mass XLF and Globular Cluster Population of X-Ray Binaries in the Low Star-forming Spiral M81

We characterize the optical counterparts to the compact X-ray source population within the nearby spiral galaxy M81 using multiband Hubble Space Telescope (HST) imaging data. By comparing the optical luminosities and colors measured for candidate donor stars and host clusters to stellar and cluster evolutionary models, respectively, we estimate the likely masses and upper age limits of the field and cluster X-ray binaries. We identify 15 low-mass X-ray binaries (i.e., donor star mass ≲ 3 M ⊙) within ancient globular clusters, as well as 42 candidate high-mass X-ray binaries (i.e., donor star mass ≳ 8 M ⊙). To estimate the likelihood of misclassifications, we inject 4000 artificial sources into the HST mosaic image and conclude that our classifications of globular clusters and high-mass X-ray binaries are reliable at the >90% level. We find that globular clusters that host X-ray binaries are on average more massive and more compact than globular clusters that do not. However, there is no apparent correlation between the X-ray brightness of the clusters and their masses or densities, nor are X-ray binary hosts more X-ray luminous than the general field population of low-mass X-ray binaries. This work represents one of the first in-depth analyses of the population of X-ray binaries within globular clusters in a spiral galaxy.

Synthetic Aperture Radar and Optical Sensor Techniques Using Google Earth Engine For Flood Monitoring and Damage Assessment – A Case Study of Mumbwa District, Zambia

Radar and Optical based satellite sensors were used in the study of the Mumbwa flood of December 2020. The Synthetic Aperture Radar (SAR) based Sentinel-1B was processed in Google Earth Engine (GEE) and utilized to generate an image mosaic from December 2020 to May 2021 to delineate flood extent. A local water histogram threshold change detection approach by image ratio was utilized to determine the flood extent with an intensity value of 1.26 dB as it fitted the study area uniquely as opposed to the global value of 1.25 dB. After extracting the initial flood water extent, it was necessary to filter out regions which inundated during the flood period. This was carried out using the following datasets and parameters: The HydroSHEDS Digital Elevation Model (DEM) was used to filter out regions with a slope value of greater than 7% and the Global Surface Water Layer was used to clip out regions with existing permanent surface water. Once the flooded areas were identified, the Optical based Sentinel-2 was used in the production of a Land Use Land Cover (LULC) Map for August 2020 in order to superimpose the flooded areas with existing land features over the study area. The map also under went pre and post processing in GEE using the Random Forest Classification Algorithm that achieved an Overall Accuracy and Kappa Coefficient of 0.957 and 0.91519 respectively. Thereafter the flood analysis and damage assessment were carried out. The quantitative damages to Landcover were found to be: Wetland 6,338.97 Ha (33.27%), Shrubland 5,117.75 Ha (26.89%), Biochar Soil 3,660.47 Ha (19.21%), Trees 3,466.37 Ha (18.19%), Bare soil 273.47 Ha (1.44%), Crop Fields 190.69 Ha (1%) and Built-Up 4.13 Ha (0.02%). Therefore the use of SAR by local histogram threshold approach with Optical datasets for LULC map production proved successful in the study of flood damage.

Mobilenetv2_CA Lightweight Object Detection Network in Autonomous Driving

A lightweight network target detection algorithm was proposed, based on MobileNetv2_CA, focusing on the problem of high complexity, a large number of parameters, and the missed detection of small targets in the target detection network based on candidate regions and regression methods in autonomous driving scenarios. First, Mosaic image enhancement technology is used in the data pre-processing stage to enhance the feature extraction of small target scenes and complex scenes; second, the Coordinate Attention (CA) mechanism is embedded into the Mobilenetv2 backbone feature extraction network, combined with the PANet and Yolo detection heads for multi-scale feature fusion; finally, a Lightweight Object Detection Network is built. The experimental test results show that the designed network obtained the highest average detection accuracy of 81.43% on the Voc2007 + 2012 dataset, and obtained the highest average detection accuracy of 85.07% and a detection speed of 31.84 FPS on the KITTI dataset. The total amount of network parameters is only 39.5 M. This is beneficial to the engineering application of MobileNetv2 network in automatic driving.

Chronology, composition, and mineralogy of mare basalts in the junction of Oceanus Procellarum, Mare Imbrium, Mare Insularum, and Mare Vaporum

The timeline of volcanic activity is critical for constraining the thermal evolution of the Moon. The spatial extents of mare basalts, major products of lunar volcanism, have been precisely extracted from LROC (Lunar Reconnaissance Orbiter Camera) image mosaics. With the maria extents newly extracted from LROC mosaics, we found that a large area of mare basalts in the junction of Oceanus Procellarum, Mare Imbrium, Mare Insularum, and Mare Vaporum (the PIV region) has not yet been dated. This study analysed the chronology, composition, and mineralogy of the PIV region, aiming to finish the picture of basaltic volcanism in the Procellarum region, which is a key puzzle of our global geological mapping of the Moon. According to the topographical, spectral, and compositional characteristics, mare units of the PIV region are defined, and the crater size frequency distribution is measured. The primary craters with diameters >0.1 km in the PIV region are measured, and the absolute model ages of basalt units between ∼3.78 Ga and ∼1.71 Ga are derived. Most western PIV basalt units are Eratosthenian-aged, while eastern units mostly formed in the Imbrian period. The spectra of 4965 small impact craters are extracted to interpret the mineral compositions of the PIV basalt units using Chandrayaan-1 Moon Mineralogy Mapper data. Using a Modified Gaussian Model, the reflectance spectra are deconvoluted, and the obtained modal proportions of mafic minerals show low olivine and high low-Ca pyroxene abundances in the eastern PIV region, while the western region features high olivine and calcic pyroxene concentrations. Three episodes of volcanic events occurring in the PIV region are identified. The first (main) occurred at approximately 3.5 Ga (Late Imbrian), with erupted lava flows with less evolved compositions covering most of the PIV area. The peak of volcanic activity in the Eratosthenian period occurred around 2.5 Ga, where mare basalts with moderately evolved compositions and mineralogy were formed. The last major eruption occurred at approximately 1.8 Ga, forming mare basalts with highly evolved compositions.

MSFA-Net: A convolutional neural network based on multispectral filter arrays for texture feature extraction

Multispectral snapshot cameras fitted with a multispectral filter array (MSFA) acquire several spectral bands in one shot and provide a raw mosaic image in which a single channel value is available at each pixel. Texture features are classically extracted from fully-defined images that are estimated by demosaicing. Such an estimation may however cause spatio-spectral artifacts. Moreover, texture feature extraction becomes computationally inefficient and yields to high-dimensional features as the number of bands increases. In this paper, we propose an original approach based on a convolutional neural network called MSFA-Net to capture spatio-spectral interactions in raw images at reduced computation costs. Experiments of multispectral image classification and outdoor image segmentation show that the proposed approach outperforms several hand-crafted and deep learning-based feature extractors.

Super multi-view display based on near-eye pinholes.

Vergence-accommodation-conflict-free super multi-view (SMV) display based on more than one near-eye pinhole group for each pupil of the viewer is developed in this paper. Two-dimensionally arranged pinholes of a group correspond to different subscreens of the display screen, with perspective views projected by each subscreen through a corresponding pinhole splicing into an image of enlarged field of view (FOV). By sequentially switching on/off different pinhole groups, more than one mosaic image is projected to each pupil of the viewer. Adjacent pinholes of a group are endowed with different timing-polarizing characteristics to generate an effective noise-free region for each pupil. In the experiment, four groups of 3×3 pinholes are configured for a proof-of-concept SMV display on a 240Hz display screen, with a diagonal FOV of 55deg and a depth of field reaching 1.2m.

Fast mosaicing method based on image resizing pre-processing

In recent years, mosaic images have found great success thanks to the increasing development in the field of imaging as well as the technological evolution of computer systems (camera, mobile, etc.). Mosaic images are obtained by merging several images of the same scene. The process incorporates several steps, each of which requires resources and execution time depending on the size, quality and resolution of the images used. In this paper, we propose a new image mosaic method that significantly reduces the execution time. The idea is to apply the stages of registration and search for the best inliers, necessary for the calculation of the geometric transformation, to the miniature of images. This allows the minimization of the overall processing time without altering the quality of the results. The experiment, on a database of images, shows that the proposed algorithm provides rapid results compared to similar methods. Also, we have extended our method to generate 360° panoramic images.

CEERS Epoch 1 NIRCam Imaging: Reduction Methods and Simulations Enabling Early JWST Science Results

We present the data release and data reduction process for the Epoch 1 NIRCam observations for the Cosmic Evolution Early Release Science Survey (CEERS). These data consist of NIRCam imaging in six broadband filters (F115W, F150W, F200W, F277W, F356W and F444W) and one medium-band filter (F410M) over four pointings, obtained in parallel with primary CEERS MIRI observations. We reduced the NIRCam imaging with the JWST Calibration Pipeline, with custom modifications and reduction steps designed to address additional features and challenges with the data. Here we provide a detailed description of each step in our reduction and a discussion of future expected improvements. Our reduction process includes corrections for known prelaunch issues such as 1/f noise, as well as in-flight issues including snowballs, wisps, and astrometric alignment. Many of our custom reduction processes were first developed with prelaunch simulated NIRCam imaging over the full 10 CEERS NIRCam pointings. We present a description of the creation and reduction of this simulated data set in the Appendix. We provide mosaics of the real images in a public release, as well as our reduction scripts with detailed explanations to allow users to reproduce our final data products. These represent one of the first official public data sets released from the Directors Discretionary Early Release Science (DD-ERS) program.

Panoramic UAV Image Mosaic Method and Its Application in Pavement Paving Temperature Monitoring

The low-altitude technology of unmanned airborne infrared detection system is used to effectively monitor the temperature segregation in the paving stage and realize the temperature uniformity control of asphalt pavement construction. The image mosaic method can splice two images with overlapping areas together to form a panoramic image. In order to solve the problems of long time-consuming and low accuracy of aerial image mosaic algorithm, the low-temperature area of the whole pavement can be obtained quickly and accurately. In this paper, threshold segmentation technology is introduced to convert the image captured by the unmanned aerial vehicle (UAV) into a binary greyscale image so as to compensate for the mosaic error caused by temperature difference. In order to improve the efficiency and accuracy of splicing, a reference plate is used, which can provide enough feature points for splicing. Finally, the image mosaic method proposed in this paper can quickly obtain the image of the whole low-temperature area of the newly paved asphalt pavement, which has practical value and positive significance for the quality control of asphalt pavement.

Multiscale structural feature transform for multi-modal image matching

Robust and reliable multi-modal image matching is an essential and challenging technique in the multi-modality involved scenarios for image fusion, image mosaic, and visual navigation. Existing image matching methods cannot simultaneously solve the scale and rotation distortion caused by different viewpoints and the nonlinear radiation distortion caused by different imaging environments or mechanisms. To address these problems, a multiscale structural feature transform method is proposed for multi-modal image matching, which consists of three blocks: multiscale structural feature detector, symmetric log-polar descriptor, and point matching with position correction. The multiscale structural feature detector constructs the phase congruency map based Difference of Gaussian image pyramid for the scale-invariant feature points detection to tackle the nonlinear radiation distortion. The feature points are detected within 10-neighborhood instead of 26-neighborhood to produce sufficient points on the phase congruency map with very sparse texture. The symmetric log-polar descriptor utilizes the multiscale structure principal direction and symmetric bins to improve the robustness of the descriptor to rotation and nonlinear radiation distortion. At last, the point matching with position correction corrects the position of feature points to compensate the position offset between different images caused by the scale and rotation distortion and the nonlinear radiation distortion, which increases the number of correct matches and improves the transformation consistency between matched points. In the experiment, the proposed method in this paper is compared with seven state-of-the-art methods. Experimental results verify the superiority of the proposed method and its robustness to the scale and rotation distortion and the nonlinear radiation distortion. In addition, we demonstrate the effectiveness of the proposed three blocks by the ablation experiments.