Mosaicking Approach Research Articles

High throughput screening system for engineered cardiac tissues.

Introduction: Three dimensional engineered cardiac tissues (3D ECTs) have become indispensable as in vitro models to assess drug cardiotoxicity, a leading cause of failure in pharmaceutical development. A current bottleneck is the relatively low throughput of assays that measure spontaneous contractile forces exerted by millimeter-scale ECTs typically recorded through precise optical measurement of deflection of the polymer scaffolds that support them. The required resolution and speed limit the field of view to at most a few ECTs at a time using conventional imaging. Methods: To balance the inherent tradeoff among imaging resolution, field of view and speed, an innovative mosaic imaging system was designed, built, and validated to sense contractile force of 3D ECTs seeded on a 96-well plate. Results: The system performance was validated through real-time, parallel contractile force monitoring for up to 3 weeks. Pilot drug testing was conducted using isoproterenol. Discussion: The described tool increases contractile force sensing throughput to 96 samples per measurement; significantly reduces cost, time and labor needed for preclinical cardiotoxicity assay using 3D ECT. More broadly, our mosaicking approach is a general way to scale up image-based screening in multi-well formats.

Robust fetoscopic mosaicking from deep learned flow fields

PurposeFetoscopic laser photocoagulation is a minimally invasive procedure to treat twin-to-twin transfusion syndrome during pregnancy by stopping irregular blood flow in the placenta. Building an image mosaic of the placenta and its network of vessels could assist surgeons to navigate in the challenging fetoscopic environment during the procedure.MethodologyWe propose a fetoscopic mosaicking approach by combining deep learning-based optical flow with robust estimation for filtering inconsistent motions that occurs due to floating particles and specularities. While the current state of the art for fetoscopic mosaicking relies on clearly visible vessels for registration, our approach overcomes this limitation by considering the motion of all consistent pixels within consecutive frames. We also overcome the challenges in applying off-the-shelf optical flow to fetoscopic mosaicking through the use of robust estimation and local refinement.ResultsWe compare our proposed method against the state-of-the-art vessel-based and optical flow-based image registration methods, and robust estimation alternatives. We also compare our proposed pipeline using different optical flow and robust estimation alternatives.ConclusionsThrough analysis of our results, we show that our method outperforms both the vessel-based state of the art and LK, noticeably when vessels are either poorly visible or too thin to be reliably identified. Our approach is thus able to build consistent placental vessel mosaics in challenging cases where currently available alternatives fail.

Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects

The miniaturization of thermal infrared sensors suitable for integration with unmanned aerial vehicles (UAVs) has provided new opportunities to observe surface temperature at ultra-high spatial and temporal resolutions. In parallel, there has been a rapid development of software capable of streamlining the generation of orthomosaics. However, these approaches were developed to process optical and multi-spectral image data and were not designed to account for the often rapidly changing surface characteristics inherent in the collection and processing of thermal data. Although radiometric calibration and shutter correction of uncooled sensors have improved, the processing of thermal image data remains difficult due to (1) vignetting effects on the uncooled microbolometer focal plane array; (2) inconsistencies between images relative to in-flight effects (wind-speed and direction); (3) unsuitable methods for thermal infrared orthomosaic generation. Here, we use thermal infrared UAV data collected with a FLIR-based TeAx camera over an agricultural field at different times of the day to assess inconsistencies in orthophotos and their impact on UAV-based thermal infrared orthomosaics. Depending on the wind direction and speed, we found a significant difference in UAV-based surface temperature (up to 2 °C) within overlapping areas of neighboring flight lines, with orthophotos collected with tail wind being systematically cooler than those with head wind. To address these issues, we introduce a new swath-based mosaicking approach, which was compared to three standard blending modes for orthomosaic generation. The swath-based mosaicking approach improves the ability to identify rapid changes of surface temperature during data acquisition, corrects for the influence of flight direction relative to the wind orientation, and provides uncertainty (pixel-based standard deviation) maps to accompany the orthomosaic of surface temperature. It also produced more accurate temperature retrievals than the other three standard orthomosaicking methods, with a root mean square error of 1.2 °C when assessed against in situ measurements. As importantly, our findings demonstrate that thermal infrared data require appropriate processing to reduce inconsistencies between observations, and thus, improve the accuracy and utility of orthomosaics.

Remote Sensing for Monitoring Photovoltaic Solar Plants in Brazil Using Deep Semantic Segmentation

Brazil is a tropical country with continental dimensions and abundant solar resources that are still underutilized. However, solar energy is one of the most promising renewable sources in the country. The proper inspection of Photovoltaic (PV) solar plants is an issue of great interest for the Brazilian territory’s energy management agency, and advances in computer vision and deep learning allow automatic, periodic, and low-cost monitoring. The present research aims to identify PV solar plants in Brazil using semantic segmentation and a mosaicking approach for large image classification. We compared four architectures (U-net, DeepLabv3+, Pyramid Scene Parsing Network, and Feature Pyramid Network) with four backbones (Efficient-net-b0, Efficient-net-b7, ResNet-50, and ResNet-101). For mosaicking, we evaluated a sliding window with overlapping pixels using different stride values (8, 16, 32, 64, 128, and 256). We found that: (1) the models presented similar results, showing that the most relevant approach is to acquire high-quality labels rather than models in many scenarios; (2) U-net presented slightly better metrics, and the best configuration was U-net with the Efficient-net-b7 encoder (98% overall accuracy, 91% IoU, and 95% F-score); (3) mosaicking progressively increases results (precision-recall and receiver operating characteristic area under the curve) when decreasing the stride value, at the cost of a higher computational cost. The high trends of solar energy growth in Brazil require rapid mapping, and the proposed study provides a promising approach.

Automated Mosaicking of Sentinel-2 Satellite Imagery

Repeat frequencies of optical remote sensing satellites have been increasing over the last 40 years, but there is still dependence on clear skies to acquire usable imagery. To increase the quality of data, composited mosaics of satellite imagery can be used. In this paper, we develop an automated method for clearing clouds and producing different types of composited mosaics suitable for use in cloud-affected countries, such as New Zealand. We improve the Tmask algorithm for cloud detection by using a parallax method to produce an initial cloud layer and by using an object-based cloud and shadow approach to remove false cloud detections. We develop several parametric scoring approaches for choosing best-pixel composites with minimal remaining cloud. The automated mosaicking approach produced Sentinel-2 mosaics of New Zealand for five successive summers, 2015/16 through 2019/20, with remaining cloud being less than 0.1%. Contributing satellite overpasses were typically of the order of 100. In comparison, manual methods for cloud clearing produced mosaics with 5% remaining cloud and from satellite overpasses typically of the order of 20. The improvements to cloud clearing enable the use of all possible Sentinel-2 imagery to produce automatic mosaics capable of regular land monitoring, at a reasonable cost.

Sonar image mosaic based on a new feature matching method

Sonar images are valuable in exploring underwater environmental information. As these images are generally limited by the viewing angle, sonar image mosaicking becomes an important research topic. By combining several frames consecutively acquired while the underwater vehicle is manoeuvring, an image with a wider view can be obtained. This study presents a fast sonar image mosaicking approach consisting of denoising, feature extraction, initial matching, splicing, and optimisation. Based on the Euclidean distance between initially matched points and dip angle of connection line, poorly matched feature point pairs are removed to avoid false matching. This way, the success rate of image mosaicking and quality of the resulting mosaicking is effectively improved.

Real-time mosaicking for infrared videos from an oblique sweeping camera

Image mosaicking is widely used in Geographic Information Systems (GISs) for large-scale ground surface analysis. However, most existing mosaicking methods can only be used in offline processing due to the enormous amounts of computation. In this paper, we propose a novel and practical algorithm for real-time infrared video mosaicking. To achieve this, a novel fast template matching algorithm based on Sum of Cosine Differences (SCD) is proposed to coarsely match the sequential images. The high speed of the proposed template matching algorithm is obtained by computing correlation with Fast Fourier Transform (FFT). We also propose a novel fast Least Squares Matching (LSM) algorithm for inter-frame fine registration, which can significantly reduce the computation without degrading the matching accuracy. In addition, the proposed fast LSM can effectively adapt for noise degradation and geometric distortion. Based on the proposed fast template matching algorithm and fine registration algorithm, we develop a practical real-time mosaicking approach which can produce seamless mosaic image highly efficiently. Experiments on synthetic and real-world datasets demonstrate that the proposed algorithm is not just computationally efficient but also robust against various noise distortions.

PEMANFAATAN GIS (GEOGRAPHIC INFORMATION SYSTEM) UNTUK PEMETAAN VERIFIKASI SISIP DAN POKOK PADA PERKEBUNAN KELAPA SAWIT DI KABUPATEN KUTAI BARAT KALIMANTAN TIMUR

Verification of oil palm plants is done to cross check with census data on the number of plants to find out the number of oil palm plants in each. Unfortunately the company still has difficulty detecting the amount of palm oil in each block. One effort to overcome this problem is to use GIS (Geographic Information System). The purpose of this study was to determine the effect and differences in the use of photogrammetric methods with the waypoint method, the role of GIS (Geographic Information System) and determine the number of oil palm trees to be planted in the blocks in Afdeling I and Afdeling IV. This research through the preparation stage by preparing a sub-block map and flight plan, data collection stage by collecting aerial photo data, waypoint, block SHP, information on block area, planting year information and initial basic information on oil palm, the data processing stage using the mosaicking approach on the Agisoft PhotoScan Propessional application, waypoint conversion with Global Mapper and overlaying aerial photos with a waypoint with ArcMap 10.3. The results showed that the waypoint using GPS had higher accuracy compared to aerial photography and had an effect on the overlay results at the planned waypoint distance with the initial principal palm oil object. GIS is useful for processing aerial photo data and waypoints from GPS, and can speed up side verification. The plan is to insert in the B37 Block as many as 462 principal, B38 Block as many as 928 principal, B39 Block as many as 1,611 principal, B40 Block 351 principal, Block D29 1,063 principal, D30 Block 1,085, A53a Block 2,665 principal, A55a Block 539 subjects , C55 Block as many as 371, C56 Block as many as 510 principal, D55 Block as many as 157 principal, D56 Block as many as 375 principal, D57 Block as many as 1,561 and Block D59 with 1,431.

Generation of Large-Scale Moderate-Resolution Forest Height Mosaic With Spaceborne Repeat-Pass SAR Interferometry and Lidar

This paper provides an overview of the scattering model, inversion approach, and validation of the application results for creating large-scale moderate-resolution (hectare-level) mosaics of forest height through using spaceborne repeat-pass SAR interferometry and lidar. By incorporating several improvements to the forest height inversion and mosaicking approach, the height estimation accuracy along with the robustness of this approach have been considerably enhanced from its originally reported accuracy of RMSE of 3–4 m at a 20-hectare aggregated pixel size to RMSE of 3–4 m on the order of 3–6 hectares. Furthermore, practical data processing schemes are provided in detail. Extensive validation results are demonstrated which include: 1) a forest height mosaic (total area of 11.6 million hectares) is generated for the U.S. states of Maine and New Hampshire using Japanese Aerospace Exploration Agency’s (JAXA) ALOS-1 InSAR correlation data and a small airborne lidar strip (44 000 hectares); 2) the mosaic height estimates are further compared with the available airborne lidar data and field measurements over both flat and mountainous areas; and 3) feasibility of using modern repeat-pass InSAR satellites with short repeat interval is also examined by using JAXA’s ALOS-2 data. This simple and efficient approach is a potential observational prototype with much smaller error budget for the future spaceborne repeat-pass L-band InSAR systems with small spatial baseline and moderate/large temporal baseline (such as NISAR) in combination with lidar (such as GEDI) on the application of large-scale forest height/biomass mapping. It also serves as a complementary tool to the spaceborne single-pass InSAR systems using InSAR/PolInSAR methods when full-pol data are not available and/or when the underlying topography slope causes problems for these approaches.

Comparative Performance Evaluation of Pixel-Level and Decision-Level Data Fusion of Landsat 8 OLI, Landsat 7 ETM+ and Sentinel-2 MSI for Crop Ensemble Classification

Crops mapping unequivocally becomes a daunting task in humid, tropical, or subtropical regions due to unattainability of adequate cloud-free optical imagery. Objective of this study is to evaluate the comparative performance between decision- and pixel-levels data fusion ensemble classified maps using Landsat 8, Landsat 7, and Sentinel-2 data. This research implements parallel and concatenation approach to ensemble classify the images. The multiclassifier system comprises of Maximum Likelihood, Support Vector Machines, and Spectral Information Divergence as base classifiers. Decision-level fusion is achieved by implementing plurality voting method. Pixel-level fusion is achieved by implementing fusion by mosaicking approach, thus appending cloud-free pixels from either Sentinel-2 or Landsat 7. The comparison is based on the assessment of classification accuracy. Overall accuracy results show that decision-level fusion achieved an accuracy of 85.4%, whereas pixel-level fusion classification attained 82.5%, but their respective kappa coefficients of 0.84 and 0.80 but are not significantly different according to Z-test at $\alpha = {\text{0.05}}$ . F1-score values reveal that decision-level performed better on most individual classes than pixel-level. Regression coefficient between planted areas from both approaches is 0.99. However, Support Vector Machines performed the best of the three classifiers. The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results. Therefore, either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales. Future work can focus on performing more comparison tests on different areas, run tests using different multiclassifier systems, and use different imagery.

An improved approach for generating globally consistent seamline networks for aerial image mosaicking

ABSTRACTSeamless image mosaicking is a crucial technique in the production of orthophotos in photogrammetry and remote sensing. There are many studies focused on generating high-quality ortho-mosaics and many algorithms have been put forward for this purpose. Some of the existing methods concentrate on guiding the seamline to bypass ground objects, while others focus on generating a seamline network with better global quality, based on a regular and consistent initial network. However, the former may result in an irregular seamline network structure and the latter cannot guarantee the seamline bypasses ground objects in some cases, due to limited optimization. In this study, an improved image mosaicking approach, taking both the local optimization of a single seamline and the global regularity and consistency of the whole seamline network into consideration, is proposed in order to generate a globally consistent seamline network for aerial images. Firstly, the seamline is selected using a coarse-to-fine strategy between adjacent overlapping orthoimages, in which the initial seamline is generated based on our previous work and further refined based on texture difference. Then, an effective mosaic polygon of each orthoimage is obtained through a novel method, before being used to compose a globally consistent seamline network. Furthermore, an evaluation index called the Area Weighted Distance is introduced to quantitatively evaluate the global consistency and regularity of the generated seamline network. Experiment results demonstrate that the proposed approach is feasible and promising in terms of its performance. Compared to the comparison methods, the proposed approach is 1.35 and 2.15 times smaller in the average and standard deviation of the area weighted distance, respectively, and 3.6–6.5 times faster in terms of efficiency.

Mosaicking UAV orthoimages using bounded Voronoi diagrams and watersheds

ABSTRACTPanoramic mosaicking is a key step in the process of aerial photogrammetry, which combines multiple images to form a single image with a wide field of view. In this article, a refined global seam-line network generation method based on bounded Voronoi diagrams and watershed segmentation algorithm is proposed for mosaicking unmanned aerial vehicle (UAV) orthoimages. First of all, valid range polygons and image centres are extracted from orthoimages. Second, an initial seam-line network is generated by a bounded Voronoi diagrams and polygon Boolean operations. Third, the seam-line network is refined by marker-controlled watershed segmentation algorithm. Finally, a photomosaic is produced by multiband blending mosaic. The panoramic mosaicking approach is comprehensively discussed in terms of data volume and degree of overlapping. Experimental studies performed on several groups of UAV orthoimages show that the proposed approach provides a feasible and promising solution to large-scale UAV photomosaic generation.

A new image mosaicking approach for the multiple camera system of the optical remote sensing satellite GaoFen1

ABSTRACTThe GaoFen1 (GF1) optical remote sensing satellite is the first in China’s series of high-resolution civilian satellites and is equipped with four wide field of view (WFV) cameras. High accuracy image mosaicking depends on the geometrical quality of the stitched satellite imagery. This letter proposes a high accuracy image mosaicking approach based on combining multiple WFV cameras into a big virtual camera (BVC). In this method, a BVC is created according to the rigorous imaging model of a single camera, from which a final stitched image and corresponding high accuracy rational function model (RFM) can be obtained simultaneously. WFV images from GF1 and reference digital elevation models from ZiYuan3 are used to validate the proposed method. The experimental results validate that seamless mosaicking can be achieved directly from the geometrical relationship of the imaging model of the BVC and WFV camera. The proposed method is able to maintain the absolute and relative internal positioning accuracy of the original images.

Image Mosaicking Approach for a Double-Camera System in the GaoFen2 Optical Remote Sensing Satellite Based on the Big Virtual Camera.

The linear array push broom imaging mode is widely used for high resolution optical satellites (HROS). Using double-cameras attached by a high-rigidity support along with push broom imaging is one method to enlarge the field of view while ensuring high resolution. High accuracy image mosaicking is the key factor of the geometrical quality of complete stitched satellite imagery. This paper proposes a high accuracy image mosaicking approach based on the big virtual camera (BVC) in the double-camera system on the GaoFen2 optical remote sensing satellite (GF2). A big virtual camera can be built according to the rigorous imaging model of a single camera; then, each single image strip obtained by each TDI-CCD detector can be re-projected to the virtual detector of the big virtual camera coordinate system using forward-projection and backward-projection to obtain the corresponding single virtual image. After an on-orbit calibration and relative orientation, the complete final virtual image can be obtained by stitching the single virtual images together based on their coordinate information on the big virtual detector image plane. The paper subtly uses the concept of the big virtual camera to obtain a stitched image and the corresponding high accuracy rational function model (RFM) for concurrent post processing. Experiments verified that the proposed method can achieve seamless mosaicking while maintaining the geometric accuracy.

一种光学遥感卫星多相机成像系统的高精度影像拼接方法

一种光学遥感卫星多相机成像系统的高精度影像拼接方法

A Mosaicking Approach for In Vivo Thickness Mapping of the Human Tympanic Membrane Using Low Coherence Interferometry.

The thickness of the human tympanic membrane (TM) is known to vary considerably across different regions of the TM. Quantitative determination of the thickness distribution and mapping of the TM is of significant importance in hearing research, particularly in mathematical modeling of middle-ear dynamics. Change in TM thickness is also associated with several middle-ear pathologies. Determination of the TM thickness distribution could therefore also enable a more comprehensive diagnosis of various otologic diseases. Despite its importance, very limited data on human TM thickness distribution, obtained almost exclusively from ex vivo samples, are available in the literature. In this study, the thickness distribution for the in vivo human TM is reported for the first time. A hand-held imaging system, which combines a low coherence interferometry (LCI) technique for single-point thickness measurement, with video-otoscopy for recording the image of the TM, was used to collect the data used in this study. Data were acquired by pointing the imaging probe over different regions of the TM, while simultaneously recording the LCI and concomitant TM surface video image data from an average of 500 locations on the TM. TM thickness distribution maps were obtained by mapping the LCI imaging sites onto an anatomically accurate wide-field image of the TM, which was generated by mosaicking the sequence of multiple small field-of-view video-otoscopy images. Descriptive statistics of the thickness measurements obtained from the different regions of the TM are presented, and the general thickness distribution trends are discussed.

The TanDEM-X DEM Mosaicking: Fusion of Multiple Acquisitions Using InSAR Quality Parameters

Since 2010, TanDEM-X and its twin satellite TerraSAR-X fly in a close orbit formation and form a single-pass synthetic aperture radar (SAR) interferometer. The formation was established to acquire a global high-precision digital elevation model (DEM) using SAR interferometry (InSAR). In order to achieve the required height accuracy of the TanDEM-X DEM, at least two global coverages have to be acquired. However, in difficult and mountainous terrain, up to five coverages are present. Here, acquisitions from ascending and descending orbits are needed to fill gaps and to overcome geometric limitations. Therefore, a strategy to properly combine the available height estimates is mandatory. The objective of this paper is the presentation of the operational TanDEM-X DEM mosaicking approach. In general, multiple InSAR DEM heights are combined by means of a weighted average with the height error as weight. Apart from this widely used mosaicking approach, one big challenge remains with the handling of larger height discrepancies between the input data, which are mainly caused by phase unwrapping errors, but also by temporal changes between acquisitions. In the case of inconsistencies, the TanDEM-X mosaicking approach performs a grouping into height levels. A priority concept is set up to evaluate the different groups of heights considering the number of DEMs and several InSAR quality parameters: the height error, the phase unwrapping method, and the height of ambiguity. This allows the identification of the most reliable height level for mosaicking. This fusion concept is verified on different test areas affected by phase unwrapping errors in flat and mountainous terrain as well as by height discrepancies in forests. The results show that the quality of the final TanDEM-X DEM mosaic benefits a lot from this mosaicking approach.

Real-time video mosaicking of the ocean floor

An approach was developed for real-time creation of video mosaics of the ocean floor. The approach uses visual correspondence during both image acquisition and consolidation to insure that there are no gaps in the mosaic. The use of visual information for mosaic creation offers an accuracy previously unachievable in real time. A complete mosaicking approach was developed, including a technique for image registration and several strategies for image acquisition control. The approach was implemented and tested experimentally. Ocean-floor mosaics were created in real time using a remotely-operated vehicle. In addition, extensive mosaicking experiments were conducted in a test-tank environment using a semiautonomous underwater robot.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>