Imaging Geometry Research Articles

Where does a glacier end? Integrated geophysical, geomorphological and photogrammetric measurements to image geometry and ice facies distribution

In this study we integrated different techniques, spanning from Ground Penetrating Radar to geomorphological and photogrammetric data, to characterize the glacial and proglacial environments of the Sforzellina Glacier (Central Alps, Italy). Direct data highlighted the presence of debris-covered ice and even of dead ice patches in front of the actual glacier terminus. Such an information was further proved by geophysical data. We try to correlate the dead ice patches occurrence with several parameters such as debris cover thickness, ground temperature, solar radiation, elevation, velocity of glacier retreat, and dip of the topographic surface without finding relevant correlations, except for the local bedrock morphology. In fact, the latter factor seems to be crucial to make the favourable conditions to dead ice patches formation and preservation with time.Since ice is not always outcropping, while dead ice patches never outcrop because they are hidden by continuous debris cover, classical glaciological monitoring techniques are not always successfully applicable.On the basis of the obtained results, we argue that estimates of glacier extension just related to surficial information, like in the case of exclusive use of remote sensing data and outcropping ice monitoring, can produce relevant underestimations. The presence of hidden ice patches, even not strictly part of the glacier body, is further important to quantify the total water equivalent, as well as to make affordable forecasts of the future glaciers evolution.

Beyond 3DMM: Learning to Capture High-Fidelity 3D Face Shape.

3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. However, previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry, which is attributed to insufficient ground-truth 3D shapes, unreliable training strategies and limited representation power of 3DMM. To alleviate this issue, this paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person. Specifically, given a 2D image as the input, we virtually render the image in several calibrated views to normalize pose variations while preserving the original image geometry. A many-to-one hourglass network serves as the encode-decoder to fuse multiview features and generate vertex displacements as the fine-grained geometry. Besides, the neural network is trained by directly optimizing the visual effect, where two 3D shapes are compared by measuring the similarity between the multiview images rendered from the shapes. Finally, we propose to generate the ground-truth 3D shapes by registering RGB-D images followed by pose and shape augmentation, providing sufficient data for network training. Experiments on several challenging protocols demonstrate the superior reconstruction accuracy of our proposal on the face shape.

Open-source software package for on-the-fly deskewing and live viewing of volumetric lightsheet microscopy data.

Oblique plane microscopy, OPM, is a form of lightsheet microscopy that permits volumetric imaging of biological samples at high temporal and spatial resolution. However, the imaging geometry of OPM, and related variants of light sheet microscopy, distorts the coordinate frame of the presented image sections with respect to the real space coordinate frame in which the sample is moved. This makes live viewing and practical operation of such microscopes difficult. We present an open-source software package that utilises GPU acceleration and multiprocessing to transform the display of OPM imaging data in real time to produce a live extended depth of field projection. Image stacks can be acquired, processed and plotted at rates of several Hz, making live operation of OPMs, and similar microscopes, more user friendly and intuitive.

A cross-polarization as a possible cause for color shift in illumination

A cross polarization could be indispensable in certain applications when scanning and digitizing highly reflective materials or when certain applications couldn’t afford following the recommended imaging geometry 0<sup>0</sup>/45<sup>0</sup> | 45<sup>o</sup>/0<sup>o</sup> for some technical reasons. However, that puts very much color fidelity in question, to which extent a cross polarization may impact the source illuminant in the first place that is consequently impacting the color appearance during the imaging and the color correction procedures. In this research we show how certain cross polarization setups are adding a chroma tint to the light source, D50 in this study, causing by that undesirable color shift of the color of the light source. Consequently, a shift in its color correlated temperature moving, in worst case scenario, from ~5000K to ~4500K and resulting in an increased DE00 as a result of the added chroma when compared against a standard D50; nearly doubled in best case scenario and nearly tripled in worst case scenario.

A Pose-Only Solution to Visual Reconstruction and Navigation.

Visual navigation and three-dimensional (3D) scene reconstruction are essential for robotics to interact with the surrounding environment. Large-scale scenarios and computational robustness are great challenges facing the research community to achieve this goal. This paper raises a pose-only imaging geometry representation and algorithms that might help solve these challenges. The pose-only representation, equivalent to the classical multiple-view geometry, is discovered to be linearly related to camera global translations, which allows for efficient and robust camera motion estimation. As a result, the spatial feature coordinates can be analytically reconstructed and do not require nonlinear optimization. Comprehensive experiments demonstrate that the computational efficiency of recovering the scene and associated camera poses is significantly improved by 2-4 orders of magnitude.

Suppressive Interference Suppression for Airborne SAR Using BSS for Singular Value and Eigenvalue Decomposition Based on Information Entropy

Suppressive interference is a common interference signal for synthetic aperture radar (SAR) that can seriously affect the target identification and imaging results of SAR. This paper proposes a method for suppressing suppressive jamming using blind source separation (BSS) for singular value and eigenvalue decomposition based on information entropy. First, we developed an airborne SAR imaging geometry model and a suppressive interference signal mixing model. Next, we perform blind signal separation of the interfered mixed signal by means of BSS based on singular value and eigenvalue decomposition. Then, we image the different signals we have extracted. Finally, we extract the features of the image domain for the separated signals and set the information entropy threshold by the difference of information entropy to identify the jamming signal and the source signal and obtain the source signal. This method uses eigenvalue and singular value decomposition for BSS and extracts the image domain features of the signal after BSS by information entropy and identifies the source signal by information entropy thresholding. This method compensates for the uncertainty in the decomposition of the signal by means of BSS. The signal loss is minimal and the similarity of the separated signal and the original signal is very high. Simulated and measured data demonstrate the feasibility of this algorithm.

S&GDA: An Unsupervised Domain Adaptive Semantic Segmentation Framework Considering Both Imaging Scene and Geometric Domain Shifts

Unsupervised domain adaptation uses labeled data from a source domain to help learn a target domain without any labeled data. Previous studies have not systematically analyzed the causes of remote sensing (RS) domain shifts, making it difficult to effectively model domain shifts caused by differences in geographic scene and platform imaging positions and attitudes. Therefore, this study conducts detailed analysis of the causes of domain shifts in RS images, and an unsupervised domain adaptive semantic segmentation framework, called S&GDA, that considers both imaging scene and geometric domain shifts is proposed. S&GDA comprised two modules: imaging scene simulation and imaging geometric simulation modules. The imaging scene simulation module is instrumental in mitigating domain shifts in geographical scenes due to variations in natural and human factors, thereby achieving cross-domain imaging scene consistency. Meanwhile, the imaging geometric simulation module allows for accurate simulation of domain shifts caused by changes in the position and attitude of a platform, ensuring cross-domain imaging geometry consistency. Note that none of these modules add additional parameters or computational complexity to the model as they only work on the input side of the data. Comprehensive experiments are conducted on the LoveDA and ISPRS datasets to evaluate S&GDA. Results indicate that S&GDA outperforms the state-of-the-art (SOTA) unsupervised domain adaptive semantic segmentation method by 3.12% of mIoU and can achieve 85% of the performance of the fully supervised method.

An Innovative Approach to Electrical Motor Geometry Generation Using Machine Learning and Image Processing Techniques

This paper presents a methodology for generating geometries for interior permanent magnet (IPM) motors in electric vehicles (EVs) through the application of artificial intelligence (AI) and image processing (IP) techniques. Due to the implementation of green agreements and policies aimed at reducing greenhouse gas emissions, EVs have become popularity. As a consequence, the improvement studies on the powertrain and battery system of EVs are focused. Especially for the powertrain, design optimization studies of e-motor have increased in the literature. One of the most widely used e-motor topologies is interior permanent magnet (IPM) motor. However, designing the IPM motor presents a challenge due to the dynamic considerations with geometric constraints. Therefore, e-motor designers encounter challenges related to determining initial geometry and the long time of the optimization process. To address these challenges, a novel approach is proposed that leverages machine learning (ML) techniques in combination with IP to generate initial geometries and design parameters for IPM motors. The proposed approach generates images of the motor geometry and extract dimensional features from the resulting images by using artificial neural networks (ANNs). The proposed method performs an analysis of the input vectors to reduce their size using techniques such as Histogram, 2D Maximum, 2D Mean, 2D Minimum, 2D Standard Deviation, and 2D Variance to enhance feature extraction. Additionally, FFT (Fast Fourier Transform) and IFFT (Inverse Fast Fourier Transform) are used to improve the neural network process in generating the image geometry. Further, the generated image geometry is improved by applying digital filtering techniques such as Log, FFT, Log+FFT, Laplacian, Sobel, and Histogram Equalization. Finally, the trained ANNs are tested to validate the results by using Ansys RMXprt and Maxwell. Eventually, the proposed method represents an innovative solution to generating initial geometries for IPM motors in EVs that satisfies desired design requirements. This approach leverages the power of AI and image processing techniques, which could lead to significant improvements in the optimization process for IPM motors, accelerate the designer’s analysis process, and enhance the performance of EVs.

Efficient and Stable Generation of High-Resolution Hair and Fur With ConvNet Using Adaptive Strand Geometry Images

Efficient and Stable Generation of High-Resolution Hair and Fur With ConvNet Using Adaptive Strand Geometry Images

Improving ultrasound-based brachytherapy needle conspicuity by applying an echogenic coating.

Applicator conspicuity in ultrasound-guided brachytherapy procedures is commonly impaired by imaging artifacts or non-ideal imaging geometry, which can slow down applicator position digitization and increase the geometric uncertainty of the delivered dose distribution. The purpose of this study was to improve the conspicuity of high-dose rate (HDR) brachytherapy needles under B-mode ultrasound imaging by applying an echogenic surface coating. Our hypothesis was that an echogenic coating would reduce artifacts and improve needle visualization within regions of signal degradation. In this study, 17-gauge, 25-cm long titanium HDR brachytherapy needles were coated with acoustically reflective microspheres over a 2.5cm region starting from the needle tip. Three coating thicknesses (27μm, 40μm, 64μm) were compared against an uncoated control needle. The coated and uncoated needles were imaged using B-mode ultrasound in a tissue-equivalent prostate phantom and in a cadaverous male pelvis using a transrectal probe. Needle conspicuity was assessed under multiple conditions: a single needle implant, an implant with multiple needles between the probe and the needle of interest, and an angled needle implant. All images were assessed qualitatively for needle conspicuity and the presence of artifacts and quantitatively using grey-scale image intensity values. The 64μm echogenic coating reduced the magnitude of reverberation artifacts by 31±14% and comet tail artifacts by 40%-70%. The echogenic coating also improved needle contrast, measured by the relative differences in signal intensity compared with the adjacent environment, when needles were angled up to 30° with respect to the transducer probe in the cadaver. The improvements in conspicuity and artifact reduction increased with increasing coating thickness. The performance of the needles coated with the 64μm thickness was qualitatively superior and yielded high-contrast, well-circumscribed signals in the cadaverous male pelvis, even under situations where a needle was acoustically shadowed by multiple other needles. An echogenic surface coating reduced imaging artifacts and improved needle conspicuity under realistic clinical conditions for ultrasound-based prostate or gynecological brachytherapy. The improved conspicuity has the potential to improve the efficiency of needle placement and the accuracy of needle position digitization during brachytherapy procedures.

Creation of a Virtual Tour .Exe Utilizing Very High-Resolution RGB UAV Data

In the last decades, developments in game engine technology led to a raised attraction to the virtual reality (VR) and augmented reality (AR) concepts which offer users an interactive synthetic environment. Also, with the travel limitations of the current COVID-19 pandemic, VR tour applications that visualize the geospatial data gained popularity more than ever. In this study, a three-dimensional (3D) VR tour application was created for Gebze Technical University (GTU) Campus by integrating unmanned aerial vehicle (UAV) data into an artificial environment by using cross-platform game development engine Unity. For creating high-quality 3D models of the Campus, different imaging geometries and flight altitudes were applied. The aerial photos were achieved with a ground sampling distance (GSD) of ≤2.2 cm with a 20 megapixel (MP) Sony Exmor RGB camera. Point cloud processing and the generation of high-quality 3D products were carried out by structure from motion (SfM) based photogrammetric software Agisoft Metashape. Using 86 well-distributed ground control points (GCPs), geometric correction accuracy of ±2 cm (~0.9 pixels) was reached as root mean square error (RMSE). Generated 3D models were imported into the Unity environment and the negative influence of high polygon data on the application performance was reduced by applying occlusion culling and space subdivision rendering optimization algorithms. The visual potential of the VR was improved by adding 3D individual object models such as trees, benches and arbors. For enhancing the information content of the VR tour, interactive information panels including the building metadata such as building name, block name and total floor area were placed. Finally, a first-person player was implemented for a realistic VR experience.

Three‐dimensional point cloud reconstruction of inverse synthetic aperture radar image sequences based on back projection and iterative closest point fusion

AbstractIn order to recover the three‐dimensional (3D) structure of the target from sequential inverse synthetic aperture radar (ISAR) images, the factorisation method is generally used. It requires a large number of high‐quality matched feature points from different ISAR images. If the number of extracted feature points is insufficient, the restored 3D structure is not obvious. Furthermore, the mismatching of feature points will greatly affect the quality of target reconstruction. However, the factorisation method only uses the information from the ISAR images, while that from imaging geometry is not sufficiently considered. ISAR imaging is a kind of projection, and the projection plane information could be taken into account for the 3D reconstruction. Hence, a new 3D reconstruction method for stable targets from sequential ISAR images is proposed in this paper. Firstly, the ISAR images are preprocessed with the CLEAN algorithm. The maximum between‐cluster variance method is applied to extract feature points from the processed images. Moreover, the image projection plane and projection equation corresponding to different ISAR images are analysed by using imaging geometry information. According to the projection equation, the feature points are back‐projected (BP) to the 3D space. Finally, the 3D point clouds obtained by the BP from multiple radar images are fused by the iterative closest point algorithm to restore the 3D structure of the target. The simulation and experiment results show the effectiveness and robustness of this method.

The representational geometry of images and concepts in perception and memory

The representational geometry of images and concepts in perception and memory

Digital geometry on a cubic stair-case mesh

• The cubic grid Z 3 plays important role in the (digital) world. • Meshes obtained by cutting Z 3 by oblique planes can display 2d images. • The dual semi-regular grid, the rhombille tiling is an oblique cubic mesh. • Shortest path algorithm and digital distance are computed on the rhombille tiling. • The rhombille tiling can be used as a digital canvas and for RGB pixel geometry. In this paper, we investigate digital geometry on the rhombille tiling, D(6,3,6,3), that is the dual of the semi-regular tiling called hexadeltille T(6,3,6,3) tiling and also known as trihexagonal tiling. In fact, this tiling can be seen as an oblique mesh of the cubic grid giving practical importance to this specific grid both in image processing and graphics. The properties of the coordinate systems used to address the tiles are playing crucial roles in the simplicity of various algorithms and mathematical formulae of digital geometry that allow to work on the grid in image processing, image analysis and computer graphics, thus we present a symmetric coordinate system. This coordinate system has a strong relation to topological/combinatorial coordinate system of the cubic grid. It is an interesting fact that greedy shortest path algorithm may not be used on this grid, despite to this, we present algorithm to provide a minimal-length path between each pair of tiles, where paths are defined as sequences of neighbor tiles (those are considered to be neighbors which share a side). We also prove closed formula for computing the digital, i.e., path-based distance, the length (the number of steps) of a/the shortest path(s). Some example pictures on this grid are also presented, as well as its possible application as pixel geometry for color images and videos on the hexagonal grid. To create your abstract, type over the instructions in the template box below. Fonts or abstract dimensions should not be changed or altered.

Semi-analytic simulation of optical wave propagation through turbulence.

Split-step wave-optical simulations are useful for studying optical propagation through random media like atmospheric turbulence. The standard method involves alternating steps of paraxial vacuum propagation and turbulent phase accumulation. We present a semianalytic approach to evaluating the Fresnel diffraction integral with onephase screen between the source and observation planes and another screen in the observation plane. Specifically, we express the first phase screen's transmittance as a Fourier series, which allows us to bring phase screen effects outside of the Fresnel diffraction integral, thereby reducing the numerical computations. This particular setup is useful for simulating astronomical imaging geometries and two-screen laboratory experiments that emulate real turbulence with phase wheels, spatial light modulators, etc. Further, this is a key building block in more general semianalytic split-step simulations that have an arbitrary number of screens. Compared with the standard angular-spectrum approach using the fast Fourier transform, the semianalytic method provides relaxed sampling constraints and an arbitrary computational grid. Also, when a limited number of observation-plane points is evaluated or when many time steps or random draws are used, the semianalytic method can compute faster than the angular-spectrum method.

Multi-perspective region-based CNNs for vertebrae labeling in intraoperative long-length images

PurposeEffective aggregation of intraoperative x-ray images that capture the patient anatomy from multiple view-angles has the potential to enable and improve automated image analysis that can be readily performed during surgery. We present multi-perspective region-based neural networks that leverage knowledge of the imaging geometry for automatic vertebrae labeling in Long-Film images – a novel tomographic imaging modality with an extended field-of-view for spine imaging. MethodA multi-perspective network architecture was designed to exploit small view-angle disparities produced by a multi-slot collimator and consolidate information from overlapping image regions. A second network incorporates large view-angle disparities to jointly perform labeling on images from multiple views (viz., AP and lateral). A recurrent module incorporates contextual information and enforce anatomical order for the detected vertebrae. The three modules are combined to form the multi-view multi-slot (MVMS) network for labeling vertebrae using images from all available perspectives. The network was trained on images synthesized from 297 CT images and tested on 50 AP and 50 lateral Long-Film images acquired from 13 cadaveric specimens. Labeling performance of the multi-perspective networks was evaluated with respect to the number of vertebrae appearances and presence of surgical instrumentation. ResultsThe MVMS network achieved an F1 score of >96% and an average vertebral localization error of 3.3 mm, with 88.3% labeling accuracy on both AP and lateral images – (15.5% and 35.0% higher than conventional Faster R-CNN on AP and lateral views, respectively). Aggregation of multiple appearances of the same vertebra using the multi-slot network significantly improved the labeling accuracy (p < 0.05). Using the multi-view network, labeling accuracy on the more challenging lateral views was improved to the same level as that of the AP views. The approach demonstrated robustness to the presence of surgical instrumentation, commonly encountered in intraoperative images, and achieved comparable performance in images with and without instrumentation (88.9% vs. 91.2% labeling accuracy). ConclusionThe MVMS network demonstrated effective multi-perspective aggregation, providing means for accurate, automated vertebrae labeling during spine surgery. The algorithms may be generalized to other imaging tasks and modalities that involve multiple views with view-angle disparities (e.g., bi-plane radiography). Predicted labels can help avoid adverse events during surgery (e.g., wrong-level surgery), establish correspondence with labels in preoperative modalities to facilitate image registration, and enable automated measurement of spinal alignment metrics for intraoperative assessment of spinal curvature.

Beam-Shape Effects and Noise Removal From THz Time-Domain Images in Reflection Geometry in the 0.25–6 THz Range

The increasing need of restoring high-resolution hyper-spectral (HS) images is determining a growing reliance on computer vision-based processing to enhance the clarity of the image content. HS images can, in fact, suffer from degradation effects or artefacts caused by instrument limitations. This article focuses on a procedure aimed at reducing the degradation effects, frequency-dependent blur and noise, in Terahertz time-domain spectroscopy (THz-TDS) images in reflection geometry. It describes the application of a joint deblurring and denoising approach that had been previously proved to be effective for the restoration of THz-TDS images in transmission geometry, but that had never been tested in reflection modality. This mode is often the only one that can be effectively used in most cases, for example, when analyzing objects that are either opaque in the THz range, or that cannot be displaced from their location (e.g., museums), such as those of cultural interest. Compared to transmission mode, reflection geometry introduces, however, further distortion to THz data, usually neglected in existing literature. In this work, we successfully implement image deblurring and denoising of both uniform-shape samples (a contemporary 1 Euro cent coin and an inlaid pendant) and samples with the uneven reliefs and corrosion products on the surface, which make the analysis of the object particularly complex (an ancient Roman silver coin). The study demonstrates the ability of image processing to restore data in the 0.25–6 THz range, spanning over more than four octaves, and providing the foundation for future analytical approaches of cultural heritage using the far-infrared spectrum still not sufficiently investigated in the literature.

Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction

Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur. • DeepSfT is a DNN fully-convolutional based on residual encoder-decoder for SfT. • We propose a semi-supervised end-to-end training procedure with synthetic and real data. • DeepSfT is a solution to cope with multiple imaging geometries, at training and inference. • We propose to combine DeepSfT with a physics-based estimation procedure.

A new phase unwrapping method for cross-track interferometric synthetic aperture radar systems

One of the essential applications of interferometric synthetic aperture radar (InSAR) systems is to estimate the ground surface height. To this end, the altitude of each pixel in the region is estimated according to the phase difference of the two received signals by two sensors. However, the sensor has an ambiguity in the relative terrain altitude due to the 2π cyclic nature of the interferometric phase, which leads to a significant error in altitude estimation. In this paper, by focusing on the phase unwrapping issue in InSAR processing, an efficient iterative approach to minimize the phase difference error is presented. For this purpose, first, according to the InSAR imaging geometry, the interferometric phase relationships are described. Second, based on the least-squares approach, the phase unwrapping problem is expressed as an optimization issue with a specific cost function. Third, the cost function is converted to a submodular convex Markov random field (MRF) function. Finally, the resulting optimization problem is solved by using a fast and efficient approach to optimize submodular convex MRF functions. The proposed method's performance is examined using several numerical studies compared to its relevant competitors. The mean square error (MSE) of elevation estimation is considered the performance metric. Numerical simulations have confirmed the efficiency of the proposed algorithm in terms of MSE of elevation estimation. As seen, the outperformance of the proposed method with respect to its best competitor is at least 2 times (3 dB). The operation times of various methods are also compared as a measure of their convergence rate. Based on simulation results, the convergence rate of the proposed algorithm is more than 50 times better than that of its closest competitor.

Gravitational lensing effects of supermassive black holes in cluster environments

ABSTRACT This study explores the gravitational lensing effects of supermassive black holes (SMBHs) in galaxy clusters. While the presence of central SMBHs in galaxies is firmly established, recent work from high-resolution simulations predict the existence of an additional population of wandering SMBHs. Though the masses of these SMBHs are a minor perturbation on the larger scale and individual galaxy scale dark matter components in the cluster, they can impact statistical lensing properties and individual lensed image configurations. Probing for these potentially observable signatures, we find that SMBHs imprint detectable signatures in rare, higher order strong lensing image configurations although they do not manifest any statistically significant detectable evidence in either the magnification distribution or the integrated shear profile. Investigating specific lensed image geometries, we report that a massive, near point-like, potential of an SMBH causes the following detectable effects: (i) image splitting leading to the generation of extra images; (ii) positional and magnification asymmetries in multiply imaged systems; and (iii) the apparent disappearance of a lensed counter image. Of these, image splitting inside the cluster tangential critical curve, is the most prevalent notable observational signature. We demonstrate these possibilities in two cases of observed giant arcs in SGAS J003341.5+024217 and RX J1347.5−1145, wherein specific image configurations seen can be reproduced with SMBHs. Future observations with high-resolution instrumentation (e.g. MAVIS-Very Large Telescope, MICADO-Extremely Large Telescope, and the upgraded ngVLA, along with data from the Euclid and Nancy Grace Roman Space Telescopes and the Rubin LSST Observatory are likely to allow us to probe these unique yet rare SMBHs lensing signatures.