Geometric Misalignment Research Articles

Improved high dynamic range imaging using multi-scale feature flows balanced between task-orientedness and accuracy

Deep learning has made it possible to accurately generate high dynamic range (HDR) images from multiple images taken at different exposure settings, largely owing to advancements in neural network design. However, generating images without artifacts remains difficult, especially in scenes with moving objects. In such cases, issues like color distortion, geometric misalignment, or ghosting can appear. Current state-of-the-art network designs address this by estimating the optical flow between input images to align them better. The parameters for the flow estimation are learned through the primary goal, producing high-quality HDR images. However, we find that this ”task-oriented flow” approach has its drawbacks, especially in minimizing artifacts. To address this, we introduce a new network design and training method that improve the accuracy of flow estimation. This aims to strike a balance between task-oriented flow and accurate flow. Additionally, the network utilizes multi-scale features extracted from the input images for both flow estimation and HDR image reconstruction. Our experiments demonstrate that these two innovations result in HDR images with fewer artifacts and enhanced quality.

A multi-view references image super-resolution framework for generating the large-FOV and high-resolution image

This paper investigates the generation of large field-of-view (FOV) and high-resolution (HR) panorama images for smartphones. Existing techniques like image stitching struggle to produce satisfactory results due to geometry misalignment and inconsistent appearance. To circumvent the inherent challenges of image stitching methods and generate high-quality panoramas, we treat the image stitching problem as a multiple-reference-based super-resolution problem. Specifically, one large-FOV low-resolution (LR) image and several overlapped small-FOV HR images are taken as inputs, where the LR image acts as a base and multi-view references provide rich HR information. Building on this foundation, a novel multi-view references image super-resolution framework (MVRefSR) is proposed. Within this framework, to address the residual geometric misalignment between the LR-Ref image pairs after coarse alignment, a flow-based RefSR network (FlowSRNet) is proposed, which super-resolves LR patches with corresponding HR references. To facilitate adaptive feature fusion and minimize distortion in structured regions, a fusion weight estimation module and a gradient branch are introduced in FlowSRNet. Finally, the large-FOV HR image is generated by combining these SR patches together. Furthermore, the lack of real-world RefSR datasets for smartphones is addressed by designing an innovative dataset construction pipeline. Extensive experiments demonstrate the superior performance of FlowSRNet and MVRefSR over the compared SR methods and image stitching software, which proves the effectiveness of generating panoramas from a new perspective.

A cluster-based disambiguation method using pose consistency verification for structure from motion

Structure from motion (SfM) recovers scene structures and camera poses based on feature matching, and faces challenges from ambiguous scenes. There are a large number of ambiguous scenes in real environment, which contain many duplicate structures and textures. The ambiguity leads to incorrect feature matches between images with similar appearance, and makes geometric misalignment in SfM. To address this problem, recent methods have focused on investigating the inconsistencies in feature topology among multi-view images. However, the feature topology is directly derived from 2D images. Thus, it is susceptible to feature occlusion caused by changes in perspective. Therefore, we propose a new method that disambiguates scenes using pose consistency rather than feature consistency. The pose consistency is conducted in 3D geometric space which is less sensitive to feature occlusion. Thus, the pose consistency is more robust than feature consistency. Our core motivation lies that the incorrect matches between ambiguous images will cause pose deviation from the global poses generated by correct matches. To detect this pose deviation, we first combine local and global information of the scene to generate the global reliable camera poses. The local information of each image is obtained by image clustering, and it strengthens the global information that is represented as the verified maximum spanning tree of clusters. Then, the global poses serve as the reference for further pose consistency verification. The global poses also enable us to perform both rotation and translation consistency verification for uncertain matches. During the pose consistency verification, the pose deviation calculated on image-level may be too small to be noticed. Thus, we propose to perform pose consistency verification at cluster-level instead of image-level to amplify the pose deviation. In the experiments, we compared our approach with several state-of-the-art methods, including COLMAP, Geodesic-SfM and TC-SfM, on both ambiguous and regular datasets. The results demonstrate that our approach achieves the best robustness, only our approach succeeds on all ambiguous image sequences (14/14). The quantitative evaluation results on image sequences with ground truth also show that our approach achieves the best accuracy (average RMSE of translation = 0.109, average RMSE of rotation = 0.827) among all methods. The source code of our approach is publicly available at https://github.com/gongyeted/MA-SfM.

Automated three-dimensional image registration for longitudinal photoacoustic imaging.

Photoacoustic tomography (PAT) has great potential in monitoring disease progression and treatment response in breast cancer. However, due to variations in breast repositioning, there is a chance of geometric misalignment between images. Further, poor repositioning can affect light fluence distribution and imaging field-of-view, making images different from one another. The net effect is that it becomes challenging to distinguish between image changes due to repositioning effects and those due to true biological variations. The aim is to develop a three-dimensional image registration framework for geometrically aligning repeated PAT volumetric images, which are potentially affected by repositioning effects such as misalignment, changed radiant exposure conditions, and different fields-of-view. The proposed framework involves the use of a coordinate-based neural network to represent the displacement field between pairs of PAT volumetric images. A loss function based on normalized cross correlation and Frangi vesselness feature extraction at multiple scales was implemented. We refer to our image registration framework as MUVINN-reg, which stands for multiscale vesselness-based image registration using neural networks. The approach was tested on a longitudinal dataset of healthy volunteer breast PAT images acquired with the hybrid photoacoustic-ultrasound Photoacoustic Mammoscope 3 imaging system. The registration performance was also tested under unfavorable repositioning conditions such as intentional mispositioning, and variation in breast-supporting cup size between measurements. A total of 13 pairs of repeated PAT scans were included in this study. MUVINN-reg showed excellent performance in co-registering each pair of images. The proposed framework was shown to be robust to image intensity shifts and field-of-view changes. Furthermore, MUVINN-reg could align vessels at imaging depths greater than 4cm. The proposed framework will enable the use of PAT for quantitative and reproducible monitoring of disease progression and treatment response.

Harnessing the synergy of grout and adhesive: Numerical prediction of load capacity in hybrid composite joints

This study examines grouted joints for offshore wind turbine systems, where gaps between steel pipes are filled with grout, relying on adhesion, friction and compression struts formed by shear keys. Concerns about the limited load carrying capacity of large diameter grouted joints in offshore structures have prompted investigation. Whilst adhesive bonds are recognized for load transfer in structural steel, the complete replacement of grouting with bonding alone is proving impractical for large gaps in offshore structures, posing challenges in terms of cost and handling. To address these challenges, the study presents an innovative hybrid grouted joint for steel structures that combines grout and adhesive layers. This novel approach replaces traditional large shear keys with distributed micro-shear keys — small granules embedded in the grout material. Under axial loading, the hybrid joint demonstrates robust performance, with a maximum nominal shear stress on the inner pipe of 30.1 MPa and consistent load capacity across tests. Average shear strength is in line with, and occasionally exceeds, expectations. Notably, the hybrid joint shows resilience in different configurations and maintains a low coefficient of variation of 5.5%, indicating consistent performance. When the influence of the adhesive material on the hybrid joint is examined, the effect on the joint stiffness is minimal, despite variations in adhesive stiffness. Sikadur 370, which has a higher modulus of elasticity than DuploTEC, shows similar deformation curves in the elastic region, emphasizing the robustness of the joint. Differences in the maximum loads are attributed to the thickness of the adhesive layer and the lower modulus of DuploTEC, resulting in different shear strengths. The proposed hybrid joint offers a promising solution to improve the performance of conventional grouts and adhesives in offshore structures. The study also formulates a failure criterion, performs numerical stress analysis, and models the effect of geometric dimensions, eccentricity, and misalignment on load capacity. The methodology’s accuracy in predicting load capacity, supported by numerical analysis and large-scale joint simulations, contributes to a comprehensive understanding of hybrid joint performance in engineering applications.

Layer-Number Engineered Momentum-Indirect Interlayer Excitons with Large Spectral Tunability.

Interlayer excitons (IXs) in type II van der Waals (vdW) heterostructures are equipped with an oriented permanent dipole moment and long lifetime and thus would allow promising applications in excitonic and optoelectronic devices. However, based on the widely studied heterostructures of transition-metal dichalcogenides (TMDs), IX emission is greatly influenced by the lattice mismatch and geometric misalignment between the constituent layers, increasing the complexity of the device fabrication. Here, we report on the robust momentum-indirect IX emission in TMD/two-dimensional (2D) perovskite vdW heterostructures, which were fabricated without considering the orientation arrangement or momentum mismatch. The IXs show a large diffusion coefficient of ∼10 cm2 s-1, and importantly the IX emission energy can be widely tuned from 1.3 to 1.6 eV via changing the layer number of the 2D perovskite or the thickness of TMD flakes, shedding light on the applications of vdW interface engineering to broad-spectrum optoelectronics.

Optimizing Local Alignment along the Seamline for Parallax-Tolerant Orthoimage Mosaicking

Orthoimage mosaicking with obvious parallax caused by geometric misalignment is a challenging problem in the field of remote sensing. Because the obvious objects are not included in the digital terrain model (DTM), large parallax exists in these objects. A common strategy is to search an optimal seamline between orthoimages, avoiding the majority of obvious objects. However, stitching artifacts may remain because (1) the seamline may still cross several obvious objects and (2) the orthoimages may not be precisely aligned in geometry when the accuracy of the DTM is low. While applying general image warping methods to orthoimages can improve the local geometric consistency of adjacent images, these methods usually significantly modify the geometric properties of orthophoto maps. To the best of our knowledge, no approach has been proposed in the field of remote sensing to solve the problem of local geometric misalignments after orthoimage mosaicking with obvious parallax. In this paper, we creatively propose a method to optimize local alignment along the seamline after seamline detection. It consists of the following main processes. First, we locate regions with geometric misalignments along the seamline based on the similarity measure. Second, for any one region, we find one-dimensional (1D) feature matches along the seamline using a semi-global matching approach. The deformation vectors are calculated for these matches. Third, these deformation vectors are robustly and smoothly propagated into the buffer region centered on the seamline by minimizing the associated energy function. Finally, we directly warp the orthoimages to eliminate the local parallax under the guidance of dense deformation vectors. The experimental results on several groups of orthoimages show that our proposed approach is capable of eliminating the local parallax existing in the seamline while preserving most geometric properties of digital orthophoto maps, and that it outperforms state-of-the-art approaches in terms of both visual quality and quantitative metrics.

Hyperspectral Image Stitching via Optimal Seamline Detection

Hyperspectral images (HSIs) with both spatial and spectral information have found broad applications. Since most cameras have narrow viewing angle, generating panoramic images is essential to show a large-range view of the environment. So far, there are few studies on HSI stitching, and the stitching result still suffers from some problems, such as blurring and ghosting, geometric misalignment, visible seam, and spectral distortion. Hence, to address the above disadvantages, we propose a novel HSI stitching strategy using optimal seamline detection approach in this letter. First, we use a fast and robust seam estimation method to determine the seamline in each single band of HSI. This method works in RGB images, and we have modified it to be used in a single-band gray-scale image of HSI. Then, to guarantee the integrity of spatial and spectral information of hundreds of bands of HSI, we propose to apply the structural similarity (SSIM) index to select the optimal one among all band candidate seamlines and use the selected optimal seamline to stitch all the remaining bands. The experimental results demonstrate that our proposed approach outperforms traditional HSI stitching approach in both spatial and spectral performances.

On the Joint Effect of Rain and Beam Misalignment in Terahertz Wireless Systems

This contribution focuses on extracting the theoretical framework for the assessment and evaluation of the joint effect of rain, beam misalignment and hardware imperfections at long-range outdoor terahertz (THz) wireless systems. In this direction, we first report an appropriate system model for outdoor THz wireless systems that take into account the impact of different design parameters, including antenna gain and transceivers hardware imperfections, atmospheric conditions, such as rain, and parameters, like temperature, humidity and pressure, as well as stochastic beam misalignment that can be caused by thermal expansion, dynamic wind loads and/ot weak earthquakes. For this model, we extract novel closed-form expressions for the probability density and cumulative distribution functions of the THz wireless channel that captures the impact of geometric loss, beam misalignment and rain attenuation. We capitalized the aforementioned expressions by presenting closed-form formulas for the outage probability and achievable throughput of the system. Finally, we document an analytical policy that returns the optimal transmission spectral efficiency that maximizes the achievable throughput.

Effects of Receiver Misalignment on the Intercept Factor of Parabolic Trough Collectors

Abstract This study developed a methodology for determining the intercept factor (γ) and reports the effects of varying the design parameters, within realistic ranges, on the optical performance of parabolic trough collectors (PTC). The design equations account for the aperture width of the concentrator, the geometric concentration ratio, rim-angle, and misalignment of the receiver, characterized by the radial and angular offsets. The Latin Hypercube Sampling (LHS)-based ray-tracing method was used. Validation of the method was performed by comparing the results obtained by tracing a single ray with measurements obtained in a two-dimensional (2D) drafting program. Then, parametric analyses were performed to determine the effects of changing the radial and angular offset, geometric concentration ratio, and rim-angle on the percentage change in intercept factor (γΔ) with respect to the γ of a PTC with a perfectly aligned receiver. Finally, contour plots depicting the γ of various standard troughs over the range of radial and angular offsets were plotted, compared, and discussed. An online tool is also presented to help designers and researchers analyze these effects quickly by providing the design parameters.

Integrated Preprocessing of Multitemporal Very-High-Resolution Satellite Images via Conjugate Points-Based Pseudo-Invariant Feature Extraction

Multitemporal very-high-resolution (VHR) satellite images are used as core data in the field of remote sensing because they express the topography and features of the region of interest in detail. However, geometric misalignment and radiometric dissimilarity occur when acquiring multitemporal VHR satellite images owing to external environmental factors, and these errors cause various inaccuracies, thereby hindering the effective use of multitemporal VHR satellite images. Such errors can be minimized by applying preprocessing methods such as image registration and relative radiometric normalization (RRN). However, as the data used in image registration and RRN differ, data consistency and computational efficiency are impaired, particularly when processing large amounts of data, such as a large volume of multitemporal VHR satellite images. To resolve these issues, we proposed an integrated preprocessing method by extracting pseudo-invariant features (PIFs), used for RRN, based on the conjugate points (CPs) extracted for image registration. To this end, the image registration was performed using CPs extracted using the speeded-up robust feature algorithm. Then, PIFs were extracted based on the CPs by removing vegetation areas followed by application of the region growing algorithm. Experiments were conducted on two sites constructed under different acquisition conditions to confirm the robustness of the proposed method. Various analyses based on visual and quantitative evaluation of the experimental results were performed from geometric and radiometric perspectives. The results evidence the successful integration of the image registration and RRN preprocessing steps by achieving a reasonable and stable performance.

Simple and efficient registration of 3D point cloud and image data for an indoor mobile mapping system.

Registration of 3D lidar point clouds with optical images is critical in the combination of multisource data. Geometric misalignment originally exists in the pose data between lidar point clouds and optical images. To improve the accuracy of the initial pose and the applicability of the integration of 3D points and image data, we develop a simple but efficient registration method. We first extract point features from lidar point clouds and images: point features are extracted from single-frame lidar and point features are extracted from images using a classical Canny operator. The cost map is subsequently built based on Canny image edge detection. The optimization direction is guided by the cost map, where low cost represents the desired direction, and loss function is also considered to improve the robustness of the proposed method. Experiments show positive results.

Video Data Extraction and Processing for Investigation of Vehicles’ Impact on the Asphalt Deformation Through the Prism of Computational Algorithms

There are numerous algorithms and solutions for car or object detection as humanity is aiming towards the smart city solutions. Most solutions are based on counting, speed detection, traffic accidents and vehicle classification. The mentioned solutions are mostly based on high-quality videos, wide angles camera view, vehicles in motion, and are optimized for good visibility conditions intervals. A novelty of the proposed algorithm and solution is more accurate digital data extraction from video file sources generated by security cameras in Bosnia and Herzegovina from M18 roadway, but not limited only to that particular source. From the video file sources, data regarding number of vehicles, speed, traveling direction, and time intervals for the region of interest will be collected. Since finding contours approach is effective only on objects that are mobile, and because the application of this approach on traffic junctions did not yield desired results, a more specific approach of classification using a combination of Histogram of Oriented Gradients (HOG) and Support Vector Machines (Linear SVM) has shown to be more appropriate as the original source data can be used for training where the main benefit is the preservation of local second-order interactions, providing tolerance to local geometric misalignment and ability to work with small data samples. The features of the objects within a frame are extracted first by standardizing the feature variables and then computing the first order gradients of the frame. In the next stage, an encoding that remains robust to small changes while being sensitive to local frame content is produced. Finally, the HOG descriptors are generated and normalized again. In this way the channel histogram and spatial vector becomes the feature vector for the Linear SVM classifier. With the following parameters and setup system accuracy was around 85 to 95%. In the next phase, after cleaning protocols on collected data parameters, data will be used to research asphalt deformation effects.

Impact of angular pointing error on BER performance of underwater optical wireless links.

Even in clear ocean water, underwater optical wireless communication (UOWC) is impaired not only by absorption and scattering, but also by oceanic turbulence and dynamic pointing errors which result in a fading channel, degrading the bit error rate (BER) performance. In this paper, for the first time, we quantify analytically the trade-off between geometric loss and misalignment in underwater scattering channels. A novel geometric loss model is developed which is used to compute the average BER in the presence of absorption and scattering over salinity-induced oceanic turbulence channels. Our findings suggest that UOWC systems are less sensitive to angular pointing errors due to jitter since scattering is able to alleviate such a fading effect at the expense of a higher attenuation due to geometric spread. Monte Carlo simulation results are further included to verify the developed BER expression which is valid over a wide range of signal-to-noise-ratio (SNR). Finally, the impact of inter-symbol interference (ISI) is also quantified by measuring the optical power penalty.

Operational Coregistration of the Sentinel-2A/B Image Archive Using Multitemporal Landsat Spectral Averages

Geometric misalignment between Landsat and Sentinel-2 data sets as well as multitemporal inconsistency of Sentinel-2A and -2B data sets currently complicate multitemporal analyses. Operational coregistration of Sentinel-2A and -2B imagery is thus required. We present a modification of the established Landsat Sentinel Registration (LSReg) algorithm. The modifications enabled LSReg to be included in an operational preprocessing workflow to automatically coregister large volumes of Sentinel-2 imagery with Landsat base images that represent multiannual monthly spectral average values. The modified LSReg was tested for the complete Sentinel-2 archive covering Crete, Greece, which is a particularly challenging region due to steep topographic gradients and high shares of water in Sentinel-2 tiles. A coregistration success rate of 87.5% of all images was obtained with a mean coregistration precision of 4.4 m. The mean shifts of 14.0 m in the x-direction and 13.4 m in the y-direction before coregistration were found, with maxima exceeding four pixels. Time series noise in locations with land cover transitions ( n = 585) was effectively reduced by 43% using the presented approach. The multitemporal geometric consistency of the Sentinel-2 data set was substantially improved, thus enabling time series analyses within the Sentinel-2 data record, as well as integrated Landsat and Sentinel-2A and -2B data sets. The modified algorithm is implemented in the Framework for Operational Radiometric Correction for Environmental monitoring (FORCE) version 3.0 (https://github.com/davidfrantz/force).

A self-adaptive alignment strategy for large components based on dynamic compliance center

PurposeAiming at the problems of geometric precision misalignment and unconsidered physical constraints between large components during the measurement-assisted assembly, a self-adaptive alignment strategy based on the dynamic compliance center (DCC) is proposed in this paper, using force information to guide alignment compliantly.Design/methodology/approachFirst, the self-adaptive alignment process of large components is described, and its geometrical and mechanical characteristics are analyzed based on six-dimensional force/torque (F/T). The setting method of DCC is studied and the areas of DCC are given. Second, the self-adaptive alignment platform of large components driven by the measured six-dimensional F/T is constructed. Based on this platform, the key supporting technologies, including principle of self-adaptive alignment, coordinate transfer, calculation of six-dimensional F/T and alignment process control, are illustrated.FindingsUsing the presented strategy, the position and orientation of large component is adjusted adaptively responding to measured six-dimensional F/T and the changes of contact states are consistent with the strategy. Through the setting of DCC, alignment process runs smoothly without jamming.Practical implicationsThis strategy is applied to the alignment experiment of large components muff coupling. The experimental results show that the proposed alignment strategy is correct and effective and meets the real-time requirement.Originality/valueThis paper proposed a novel way to apply force information in large component self-adaptive alignment, and the setting method of DCC was presented to make the alignment process more feasible.

A low-cost and easy-to-use phantom for cone-beam geometry calibration of a tomographic X-ray system

Knowledge of the acquisition geometry is key for tomographic reconstruction. Before image reconstruction algorithms can be applied to compute a 3D image from a set of 2D projections, calibration must be carried out to correct geometrical inaccuracies. The main source of geometric misalignment can be attributed to possible mechanical instability and slight offsets in rotation and translation of the source, detector and/or the sample stage themselves from the measured parameters. Although, many studies have been dealing with the calibration problems for a specific X-ray CT system, most of those methods require specificallydesigned and/or expensive phantoms. In this work, we introduce a low-cost, easy-to-use and readily available phantom, built from LEGO bricks that serves as a structure to hold small, ‘metal’ beads for geometric calibration of a tomographic X-ray system.

3-D FE analyses of WRS and simplified elastic-plastic fracture mechanics assessment of a repaired weld

Abstract An effective excavation and re-welding process is often needed to correct the installation slope and geometric misalignment in the piping, where significant local plastic strain damage and complex welding residual stresses (WRS) often occur during the repair process. At the same time, it is complex to perform elastic-plastic fracture assessment for combined loads such as the WRS, bending moment and internal pressure. In this paper, three-dimensional (3-D) finite element (FE) analyses of WRS of the repair welding are performed, and the influence of weld bead sequencing as well as the displacement caused by external loads is simulated. In the study, the plastic strain damage is also assessed to prevent void formation. To avoid the complex elastic-plastic fracture assessment, a simplified elastic-plastic fracture study is carried out based on the failure assessment diagram (FAD) method. The results show that the repair process is effective, and the strain damage caused by the weld repair is acceptable. There are clear differences between WRS caused by initial weld and that of the re-weld, and there is palpable influence of the repair process on the WRS for the unrepaired/unexcavated part of the pipe. Based on the simplified elastic-plastic FAD method, the repair process has obviously influence on the margin of safety. The repair process described in this paper is generic and thus can be applied in similar pipe deflection correction applications.

Mode characteristics investigations on a side pumped electro-optic Q-switched Nd:YAG laser

The influence of representative aberration on the performance of a side pumped electro-optic Q-switched Nd:YAG lasers was demonstrated. The spatial and temporal behavior of the laser with intracavity disturbation was modeled using a numerical calculation of the diffraction integral equations and then experimentally characterized. The eigenmode properties, including the coaxial state with thermal effect of the active medium and geometric misalignment of the cavity OC mirror have been analyzed. Zernike polynomials are further adopted to describe beam distortions by combining the model wavefront reconstruction. Experiment illustrates that the lower-order Zernike aberration such as Z3, Z4 Z5 and Z10, etc. caused by the typical phase disturbation will significantly affect the mode characteristics and the uniform symmetry of wavefront. Other related beam properties e.g. PSF can also be acquired by further calculation. In addition, the proposed method has the merit of working stability and high efficiency. Corresponding strategies are suggested to consider active compensation of intracavity aberration in such lasers.

Alignment and disruption robust binary mapper for optical Braille recognition

Braille is the system invented for visually impaired peoples to capable them for written communication. The basic constructs of each natural language can be transited to corresponding Braille characters. Optical Braille recognition system is capable to read the Braille pattern images and convert it to corresponding language text. The images captures from portable camera devices and scanners are generally geometrically and visually disturbed images. In this paper, a disruption robust model is presented to improve the scope and accuracy of Braille character recognition system. The proposed framework improved the image visibility using SD adaptive filter and used the polar cosine transformation to resolve geometric misalignment. Later on grid mapper and binary encoder are applied on emboss enhanced character image for transforming it to binary coded array. Finally, the peered map on this binary character is implied to recognize the corresponding language text. The proposed framework is implemented on three different sample sets of Braille character and document images collected randomly from the web. The comparative evaluation against PCA (principal component analysis) and LDA (linear discriminant analysis) method shows that the proposed model significantly improved the accuracy of Braille character and document recognition.