3D Affine Transformation Research Articles

Determination of Black Sea Coastline Length with Oblique Stereographic Projection Using Affine Transformation

The Black Sea basin has rich oil and natural gas resources. As a result, determining the continental shelf in international relations and maritime law is a critical issue for countries with a shoreline on the Black Sea, which is a semi-enclosed sea. Global projections are generally used for the projection of satellites used to image the earth. The use of global projections causes increased deformation in applications in local areas. For this reason, the deformation of satellite images used as a base can be reduced by converting them to the appropriate projection. In this study, the coastline lengths of the countries neighboring the Black Sea were calculated by using remote sensing images and selecting the appropriate cartographic projection due to the aim of minimum map deformation. By using different numbers of control points to determine the six parameters of the 2D Affine transformation used for coordinate transformation, the change in the accuracy of the transformation depending on the number of control points was examined. The Black Sea coastline, which was digitized in the local coordinate system via the MODIS satellite image, was transformed into coordinates in the oblique stereographic projection system with 2D affine transformation. 11 test points were used in the affine transformation parameters calculated using different control points, and root mean square error (RMSE) of approximately 6 km on the X axis and approximately 10 km on the Y axis was achieved using 25 control points. As a result of the transformation, the coastline lengths of each country bordering the Black Sea were determined in oblique stereographic projection.

3D Class A Bézier curves with monotone curvature

This paper introduces a special class of 3D Bézier curves that are defined by their degree, a starting point, the first leg of their control polygons, and a 3D affine transformation composing of a uniform scaling and a rotation. We present new formulas for the curvature of such Bézier curves and based on the new formulas we derive sufficient conditions for the curves to have monotonic curvature. The conditions are expressed by a simple constraint on the rotation angle and the scaling factor. This facilitates constructing 3D Class A Bézier curves that are a generalization of planar typical curves proposed by Mineur et al. (1998), which are often used in automotive and other design applications. Some examples are provided to demonstrate the effectiveness of our construction.

Patient-specific Cardio-respiratory Motion Prediction in X-ray Angiography using LSTM Networks

Objective. To develop a novel patient-specific cardio-respiratory motion prediction approach for X-ray angiography time series based on a simple long short-term memory (LSTM) model. Approach. The cardio-respiratory motion behavior in an X-ray image sequence was represented as a sequence of 2D affine transformation matrices, which provide the displacement information of contrasted moving objects (arteries and medical devices) in a sequence. The displacement information includes translation, rotation, shearing, and scaling in 2D. A many-to-many LSTM model was developed to predict 2D transformation parameters in matrix form for future frames based on previously generated images. The method was developed with 64 simulated phantom datasets (pediatric and adult patients) using a realistic cardio-respiratory motion simulator (XCAT) and was validated using 10 different patient X-ray angiography sequences. Main results. Using this method we achieved less than 1 mm prediction error for complex cardio-respiratory motion prediction. The following mean prediction error values were recorded over all the simulated sequences: 0.39 mm (for both motions), 0.33 mm (for only cardiac motion), and 0.47 mm (for only respiratory motion). The mean prediction error for the patient dataset was 0.58 mm. Significance. This study paves the road for a patient-specific cardio-respiratory motion prediction model, which might improve navigation guidance during cardiac interventions.

Proposal of fully augmented complex-valued neural networks

In this paper, we propose a complex-valued neural network based on the widely linear estimation taking advantage of its geometric structure, and show that it can learn geometric transformations that the conventional neural networks cannot. First we formulate a fully augmented complex-valued neuron model based on the widely linear estimation. It is a generalized complex-valued neuron model that includes a usual complex-valued neuron and a degenerated case called a degenerated fully augmented complex-valued neuron. A fully augmented complex-valued neural network consists of such fully augmented complex-valued neurons. Secondly, we derive the back-propagation learning algorithms for the multi-layered fully augmented complex-valued neural networks. Finally, we find out via experiments that the multi-layered fully augmented complex-valued neural network has the different ability to learn 2D affine transformation from that of the usual complex-valued neural network.

ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning

The central idea of contrastive learning is to discriminate between different instances and force different views from the same instance to share the same representation. To avoid trivial solutions, augmentation plays an important role in generating different views, among which random cropping is shown to be effective for the model to learn a generalized and robust representation. Commonly used random crop operation keeps the distribution of the difference between two views unchanged along the training process. In this work, we show that adaptively controlling the disparity between two augmented views along the training process enhances the quality of the learned representations. Specifically, we present a parametric cubic cropping operation, ParamCrop, for video contrastive learning, which automatically crops a 3D cubic by differentiable 3D affine transformations. ParamCrop is trained simultaneously with the video backbone using an adversarial objective, so that it learns to increase the contrastive loss and thus gradually reduces the shared contents between two cropped views. Experiments show that this adaptive and gradual increase in the disparity yielded by ParamCrop is beneficial to learning a strong and generalized representation for downstream tasks, which is shown to be effective on multiple contrastive learning frameworks and video backbones.

SIMILARITY LIMITS OF ORIENTATION CHANGES IN AFFINE TRANSFORMATIONS WITH APPLICATIONS TO PLANAR PATTERN MATCHING

Besides descriptor designing, Factoring out the categorical effects of affine transformations is also an effective way to match features. This paper proposed and demonstrated a hypothesis that any coplanar orientations have limited geometrical changes when observed in the camera screen under given ranges of 3D affine transformations.Confirmatory experiments are accomplished by Matlab simulations focusing on specific influences of 3D affine transformations on similarity biases between concerned orientation changes. Statistical analyses and results show that rotations about plane axes mainly affect the geometrical biases of orientation changes. A set of fit limits standing for the biggest influences of most affine transformations in specified ranges are numerically approximated by total sample space distribution and experimental extreme values. We use these Similarity Limits of Orientation Changes (SLOC) as optimization constraints in matching problems, and an application example of planar pattern matching is given. The effectiveness and efficiency of SLOC are proved by the experimental results in Mikolajczyk and HPatches testbed.

Fast 3D Analytical Affine Transformation for Polygon-Based Computer-Generated Holograms

We present a fast 3D analytical affine transformation (F3DAAT) method to obtain polygon-based computer-generated holograms (CGHs). CGHs consisting of tens of thousands of triangles from 3D objects are obtained by this method. We have attempted a revised method based on previous 3D affine transformation methods. In order to improve computational efficiency, we have derived and analyzed our proposed affine transformation matrix. We show that we have further increased the computational efficiency compared with previous affine methods. We also have added flat shading to improve the reconstructed image quality. A 3D object from a 3D camera is reconstructed holographically by numerical and optical experiments.

Establishment of a corrective geoid surface by spline approximation of Iranian GNSS/levelling network

The performance of the gravimetric geoid models is usually evaluated by comparison of geoid heights with the GNSS/levelling derived geoid. But the GNSS/levelling network can be infected by significant systematic biases and random errors, especially in large and uneven areas. This contribution addresses the challenging problem of the corrector surface development along with the elimination of biases. To this end, fitting an appropriate geometric surface to the GNSS/levelling geoid heights is required, which is accomplished by applying the least squares B-spline approximation theory to the GNSS/levelling data. In addition, the 3D affine transformation is used to detect systematic effects of the GNSS/levelling network compared to a global geoid model. This strategy is applied to the adjustment of the Iranian GNSS/levelling network. A significant tilt is observed across the country ranging from −0.35 m to 0.04 m. The entire study area is divided into four zones and the corrector surfaces are obtained in each zone. The accuracy of three global geoid models, EGM2008, EIGEN-6C4 and SGG-UGM-1, along with that of the Iranian regional geoid model IRG2016, are then investigated based on the raw GNSS/levelling heights and the corrector surfaces. The evaluations show the reliable results regarding the corrector surfaces against the raw data set. All geoid models show their maximum RMSE values of discrepancies in the mountainous zones and their minimum RMSEs in the zones having lower variety in topography. This indicates that the irregular topographies are not well detected by the global models over the study area.

A Rapid Dual Feature Tracking Method for Medical Equipments Assembly and Disassembly in Markerless Augmented Reality

Markerless Augmented Reality (MAR) is a superior technology that is currently used by the medical device assembler with aid in design, assembly, disassembly and maintenance operations. The medical assembler assembles the medical equipment based on the doctors requirement, they also maintains quality and sanitation of the equipment. The major research challenges in MAR are as follows: establish automatic registration parts, find and track the orientation of parts, and lack of depth and visual features. This work proposes a rapid dual feature tracking method i.e., combination of Visual Simultaneous Localization and Mapping (SLAM) and Matched Pairs Selection (MAPSEL). The main idea of this work is to attain high tracking accuracy using the combined method. To get a good depth image map, a Graph-Based Joint Bilateral with Sharpening Filter (GRB-JBF with SF) is proposed since depth images are noisy due to the dynamic change of environmental factors that affects tracking accuracy. Then, the best feature points are obtained for matching using Oriented Fast and Rotated Brief (ORB) as a feature detector, Fast Retina Key point with Histogram of Gradients (FREAK-HoG) as a feature descriptor, and Feature Matching using Rajsk’s distance. Finally, the virtual object is rendered based on 3D affine and projection transformation. This work computes the performance in terms of tracking accuracy, tracking time, and rotation error for different distances using MATLAB R2017b. From the observed results, it is perceived that the proposed method attained the least position error value about 0.1 cm to 0.3 cm. Also, rotation error is observed as minimal between 2.40 (Deg) to 3.10 and its average scale is observed as 2.7140. Further, the proposed combination consumes less time against frames compared with other combinations and obtained a higher tracking accuracy of about 95.14% for 180 tracked points. The witnessed outcomes from the proposed scheme display superior performance compared with existing methods.

Polygon-based computer-generated holography: a review of fundamentals and recent progress [Invited

In this review paper, we first provide comprehensive tutorials on two classical methods of polygon-based computer-generated holography: the traditional method (also called the fast-Fourier-transform-based method) and the analytical method. Indeed, other modern polygon-based methods build on the idea of the two methods. We will then present some selective methods with recent developments and progress and compare their computational reconstructions in terms of calculation speed and image quality, among other things. Finally, we discuss and propose a fast analytical method called the fast 3D affine transformation method, and based on the method, we present a numerical reconstruction of a computer-generated hologram (CGH) of a 3D surface consisting of 49,272 processed polygons of the face of a real person without the use of graphic processing units; to the best of our knowledge, this represents a state-of-the-art numerical result in polygon-based computed-generated holography. Finally, we also show optical reconstructions of such a CGH and another CGH of the Stanford bunny of 59,996 polygons with 31,724 processed polygons after back-face culling. We hope that this paper will bring out some of the essence of polygon-based computer-generated holography and provide some insights for future research.

An adaptive registration algorithm for zebrafish larval brain images

Background and ObjectiveZebrafish (Danio rerio) in their larval stages have grown increasingly popular as excellent vertebrate models for neurobiological research. Researchers can apply various tools in order to decode the neural structure patterns which can aid the understanding of vertebrate brain development. In order to do so, it is essential to map the gene expression patterns to an anatomical reference precisely. However, high accuracy in sample registration is sometimes difficult to achieve due to laboratory- or protocol-dependent variabilities. MethodsIn this paper, we propose an accurate adaptive registration algorithm for volumetric zebrafish larval image datasets using a synergistic combination of attractive Free-Form-Deformation (FFD) and diffusive Demons algorithms. A coarse registration is achieved first for 3D volumetric data using a 3D affine transformation. A localized registration algorithm in form of a B-splines based FFD is applied next on the coarsely registered volume. Finally, the Demons algorithm is applied on this FFD registered volume for achieving fine registration by making the solution noise resilient. ResultsResults Experimental procedures are carried out on a number of 72 hpf (hours post fertilization) 3D confocal zebrafish larval datasets. Comparisons with state-of-the-art methods including some ablation studies clearly demonstrate the effectiveness of the proposed method. ConclusionsOur adaptive registration algorithm significantly aids Zebrafish imaging analysis over current methods for gene expression anatomical mapping, such as Vibe-Z. We believe the proposed solution would be able to overcome the requirement of high quality images which currently limits the applicability of Zebrafish in neuroimaging research.

IFC TO CITYGML CONVERSION ALGORITHM BASED ON GEOMETRY AND SEMANTIC MAPPING

Abstract. Geographic information system (GIS) is known traditionally for the modelling of two-dimensional (2D) geospatial analysis and therefore present information about the extensive spatial framework. On the other hand, building information modelling (BIM) is digital representation of building life cycle. The increasing use of both BIM and GIS simultaneously because of their mutual relationship, as well as their similarities, has resulted in more relationships between both worlds, therefore the need for their integration. A significant purpose of these similarities is importing BIM data into GIS to significantly assist in different design-related issues. However, currently this is challenging due to the diversity between the two worlds which includes diversity in coordinate systems, three-dimensional (3D) geometry representation, and semantic mismatch. This paper describes an algorithm for the conversion of IFC data to CityGML in order to achieve the set goal of sharing information between BIM and GIS domains. The implementation of the programme developed using python was validated using an IFC model (block HO2) of a student’s hostel, Kolej Tun Fatima (KTF). The conversion is based on geometric and semantic information mapping and the use of 3D affine transformation of IFC data from local coordinate system (LCS) to CityGML world coordinate system (WCS) (EPSG:4236). In order to bridge the gap between the two data exchange formats of BIM and GIS, we conducted geometry and semantic mapping. In this paper, we limited the conversion of the IFC model on level of details 2 (LOD2). The conversion will serve as a bridge toward the development of a software that will perform the conversion to create a strong synergy between the two domains for purpose of sharing information.

See360: Novel Panoramic View Interpolation.

We present See360, which is a versatile and efficient framework for 360° panoramic view interpolation using latent space viewpoint estimation. Most of the existing view rendering approaches only focus on indoor or synthetic 3D environments and render new views of small objects. In contrast, we suggest to tackle camera-centered view synthesis as a 2D affine transformation without using point clouds or depth maps, which enables an effective 360° panoramic scene exploration. Given a pair of reference images, the See360 model learns to render novel views by a proposed novel Multi-Scale Affine Transformer (MSAT), enabling the coarse-to-fine feature rendering. We also propose a Conditional Latent space AutoEncoder (C-LAE) to achieve view interpolation at any arbitrary angle. To show the versatility of our method, we introduce four training datasets, namely UrbanCity360, Archinterior360, HungHom360 and Lab360, which are collected from indoor and outdoor environments for both real and synthetic rendering. Experimental results show that the proposed method is generic enough to achieve real-time rendering of arbitrary views for all four datasets. In addition, our See360 model can be applied to view synthesis in the wild: with only a short extra training time (approximately 10 mins), and is able to render unknown real-world scenes. The superior performance of See360 opens up a promising direction for camera-centered view rendering and 360° panoramic view interpolation.

3D LEAST SQUARES MATCHING APPLIED TO MICRO-TOMOGRAPHY DATA

Abstract. The paper introduces 3D least squares matching as a technique to analyze multi-temporal micro-tomography data in civil engineering material testing. Time series of tomography voxel data sets are recorded during an in-situ tension test of a strain-hardening cement-based composite probe at consecutive load steps. 3D least squares matching is a technique to track cuboids in consecutive voxel data sets minimizing the sum of the squares of voxel value differences after a 12-parameter 3D affine transformation. For a regular grid of locations in each voxel data set of the deformed states, a subvoxel-precise 3D displacement vector field is computed. Discontinuities in these displacement vector fields indicate the occurrence of cracks in the probes during the load tests. These cracks are detected and quantitatively described by the computation of principal strains of tetrahedrons in a tetrahedral mesh, that is generated between the matching points. The subvoxel-accuracy potential of the technique allows the detection of very small cracks with a width much smaller than the actual voxel size.

Alternative formulae for robust Weighted Total Least-Squares solutions for Errors-In-Variables models

Weighted Total Least-Squares (WTLS) can optimally solve the issue of parameter estimation in the Errors-In-Variables (EIV) model; however, this method is relatively sensitive to outliers that may exist in the observation vector and/or the coefficient matrix. Hence, an attempt to identify/suppress those outliers is in progress and will ultimately lead to a novel robust estimation procedure similar to the one used in the Gauss-Markov model. The method can be considered as a follow-up to the WTLS solution formulated with the standard Least-Squares framework. We utilize the standardized total residuals to construct the equivalent weights, and apply the median method to obtain a robust estimator of the variance to provide good robustness in the observation and structure spaces. Moreover, a preliminary analysis for the robustness of related estimators within the EIV model is conducted, which shows that the redescending M-estimates are more robust than the monotonic ones. Finally, the efficacy of the proposed algorithm is demonstrated through two applications, i.e. 2D affine transformation and linear regression on simulated data and on real data with some assumptions. Unfortunately, the proposed algorithm may not be reliable for detecting multiple outliers. Therefore, MM-estimates within the EIV model need to be investigated in further research.

Fast Numerical Computation of Two-Dimensional Non-Separable Linear Canonical Transform Based on Matrix Decomposition

Two-dimensional non-separable linear canonical transform (2D NSLCT), as a generalized form of linear canonical transform (LCT) and 2D separable linear canonical transform (2D SLCT), has important applications in many engineering fields. Therefore, accurate and efficient digital implementation is urgent. In this paper, a new four-stage parameter matrix decomposition method is proposed for the efficient implementation of 2D NSLCT. According to the theory of matrix decomposition, we construct a set of decomposition forms which are composed of 2D chirp multiplication (2D CM), 2D Fourier transform (2D FT), 2D affine transformation (2D AT) and 2D CM. The decomposition not only keeps the form simple, but also achieves the ideal effect in a larger range of conditions. Compared with previous methods, the proposed method shows much smaller error and higher accuracy level when the adjustable sampling interval is greater than or equal to 0.05 in the intermediate stage of the algorithm. What's more, simulation results show that our method has conspicuous advantages when changing different parameters and input functions. In addition, the additivity and reversibility of 2D NSLCT also have good performances. Finally, we construct a new optical system and apply our method to it. Simulation results confirm that the proposed method is still more efficient and exact in optical system analysis.

New First-Order Approximate Precision Estimation Method for Parameters in an Errors-in-Variables Model

To evaluate the posterior precision of weighted total least-squares (WTLS) estimates in an errors-in-variables model, first-order approximate precision estimation (FOA) methods are usually used. However, FOAs might not be valid if the underlying assumption is invalid, and this assumption has not been sufficiently proven. Therefore, this paper investigates the validity of the latent assumption and proposes a new first-order approximate (NFOA) precision estimation method to avoid the underlying assumption and design a corresponding algorithm. The difference between NFOA and FOA is formulated and analyzed. The proposed NFOA method is tested by a simulated classic straight-line fitting example with six scenarios and a simulated three-dimensional (3D) affine transformation experiment with four scenarios, and the mean values of the standard deviation of true errors (MSDTE) and FOA are also calculated for comparison. The results numerically indicate that NFOA works better than FOA and is close to the MSDTE, which means that NFOA can evaluate the precision of estimated parameters more reasonably and accurately.

An iterated reweighting total least squares algorithm formulated by standard least-squares theory

If both the coefficient matrix and the observation vector are affected by noise, a total least-squares algorithm should be applied to obtain the solution. However, if they are also contaminated by outliers, the solution of the algorithm will seriously deviate from the true values. Therefore, the effect of outliers needs to be eliminated. For this purpose, the robust estimation is introduced into the total least-squares algorithm, developing a new robust weighted total least-squares algorithm. Simultaneously, considering the robustness of structure space, the standardized residuals are utilized to construct the weight function. Finally, the robustness and efficiency of this algorithm are verified by three experiments involving line-fitting and 2D affine transformation.

PARTICIPATORY IMAGE-BASED MODELS’ ALIGNMENT FOR RECONSTRUCTING A LARGE-SCALE INDOOR MAPPING

Abstract. In this paper, we introduced a recently developed image-based model alignment technique for 3D reconstruction of large-scale indoor corridors. The proposed participatory model alignment technique enables crowd source single image-based modeling since it allows various participants to incorporate their images taken from different cameras for large-scale indoor mapping. This technique is robust against changes of camera orientation and prevents miss-association of a newly generated 3D model to the previously integrated models. To investigate the possibility of aligning two individual 3D models, their respective corridor topological graphs must match, and they need to geometrically transform into the same object space. Here 3D affine transformation is applied, and the transformation parameters are estimated through corresponding vertices of both 3D models. Having integrated two models in the same 3D space, they will be back projected into the image space for evaluation using Direct Linear Transformation. Note that the proposed method performs layout model matching in image space and considers information including layout topology and geometry as well as image information to address model alignment. The advantages of using layout information in the proposed alignment technique are twofold. First, a metric constraint is imposed to insure topological model consistency and balance 3D models scale issues. Second, it will reduce alignment ambiguity related to indoor corridor scenes, where the scene is enriched with multiple structural elements including various corridors junctions. To evaluate the performance of the proposed method, we have performed the experiments on a data set collected from Ross building corridors at York University. This dataset includes single images captured by a handheld wide-angle camera. The obtained results present the ability of the proposed method in alignment of single image-based 3D models while producing limited geometric errors.

Model-free Cardiorespiratory Motion Prediction from X-ray Angiography Sequence with LSTM Network.

We present a novel model-free approach for cardiorespiratory motion prediction from X-ray angiography time series based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN). Cardiorespiratory motion prediction is defined as a problem of estimating the future displacement of the coronary vessels in the next image frame in an X-ray angiography sequence. The displacement of the vessels is represented as a sequence of 2D affine transformation matrices allowing 2D X-ray registrations in a sequence. The new displacement parameters from a sequence of transformation matrices are predicted using an LSTM model. LSTM is a particular form of Recurrent Neural Network (RNN) architecture suitable for learning sequential data and predicting time series. The method was developed and validated by simulated data using a realistic cardiorespiratory motion simulator (XCAT). The results show that this method converges quickly and can predict the complex motion in the angiography sequences with irregularities. The mean values of prediction error over all the patients are approximately 0.29 mm (2 pixels) difference for the combination of both motions, 0.51 mm (3.5 pixels) difference for cardiac motion and 0.44 mm (3 pixels) difference for respiratory motion.