Matching Of Control Points Research Articles

A high-precision image registration method for multi-channel single molecule localization

Single-molecule localization technology has been widely used in single-particle tracking and super-resolution imaging of biological samples, as it can bypass the diffraction limit of optical systems. Multi-channel single-molecule localization uses multiple imaging channels to simultaneously track different targets or perform multi-color super-resolution imaging, and can also improve the axial depth of single-particle tracking or achieve higher localization precision and density for super-resolution imaging. However, the difference between images in each channel can affect collaborative localization or quantitative analysis, so image registration is a key step in its image data preprocessing. Moreover, due to the high precision of single-molecule localization, its requirements for multi-channel image registration accuracy are also high. Existing technologies generally use control point-based registration methods and often use complicated and precise methods to obtain fiducial images for locating control point pairs to achieve high-precision image registration, which involves high sample or experimental equipment requirements and is difficult to directly extend to other systems. Therefore, developed in this work, is a high-precision image registration method that can directly use randomly distributed fluorescent beads as fiducial samples based on local nonlinear transformation and elimination of mismatched points. By monitoring and iteratively filtering control points in the process of feature matching and transformation model parameter estimation to eliminate control point pairs that are not accurately matched due to inaccurate or poor precision of single-molecule localization, the adverse effects on accurate acquisition and precise matching of control points when using randomly distributed fluorescent beads as fiducial samples are eliminated. At the same time, a second-order polynomial fitting based on local weighted mean is used for estimating the transformation model parameter to better adapt to the existence of local nonlinear deformation between different channels. The results show that using this method only requires three iterations to find and eliminate control point pairs that are not accurately located and matched, thereby achieving more accurate transformation model parameter and improving the registration accuracy by an order of magnitude, achieving a registration accuracy of about 6 nm in a complex dual-channel single-molecule localization imaging system based on orthogonal astigmatism.

The Impact of Control Point Dispersion on Measurement Accuracy in a New Type of Light-Pen Coordinate Measuring System

Light-pen coordinate measuring machines are widely used due to their portability; however, the measurement accuracy is affected by camera calibration, light-pen probe calibration, control points, etc. To study the influence of control point dispersion on measurement accuracy, a light-pen coordinate measuring system is constructed in this paper. In this system, a new type of control point distribution is designed. Based on this distribution, five light-pens with different control point dispersions are created and an algorithm is proposed to match control points with their images. The experimental repeatability results for the five light-pen probes indicate satisfactory repeatability, which ensures the effectiveness of the measurement results. The experimental measurement results for five light-pens demonstrate that the measurement error decreases with the decline of the control point dispersion at first, but begins to increase when the control point dispersion reaches a certain value.

Calibrating stereo-digital image correlation system using synthetic speckle-pattern calibration target

We describe an alternative stereo calibration method for stereo-digital image correlation (stereo-DIC) using computer-simulated speckle-pattern calibration targets and a DIC-based control points matching technique. The present work refines and extends the recently proposed camera calibration method to stereo calibration, which is an essential part of stereo-DIC measurement. In the proposed calibration method, the calibration target is generated by displaying a synthetic speckle pattern on a commercial tablet. On the simulated speckle pattern, a group of control points is specified with flexible location and number, which is further precisely matched to the captured calibration images using the well-developed subset-based DIC algorithm. These precisely matched control points are subsequently used to calibrate the intrinsic and extrinsic parameters of the stereo-DIC system with stereo calibration algorithm. For better accuracy, the error in the manufacture of the calibration target is also considered and compensated. As a flexible number of control point matches are registered with high accuracy and strong robustness, the presented stereo calibration method using synthetic speckle pattern calibration target offers higher calibration accuracy than that based on a calibration target with a limited number of designated features, such as checkerboard target and circular point target. After being demonstrated by a set of simulated calibration tests, the proposed stereo calibration method was applied to calibrate a real stereo-DIC system, which was then used to measure 3D full-field shapes and displacements. The results confirm the effectiveness and accuracy of the proposed stereo calibration method for stereo-DIC applications.

Investigation of image registration method for the multi-directional image fusion of woven fabrics

Traditional single-side scanning or single-vision image acquisition methods have the limitation of incomplete information caused by the existence of blind spots. To collect the complete texture information of fabric images, a new multi-vision image acquisition and the related fusion method is developed to solve this problem. However, linear addition of image sequences acquired from multiple directions cannot achieve a good result of image fusion, it is necessary to conduct the image fusion based on the image registration between images at pixel level. Therefore, a new multi-directional digital image acquisition system for woven fabrics is established in this article; one set of image fusion algorithm based on image registration is proposed for the image enhancement of fabric. Fabric texture images are digitized by means of multi-directional vision imaging instead of unidirectional imaging, the structural information of fabric texture could be enhanced using image registration and fusion technology and the indexing and localization of texture corresponding points could be controlled using matching points or control points. Our experimental results show that the proposed method could be used to merge the effective information from the multi-directional vision images completely, it has the potential application for the rendering of woven fabrics using image driven virtual reality enhancement.

THERMAL REMOTE SENSING WITH UAV-BASED WORKFLOWS

Abstract. Climate change will have a significant influence on vegetation health and growth. Predictions of higher mean summer temperatures and prolonged summer draughts may pose a threat to agriculture areas and forest canopies. Rising canopy temperatures can be an indicator of plant stress because of the closure of stomata and a decrease in the transpiration rate. Thermal cameras are available for decades, but still often used for single image analysis, only in oblique view manner or with visual evaluations of video sequences. Therefore remote sensing using a thermal camera can be an important data source to understand transpiration processes. Photogrammetric workflows allow to process thermal images similar to RGB data. But low spatial resolution of thermal cameras, significant optical distortion and typically low contrast require an adapted workflow. Temperature distribution in forest canopies is typically completely unknown and less distinct than for urban or industrial areas, where metal constructions and surfaces yield high contrast and sharp edge information. The aim of this paper is to investigate the influence of interior camera orientation, tie point matching and ground control points on the resulting accuracy of bundle adjustment and dense cloud generation with a typically used photogrammetric workflow for UAVbased thermal imagery in natural environments.

Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients.

With availability of different retinal imaging modalities such as fundus photography and spectral domain optical coherence tomography (SD-OCT), having a robust and accurate registration scheme to enable utilization of this complementary information is beneficial. The few existing fundus-OCT registration approaches contain a vessel segmentation step, as the retinal blood vessels are the most dominant structures that are in common between the pair of images. However, errors in the vessel segmentation from either modality may cause corresponding errors in the registration. In this paper, we propose a feature-based registration method for registering fundus photographs and SD-OCT projection images that benefits from vasculature structural information without requiring blood vessel segmentation. In particular, after a preprocessing step, a set of control points (CPs) are identified by looking for the corners in the images. Next, each CP is represented by a feature vector which encodes the local structural information via computing the histograms of oriented gradients (HOG) from the neighborhood of each CP. The best matching CPs are identified by calculating the distance of their corresponding feature vectors. After removing the incorrect matches the best affine transform that registers fundus photographs to SD-OCT projection images is computed using the random sample consensus (RANSAC) method. The proposed method was tested on 44 pairs of fundus and SD-OCT projection images of glaucoma patients and the result showed that the proposed method successfully registers the multimodal images and produced a registration error of 25.34 ± 12.34 μm (0.84 ± 0.41 pixels).

Changes in Salivary and Plasma Markers during and Following Short-Term Maximal Aerobic Exercise Assessed during Cognitive Assessment

This study assessed multiple salivary and plasma markers before and after incremental short-term maximal aerobic exercise and in a non-exercising control in conjunction with cognitive testing. Subjects: Apparently healthy 18 - 30 years old low CVD risk females participated (n = 19). Methods: Subjects completed two conditions: 1) exercise: short maximal treadmill exercise and cognitive assessment pre- and post-exercise and, 2) non-exercise: with cognitive assessment timed to match testing in the exercising condition. Non-stimulated, timed salivary samples and venous blood were collected before and after exercise and after recovery. Results: Saliva: Over time α-amylase increased in both exercise and non-exercising conditions. Exercise had increases in α-amylase at time matched control points up to 36% greater than the non-exercising conditions. Following exercise and recovery from exercise α-amylase in-creased compared to baseline (ranging from 47% to 290%). Baseline cortisol was 33% higher than post-exercise and 59% higher than recovery irrespective of exercise. Plasma: NEFA was 50% higher at post-exercise and recovery compared to baseline without exercise and 36% higher at post-exercise and recovery compared to baseline with exercise. Glucose and lactate were, 18% and 50% higher respectively, after exercise compared to baseline and recovery with exercise. Post-exercise glycerol was 11% higher than recovery. Differences between Conditions: Post-exercise glucose and lactate were 20% and 40% higher respectively with exercise. Glycerol was 11% lower after exercise. Conclusions: We demonstrated that acute exercise coupled with cognitive task increased α-amylase levels, but not cortisol, potentially due to a differential stress response, but most likely due to the timing of sample collection.

A novel method for multi-angle SAR image matching

Multi-angle synthetic aperture radar (SAR) image matching is very challenging, because the same object may cause different backscattering patterns, heavily depending on the radar incident angle. A technique based on the relations between the invariant positions of ground targets among the reference and sensed images is proposed to accommodate the nonmatching patterns. It involves a target extraction using wavelet coefficient fusion, as well as a geometric voting matching routine for searching the matched control points (CPs) in the reference and sensed images, respectively. To accelerate the speed of the search, a robust, rapidly corresponding CPs determination strategy is exploited by utilizing the global spatial transformation model, as well as a procedure of outlier removal to ensure the desired accuracy. Meanwhile, the positions of the matched point pairs are relocated using mutual information. The final warping of the images according to the CPs is performed by using a polynomial function. The results show the possibility of matching multi-angle SAR images in general cases.

영상등록을 위한 Mutual Information 기반의 원형 템플릿 정합

본 논문에서는 영상 등록을 위한 유사도 계산에 사용되는 원형 템플릿의 설계 방법을 제안한다. 원형 템플릿은 영상의 이동 및 회전 변환에 불변한 성질을 가지고 있어 기준 영상 및 관측 영상 사이에 이동 및 회전 변환이 존재하더라도 영상 등록 제어점을 정확하게 정합하는 장점이 있다. 기준 영상의 제어점을 중심으로 일정한 거리 이내에 다수의 원주를 구성하고 각 원주 위에 일정한 간격으로 위치하는 화소들로 이루어지는 원형 템플릿을 생성하고 이를 이차원 이산 극좌표 행렬(Discrete Polar Coordinate Matrix, DPCM)으로 구성한다. 관측 영상에서도 동일한 형태의 원형 템플릿을 생성하고 탐색 범위 내의 각 위치에서 관측 영상의 원형 템플릿을 0도에서 360도 범위 내에서 일정 각도 간격으로 회전시키면서 극좌표 행렬을 생성하고 기준 영상의 극좌표 행렬과의 유사도를 Mutual Information을 이용해서 계산한다. 탐색 범위 내의 각 위치와 회전 각도에 대한 Mutual Information이 최대가 되는 화소를 정합쌍으로 결정한다. 제안한 알고리즘은 서로 다른 두 시기에 촬영한 KOMPSAT-2 영상에 적용하여 영상의 회전 변화 조건하에서 우수한 정합 성능을 보임을 확인하였다. This paper presents a method for designing circular template used in similarity measurement for image registration. Circular template has translation and rotation invariant property, which results in correct matching of control points for image registration under the condition of translation and rotation between reference and sensed images. Circular template consisting of the pixels located on the multiple circumferences of the circles whose radii vary from zero to a certain distance, is converted to two-dimensional Discrete Polar Coordinate Matrix (DPCM), whose elements are the pixels of the circular template. For sensed image, the same type of circular template and DPCM are created by rotating the circular template repeatedly by a certain degree in the range between 0 and 360 degrees and then similarity is calculated using mutual information of the two DPCMs. The best match is determined when the mutual information for each rotation angle at each pixel in search area is maximum. The proposed algorithm was tested using KOMPSAT-2 images acquired at two different times and the results indicate high accurate matching performance under image rotation.

GPU-accelerated large-size VHR images registration via coarse-to-fine matching

This paper presents a coarse-to-fine large-size very high resolution image registration method. This method uses compute unified device architecture to speed up the acquisition of control points and image rectification. In coarse registration, the scale-invariant feature transform algorithm is used to match control points to estimate the initial global transformation parameters between images to be registered. The initial parameters are then used to guide Oriented FAST and Rotated BRIEF (ORB) feature matching in fine registration. To fix local distortion, image rectification is based on a linear mapping function computed from triangulations, and a self-adaptive scan filling algorithm is proposed to determine which triangle each pixel belongs to. Experiments are conducted with large-size satellite images. Results show that a graphics processing unit can obtain significant-acceleration factors while maintaining registration accuracy.

People Tracking using Color Control Points and Skin Color

This paper presents a new approach to be used for people detection and tracking in image sequences based on color control points and skin color modeling. This method aims to track these people in complex situations such as players on a soccer field. Each person in the image is represented by several control points which are obtained using a color version of the Harris algorithm detector. Each control point is characterized by the local appearance which is a vector of local characteristics. Then we determine the rules that define the skin regions in three different color spaces such as RGB, HSV and YCbCr, and we apply these rules to our images to segment skin regions. Using a set of control points and skin regions allows us to track a person by matching control points based on the measure of ZNCC correlation «Zero mean Normalized Cross Correlation». The simulations and experimental results show the robustness of our algorithms in terms of stability and convergence. Performance is illustrated by some examples. Thus, our method fits well with the noise conditions.

A Piecewise Monotone Subgradient Algorithm for Accurate <formula formulatype="inline"><tex Notation="TeX">${\rm L}^{1}$</tex></formula>-TV Based Registration of Physical Slices With Discontinuities in Microscopy

Image registration tasks are often formulated in terms of minimization of a functional consisting of a data fidelity term penalizing the mismatch between the reference and the target image, and a term enforcing smoothness of shift between neighboring pairs of pixels (a min-sum problem). Most methods for deformable image registration use some form of interpolation between matching control points. The interpolation makes it impossible to account for isolated discontinuities in the deformation field that may appear, e.g., when a physical slice of a microscopy specimen is ruptured by the cutting tool. For registration of neighboring physical slices of microscopy specimens with discontinuities, Janácek proposed an L¹-distance data fidelity term and a total variation (TV) smoothness term, and used a graph-cut (GC) based iterative steepest descent algorithm for minimization. The L¹-TV functional is nonconvex; hence a steepest descent algorithm is not guaranteed to converge to the global minimum. Schlesinger presented transformation of max-sum problems to minimization of a dual quantity called problem power, which is--contrary to the original max-sum functional--convex. Based on Schlesinger's solution to max-sum problems we developed an algorithm for L¹-TV minimization by iterative multi-label steepest descent minimization of the convex dual problem. For Schlesinger's subgradient algorithm we proposed a novel step control heuristics that considerably enhances both speed and accuracy compared with standard step size strategies for subgradient methods. It is shown experimentally that our subgradient scheme achieves consistently better image registration than GC in terms of lower values both of the composite L¹-TV functional, and of its components, i.e., the L¹ distance of the images and the transformation smoothness TV, and yields visually acceptable results even in cases where the GC based algorithm fails. The new algorithm allows easy parallelization and can thus be sped up by running on multi-core graphic processing units.

AN ACCURACY ASSESSMENT OF GEOREFERENCED POINT CLOUDS PRODUCED VIA MULTI-VIEW STEREO TECHNIQUES APPLIED TO IMAGERY ACQUIRED VIA UNMANNED AERIAL VEHICLE

Abstract. Low-cost Unmanned Aerial Vehicles (UAVs) are becoming viable environmental remote sensing tools. Sensor and battery technology is expanding the data capture opportunities. The UAV, as a close range remote sensing platform, can capture high resolution photography on-demand. This imagery can be used to produce dense point clouds using multi-view stereopsis techniques (MVS) combining computer vision and photogrammetry. This study examines point clouds produced using MVS techniques applied to UAV and terrestrial photography. A multi-rotor micro UAV acquired aerial imagery from a altitude of approximately 30–40 m. The point clouds produced are extremely dense (<1–3 cm point spacing) and provide a detailed record of the surface in the study area, a 70 m section of sheltered coastline in southeast Tasmania. Areas with little surface texture were not well captured, similarly, areas with complex geometry such as grass tussocks and woody scrub were not well mapped. The process fails to penetrate vegetation, but extracts very detailed terrain in unvegetated areas. Initially the point clouds are in an arbitrary coordinate system and need to be georeferenced. A Helmert transformation is applied based on matching ground control points (GCPs) identified in the point clouds to GCPs surveying with differential GPS. These point clouds can be used, alongside laser scanning and more traditional techniques, to provide very detailed and precise representations of a range of landscapes at key moments. There are many potential applications for the UAV-MVS technique, including coastal erosion and accretion monitoring, mine surveying and other environmental monitoring applications. For the generated point clouds to be used in spatial applications they need to be converted to surface models that reduce dataset size without loosing too much detail. Triangulated meshes are one option, another is Poisson Surface Reconstruction. This latter option makes use of point normal data and produces a surface representation at greater detail than previously obtainable. This study will visualise and compare the two surface representations by comparing clouds created from terrestrial MVS (T-MVS) and UAV-MVS.

Multispectral Image Matching Using Rotation-Invariant Distance

Normalized cross correlation (NCC) has been widely used to match control points (CP) in image alignment. This method will produce a lot of incorrect matches owing to the significant difference in the image intensity between multispectral image pairs, and furthermore, it is very computationally expensive to handle rotational displacement. This letter presents a method using rotation-invariant distance to match CPs; a local descriptor matrix is built to describe each CP, and fast Fourier transform is introduced to compute the rotation-invariant distance between the matrices. The computational load is sharply decreased by rotation-invariant distance compared to NCC, and furthermore, the load will remain unchanged in circumstance with arbitrary rotational angle. Experimental results indicate that the proposed method improves the match performance compared to other state-of-the-art methods in terms of correct match rate and aligning accuracy.

Influences of forest roads on the spatial pattern of wildfire boundaries

Although many studies have examined fire ignition and fire spread, the cessation of fire is largely unexplored at a landscape scale. Therefore, we sought to examine the effects of forest roads on fire boundary locations. We analysed six fires within the Wenatchee National Forest in Washington State. Geographic Information System datasets of fire boundaries and environmental variables were obtained from government agencies. Case points were sampled on the fire boundaries, and five matching control points were sampled inside the fire boundary for each case. Matched case–control logistic regression was performed to identify the environmental differences between each case and its set of control points. We found that for four of the fires, fire boundaries were significantly closer to roads than to random control points. Various terrain and vegetation attributes were also correlated with fire boundary locations. In the fires where road effects were significant, they had stronger effects than any of the other environmental constraints on fire boundary locations. These results suggest that road effects on fire cessation should be incorporated more explicitly into landscape-level assessment of wildfire risk and analyses of fuel treatment effectiveness.

AISIR: Automated inter-sensor/inter-band satellite image registration using robust complex wavelet feature representations

An automated registration system named AISIR (Automated inter-sensor/inter-band satellite image registration) has been designed and implemented for the purpose of registering satellite images acquired using different sensors and spectral bands. Sensor and environmental noise, contrast non-uniformities, and inter-sensor and inter-band intensity mapping differences are addressed in the AISIR system. First, a novel modified Geman–McClure M-estimation scheme using a robust phase-adaptive complex wavelet feature representation is introduced for robust control point matching. Second, an iterative refinement scheme is introduced in the AISIR system for improved control point pair localization. Finally, the AISIR system introduces a robust mapping function estimation scheme based on the proposed modified Geman–McClure M-estimation scheme. The AISIR system was tested using various multi-spectral optical, LIDAR, and SAR images and was shown to achieve better registration accuracy than state-of-the-art M-SSD and ARRSI registration algorithms for all of the test sets.

Change detection deconfliction process for sonar clutter items.

When resurveying a seafloor area of interest during change detection operations, an automated method to match found bottom objects with objects detected in a previous survey allows the surveyor to quickly sort new objects from old objects. Here we will demonstrate a software system that accomplishes change detection. The change detection system contains modules for automatic object detection by geospatial bitmap technique, object collocation, feature matching using shadow outlining, scene matching by control point matching, and visualization and filtering capabilities. Emphasis will be placed on the new elements of the system, namely, shadow outlining and optional spatial filtering.

Automatic image registration by local descriptors in remote sensing

Registration of multiresolution and multisensor images is a challenging problem in the research area of remote sensing. Conventional image registration algorithms have greatly suffered from control points selected manually. In this study, we propose a novel method for automatic affine image registration based on local descriptors, called automatic image registration by local descriptors (AIRLD). In this algorithm, we first apply the Harris-affine method to extract the interesting points of the given images, and then they are used to calculate the affine invariant feature descriptors. These descriptors are combined through the normalized mutual information (NMI) method to obtain robust matching control points in both reference and sensed images automatically. Finally, we show that subpixel registration accuracy can be reached by means of a least-square fitting of a paraboloid function to the surface generated by the NMIs of the control point neighbors. Experimental results show that our method can provide better accuracy than the conventional registration process.

Automated change detection with area matching

When resurveying a geographic area of the seafloor during sidescan change detection operations, an automated method to match bottom objects imaged previously with objects imaged in the resurvey can increase efficiency and accuracy. The geographic position of a new object relative to a historical object is a good indicator of a match. However, due to position error within either survey, there may be more than one spatially-close object in the new imagery. To complicate matters further, the reflected energy from the new object may be significantly different given a different incidence angle in the resurvey or the partial burial of the object. In addition, the resurveyed object image may be below the threshold set for automatic recognition and falsely eliminated. This presentation will address these problems and suggest possible methods for matching “constellations” of bottom objects by Dijkstra's minimum cost - maximum flow algorithm, control point matching, and the data-association procedure.

SU‐FF‐I‐64: Breast Image Registration Using SIFT and TPS Hybrid Method

Purpose/Objectives(s): Breast images compose of soft tissues with low contrast in MRI imaging modality. However, even small amounts of motion can have large effects on clinical analysis, since each individual pixel needs to be careful checked. We investigate a strategy to obtain high precision deformable registration method for breast images. Materials: The data set contains 3 series. Each series contains 32 slices, with adifferent number of time points. Series 1 contains 15 time points (15×32 = 480 images), Series 2 contains 20 time points (20×32 = 640 images), and Series 3 contains 20 time points (20×32 = 640 images). Some potential complicating factors are: 1) In Series 1, the MRI flip angle is different for time points 1–5, 6–10, and 11–15, which causes each set of 5 time points to potentially have different contrast. 2) Contrast agent is administered in series 2, so the contrast changes over time. 3) There is a space of a few minutes between each series, when the patient may move. 4) Series 3 is a “wash‐out” series. As the contrast agent washes out, the image contrast may change. Methods: Our calculation consists of two natural steps. First, a number of control points are auto‐searched based on tissue intensity information. Control points in fixed image and moving image are matched together using SIFT (Scale Invariance Feature Transformation) method. In the second stage, the matched control points serve as a priori information for the TPS (Thin Plate Spline) deformable registration calculation. Results: Breast images are matched well using this hybrid registration method. The subtraction images are presented to show the registration results. Conclusions: The proposed method can be used in the soft tissue images and large deformation images. Comparison to the conventional deformable registration, this method can effectively increase the local registration quality.