Image Space Coordinates Research Articles

Georeferencing oblique PhenoCam imagery

The high temporal and spatial resolution of ecosystem data captured by tower-mounted PhenoCams have established these instruments as fundamental tools in phenological studies and positioned them as a critical mid-step between airborne or spaceborne and in-situ data in ecological research. However, adding spatial precision can further expand PhenoCam network applications and attract more users, such as drawing more phenological scientists to the near-surface remote sensing research field. In this study, a georeferencing approach was established to enhance research infrastructure for PhenoCams. Advanced photogrammetric techniques were applied to the camera field of view to geo-enable all pixels, tying them to their location on Earth and adding more usable information to datasets in addition to the current “region of interest” (ROI) level data. The georeferencing method is presented along with the photogrammetric equations that enable going from object space coordinates (3D) to image space coordinates (2D). This method was tested and demonstrated on PhenoCam data at Ordway Swisher Biological Station (OSBS), located in Melrose, Florida, USA. Statistical and sensitivity analyses show that projected pixel-location can be as accurate as 1.5 pixels RMSE for the presented case study, corresponding to object space accuracy of 10 cm, 20 cm, and 30 cm at distances of 100 m, 200 m, and 300 m, respectively. In addition, geo-located PhenoCam data at OSBS was co-located with Moderate Imaging Spectrometer (MODIS) data and characterized. These results demonstrate that the techniques presented reliably provide additional data from PhenoCams that are useful for ecosystem-level studies. By providing each pixel’s absolute location corresponding to its place in the real world efficiently, this research introduces a higher degree of spatial precision to every phenological observation from the PhenoCam at OSBS. This presentation of reproducible steps and analysis facilitates implementation for other PhenoCam data as well as other obliquely mounted cameras.

Mobile Robot Control Based on 3D Visual Servoing: A New Approach Combining Pose Estimation by Neural Network and Differential Flatness

This paper deals with 3D visual servoing applied to mobile robots in the presence of measurement disturbances, caused in particular by target occlusion. We propose a new approach based on the flatness concept. In 3D visual servoing, the task is performed out of image coordinate space and targets may leave the camera field of view during navigation (servoing). Forced to navigate blindly during one or more periods of time, the robot will use our new open-loop control algorithm inspired by the flatness concept. The 3D visual servoing method is performed using robot pose estimation. This estimation generally contains some errors. The exact position of the robot is therefore not guaranteed, and robust feedback control is necessary to reject these errors in the input. To solve this problem, we propose a new pose estimation method that uses neural networks. We reduce the complexity of the architecture of the neural networks used (the number of variables to estimate) by proving that the location and the orientation of the robot can be ensured by using a single point in the image coordinate space for mobile robots with two degrees of freedom. To show the efficiency of the proposed algorithm, we use the RVCTOOLS MATLAB toolbox.

Development of a mechatronic guidance system for targeted ultrasound-guided biopsy under high-resolution positron emission mammography localization.

Image-guided needle biopsy of small, detectable lesions is crucial for early-stage diagnosis, treatment planning, and management of breast cancer. High-resolution positron emission mammography (PEM) is a dedicated functional imaging modality that can detect breast cancer independent of breast tissue density, but anatomical context and real-time needle visualization are not yet available to guide biopsy. We propose a mechatronic guidance system integrating an ultrasound (US)-guided core-needle biopsy (CNB) with high-resolution PEM localization to improve the spatial sampling of breast lesions. This paper presents the benchtop testing and phantom studies to evaluate the accuracy of the system and its constituent components for targeted PEM-US-guided biopsy under simulated high-resolution PEM localization. A mechatronic guidance system was developed to operate with the Radialis PEM system and a conventional US system. The system includes a user-operated guidance arm and end-effector biopsy device, integrating a US transducer and CNB gun, with its needle focused on a remote center of motion (RCM). Custom software modules were developed to track, display, and guide the end-effector biopsy device. Registration of the mechatronic guidance system to a simulated PEM detector plate was performed using a landmark-based method. Testing was performed with fiducials positioned in the peripheral and central regions of the simulated detector plate and registration error was quantified. Breast phantom experiments were performed under ideal detection and localization to evaluate for bias in the end-effector biopsy device. The accuracy of the complete mechatronic guidance system to perform targeted breast biopsy was assessed using breast phantoms with simulated lesions. Three-dimensional positioning error was quantified, and principal component analysis assessed for directional trends in 3D space within 95% prediction intervals. Targeted breast biopsies with test phantoms were performed and an overall in-plane needle targeting error was quantified. The mean registration errors were 0.63mm (N=44) and 0.73mm (N=72) in the peripheral and central regions of the simulated PEM detector plate, respectively. A 3D 95% prediction ellipsoid shows an error volume <2.0mm in diameter, centered on the mean registration error. Under ideal detection and localization, targets <1.0mm in diameter can be sampled with 95% confidence. The complete mechatronic guidance system was able to successfully spatially sample simulated breast lesions, 4mm and 6mm in diameter and height (N=20) in known 3D positions in the PEM image coordinate space. The 3D positioning error was 0.85mm (N=20) with 0.64mm in-plane and 0.44mm cross-plane component errors. Targeted breast biopsies resulted in a mean in-plane needle targeting error of 1.08mm (N=15) allowing for targets 1.32mm in radius to be sampled with 95% confidence. We demonstrated the utility of our mechatronic guidance system for targeted breast biopsy under high-resolution PEM localization. Breast phantom studies showed the ability to accurately guide, position, and target breast lesions with the accuracy to spatially sample targets <3.0mm in diameter with 95% confidence. Future work will integrate the developed system with the Radialis PEM system toward combined PEM-US-guided breast biopsy.

Natural reconstruction coordinates for imperfect TRASE MRI

TRansmit Array Spatial Encoding (TRASE) Magnetic Resonance Imaging (MRI) encodes image information in the Nuclear Magnetic Resonance (NMR) signal using spatially varying phases in a multiple spin-echo radio frequency (RF) pulse sequence. When the TRASE transmit coils are perfect, the spatial isophase RF contours produced are straight and orthogonal for a UXY TRASE sequence and the data produced fall into a well-understood k-space of spatial frequencies. When the TRASE coils are imperfect the spatial isophase RF contours will be non-orthogonal and/or curved. In that case the data produced by the TRASE sequence no longer falls into the traditional k-space. Here we show that the data fall into an alternative k-space relative to non-Cartesian coordinates in the image space that are naturally defined by the spatial isophase RF contours produced by the imperfect TRASE coils. With the construction of these natural image space coordinates, a direct image reconstruction transform may be defined. Simulated TRASE MRI cases with imperfect virtual transmit coils were computed and images reconstructed using the direct image reconstruction transform. The mathematical set-up here is a new one for MRI and may be amenable to further mathematical development as TRASE technology evolves.

Jitter compensation of ZiYuan-3 satellite imagery based on object point coincidence

ABSTRACTSatellite jitter is a very important factor that affects the sensor orientation of ZiYuan-3 imagery based on a rational function model (RFM). The conventional affine transformation model is unable to compensate such periodic jitters. The sensor orientation accuracy is thereby worse than expected. To eliminate the influence of jitters and improve the orientation accuracy, a feasible jitter compensation method for ZiYuan-3 imagery based on object point coincidence is presented in this study. In this method, no actual ground control points (AGCPs) are required to estimate the jitter compensation parameters. First, numerous virtual object points are projected onto the image by using the RFM. Then, the residual errors between the image-space coordinates of the projected and corresponding points are used to detect the satellite jitters. Finally, two sinusoidal functions are used to model and compensate the jitters. Experimental results of the three ZiYuan-3 satellite images show that before the jitter compensation, the residual errors of the independent check points obviously show a sinusoidal pattern. These periodic errors cannot be effectively compensated by the affine transformation model even if the number of AGCPs is increased from 4 to 16. After the jitters are compensated with the estimated sinusoidal coefficients, the influence of jitters can be eliminated. The sensor orientation accuracies of the three images reach 0.852 pixels, 0.798 pixels, and 0.921 pixels, which are respectively 49.7%, 55.1%, 65.7% better than those achieved before the jitter compensation.

Feature-based phase retrieval wavefront sensing approach using machine learning.

A feature-based phase retrieval wavefront sensing approach using machine learning is proposed in contrast to the conventional intensity-based approaches. Specifically, the Tchebichef moments which are orthogonal in the discrete domain of the image coordinate space are introduced to represent the features of the point spread functions (PSFs) at the in-focus and defocus image planes. The back-propagation artificial neural network, which is one of most wide applied machine learning tool, is utilized to establish the nonlinear mapping between the Tchebichef moment features and the corresponding aberration coefficients of the optical system. The Tchebichef moments can effectively characterize the intensity distribution of the PSFs. Once well trained, the neural network can directly output the aberration coefficients of the optical system to a good precision with these image features serving as the input. Adequate experiments are implemented to demonstrate the effectiveness and accuracy of proposed approach. This work presents a feasible and easy-implemented way to improve the efficiency and robustness of the phase retrieval wavefront sensing.

Computing RPC using robust selection of GCPs

The Rational Function Model (RFM) has gained widespread acceptance in photogrammetric processing, e.g. georeferencing, Digital Elevation Model (DEM) generation, 3D reconstruction, ortho-rectification etc. Most satellite image vendors provide the Rational Polynomial Coefficients (RPCs) instead of the rigorous sensor model. In this paper we present a novel approach to estimate RPCs using only Ground Control Points (GCPs). We ensured the RPC computing and validation with the aid satellite image and some number GCPs for which image coordinates and 3D ground coordinates are known. We obtained the system of the nonlinear equations on these well-established GCPs. Each equation describes the relationship between the image space coordinates and the object space coordinates. Then, we examined the RFM accuracy using well-established ground check points. This accuracy heavily depends on the distribution of GCPs. We propose a new method of computing RPCs using the robust selection of well-established GCPs. Our method is based on the analysis of spectral properties of the system of equations generated from these points.

Moving Target Tracking in SAR Data Using Combined Exo- and Endo-Clutter Processing

Detecting and tracking moving targets in synthetic aperture radar (SAR) data is a challenging task, demanding state-of-the-art processing methods and advanced SAR systems. Current approaches concentrate on the problem of either endo-clutter moving target tracking or exo-clutter moving target tracking, neglecting the advantages of a joint tracking framework. We present an approach relying on a combined exo- and endo-clutter processing scheme using SAR data with a high pulse repetition frequency. The main processing chain is subdivided into four major steps: 1) focusing of temporal and spatial overlapping SAR images; 2) extracting image statistics for each of these subaperture images in the endo- and exo-clutter domains; 3) subsequent tracking of both endo- and exo-clutter observations using multitarget unscented Kalman filtering; and 4) calculating real-world speeds and positions from the SAR image space coordinates using a road network. The results of this approach are validated and compared with ground-based measurements, and it is found that 100% of the vehicles were detected correctly with an accuracy in speed of 0.02 ± 0.31 m/s and an average tracking time of ~28 s.

VSP wavefield separation using the gray-scale Hough transform: synthetic data

The principal objective in Vertical Seismic Profile (VSP) processing is the separation of the downgoing and upgoing wavefields. Several methods have been suggested in this area. This paper presents a new approach based on the gray-scale Hough transform (GSHT) which is an extension of the conventional Hough transform used to detect straight lines and other curves. The technique, we suggest here, directly maps the gray-scale VSP image, including the downgoing and upgoing linear events, in image coordinate space (x,t,g) to the gray Hough parameter counting space (θ,ρ,g), where θ and ρ are the polar parameters and g is the gray-scale value. In this new space, the downgoing events appear in the negative angles θ quadrant and the upgoing in the positive quadrant, owning to their opposite apparent velocities. The inverse GSHT algorithm, we developed in this study, is performed for extracting separately these two wavefields by considering the straight lines that satisfy the corresponding filtering conditions. The experimental results on synthetic VSP datasets are convincing. The wave separation is well performed, even in the presence of loud noise levels, with signal to noise ratio improvement and amplitude preservation, in contrast to median filtering.

Design and Verification of Novel Low-Cost MR-Guided Small-Animal Stereotactic System

Stereotactic systems have been used for decades to position surgical instruments in important preclinical and clinical studies. We developed and verified a low-cost magnetic-resonance-guided system for accurate preclinical stereotactic operations on small animals. The system consists of both commercially available and custom components, including in-house software written in MATLAB to register coordinate spaces. The software registers target the image-space coordinates from magnetic resonance imaging to the physical space through a rigid-body transform. The transform uses singular value decomposition to determine rotation and translation matrices that are optimal in a least squares sense. Needle positioning and core biopsy studies were performed to verify the accuracy of the system. The needle positioning study demonstrated a sub-millimeter positioning error of 0.294 ± 0.156 mm while using a 26 s gauge Hamilton needle. Biopsy studies demonstrated >94 % success in accessing sub-millimeter stereotactic targets embedded in agarose gel phantoms. These favorable results confirm that despite its relatively low hardware cost (US$18,730) and easy assembly, the system is able to consistently access stereotactic targets on the spatial scale of a small animal.

Visual Servoing of Quadrotors for Perching by Hanging From Cylindrical Objects

This letter addresses vision-based localization and servoing for quadrotors to enable autonomous perching by hanging from cylindrical structures using only a monocular camera. We focus on the problems of relative pose estimation, control, and trajectory planning for maneuvering a robot relative to cylinders with unknown orientations. We first develop a geometric model that describes the pose of the robot relative to a cylinder. Then, we derive the dynamics of the system, expressed in terms of the image features. Based on the dynamics, we present a controller, which guarantees asymptotic convergence to the desired image space coordinates. Finally, we develop an effective method to plan dynamically feasible trajectories in the image space, and we provide experimental results to demonstrate the proposed method under different operating conditions such as hovering, trajectory tracking, and perching.

Image Space Coordinates Extraction Algorithm Based on the Perspective Projection Transformation Matrix

The paper analyses the 3D graphics transformation in computer graphics and its displays process. The method of graphics transformation is introduced. The analysis focuses on the perspective projection transformation matrix theory and its derivation. The calculation method of image space coordinates in the image point coordinates systems is introduced.

3D gradient enhancement

Enhancement can exaggerate visual details in both image processing and 3D rendering. In this paper, we adapt the gradient enhancement technique from image processing to 3D rendering through differentiating the rendering result with respect to the image space coordinates under point lighting. In this way, we can achieve 3D enhancement taking into account the gradient of geometry, projection transform, visibility and highlight. We also propose a tunable shape descriptor for users to achieve rendering results in different enhancement extent. Moreover, we extend this method to the environment lighting with some simplifications. Finally, we demonstrate that our method can handle the grazing angle area and the edges of sharp slope better than the previous method due to the gradient of the projection transform.

IMAGE DESCRIPTION WITH NONSEPARABLE TWO-DIMENSIONAL CHARLIER AND MEIXNER MOMENTS

This paper presents two new sets of nonseparable discrete orthogonal Charlier and Meixner moments describing the images with noise and that are noise-free. The basis functions used by the proposed nonseparable moments are bivariate Charlier or Meixner polynomials introduced by Tratnik et al. This study discusses the computational aspects of discrete orthogonal Charlier and Meixner polynomials, including the recurrence relations with respect to variable x and order n. The purpose is to avoid large variation in the dynamic range of polynomial values for higher order moments. The implementation of nonseparable Charlier and Meixner moments does not involve any numerical approximation, since the basis function of the proposed moments is orthogonal in the image coordinate space. The performances of Charlier and Meixner moments in describing images were investigated in terms of the image reconstruction error, and the results of the experiments on the noise sensitivity are given.

Technique for Targeting Arteriovenous Malformations Using Frameless Image-Guided Robotic Radiosurgery

To integrate three-dimensional (3D) digital rotation angiography (DRA) and two-dimensional (2D) digital subtraction angiography (DSA) imaging into a targeting methodology enabling comprehensive image-guided robotic radiosurgery of arteriovenous malformations (AVMs). DRA geometric integrity was evaluated by imaging a phantom with embedded markers. Dedicated DSA acquisition modes with preset C-arm positions were configured. The geometric reproducibility of the presets was determined, and its impact on localization accuracy was evaluated. An imaging protocol composed of anterior-posterior and lateral DSA series in combination with a DRA run without couch displacement between acquisitions was introduced. Software was developed for registration of DSA and DRA (2D-3D) images to correct for: (a) small misalignments of the C-arm with respect to the estimated geometry of the set positions and (b) potential patient motion between image series. Within the software, correlated navigation of registered DRA and DSA images was incorporated to localize AVMs within a 3D image coordinate space. Subsequent treatment planning and delivery followed a standard image-guided robotic radiosurgery process. DRA spatial distortions were typically smaller than 0.3 mm throughout a 145-mm × 145-mm × 145-mm volume. With 2D-3D image registration, localization uncertainties resulting from the achievable reproducibility of the C-arm set positions could be reduced to about 0.2 mm. Overall system-related localization uncertainty within the DRA coordinate space was 0.4 mm. Image-guided frameless robotic radiosurgical treatments with this technique were initiated. The integration of DRA and DSA into the process of nidus localization increases the confidence with which radiosurgical ablation of AVMs can be performed when using only an image-guided technique. Such an approach can increase patient comfort, decrease time pressure on clinical and technical staff, and possibly reduce the number of cerebral angiograms needed for a particular patient.

Application accuracy in frameless image-guided neurosurgery: a comparison study of three patient-to-image registration methods

The aim of this study was to compare three patient-to-image registration methods in frameless stereotaxy in terms of their application accuracy (the accuracy with which the position of a target can be determined intraoperatively). In frameless stereotaxy, imaging information is transposed to the surgical field to show the spatial position of a localizer or surgical instrument. The mathematical relationship between the image volume and the surgical working space is calculated using a rigid body transformation algorithm, based on point-pair matching or surface matching. Fifty patients who were scheduled to undergo a frameless image-guided neurosurgical procedure were included in the study. Prior to surgery, the patients underwent either computerized tomography (CT) scanning or magnetic resonance (MR) imaging with widely distributed adhesive fiducial markers on the scalp. An extra fiducial marker was placed on the head as a target, as near as possible to the intracranial lesion. Prior to each surgical procedure, an optical tracking system was used to perform three separate patient-to-image registration procedures, using anatomical landmarks, adhesive markers, or surface matching. Subsequent to each registration, the target registration error (TRE), defined as the Euclidean distance between the image space coordinates and world space coordinates of the target marker, was determined. Independent of target location or imaging modality, mean application accuracy (+/- standard deviation) was 2.49 +/- 1.07 mm when using adhesive markers. Using the other two registration strategies, mean TREs were significantly larger (surface matching, 5.03 +/- 2.30 mm; anatomical landmarks, 4.97 +/- 2.29 mm; p < 0.001 for both). The results of this study show that skin adhesive fiducial marker registration is the most accurate noninvasive registration method. When images from an earlier study are to be used and accuracy may be slightly compromised, anatomical landmarks and surface matching are equally accurate alternatives.

Robot-assisted needle placement in open MRI: System architecture, integration and validation

In prostate cancer treatment, there is a move toward targeted interventions for biopsy and therapy, which has precipitated the need for precise image-guided methods for needle placement. This paper describes an integrated system for planning and performing percutaneous procedures with robotic assistance under MRI guidance. A graphical planning interface allows the physician to specify the set of desired needle trajectories, based on anatomical structures and lesions observed in the patient's registered pre-operative and pre-procedural MR images, immediately prior to the intervention in an open-bore MRI scanner. All image-space coordinates are automatically computed, and are used to position a needle guide by means of an MRI-compatible robotic manipulator, thus avoiding the limitations of the traditional fixed needle template. Automatic alignment of real-time intra-operative images aids visualization of the needle as it is manually inserted through the guide. Results from in-scanner phantom experiments are provided.

Robot-assisted needle placement in open MRI: System architecture, integration and validation

This work describes an integrated system for planning and performing percutaneous procedures-such as prostate biopsy-with robotic assistance under MRI-guidance. The physician interacts with a planning interface in order to specify the set of desired needle trajectories, based on anatomical structures and lesions observed in the patient's MR images. All image-space coordinates are automatically computed, and used to position a needle guide by means of an MRI-compatible robotic manipulator, thus avoiding the limitations of the traditional fixed needle template. Direct control of real-time imaging aids visualization of the needle as it is manually inserted through the guide. Results from in-scanner phantom experiments are provided.

Fast Computation of Chebyshev Moments

Chebyshev moments were recently proposed to address problems of moments of continuous orthogonal basis. They are moments of a discrete basis and are orthogonal in the domain of the image coordinate space, a feature which presents a number of advantages over moments of continuous orthogonal basis. No effort has been devoted yet to the efficiency of their calculation. In this paper, we present a hardware architecture which enables real-time processing of binary and grayscale images of up to 4 Mpixels. This architecture is realized in a field-programmable gate array of typical clock frequency of 100 MHz

Creation of Three-dimensional Crop Maps based on Aerial Stereoimages

Geographical information systems (GIS) are currently migrating from two dimensions (2D) to three dimensions (3D). A three-dimensional representation of a field scene can improve the presentation of crop information in GIS and also provide more information for controlling autonomous operations inside agricultural fields. This research investigated the use of a compact stereocamera mounted on a remote-controlled helicopter to acquire stereoimages of a maize field and represent the crop information in a 3D crop map. In this 3D crop map generation method, individual images were fused in a global map by transforming image space coordinates into global East-North coordinates. A global positioning system (GPS) receiver and an inertial measurement unit (IMU) were used to record position and orientation data for every stereoimage. This paper presents a general methodology for creating 3D crop maps based on the information gathered with a stereoscopic camera, an inertial sensor and a GPS receiver to represent more informative crop status.