3D Recovery Research Articles

Registration of Images To Lidar and GIS Data Without Establishing Explicit Correspondences

Recovering the camera orientation is a fundamental problem in photogrammetry for precision 3D recovery, orthophoto generation, and image registration. In this paper, we achieve this goal by fusing the image information with information extracted from different modalities, including lidar and <small>GIS</small>. In contrast to other approaches, which require feature correspondences, our approach exploits edges across the modalities without the necessity to explicitly establish correspondences. In the proposed approach, extracted edges from different modalities are not required to have analytical forms. This flexibility is achieved by minimizing a new cost function using a Bayesian approach, which takes the Euclidean distances between the projected edges extracted from the other data source and the edges extracted from the reference image as its random variable. The proposed formulation minimizes the overall distances between the sets of edges iteratively, such that the end product results in the correct camera parameters for the reference image as well as matching features across the modalities. The initial solution can be obtained from <small>GPS/IMU</small> data. The formulation is shown to successfully handle noise and missing observations in edges. Point matching methods may fail for oblique images, especially high oblique images. We eliminate the requirement for exact point-to-point matching. The feasibility of the method is experimented with nadir and oblique images.

How Common Is Signal-Intensity Increase in Optic Nerve Segments on 3D Double Inversion Recovery Sequences in Visually Asymptomatic Patients with Multiple Sclerosis?

In postmortem studies, subclinical optic nerve demyelination is very common in patients with MS but radiologic demonstration is difficult and mainly based on STIR T2WI. Our aim was to evaluate 3D double inversion recovery MR imaging for the detection of subclinical demyelinating lesions within optic nerve segments. The signal intensities in 4 different optic nerve segments (ie, retrobulbar, canalicular, prechiasmatic, and chiasm) were evaluated on 3D double inversion recovery MR imaging in 95 patients with MS without visual symptoms within the past 3 years and in 50 patients without optic nerve pathology. We compared the signal intensities with those of the adjacent lateral rectus muscle. The evaluation was performed by a student group and an expert neuroradiologist. Statistical evaluation (the Cohen κ test) was performed. On the 3D double inversion recovery sequence, optic nerve segments in the comparison group were all hypointense, and an isointense nerve sheath surrounded the retrobulbar nerve segment. At least 1 optic nerve segment was isointense or hyperintense in 68 patients (72%) in the group with MS on the basis of the results of the expert neuroradiologist. Student raters were able to correctly identify optic nerve hypersignal in 97%. A hypersignal in at least 1 optic nerve segment on the 3D double inversion recovery sequence compared with hyposignal in optic nerve segments in the comparison group was very common in visually asymptomatic patients with MS. The signal-intensity rating of optic nerve segments could also be performed by inexperienced student readers.

3-4 固視微動型カメラ回転による画像ボケからの3D復元における低周波成分抑圧の効果(第3部門 コンピュータビジョン)

3-4 固視微動型カメラ回転による画像ボケからの3D復元における低周波成分抑圧の効果(第3部門 コンピュータビジョン)

Experimental evaluation of a 3D wavelet-based phase recovery method in temporal speckle pattern interferometry.

We test the performance of a phase recovery method based on a three-dimensional directional wavelet transform applied to the intensity signal measured by temporal speckle pattern interferometry (TSPI). We present and discuss several sources of uncertainty by analyzing experimental datasets recorded for an in-plane interferometer without introducing a temporal carrier. The dynamic phase data measured with the proposed method are compared with those obtained from the well-known one-dimensional Fourier transform phase recovery technique. In the Fourier method, the filtered Fourier transform for each intensity pixel is evaluated along the temporal direction. In contrast, the three-dimensional directional wavelet transform method uses the information of adjacent pixels and then increases the performance of the recovered dynamic phase results. The advantages and limitations of the three-dimensional directional wavelet transform approach are discussed, and a summary of conclusions from the analysis of TSPI data is also given.

A survey of structure from motion.

The structure from motion (SfM) problem in computer vision is to recover the three-dimensional (3D) structure of a stationary scene from a set of projective measurements, represented as a collection of two-dimensional (2D) images, via estimation of motion of the cameras corresponding to these images. In essence, SfM involves the three main stages of (i) extracting features in images (e.g. points of interest, lines,etc.) and matching these features between images, (ii) camera motion estimation (e.g. using relative pairwise camera positions estimated from the extracted features), and (iii) recovery of the 3D structure using the estimated motion and features (e.g. by minimizing the so-calledreprojection error). This survey mainly focuses on relatively recent developments in the literature pertaining to stages (ii) and (iii). More specifically, after touching upon the early factorization-based techniques for motion and structure estimation, we provide a detailed account of some of the recent cameralocationestimation methods in the literature, followed by discussion of notable techniques for 3D structure recovery. We also cover the basics of thesimultaneous localization and mapping(SLAM) problem, which can be viewed as a specific case of the SfM problem. Further, our survey includes a review of the fundamentals of feature extraction and matching (i.e. stage (i) above), various recent methods for handling ambiguities in 3D scenes, SfM techniques involving relatively uncommon camera models and image features, and popular sources of data and SfM software.

Binocular 3D Object Recovery Using a Symmetry Prior

We present a new algorithm for 3D shape reconstruction from stereo image pairs that uses mirror symmetry as a biologically inspired prior. 3D reconstruction requires some form of prior because it is an ill-posed inverse problem. Psychophysical research shows that mirror-symmetry is a key prior for 3D shape perception in humans, suggesting that a general purpose solution to this problem will have many applications. An approach is developed for finding objects that fit a given shape definition. The algorithm is developed for shapes with two orthogonal planes of symmetry, thus allowing for straightforward recovery of occluded portions of the objects. Two simulations were run to test: (1) the accuracy of 3D recovery, and (2) the ability of the algorithm to find the object in the presence of noise. We then tested the algorithm on the Children’s Furniture Corpus, a corpus of stereo image pairs of mirror symmetric furniture objects. Runtimes and 3D reconstruction errors are reported and failure modes described.

MR Neurography of Brachial Plexus at 3.0 T with Robust Fat and Blood Suppression.

Purpose To develop and evaluate magnetic resonance (MR) neurography of the brachial plexus with robust fat and blood suppression for increased conspicuity of nerves at 3.0 T in clinically feasible acquisition times. Materials and Methods This prospective study was HIPAA compliant, with institutional review board approval and written informed consent. A low-refocusing-flip-angle three-dimensional (3D) turbo spin-echo (TSE) sequence was modified to acquire both in-phase and out-of-phase echoes, required for chemical shift (Dixon) reconstruction, in the same repetition by using partial echoes combined with modified homodyne reconstruction with phase preservation. This multiecho TSE modified Dixon (mDixon) sequence was optimized by using simulations and phantom studies and in three healthy volunteers. The sequence was tested in five healthy volunteers and was evaluated in 10 patients who had been referred for brachial plexopathy at 3.0 T. The images were evaluated against the current standard of care, images acquired with a 3D TSE short inversion time inversion recovery (STIR) sequence, qualitatively by using the Wilcoxon signed-rank test and quantitatively by using the Friedman two-way analysis of variance, with P < .05 considered to indicate a statistically significant difference. Results Multiecho TSE-mDixon involving partial-echo and homodyne reconstruction with phase preservation achieved uniform fat suppression in half the imaging time compared with multiacquisition TSE-mDixon. Compared with 3D TSE STIR, fat suppression, venous suppression, and nerve visualization were significantly improved (P < .05), while arterial suppression was better but not significantly so (P = .06), with increased apparent signal-to-noise ratio in the dorsal nerve root ganglion and C6 nerve (P < .001) with the multiecho TSE-mDixon sequence. Conclusion The multiecho 3D TSE-mDixon sequence provides robust fat and blood suppression, resulting in increased conspicuity of the nerves, in clinically feasible imaging times and can be used for MR neurography of the brachial plexus at 3.0 T. © RSNA, 2016 Online supplemental material is available for this article.

Digitally reconstructing the Great Parchment Book: 3D recovery of fire-damaged historical documents

The Great Parchment Book of the Honourable the Irish Society is a major surviving historical record of the estates of the county of Londonderry (in modern day Northern Ireland). It contains key data about landholding and population in the Irish province of Ulster and the city of Londonderry and its environs in the mid-17th century, at a time of social, religious, and political upheaval. Compiled in 1639, it was severely damaged in a fire in 1786, and due to the fragile state of the parchment, its contents have been mostly inaccessible since. We describe here a long-term, interdisciplinary, international partnership involving conservators, archivists, computer scientists, and digital humanists that developed a low-cost pipeline for conserving, digitizing, 3D-reconstructing, and virtually flattening the fire-damaged, buckled parchment, enabling new readings and understanding of the text to be created. For the first time, this article presents a complete overview of the project, detailing the conservation, digital acquisition, and digital reconstruction methods used, resulting in a new transcription and digital edition of the text in time for the 400th anniversary celebrations of the building of Londonderry’s city walls in 2013. We concentrate on the digital reconstruction pipeline that will be of interest to custodians of similarly fire-damaged historical parchment, whilst highlighting how working together on this project has produced an online resource that has focussed community reflection upon an important, but previously inaccessible, historical text.

The "White Gray Sign" Identifies the Central Sulcus on 3T High-Resolution T1-Weighted Images.

The central sulcus is an important anatomic landmark, but most methods of identifying it rely on variable gyral and sulcal patterns. We describe and assess the accuracy of reduced gray-white contrast along the central sulcus, an observation we term the "white gray sign." We conducted a retrospective review of 51 fMRIs with a T1-weighted 3D inversion recovery fast-spoiled gradient-echo and concomitant hand-motor fMRI, which served as confirmation for the location of the central sulcus. To measure gray-white contrast across the central and adjacent sulci, we performed a quantitative analysis of 25 normal hemispheres along the anterior and posterior cortices and intervening white matter of the pre- and postcentral gyri. 3D inversion recovery fast-spoiled gradient-echo axial images from 51 fMRIs were then evaluated by 2 raters for the presence of the white gray sign as well as additional established signs of the central sulcus: the bracket, cortical thickness, omega, and T signs. The mean gray-white contrast along the central sulcus was 0.218 anteriorly and 0.237 posteriorly, compared with 0.320 and 0.295 along the posterior precentral and anterior postcentral sulci, respectively (P < .001). Both raters correctly identified the central sulcus in all 35 normal and 16 abnormal hemispheres. The white gray sign had the highest agreement of all signs between raters and was rated as present the most often among all the signs. Reduced gray-white contrast around the central sulcus is a reliable sign for identification of the central sulcus on 3D inversion recovery fast-spoiled gradient-echo images.

3D whole-heart phase sensitive inversion recovery CMR for simultaneous black-blood late gadolinium enhancement and bright-blood coronary CMR angiography

BackgroundPhase sensitive inversion recovery (PSIR) applied to late gadolinium enhancement (LGE) imaging is widely used in clinical practice. However, conventional 2D PSIR LGE sequences provide sub-optimal contrast between scar tissue and blood pool, rendering the detection of subendocardial infarcts and scar segmentation challenging. Furthermore, the acquisition of a low flip angle reference image doubles the acquisition time without providing any additional diagnostic information. The purpose of this study was to develop and test a novel 3D whole-heart PSIR-like framework, named BOOST, enabling simultaneous black-blood LGE assessment and bright-blood visualization of cardiac anatomy.MethodsThe proposed approach alternates the acquisition of a 3D volume preceded by a T2-prepared Inversion Recovery (T2Prep-IR) module (magnitude image) with the acquisition of a T2-prepared 3D volume (reference image). The two volumes (T2Prep-IR BOOST and bright-blood T2Prep BOOST) are combined in a PSIR-like reconstruction to obtain a complementary 3D black-blood volume for LGE assessment (PSIR BOOST). The black-blood PSIR BOOST and the bright-blood T2Prep BOOST datasets were compared to conventional clinical sequences for scar detection and coronary CMR angiography (CMRA) in 18 patients with a spectrum of cardiovascular disease (CVD).ResultsDatasets from 12 patients were quantitatively analysed. The black-blood PSIR BOOST dataset provided statistically improved contrast to noise ratio (CNR) between blood and scar when compared to a clinical 2D PSIR sequence (15.8 ± 3.3 and 4.1 ± 5.6, respectively). Overall agreement in LGE depiction was found between 3D black-blood PSIR BOOST and clinical 2D PSIR acquisitions, with 11/12 PSIR BOOST datasets considered diagnostic. The bright-blood T2Prep BOOST dataset provided high quality depiction of the proximal coronary segments, with improvement of visual score when compared to a clinical CMRA sequence. Acquisition time of BOOST (~10 min), providing information on both LGE uptake and heart anatomy, was comparable to that of a clinical single CMRA sequence.ConclusionsThe feasibility of BOOST for simultaneous black-blood LGE assessment and bright-blood coronary angiography was successfully tested in patients with cardiovascular disease. The framework enables free-breathing multi-contrast whole-heart acquisitions with 100% scan efficiency and predictable scan time. Complementary information on 3D LGE and heart anatomy are obtained reducing examination time.

Chiral thiophosphoramide catalyzed asymmetric aryl transfer reactions for the synthesis of functional diarylmethanols

In this investigation, chiral thiophosphoramide 3d was easily prepared from chiral (1R,2R)-1,2-diphenylethylenediamine and then applied as an efficient chiral ligand in the catalytic asymmetric arylation reactions of various aromatic aldehydes. The corresponding diarylmethanol products were produced with good to excellent yields (up to 98%) and enantioselectivities (up to 94%). The recovery of chiral ligand 3d could be as high as 96%.

Single step full volumetric reconstruction optical coherence tomography utilizing compressive sensing

3D optical coherence tomography imaging (OCT) combined with compressive sensing (CS) has been proved to be an attractive and effective tool in a variety of fields, such as medicine and biology. To achieve high quality imaging while using as less CS sampling rate as possible is the goal of this approach. Here we present an innovative single step fully 3D CS-OCT volumetric image recovery method, in which 3D OCT volumetric image of the object is compressively sampled via our proposed CS coding strategies in all three dimensions while its sparsity is simultaneously taken into consideration in every direction. The object can be directly recovered as the whole volume reconstruction via our advanced full 3D CS reconstruction algorithm. The numerical simulations of a human retina OCT volumetric image reconstruction by our method demonstrate a PSNR of as high as 38dB at a sampling rate of less than 10%.

Simultaneous acquisition of high-contrast and quantitative liver T1 images using 3D phase-sensitive inversion recovery: a feasibility study.

Background Tumor-to-liver contrast is low in images of chronically diseased livers because gadolinium-based hepatocyte-specific contrast agents (Gd-EOB-DTPA) accumulate less to hepatocytes. Purpose To determine whether phase-sensitive inversion recovery (PSIR) could improve the T1 contrasts of Gd-based contrast agents and liver parenchyma and simultaneously provide accurate T1 values for abdominal organs. Material and Methods The image contrasts of phantoms with different Gd concentrations that were obtained using PSIR were compared to conventional turbo field echo (TFE) results. T1 value was estimated using PSIR by performing iterations to investigate the two IR magnetization evolutions. The estimated T1 values were validated using IR-spin echo (IR-SE) and Look-Locker (L-L) sequences. In an invivo study, the liver-to-spleen and liver-to-muscle contrasts of the PSIR and TFE images of seven volunteers were compared, as were the T1 values of liver parenchyma, spleen, and muscle obtained using PSIR and L-L sequences. Results The PSIR images showed T1 contrasts higher than those in the TFE results. The PSIR and IR-SE T1 values were linearly correlated. Additionally, the R1 estimated using PSIR were correlated with those measured using IR-SE and L-L. In the invivo study, the liver-to-spleen and liver-to-muscle contrasts of PSIR were significantly higher than those obtained using TFE. T1 values of abdominal organs obtained using PSIR and L-L were clearly correlated. Conclusion PSIR may be capable of improving liver image T1 contrasts when Gd-based contrast agents are employed and simultaneously yielding accurate T1 values of abdominal organs.

GRAPE: a graphical pipeline environment for image analysis in adaptive magnetic resonance imaging.

We present a platform, GRAphical Pipeline Environment (GRAPE), to facilitate the development of patient-adaptive magnetic resonance imaging (MRI) protocols. GRAPE is an open-source project implemented in the Qt C++ framework to enable graphical creation, execution, and debugging of real-time image analysis algorithms integrated with the MRI scanner. The platform provides the tools and infrastructure to design new algorithms, and build and execute an array of image analysis routines, and provides a mechanism to include existing analysis libraries, all within a graphical environment. The application of GRAPE is demonstrated in multiple MRI applications, and the software is described in detail for both the user and the developer. GRAPE was successfully used to implement and execute three applications in MRI of the brain, performed on a 3.0-T MRI scanner: (i) a multi-parametric pipeline for segmenting the brain tissue and detecting lesions in multiple sclerosis (MS), (ii) patient-specific optimization of the 3D fluid-attenuated inversion recovery MRI scan parameters to enhance the contrast of brain lesions in MS, and (iii) an algebraic image method for combining two MR images for improved lesion contrast. GRAPE allows graphical development and execution of image analysis algorithms for inline, real-time, and adaptive MRI applications.

Structural and recovery mechanisms of 3D dental pulp cell microtissues challenged with Streptococcusmutans in extracellular matrix environment.

Cariopathogen Streptococcus mutans exists in infected dental pulp of deciduous teeth and is frequently linked with heart diseases. Organotypic (3D) dental pulp stem cell (DPSC) cultures/microtissues, developed to mimic the physiological conditions in vivo, were utilized to assess the bacterial impact on their (i) 3D structural configuration and (ii) recovery mechanisms. The cultures, developed in extracellular matrix (ECM) bio-scaffold (Matrigel™), interacted with WT and GFP-tagged bacterial biofilms by permitting their infiltration through the ECM. Challenged cell constructs were visualized by F-actin/nuclei staining. Their pluripotency (Sox2) and differentiation (osteocalcin) markers were assessed by immunocytochemistry. Secreted mineral was detected by alizarin red, and 3D structural arrangements were analysed by epi-fluorescence and confocal scanning microscopy. Bacterial biofilm/ECM-embedded DPSC interactions appeared in distinct areas of the microtissues. Bacterial attachment to the cell surface occurred without evidence of invasion. Surface architecture of the challenged versus unchallenged microtissues was apparently unaltered. However, significant increases in thickness (138.42 vs 106.51 µm) and bacterial penetration were detected in challenged structures causing canal-like microstructures with various diameters (12.94 -42.88 µm) and average diameter of 20.66 to 33.42 µm per microtissue. Challenged constructs expressed pluripotency and differentiation markers and secreted the mineral. Presented model shows strong potential for assessing pulp-pathogen interactions in vivo. S. mutans infiltrated and penetrated the microtissues but did not invade the cells or compromise major cell repair mechanisms. These findings would suggest reexamining the role of S. mutans as an endodontic pathogen and investigating DPSC resistance to its pathogenicity.

3D differential phase contrast microscopy.

We demonstrate 3D phase and absorption recovery from partially coherent intensity images captured with a programmable LED array source. Images are captured through-focus with four different illumination patterns. Using first Born and weak object approximations (WOA), a linear 3D differential phase contrast (DPC) model is derived. The partially coherent transfer functions relate the sample's complex refractive index distribution to intensity measurements at varying defocus. Volumetric reconstruction is achieved by a global FFT-based method, without an intermediate 2D phase retrieval step. Because the illumination is spatially partially coherent, the transverse resolution of the reconstructed field achieves twice the NA of coherent systems and improved axial resolution.

Visualization of endolymphatic hydrops after intratympanic injection of Gd-DTPA: a comparison of high resolution three dimensional-real inversion recovery and three dimensional fluid-attenuated inversion recovery with a variable flip angle sequence

Objective To compare the value of high resolution 3D-real inversion recovery(3D-real IR) and 3D fluid-attenuated inversion recovery with a variable flip angle (3D-Flair-VFL) sequence in the visualization of endolymphatic hydrops. Methods 52 ears in 40 patients with Meniere's disease or delayed endolymphatic hydrops were enrolled and performed by axial 3D-real IR and 3D-Flair MRI scans 24 hours after intratympanic injection of eight-fold diluted Gd-DTPA. Signal to noise ratio (SNR) and contrast-to-noise ratio (CNR) of the two sequences were calculated and compared by t test. Whether there were endolymphatic hydrops in the basal turn, middle turn, apical turn of the cochlear and vestibule was further evaluated. Results Both the SNR and CNR of 3D-real IR images were statistically higher than 3D-Flair images (SNR: 28.06±12.71 vs. 17.64±6.38, P<0.05; CNR: 33.79±13.52 vs. 15.40±6.04, P<0.05). Endolymphatic hydrops of the basal turn, second turn, apical turn of the cochlear and vestibule detected by 3D-Flair-VFL images were 33, 37, 10 and 41 cases, respectively, while by 3D-Flair-VFL images were 46, 46, 46 and 49 cases, respectively. There were significant differences between the two sequences in evaluating basal turn, second turn, apical turn of the cochlear except vestibule (P<0.05). Conclusion 3D-real IR images are clearer than 3D-Flair-VFL images and can display endolymphatic hydrops more precisely. Key words: Magnetic resonance imaging; Ear, inner; Meniere disease

Generalized recovery algorithm for 3D super-resolution microscopy using rotating point spread functions.

Super-resolution microscopy with phase masks is a promising technique for 3D imaging and tracking. Due to the complexity of the resultant point spread functions, generalized recovery algorithms are still missing. We introduce a 3D super-resolution recovery algorithm that works for a variety of phase masks generating 3D point spread functions. A fast deconvolution process generates initial guesses, which are further refined by least squares fitting. Overfitting is suppressed using a machine learning determined threshold. Preliminary results on experimental data show that our algorithm can be used to super-localize 3D adsorption events within a porous polymer film and is useful for evaluating potential phase masks. Finally, we demonstrate that parallel computation on graphics processing units can reduce the processing time required for 3D recovery. Simulations reveal that, through desktop parallelization, the ultimate limit of real-time processing is possible. Our program is the first open source recovery program for generalized 3D recovery using rotating point spread functions.

The changing fortunes of pattern recognition and computer vision

As someone who had been attending conferences on pattern recognition and computer vision since 1978, I have watched with interest the ups and downs of pattern recognition and computer vision areas and how they have been presented at various conferences such as PRIP, CVPR, ECCV, ICCV, ACCV, ICPR, IJCAI, AAAI, NIPS, ICASSP and ICIP. Given the recent successes of deep learning networks, it appears that the scale is tilting towards pattern recognition as is commonly understood. A good number of papers in recent vision conferences seem to push data through one or other deep learning networks and report improvements over state of the art. While one cannot argue against the remarkable (magical?) performance improvements obtained by deep learning network-based approaches, I worry that five years from now, most students in computer vision will only be aware of software that implements some deep learning network. After all, 2-D based detection and recognition problems for which the deep learning networks have shown their mettle are only a subset of the computer vision field. While enjoying the ride, I would like to caution that understanding of scene and image formation, invariants, interaction between light and matter, human vision, 3D recovery, and emerging concepts like common sense reasoning are too important to ignore for the long-term viability of the computer vision field. It will be a dream come true if we manage to integrate these computer vision concepts into deep learning networks so that more robust performance can be obtained with less data and cheaper computers.

High-accuracy dithering-based three-dimensional data compression algorithms by phase correction.

The Holoimage technique can substantially reduce three-dimensional (3D) shape data sizes by converting the data into regular 24-bit 2D images. A dithering-based compression algorithm was recently proposed to further reduce the data size by only utilizing three bits to represent 3D shape data. However, our research finds that the recovery accuracy of the dithering-based algorithm is not very high due to the loss of information in the dithering process. In addition, we find that the final 3D recovery accuracy is highly related to the phase information. In this paper, to address the low accuracy problem, we propose two different phase-correction methods to store the phase-error information and retrieve the absolute phase with higher accuracy. Both methods can provide recovered 3D results with much better quality. Experimental results are presented to verify the performance of the proposed methods.