Large Field Of View Imaging Research Articles

A new resolution enhancement method for sandstone thin-section images using perceptual GAN

The thin section is a sliver cut of rock sample and photographed by a microscope to help researchers investigate the mineral, structure, and geology details of a rock sample. For the purpose of photographing using the microsctope, only a small portion of the thin section can be visualized. The results, however, can hardly satisfy the requirement of a high resolution and larger scale visualization. The thin section is then trapped into a dilemma of contradiction between resolution and field of view. To solve this problem, we propose the Generative Adversarial Networks（GAN） based architectures representing the perceptual SR in this paper. Four perceptual super-resolution methods and two pixel-wise super-resolution methods are trained and tested for comparison with the dataset of sandstone thin-section. The perceptual GAN based super-resolution method is demonstrated to compensate for this limitation by enhancing the resolution of a large field of view image, which will provide a higher resolution detail recovery and larger field of view simultaneously. Based on the digital experiment analysis and comparison, thin-section perceptual loss with Residual-in-Residual Dense Block (RRDB) and Relativistic average GANs (RaGAN) shows a more realistic texture and evident edge for minerals and pores than other methods. With the new perceptual GAN based image enhancement method, the petroleum geologist can significantly improve the accuracy and reliability of all their thin section based research quality.

특징점의 구 사상을 활용한 전방향 영상으로부터의 3 차원 영상 지도 생성 기법

In this paper, we propose a robust and fast 3D data reconstruction method for large FoV(Field-of-View) cameras. Large FoV cameras are beneficial in many vision systems because they increase visual overlap and capture more pixels of data from the environment. However, most vision systems operate based on the pinhole camera model. This leads to problems when using large FoV cameras, such as distortion or decreased resolution. Other methods address this problem by using different camera models (e.g., mapping overall images onto a cube surface), but this increases the amount of data processing required, and distortion is not completely eliminated. To address these issues, we propose a method in which 3D data is generated from large FoV images using sphere mapping of features. The proposed method has two advantages. The first is robustness since the sphere model uses the full FoV without distorting the images. The second is speed, arising from the fact that only the feature points are mapped and not the entire image. The proposed system is compared with others on various image sequences. The results show that, within the same amount of processing time, our system can generate a greater number of successful mappings, with greater accuracy, compared to the other approaches.

Color Fourier ptychographic microscopy based on symmetrical illumination and wavelength multiplexing

Fourier ptychographic microscopy (FPM), by using a LED array as its light source and each LED being lit up sequentially, can recover a high resolution (HR) and large field of view image from a sequence of low resolution (LR) images. It is difficult to obtain a perfect high resolution color image by a monochrome camera because of non-overlapping focal planes of different color channels resulting from the residual chromatic aberration of the microscopic objective. In this paper, a kind of method, based on symmetrical LED illumination and wavelength multiplexing, is applied in a FPM system to collect LR images and recover HR color images. Compared with single LED illumination, symmetrical illumination has higher defocus tolerance. It is proven by simulated and experimental results that higher-quality HR color images can be obtained based on wavelength multiplexing and symmetrical illumination by a monochrome camera when defocus or chromatic aberration exist. It is demonstrated that symmetrical illumination is helpful in chromatic aberration correction in color FPM imaging.

Generation of iPS clones using automated live-imaging, picking and in-process quality controls

Background & Aim Induced pluripotent stem (iPS) cells have great potential as tools for regenerative medicine, disease modeling and drug screening applications. However, current manufacturing remain highly dependent on manual tasks and subjective human decisions about which colonies to choose and progress in large-scale cell expansion process. The goal of this work is to develop automated workflow options using the Cell X™ robotic platform for: 1) Cell Source Selection based on quantitative reproducible quality attributes, 2) Precision Automation of iPS clone “picking”, “weeding”, passage and expansion, and 3) quantitative documentation of cell and clone attributes over time to define critical process parameters and critical quality attributes that can predict future performance. Methods, Results & Conclusion The Cell X™ robotic platform was used to develop and characterize imaging and analysis tools relevant to iPS populations using the DF6-9-9T.B hiPSC cell line, reprogramming of human skin fibroblasts. The Cell X platform (Fig. 1) is a multifaceted robotic device that enables: automated large field of view (LFOV) imaging using phase contrast or fluorescence; serial imaging (videomicroscopy); quantitative LFOV image analysis (Colonyze™); media changes; cell and colony “picking” (isolation and transplantation of desired cells), “weeding” (removal of undesired cells); or “biopsy (sampling of defined cells for analysis). Feasibility was demonstrated in the capture LFOV images over the reprogramming phase to track quantitative metrics of evolution and morphology of individual iPS clones (e.g. size, shape, area, perimeter roughness, texture, growth rate) (Fig. 2 & 3). Automated image processing tools were defined to segment regions of undifferentiated iPS cells from regions of differentiating and potentially differentiating cells, which could be used to guide “picking”, “weeding” or “biopsy” interventions. Picking operations were effective, and transplanted cells continued to proliferate effectively with preservation of iPS morphology. The integrated Cell X™ platform enables both fully automated iPS clone “picking” and expansion in a hands free, reproducible, and environmentally controlled setting. This system represent a potential valuable tool to automate and scale iPS fabrication with reduced expense and variation, and with rigorous documentation. Cell X™ methods can be refined for lab-specific application, core facility, and integrated in fully automated industrial GMP manufacturing facility.

Sparse Scanning Electron Microscopy Data Acquisition and Deep Neural Networks for Automated Segmentation in Connectomics.

With the growing importance of three-dimensional and very large field of view imaging, acquisition time becomes a serious bottleneck. Additionally, dose reduction is of importance when imaging material like biological tissue that is sensitive to electron radiation. Random sparse scanning can be used in the combination with image reconstruction techniques to reduce the acquisition time or electron dose in scanning electron microscopy. In this study, we demonstrate a workflow that includes data acquisition on a scanning electron microscope, followed by a sparse image reconstruction based on compressive sensing or alternatively using neural networks. Neuron structures are automatically segmented from the reconstructed images using deep learning techniques. We show that the average dwell time per pixel can be reduced by a factor of 2-3, thereby providing a real-life confirmation of previous results on simulated data in one of the key segmentation applications in connectomics and thus demonstrating the feasibility and benefit of random sparse scanning techniques for a specific real-world scenario.

High-resolution mobile bio-microscope with smartphone telephoto camera lens

Bio-microscopy has been widely applied in clinical pathological diagnosis and life science research. At present, bio-microscope mainly used in laboratories is relatively large. With the development of mobile communication technology, the internet of things (IoT) is introduced as a ubiquitous network interconnection of all things. If bio-microscope can be utilized as an IoT sensing node, biomedical point-of-care testing can be carried out anywhere, especially in remote areas. Commercial off-the-shelve optical devices can be widely accessible for micro-imaging, but optical cameras suitable for microscopes are not easily accessible. In this work, smartphone with telephoto camera lens can be handily modified into a mobile microscope by attaching an infinite correction objective before the telephoto camera lens. Undoubtedly, this microscope naturally has wireless communication capabilities. Furthermore, we applied this mobile bio-microscope to perform bright-field and fluorescence microscopic imaging, and the optical resolution can reach 4.4 μm with 40x objective lens. The bright-field microscopic images can be transmitted to a computer through wireless transmission, and be stitch into a large field of view images via digital image algorithms. The fluorescence images can be observed in real time. In the future, combined with wireless network technology, mobile microscopy will become a sensing node for medical big data terminal, providing an important optical platform for on-line biomedical diagnosis.

Bioinspired Zoom Compound Eyes Enable Variable-Focus Imaging.

Natural compound eyes provide the inspiration for developing artificial optical devices that feature a large field of view (FOV). However, the imaging ability of artificial compound eyes is generally based on the large number of ommatidia. The lack of a tunable imaging mechanism significantly limits the practical applications of artificial compound eyes, for instance, distinguishing targets at different distances. Herein, we reported zoom compound eyes that enable variable-focus imaging by integrating a deformable poly(dimethylsiloxane) (PDMS) microlens array (MLA) with a microfluidic chamber. The thin and soft PDMS MLA was fabricated by soft lithography using a hard template prepared by a combined technology of femtosecond laser processing and wet etching. As compared with other mechanical machining strategies, our combined technology features high flexibility, efficiency, and uniformity, as well as designable processing capability, since the size, distribution, and arrangement of the ommatidia can be well controlled during femtosecond laser processing. By tuning the volume of water injected into the chamber, the PDMS MLA can deform from a planar structure to a hemispherical shape, evolving into a tunable compound eye of variable FOV up to 180°. More importantly, the tunable chamber can functionalize as the main zoom lens for tunable imaging, which endows the compound eye with the additional capability of distinguishing targets at different distances. Its focal length can be turned from 3.03 mm to infinity with an angular resolution of 3.86 × 10-4 rad. This zoom compound eye combines the advantages of monocular eyes and compound eyes together, holding great promise for developing advanced micro-optical devices that enable large FOV and variable-focus imaging.

Limits on the Electromagnetic Counterpart of Binary Black Hole Coalescence at Visible Wavelengths

Abstract We used the Télescope à Action Rapide pour les Objets Transitoires network of telescopes to search for the electromagnetic counterparts of GW150914, GW170104, and GW170814, which were reported to originate from binary black hole merger events by the Laser Interferometer Gravitational-wave Observatory and Virgo collaborations. Our goal is to constrain the emission from a binary black hole coalescence at visible wavelengths. We developed a simple and effective algorithm to detect new sources by matching the image data with the Gaia catalog Data Release 1. Machine learning was used and an algorithm was designed to locate unknown sources in a large field of view image. The angular distance between objects in the image and in the catalog was used to find new sources; we then process the candidates to validate them as possible new unknown celestial objects. Though several possible candidates were detected in the three gravitational-wave source error boxes studied, none of them were confirmed as a viable counterpart. The algorithm was effective for the identification of unknown candidates in a very large field and provided candidates for GW150914, GW170104, and GW170814. The entire 90% GW170814 error box was surveyed extensively within 0.6 days after the gravitational-wave emission resulting in an absolute limiting R magnitude of −23.8. This strong limit excludes to a great extent a possible emission of a gamma-ray burst with an optical counterpart associated with GW170814.

Light-sheet microscopy of cleared tissues with isotropic, subcellular resolution.

We present cleared tissue Axially Swept Light-Sheet Microscopy (ctASLM), which enables isotropic, subcellular resolution, high optical sectioning capability, and large field of view imaging over a broad range of immersion media. ctASLM can image live, expanded, and both aqueous and organic chemically cleared tissue preparations. Depending on the optical configuration, ctASLM provides up to 260 nm axial resolution, an improvement over confocal and other reported cleared tissue light-sheet microscopes by a factor 3–10. We image millimeter-scale tissues with subcellular 3D resolution, which enabled us to automatically detect with computer vision multicellular tissue architectures, individual cells, synaptic spines, and rare cell-cell interactions.

Ultrahigh Resolution Polarization Sensitive Optical Coherence Tomography of the Human Cornea with Conical Scanning Pattern and Variable Dispersion Compensation.

Noninvasive corneal imaging is essential for the diagnosis and treatment control of various diseases affecting the anterior segment of the eye. This study presents an ultrahigh resolution polarization sensitive optical coherence tomography instrument operating in the 840 nm wavelength band that incorporates a conical scanning design for large field of view imaging of the cornea. As the conical scanning introduces a dispersion mismatch depending on the scanning angle, this study implemented variable, location dependent, numerical dispersion compensation in order to achieve high axial resolution throughout the imaged volume. The corneal images were recorded in vivo in healthy volunteers showing various details of corneal structures.

Microprism for a capsule endoscope with wafer level optics

Capsule endoscopes are pill-sized devices that physicians can utilize for diagnosis of a patient’s small intestine. For diagnostic requirements, excellent image quality, large field of view (FOV) and compact size are crucial. In this paper, a multi-camera capsule endoscope (MCCE) composed of two wafer level optic modules and microprism elements which can offer both a large FOV and clear image quality within limited space conditions is described. The properties of the microprism needed to ensure the necessary image quality are further discussed, including working efficiency, the ability to increase the FOV, and the inhibition of chromatic aberration. Finally, the functioning of the MCCE is verified through an optical simulation and preliminary experiments. The results show that the max half FOV of the MCCE could be increased from 28° to 37°, and the relative illumination could be kept at 80% without a dark corner effect.

High throughput performance-based selection and expansion of clonal progenitor populations resident in human articular cartilage to characterize sub-populations for effective cartilage stem cell therapies

Background & Aim Enormous heterogeneity in autogenous tissue sources of mesenchymal stromal cells (MSCs) with respect to the quantity and quality of the stem and progenitor cells that serve as the starting materials for cellular therapeutics limits delivery of safe, effective and reliable cellular therapies. The goal of this study is to identify the subset of the progenitors with systematically characterized critical quality attributes (CQAs) that can be selectively expanded to obtain clonal cell populations with better quality and reproducibility that can help improve stem cell therapies. Methods, Results & Conclusion Articular cartilage (Outerbridge grade 1-2) obtained from six knee arthroplasty patient's were enzymatically digested to isolate cells for 2-D cell culture assay. Large field of view images were acquired daily to capture images of the progenitors and their progeny for standardized ASTM-based automated image analysis to define and quantify CQAs of the stem/progenitor cells. Based on the CQAs identified, 24 clonal populations from each patient were picked using Cell XTMrobotic device in rapid, precise, repeatable and rigorously documented manner. Of the 24 clonal colonies, twelve fastest growing clones were expanded to 20 doublings for trilineage differentiation assay and RNA sequencing analysis. Preliminary assessment indicated wide variation in morphological and biological characteristics of progenitor and their progeny: circularity (median: 0.665; range:0.15-0.93), area (median: 116.4µm2; range:51.9-204.2µm2), doubling time (median:32.9h; rangse:26.9-41.2h) and colony density (median:6.5%;range:2.3-20.7%). Differences seen between two representative clonal populations with respect to the trilineage differentiation potential and gene expression seen at 20 doublings is shown in Figure 1 and 2 respectively. A complete detailed assessment of all the clonal populations and their culture-expanded population (n=72) will help us identify the CQAs that will aid in selection of quality clones. An improved understanding of the heterogeneity of progenitor cells resident in adult cartilage will also improve the rigor of cell sourcing and targeting decisions for pharmacological and cellular therapies. The automated methods for rapid and high precision analysis and management of cell populations that we have built into Cell X™ represent an important advancement in the tools that are available in the field for stem cell research and manufacturing.

Super-resolution imaging and field of view extension using a single camera with Risley prisms.

A novel imaging method using Risley prisms is proposed to achieve super-resolution imaging and field of view (FOV) extension. The mathematical models are developed, and the solutions to sub-pixel imaging for super-resolution reconstruction are presented. Simulations show that the proposed method can enhance the image resolution up to optical diffraction limit of the optical system for imaging systems whose resolution is limited by pixel size. A prototype is developed. Experimental results show that the scene resolving capacity can be enhanced by 2.0 times with a resolution improvement factor of 4, and the FOV extension results accord with the simulations, providing a promising approach for super-resolution reconstruction, large FOV imaging, and foveated imaging with low cost and high efficiency.

High-Throughput Super-Resolution Microscopy for Revealing Molecular Architecture

Single molecule localization microscopies (SMLM) occupy a special niche in the biologist's toolbox because they can achieve among the highest resolutions in fluorescence microscopy. Yet, many important biological questions remain out of reach due to challenges in acquiring and analyzing statistically significant SMLM datasets. Previously, we created high-throughput PALM by building an automated microscope to image hundreds of bacteria cells, live, 3D, and across cell cycle. To complement this, we created a uniform illumination scheme to enable large field of view images. More recently, we have been working toward autonomous microscopy, reducing the number of user inputs and increasing the responsiveness of the microscope to the sample using machine learning and engineering approaches. We demonstrate the power of this approach for studying macromolecular complexes within cells. To study the organization of such complexes, particle-based analysis has proven to be powerful, but has been limited so far by difficulties in generating large multi-color particle libraries, as well as the complexity of orientational alignment. We have addressed both challenges and, as a result, present a novel framework and software for deciphering the 3D organization of protein complexes composed of multiple components.

Symmetrical illumination based extending depth of field in Fourier ptychographic microscopy.

Fourier ptychographic microscopy (FPM), simply replacing the traditional light source of a microscope with a LED array, can obtain high resolution and large field of view images simultaneously. A symmetrical LED illumination FPM is presented to improve the quality of high resolution images for defocus or thick samples in this paper. It is proven theoretically that symmetrical LED illumination had higher defocus tolerance than that of single LED illumination at first. Then, it is confirmed that symmetrical LED illumination FPM has better low-resolution images and reconstruction effect for defocus samples than that of single LED illumination by simulated and experimental results. Compared with the single illumination FPM, it is demonstrated that the depth of field of symmetrical FPM is more than doubled.

Fourier conjugate adaptive optics for deep-tissue large field of view imaging.

Light microscopy enables multifunctional imaging of biological specimens at unprecedented depths and resolutions. However, the performance of all optical methods degrades with the imaging depth due to sample-induced aberrations. Methods of adaptive optics (AO), which are aimed at pre-compensation of these distortions, still suffer from a limited field of view and imaging depth as well as inconvenient microscope design. Here, I propose and investigate a new approach to overcome these limitations: Fourier image plane conjugate AO. Two experimental designs of the new approach are carefully studied, and an accurate comparison between different methods of AO is presented. Fourier conjugate AO provides a larger field of view, which can only be limited by the angular memory effect, and allows the optimal use of the spatial light modulator. Moreover, theoretically possible imaging depth of Fourier conjugate AO is limited only by the working distance of the objective and not by the microscope design.

Biomass Pretreatment and Enzymatic Hydrolysis Dynamics Analysis Based on Particle Size Imaging.

Parameters such as pretreatment method, enzyme type and concentration, determine the conversion efficiency of biomass' cellulose and hemicellulose to glucose and mainly xylose in biomass-based fuel production. Chemical quantification of these processes offers no information on the effect of enzymatic hydrolysis (EH) on particle morphology. We report on the development of a microscopy method for imaging pretreated biomass particles at different EH stages. The method was based on acquiring large field of view images, typically 20×10 mm2 containing thousands of particles. Morphology of particles with lengths between 2 μm and 5 mm could be visualized and analyzed. The particle length distribution of corn stover samples, pretreated with increasing amounts of sulfuric acid at different EH stages, was measured. Particle size was shown to be dependent on pretreatment severity and EH time. The methodology developed could offer an alternative method for characterization of EH of biomass for second generation biofuels and visualization of recalcitrant structures.

Single-shot large field of view imaging with scattering media by spatial demultiplexing.

Benefiting from the memory effect (ME) for speckle intensity correlations, only one single-shot speckle pattern can be used for the high-quality recovery of objects. However, ME gives a restriction to the field of view (FOV) for imaging with scattering media. Objects beyond the ME region cannot be recovered and produce unwanted speckle patterns, which cause reduction in the speckle contrast and recovery quality. Nevertheless, all the spatial information from a large object is embedded in a single speckle image. Here, we extract the spatial information from these unavoidable speckle patterns and enlarge the FOV of the imaging system with scattering media. Regional point spreading functions, which are fixed and only need to be recorded once for all-time use, are employed to recover corresponding spatial regions of an object by deconvolution. Then, an automatic weighted averaging in an iterative process is performed to obtain the object with significantly enlarged FOV. Our results present an important advancement of imaging techniques with strongly scattering media.

Small animal large field of view magnetic resonance imaging with metamaterial-inspired resonator

Large field of view magnetic resonance imaging is often required during small animal imaging to cover the entire body of an animal. Large field of view imaging should nevertheless be performed with sufficient signal to noise ratio and resolution. A combination of large signal to noise ratio and high resolution is usually achieved via using small loop coils with small field of view, which is inapplicable in case whole animal or a large portion of its body needs to be imaged. Here, a metamaterial-inspired coil based on an array of parallel wires is employed to perform large field of view imaging. A number of mice are imaged in a 7 T preclinical magnetic resonance scanner. Properties of the resulting images are compared to the ones acquired with commercial coil designated for whole-body imaging as well as a with a commercial small loop antenna designated to provide high signal to noise ratio images in a limited area. The signal to noise ratio is compared between the coils in different tissues and on different distances from the metamaterial-inspired coil plane. The coil is found to provide signal to noise ratio a few times higher than the commercial coil in the optimal reception region and comparable signal to noise ratio off the optimal observation region. The metamaterial-inspired coil is also shown to provide rivaling signal to noise ratio in small field-of-view application, when compared to a small-region loop coil.

Multi-modal chip-based fluorescence and quantitative phase microscopy for studying inflammation in macrophages.

Total internal reflection fluorescence (TIRF) microscopy benefits from high-sensitivity, low background noise, low photo-toxicity and high-contrast imaging of sub-cellular structures close to the membrane surface. Although, TIRF microscopy provides high-contrast imaging it does not provide quantitative information about morphological features of the biological cells. Here, we propose an integrated waveguide chip-based TIRF microscopy and label-free quantitative phase imaging (QPI). The evanescent field present on top of a waveguide surface is used to excite the fluorescence and an upright microscope is used to collect the signal. The upright microscope is converted into a Linnik-type interferometer to sequentially extract both the quantitative phase information and TIRF images of the cells. Waveguide chip-based TIRF microscopy benefits from decoupling of illumination and collection light path, large field of view imaging and pre-aligned configuration for multi-color TIRF imaging. The proposed multi-modal microscopy is used to study inflammation caused by lipopolysaccharide (LPS) on rat macrophages. The TIRF microscopy showed that LPS inflammatory molecule disrupts the cell membrane and causes cells to significantly expand across a substrate. While, QPI module quantified changes in the sub-cellular content of the LPS challenged macrophages, showing a net decrease in its maximum phase values.