Programmable Aperture Research Articles

Light-field photography using differential high-speed aperture coding

Programmable aperture light-field photography enables the acquisition of angular information without compromising spatial resolution. However, direct current (DC) background noise is unavoidable in images recorded by programmable aperture light-field photography, leading to reducing the contrast of reconstructed images. In addition, it requires sacrificing temporal resolution to obtain angular information, making it a challenge to capture dynamic scenes. In this paper, we propose programmable aperture light-field photography using differential high-speed aperture coding. This method effectively reduces DC noise and produces high-contrast refocused images. Furthermore, we build a light-field camera based on a 1250 Hz spatial light modulator and a 1250 fps high-speed camera, achieving dynamic light-field photography at 1110(H)×800(V) resolution and 24 fps. Our results demonstrate significant improvements in image contrast and exhibit considerable promise for diverse applications.

Programmable Aperture Light‐Field Microscopy

AbstractA computational three‐dimensional (3D) microscopy technique, termed programmable aperture light‐field microscopy (PALFM), for motion‐free, high‐resolution volumetric imaging of fluorescent or self‐luminous samples is proposed. The well‐known Fourier slice theorem is extended to incoherent tomographic imaging, which states that a detected image under an ideal aperture corresponds to a central slice in the 3D object spectrum, so the spectrum coverage can be accomplished based on the motion‐free aperture modulation. When further considering frequency extension and coverage, a hybrid aperture modulation scheme is designed consisting of non‐centrosymmetric circular and annular apertures for high‐efficiency, non‐ambiguous depth discrimination. A PALFM system with an easy‐to‐build programmable aperture module attached to an off‐the‐shelf inverted fluorescence microscope is constructed, where annular apertures can manipulate Bessel‐like beams for spectrum modulation. Experimental results on near‐diffraction‐limited imaging of a resolution target across a large depth range and high‐resolution, multi‐color 3D imaging of a mouse kidney section verify the validity and effectiveness of PALFM. High‐speed, long‐term time‐lapse volumetric imaging of HeLa cells in vitro further demonstrates that PALFM is a promising tomographic imaging tool for studying dynamic cellular processes and events without requiring complicated sample rotation or beam scanning.

Programmable Aperture Using a Digital Micromirror Device for In-Line Holographic Microscopy

We present the novel use of a digital micromirror device (DMD) as a digitally adjustable and programmable aperture for lens-free digital in-line holographic microscopy (LFDIHMY). The proposed micro-electro-mechanical system (MEMS) based DMD aperture can be used as a compact and inexpensive replacement for existing mechanical based aperture control methods required for many resolution enhancement methods. Simulation of the DMD aperture in an in-line holography setup is presented using a general DMD model from literature. A hardware prototype was successfully implemented and demonstrated. The resolution of both simulated and experimental holograms recorded with a conventional pinhole aperture and the proposed DMD aperture are compared using a 1951 USAF resolution test target. We show that a resolution of 2.46 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm { \mu {\textbf {m}}}$ </tex-math></inline-formula> , close to the sensor pixel limit, can be obtained with each aperture. This demonstrates that a DMD can be used as a programmable replacement for a pinhole aperture.

High resolution, programmable aperture light field laparoscope for quantitative depth mapping.

Recent applications have shown that light field imaging can be useful for developing uniaxial three-dimensional (3D) endoscopes. The immediate challenges in implementation are a tradeoff in lateral resolution and acquiring enough depth information in the physically limited environment of minimally invasive surgery. Here we propose using programmable aperture light field imaging in laparoscopy to capture 3D information without sacrificing the camera sensor's native, high spatial resolution. This hybrid design utilizes a programmable aperture to preserve the conventional laparoscope's functionality and, upon demand, to compute a depth map for surgical guidance. A working prototype is demonstrated.

Development of a microfluidic design for an automatic lab-on-chip operation

Simple and easy to use are the keys for developing lab-on-chip technology. Here, a new microfluidic circuit has been designed for an automatic lab-on-chip operation (ALOCO) device. This chip used capillary forces for controlled and precise manipulation of liquids, which were loaded in sequence from different flowing directions towards the analysis area. Using the ALOCO design, a non-expert user is able to operate the chip by pipetting liquids into suitable inlet reservoirs. To test this design, microfluidic devices were fabricated using the programmable proximity aperture lithography technique. The operation of the ALOCO chip was characterized from the flow of red-, blue- and un-dyed deionized water. Experimental result indicated that red water, which filled first the analysis area, was substituted entirely with blue water. Controlled sequential flows of these water in the ALOCO device are demonstrated in this paper.

Programmable aperture microscopy: A computational method for multi-modal phase contrast and light field imaging

We demonstrate a simple and cost-effective programmable aperture microscope to realize multi-modal computational imaging by integrating a programmable liquid crystal display (LCD) into a conventional wide-field microscope. The LCD selectively modulates the light distribution at the rear aperture of the microscope objective, allowing numerous imaging modalities, such as bright field, dark field, differential phase contrast, quantitative phase imaging, multi-perspective imaging, and full resolution light field imaging to be achieved and switched rapidly in the same setup, without requiring specialized hardwares and any moving parts. We experimentally demonstrate the success of our method by imaging unstained cheek cells, profiling microlens array, and changing perspective views of thick biological specimens. The post-exposure refocusing of a butterfly mouthpart and RFP-labeled dicot stem cross-section is also presented to demonstrate the full resolution light field imaging capability of our system for both translucent and fluorescent specimens.

Eye Vision Testing System and Eyewear Using Micromachines

Proposed is a novel eye vision testing system based on micromachines that uses micro-optic, micromechanic, and microelectronic technologies. The micromachines include a programmable micro-optic lens and aperture control devices, pico-projectors, Radio Frequency (RF), optical wireless communication and control links, and energy harvesting and storage devices with remote wireless energy transfer capabilities. The portable lightweight system can measure eye refractive powers, optimize light conditions for the eye under testing, conduct color-blindness tests, and implement eye strain relief and eye muscle exercises via time sequenced imaging. A basic eye vision test system is built in the laboratory for near-sighted (myopic) vision spherical lens refractive error correction. Refractive error corrections from zero up to −5.0 Diopters and −2.0 Diopters are experimentally demonstrated using the Electronic-Lens (E-Lens) and aperture control methods, respectively. The proposed portable eye vision test system is suited for children’s eye tests and developing world eye centers where technical expertise may be limited. Design of a novel low-cost human vision corrective eyewear is also presented based on the proposed aperture control concept. Given its simplistic and economical design, significant impact can be created for humans with vision problems in the under-developed world.

LCD-based digital eyeglass for modulating spatial-angular information.

Using programmable aperture to modulate spatial-angular information of light field is well-known in computational photography and microscopy. Inspired by this concept, we report a digital eyeglass design that adaptively modulates light field entering human eyes. The main hardware includes a transparent liquid crystal display (LCD) and a mini-camera. The device analyzes the spatial-angular information of the camera image in real time and subsequently sends a command to form a certain pattern on the LCD. We show that, the eyeglass prototype can adaptively reduce light transmission from bright sources by ~80% and retain transparency to other dim objects meanwhile. One application of the reported device is to reduce discomforting glare caused by vehicle headlamps. To this end, we report the preliminary result of using the reported device in a road test. The reported device may also find applications in military operations (sniper scope), laser counter measure, STEM education, and enhancing visual contrast for visually impaired patients and elderly people with low vision.

Motion-invariant Coding Using a Programmable Aperture Camera

A moving object or camera causes motion blur in a conventional photograph, which is a fundamental problem of a camera. In this research, we propose to code a motion-invariant blur using a programmable aperture camera. The camera can realizes virtual camera motion by translating the opening, and as a result, we obtain a coded image in which motion blur is invariant with the object velocity. Thereby, we recover motion blurs without estimation of the motion blur kernels or knowledge of the object speeds. We model the projection of the programmable aperture camera, and also demonstrate that our proposed coding works for a prototype camera.

Measuring optical spatial coherence by using a programmable aperture

The modulus of optical spatial coherence established by an incoherent light source was measured with a revised experimental setup of the Thompson and Wolf experiment. We used a liquid crystal display as a programmable aperture, which replaced the mechanical aperture controls used previously and provided a robust way to address the position-dependent spatial coherence. As a demonstration, the angular diameter, 0.982′, of an extended light source was estimated from a visibility analysis of the diffraction fringes.

Programmable Aperture Camera Using LCoS

Since 1960s, aperture patterns have been studied extensively and a variety of coded apertures have been proposed for various applications, including extended depth of field, defocus deblurring, depth from defocus, light field acquisition, etc. Researches have shown that optimal aperture patterns can be quite different due to different applications, imaging conditions, or scene contents. In addition, many coded aperture techniques require aperture patterns to be temporally changed during capturing. As a result, it is often necessary to have a programmable aperture camera whose aperture pattern can be dynamically changed as needed in order to capture more useful information. In this paper, we propose a programmable aperture camera using a Liquid Crystal on Silicon (LCoS) device. This design affords a high brightness contrast and high resolution aperture with a relatively low light loss, and enables one change the pattern at a reasonably high frame rate. We build a prototype camera and evaluate its features and drawbacks comprehensively by experiments. We also demonstrate three coded aperture applications in defocus deblurring, depth from defocus and light field acquisition.

Direct Writing of Channels for Microfluidics in Silica by MeV Ion Beam Lithography

The lithographic exposure characteristic of amorphous silica (SiO2) was investigated for 6.8 MeV16O3+ions. A programmable proximity aperture lithography (PPAL) technique was used for the ion beam exposure. After exposure, the exposed pattern was developed by selective etching in 4% v/v HF. Here, we report on the development of SiO2in term of the etch depth dependence on the ion fluence. This showed an exponential approach towards a constant asymptotic etch depth with increasing ion fluence. An example of microfluidic channels produced by this technique is demonstrated.

Projection Charged Particle Nanolithography and Nanopatterning

The projection charged particle multi-beam techniques of IMS Nanofabrication are based on illuminating a programmable aperture plate system [APS; consisting of an aperture plate and a blanking plate with integrated complementary metal–oxide–semiconductor (CMOS) electronics] with a telecentric broad beam, to separate this beam into thousands of micrometer sized beams and to demagnify the beamlets thus formed to nanometer dimensions, using a projection charged particle optics with 200× reduction. Only beams which are unaffected by the blanking plate are projected to the substrate whereas beams which are slightly deflected are filtered out at a contrast aperture at the second cross-over of the projection optics. Proof-of-concept test systems were realized for these techniques. Inserting a 43k-APS providing 43 thousand programmable 12.5 nm sized beams complex patterns were realized in 20 ×20 µm2 exposure fields when using 10 keV H3+ ions. For the electron multi-beam test system the beam at the APS was limited to 2 mm diameter, thus realizing 2500 programmable electron beams of 12.5 nm size and 50 keV energy within an exposure field of ca. 7 ×7 µm2. Application fields of projection charged particle multi-beam techniques are CHARPAN (Charged Particle Nanopatterning, with ion multi-beams), PML2 (Projection Mask-Less Lithography), and eMET (electron Mask Exposure Tool).

Charged Particle Nanopatterning (CHARPAN) of 2D and 3D masters for flexible replication in Substrate Conformal Imprint Lithography (SCIL)

Recently, the first generation programmable Aperture Plate System with integrated CMOS electronics (CMOS-APS) featuring 43 thousand switchable beams has been inserted into a Charged Particle Nanopatterning (CHARPAN) tool. Using this configuration, the first (2D) exposure results in Hydrogen Silsesquioxane (HSQ) resist, employing 10 keV Hydrogen ion ( H 3 + ) parallel beams of 12.5 nm spot size, show that (at least) a 20 nm resolution is feasible in this system. These CHARPAN patterns have been used as masters for PDMS stamp casting. The flexible PDMS stamps have been implemented in Substrate Conformal Imprint Lithography (SCIL™). The original 20 nm resolution features have also been observed in the imprinted sol–gel material. The CHARPAN tool can be operated with heavier ions (Ar +, Xe +) as well, enabling maskless and resistless 3D nanopatterning. Here again, a flexible PDMS stamp has been cast and these deep 3D structures have been successfully applied in SCIL. The combination of CHARPAN and SCIL opens up new possibilities for low cost, fast and flexible 2D and 3D manufacturing of nano-devices.

Projection maskless patterning for nanotechnology applications

Projection maskless patterning (PMLP) is based on a programmable aperture plate system and on charged particle projection optics with 200× reduction, providing thousands of electron or ion (H+, He+, Ar+, Xe+, C60−) beams working in parallel on the substrate. As part of the European CHARPAN project a PMLP proof-of-concept tool has been realized. Using resolution templates, with 10keV H+ multibeams a resolution of 16nm lines and spaces was achieved in HSQ resist across the proof-of-concept tool 25×25μm2 exposure field at an exposure dose of 25μC∕cm2. Enhancing the dose by 10% there was &amp;lt;1nm increase in linewidth. With 10keV Ar+ multibeams resistless nanopatterning of various materials was accomplished. Inserting a wired programmable aperture plate system providing ∼4000 beams, first HSQ resist exposure and patterning results have been accomplished, implementing gray scale exposure techniques. The system is being upgraded to a PMLP engineering tool integrating an aperture plate system with complementary metal oxide semiconductor electronics providing ∼40000 programmable beams, a precursor gas injection system for in situ ion multibeam induced etching and deposition, and a laser-interferometer controlled high-precision vacuum stage. Industrial PMLP nanotechnology applications are discussed.

Programmable aperture photography

In this paper, we present a system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition. The shape of the programmable aperture can be adjusted and used to capture light field at full sensor resolution through multiple exposures without any additional optics and without moving the camera. High acquisition efficiency is achieved by employing an optimal multiplexing scheme, and quality data is obtained by using the two post-processing algorithms designed for self calibration of photometric distortion and for multi-view depth estimation. View-dependent depth maps thus generated help boost the angular resolution of light field. Various post-exposure photographic effects are given to demonstrate the effectiveness of the system and the quality of the captured light field.

Three-dimensional projection mask-less patterning (PMLP) of micro-lenses and cones: Monitoring and modelling of ion multi-beam kinetic sputtering in GaAs

A projection mask-less patterning (PMLP) proof-of-concept tool, realised as part of the European FP6-NMP integrated project CHARPAN (Charged Particle Nanotech), has been applied for three-dimensional patterning of a GaAs surface, using a stencil mask with a pre-defined opening distribution. A 10×10 matrix of micro-lenses was used as test structure for argon ion multi-beam structuring, while studying the time evolution of the sputtered structures. The kinetic sputtering and re-deposition have been modelled in simulation software, in order to be able to correct for such effects beforehand in the design of datasets to be fed to the future programmable aperture plate.

Cryogenic characterization and testing of magnetically-actuated microshutter arrays for the James Webb Space Telescope

Two-dimensional MEMS microshutter arrays (MSA) have been fabricated at the NASA Goddard Space Flight Center (GSFC) for the James Webb Space Telescope (JWST) to enable cryogenic (∼35 K) spectrographic astronomy measurements at near-infrared wavelengths. Functioning as a focal plane object selection device, the MSA is a 2D programmable aperture mask with fine resolution, high efficiency and high contrast. The MSA are close-packed silicon nitride shutters (cell size of 100 µm × 200 µm) patterned with a torsion flexure to allow their opening to 90°. A layer of magnetic material is deposited onto each shutter to permit magnetic actuation. Two electrodes are deposited, one onto each shutter and another onto the support structure side-wall, permitting electrostatic latching and 2D addressing. New techniques were developed to test MSA under mission-similar conditions (8 K ⩽ T < 300 K). The ‘magnetic rotisserie’ has proven to be an excellent tool for rapid characterization of MSA. Tests conducted with the magnetic rotisserie method include accelerated cryogenic lifetesting of unpackaged 128 × 64 MSA and parallel measurement of the magneto-mechanical stiffness of shutters in ‘pathfinder’ test samples containing multiple MSA designs. Lifetest results indicate a logarithmic failure rate out to ∼106 shutter actuations. These results have increased our understanding of failure mechanisms and provide a means to predict the overall reliability of MSA devices.

Programmable Microshutter Arrays for the JWST NIRSpec: Optical Performance

Two-dimensional microshutter arrays (MSAs) are being developed at the NASA Goddard Space Flight Center for the James Webb Space Telescope (JWST) for use as a programmable aperture mask for object selection for the Near Infrared Multiobject Spectrograph (NIRSpec). The MSAs are designed to provide high transmission efficiency for the selected objects and high on to off contrast ratio at the /spl sim/35 K operating temperature of JWST. The arrays of shutters are produced from silicon nitride membranes on a 100/spl times/200 /spl mu/m pitch. Individual shutters consist of a shutter blade of silicon nitride suspended from the shutter frame by a nitride torsion flexure. The shutters are normally closed. All shutters in the array are opened by the scanning magnetic field, and are held open by an electrostatic potential applied between the open shutters and the shutter support grid electrodes. To close the required shutters for a specific configuration, the potential between the shutter to be deselected and the support frame is set to zero, allowing the shutter to close. In this way, full random access addressing is achieved. We have produced such shutters and have demonstrated mechanical actuation and selection. Optical tests of open and closed shutters have demonstrated the required contrast for the JWST application. The MSA is a pioneering technology that provides the most capable possible multiobject spectrograph for JWST. It provides high contrast selection, high transmission efficiency, and can meet the environmental requirements for JWST.

Hartmann wave-front scanner

A wave-front sensor is described that uses a programmable moving aperture to scan an incoming wave front. The position of the diffraction spot is recorded behind an objective lens with a two-dimensional sensor and gives an estimate of the local slope at the aperture position. Then the wave front is reconstructed by processing of the slope data. The device is basically a programmable Hartmann wave-front sensor. Compared with a microlens Shack-Hartmann wave-front sensor, its much longer focal length provides higher resolution, although real-time operation is lost. A practical implementation of the new scanner with a liquid-crystal television as the programmable aperture is presented.