Structured Illumination Patterns Research Articles

Observing multi-frequency structured illumination patterns based on an evanescent field in a millimeter-scale polymer slide.

Millimeter-scale slide optical waveguides (OWGs) show the potential to break the barrier of easy-to-use and versatility for total internal reflection (TIR) fluorescence technology. In this paper, multi-frequency structured illumination (SI) patterns resulting from the evanescent field (EF) on the surface of a millimeter-scale polymer slide OWG are observed by measuring the fluorescence intensity distribution of fluorescent dyes deposited on the top of the OWG. The frequency, intensity, and stability of the SI patterns show a strong dependence on the coupling angle of the incident light (changing with the incident position). The distribution of multi-frequency SI patterns in the frequency space is demonstrated for different numerical aperture (NA) imaging systems (NA = 0.3, 0.6, and 0.8), indicating the potential for enhanced resolution for low NA systems with a simple and cheap polymer slide.

All-dielectric super-lattice metasurfaces with fivefold spatial resolution enhancement for structured illumination microscopy.

Structured illumination microscopy (SIM) is a promising imaging technique for high-resolution imaging with a wide field of view. Although a periodic nanostructure is a versatile platform for engineering the spatial frequency of structured illumination patterns in SIM, challenges remain, including artifacts from Fourier space gaps. We designed an all-dielectric super-lattice metasurface (ADSLM) to generate structured illumination patterns with enhanced spatial frequency and broadened spatial frequency coverage with no intermediate frequency gaps. Our numerical simulations reveal that ADSLM-based image reconstruction is capable of producing high-contrast, artifact-free images, resulting in enhanced spatial resolution up to 5.7-fold for coherent SIM at 450 nm. Our results show that the ADSLM-SIM technique may facilitate high-resolution imaging using CMOS-compatible substrates, offering potential for compact miniaturized imaging applications.

Ghost polarimetry with a spatial light modulator for creation of structured illumination patterns

Subject of study. The spatial distribution of polarization properties for objects with linear dichroism is studied. Aim of study. A ghost polarimetry system using a spatial light modulator to measure the optical fields illuminating an object with linear dichroism is developed. Method. Ghost polarimetry is used to measure the integral intensity of radiation passing through an object. The intensity correlation functions are numerically calculated for two orthogonally polarized sets of optical fields for subsequent calculation of the modulus of the anisotropy azimuth for an object with linear dichroism. Main results. A ghost polarimetry device using a spatial light modulator is developed to measure the characteristics of structured optical fields. Images showing the polarization properties of three objects with linear dichroism are obtained. Practical significance. Ghost polarimetry has several advantages over traditional methods of obtaining polarization images when imaging objects with low radiation intensity, when imaging in high-turbulence environments, or when imaging in spectral ranges for which traditional measuring instruments are expensive or unavailable. The inclusion of a spatial light modulator in the ghost polarimetry device means it is no longer necessary to record the fields generated. This modification will enable the characteristics of optical fields for specific objects to be varied for microbiological and medical research.

Optimizing microlens arrays for incoherent HiLo microscopy

HiLo microscopy is a powerful, low-cost, and easily configurable technique for acquiring high-contrast optically-sectioned images. However, traditional HiLo microscopes are based on coherent light sources with diffusive glass plates or incoherent light sources with digital mirror devices (DMD) and spatial light modulators (SLM), which are more expensive. Here, we propose a new low-cost HiLo microscopy technique using MLAs and incoherent LED light sources. We simulated structured illumination (SI) patterns and HiLo image generation. To observe how MLAs affect HiLo images, we used three common MLAs with specific microlens pitch and numerical aperture (NA) to generate periodic illumination patterns. Our simulations concluded that the MLA's NA does not significantly affect HiLo images, and a larger lens pitch can bring a higher image contrast. However, there is an optimized lens pitch. If the lens pitch is equal to or higher than 140 μm, artifacts are observed in HiLo images. Based on our simulations, employing the cross-type MLA with a 0.01NA and a period of 120 μm yields the optimal HiLo image (94.6 % contrast). In our quantitative analysis with the traditional HiLo, where SI patterns were set with the same period, we calculated the SSIM index between each HiLo-Ground Truth image pair. The cross-type MLA HiLo performed slightly better (SSIM = 0.9554) than the traditional HiLo (SSIM = 0.9396). However, when using the hexagon-type MLA, artifacts were evident in the final HiLo image (SSIM = 0.9054). The cylinder-type MLA HiLo result (SSIM = 0.9366) is similar to the traditional HiLo. To our knowledge, this is the first numerical study about MLA-based HiLo microscopy. This study can benefit researchers using MLAs and incoherent light sources to configure their low-cost HiLo microscopes.

Snapshot compressive structured illumination microscopy.

We propose a snapshot compressive structured illumination microscopy (SoSIM) system to increase the number of reconstructed resolution-enhanced (RE) images per second and reduce the data bandwidth by capturing compressed measurements. In this system, multiple low-resolution images are encoded by a high-speed digital micro-mirror device with random binary masks. These images are then captured by a low-speed camera as a snapshot compressed measurement. Following this, we adopt an efficient deep neural network to reconstruct nine images with different structured illumination patterns from a single measurement. The reconstructed images are then combined into a single-frame RE image using the method of spectral synthesis in the frequency domain. When the camera operates at 100 frames per second (fps), we can eventually recover dynamic RE videos at the same speed with 100 fps.

ZIMFLUX: Single molecule localization microscopy with patterned illumination in 3D

Three dimensional modulation-enhanced single-molecule localization techniques, such as ModLoc, offer advancements in axial localization precision across the entire field of view and axial capture range, by applying phase shifting to the illumination pattern. However, this improvement is limited by the pitch of the illumination pattern that can be used and requires registration between separate regions of the camera. To overcome these limitations, we present ZIMFLUX, a method that combines astigmatic point-spread-function (PSF) engineering with a structured illumination pattern in all three spatial dimensions. In order to achieve this we address challenges such as optical aberrations, refractive index mismatch, supercritical angle fluorescence (SAF), and imaging at varying depths within a sample, by implementing a vectorial PSF model. In scenarios involving refractive index mismatch between the sample and immersion medium, the astigmatic PSF loses its ellipticity at greater imaging depths, leading to a deterioration in axial localization precision. In contrast, our simulations demonstrate that ZIMFLUX maintains high axial localization precision even when imaging deeper into the sample. Experimental results show unbiased localization of 3D 80 nm DNA-origami nanostructures in SAF conditions with a 1.5-fold improvement in axial localization precision when comparing ZIMFLUX to conventional SMLM methods that rely solely on astigmatic PSF engineering.

Multi-target object scattering imaging with intensity correlation of structured illumination.

Imaging through scattering layers based on the optical memory effect (OME) concept has been widely investigated in recent years. Among many scattering scenarios, it is very important to recover hidden targets with proper spatial distribution in the scene where multiple targets out of the OME range exist. In this Letter, we put forward a method for multi-target object scattering imaging. With the help of intensity correlation between the structured illumination patterns and recorded speckle images, the relative position of all hidden targets can be obtained and the movement of the targets within the OME range can be tracked. We experimentally implement scattering imaging with 16 targets and the motion tracking of them. Our results present a significant advance in a large field of view scattering imaging with multiple targets.

Ultrasonic subwavelength imaging with blind structured illumination

The resolution of ultrasonic imaging systems suffers from the diffraction limit. To solve this problem, we develop a blind structured illumination microscopy (blind-SIM) system that achieves far-field subwavelength resolution in underwater ultrasonic imaging systems. By illuminating the object with multiple structured illumination (SI) patterns generated by scattering media with subwavelength features, the spatial frequency mixing between the object and the SIs converts evanescent waves to propagate waves that can reach sensors in the far field. In this work, the acoustic SIs are generated by randomly distributed micro particles in water. The image of the object is then reconstructed from multiple measurements (each produced by a SI pattern) utilizing computational reconstruction algorithms where the precise knowledge of the SIs is not required. The developed ultrasonic blind-SIM system has the potential in improving the ultrasonic imaging resolution in medical diagnoses, underwater acoustic imaging, and communications. Our system largely reduces the imaging hardware complexity and improves the imaging speed compared with existing ultrasonic subwavelength technologies that mostly rely on the precise localization of time-varying contrast agents.

Fisher information and the Cramér-Rao lower bound in single-pixel localization microscopy with spatiotemporally modulated illumination.

Single-pixel imaging, the concept that an image can be captured via a single-pixel detector, is a cost-effective yet powerful technique to reduce data acquisition duration without sacrificing image resolution when properly structured illumination patterns are introduced. Normally, the image reconstruction process is subject to the diffraction limit. Here, we study the possibility of exploiting the information contained in the illumination patterns to enable a form of single-pixel localization microscopy (SPLM) for super-resolution. This concept is inspired by coherent holographic image reconstruction by phase transfer (CHIRPT) microscopy. CHIRPT microscopy is a single-pixel imaging technique that uses structured illumination that is spatiotemporally modulated (STM) so that a unique temporal modulation pattern is imparted to each point within a large illumination volume. The fluorescent light emitted by molecules contains the same temporal modulations as the illumination patterns at the locations of the molecules. By recording a portion of the total emitted fluorescent power, the signal may be numerically processed to form an image. Unique temporal modulation patterns that excite fluorescent probes at each point can also be used to localize individual molecules by matching their particular temporal light emission patterns to the measured temporal signal. This paper evaluates the feasibility of SPLM with STM illuminations used in and inspired by CHIRPT microscopy via the information content its data carry about the emitter location(s). More specifically, we provide the mathematical formalism of Fisher information (FI) and the Cramér-Rao lower bound (CRLB) associated with the location parameters of the emitter(s). The FI and CRLB are then numerically evaluated under different experimental assumptions to assess the effects of experimental parameters on localization precision. Last, we compare the single-pixel CRLB to that from camera-based single-molecule localization microscopy in the localization of a single emitter. We show that SPLM has several distinguishing characteristics that provide certain advantages, such as relatively constant CRLB over a very large illumination volume and improved CRLB for 3D localization due to the information coupling introduced by simultaneous modulations of the transverse axes.

High-Resolution Quantitative Phase Microscopy with Second-Harmonic Generation and Structured Illumination from a k -Space Formulation

We propose a high-resolution label-free two-dimensional imaging method by introducing second-harmonic generation (SHG) and structured illumination in quantitative phase microscopy (QPM), termed ${\mathrm{S}}^{2}\mathrm{QPM}$, which can achieve a fourfold lateral resolution improvement compared with normal incident illumination for noncentrosymmetric structures. Unlike traditional QPM only obtaining information about the optical path difference, ${\mathrm{S}}^{2}\mathrm{QPM}$ can additionally recover structural information from the ${\ensuremath{\chi}}^{(2)}$ distribution. Analytical solutions for SHG fields are obtained by solving the inhomogeneous and nonlinear Helmholtz equation in the $\mathit{k}$-space in a both concise and accurate way. With the $\mathit{k}$-space SHG field solutions for different structured illumination patterns, an object reconstruction model is developed to retrieve the high-resolution normalized ${\ensuremath{\chi}}^{(2)}$ distributions and quantitative phase maps. We simulate high-resolution reconstructions of the normalized ${\ensuremath{\chi}}^{(2)}$ distributions and quantitative phase maps using various resolution targets. The lateral resolution improvement is quantified by synthesizing the object spectrum function and profiling resolution targets.

Multi-color flow cytometer with PIC-based structured illumination.

We demonstrate a flow cytometer in which structured light illumination is used to attribute fluorescent and scattering signals to their excitation wavelength. A suitable multi-color light source emitting structured illumination patterns at 405, 488, 561 and 640 nm is developed based on a silicon nitride photonic integrated circuit and cytometry experiments are conducted with calibration beads. Performance metrics of the novel cytometer are compared with those of a mature, commercial device. While the experimental device still features a slightly higher sensitivity floor than the commercial one, all but the most weakly stained beads can be categorized. These promising results validate the feasibility of the proposed concept.

Cosinusoidal encoding multiplexed structured illumination multispectral ghost imaging.

The information dimension obtained by multispectral ghost imaging is more abundant than in single-band ghost imaging. Existing multispectral ghost imaging systems still meet some shortages, such as complex structure or reconstruction time-consuming. Here, an approach of cosinusoidal encoding multiplexed structured illumination multispectral ghost imaging is proposed. It can capture the multispectral image of the target object within one projection cycle with a single-pixel detector while maintaining high imaging efficiency and low time-consuming. The core of the proposed approach is the employed novel encoding strategy which is apt to decode and reconstruct the multispectral image via the Fourier transform. Specifically, cosinusoidal encoding matrices with specific frequency characteristics are fused with the orthogonal Hadamard basis patterns to form the multiplexed structured illumination patterns. A broadband photomultiplier is employed to collect the backscattered signals of the target object interacted by the corresponding structured illumination. The conventional linear algorithm is applied first to recover the mixed grayscale image of the imaging scene. Given the specific frequency distribution of the constructed cosinusoidal encoding matrices, the mixed grayscale image can be converted to the frequency domain for further decoding processing. Then, the pictures of multiple spectral components can be obtained with some manipulations by applying Fourier transform. A series of numerical simulations and experiments verified our proposed approach. The present cosinusoidal encoding multiplexed structured illumination can also be introduced in many other fields of high-dimensional information acquisition, such as high-resolution imaging and polarization ghost imaging.

A quantitative analysis of various patterns applied in lattice light sheet microscopy

Light sheet microscopes reduce phototoxicity and background and improve imaging speed compared to widefield and confocal microscopes. However, when equipped with Gaussian beams, the axial resolving power of a light sheet microscope and the observable field of view are inversely related. Light sheets based on dithered optical lattices improve axial resolution and beam uniformity compared Gaussian beams by using axially structured illumination patterns. However, these advantages come at the expense of an increased total illumination to the specimen and a decreased axial confinement of the illumination pattern. Using simulations and experimental measurements in fixed and live cells, we quantify the differences between Gaussian and lattice light sheets on beam uniformity, axial resolution, lateral resolution, and photobleaching. We demonstrate how different optical lattice illumination patterns can be tuned to prioritize either axial resolution or optical sectioning. Finally, we introduce an approach to spectrally fuse sequential acquisitions of different lattice light sheet patterns with complementary optical properties to achieve both high resolution and low background images.

Integrated optical device for Structured Illumination Microscopy.

Structured Illumination Microscopy (SIM) is a key technology for high resolution and super-resolution imaging of biological cells and molecules. The spread of portable and easy-to-align SIM systems requires the development of novel methods to generate a light pattern and to shift it across the field of view of the microscope. Here we show a miniaturized chip that incorporates optical waveguides, splitters, and phase shifters, to generate a 2D structured illumination pattern suitable for SIM microscopy. The chip creates three point-sources, coherent and controlled in phase, without the need for further alignment. Placed in the pupil of a microscope's objective, the three sources generate a hexagonal illumination pattern on the sample, which is spatially translated thanks to thermal phase shifters. We validate and use the chip, upgrading a commercial inverted fluorescence microscope to a SIM setup and we image biological sample slides, extending the resolution of the microscope.

Far-Field Acoustic Subwavelength Imaging with Blind Structured Illumination and Joint-Sparsity Reconstruction

The resolution of far-field imaging systems suffers from the diffraction limit. In this work, a blind structured illumination microscopy (blind-SIM) system that achieves subwavelength resolution in acoustic far-field imaging is developed. In blind-SIM, precise knowledge of the structured illumination (SI) patterns is not required. Instead, the acoustic SIs are generated by a randomly located acoustic diffraction grating with a subwavelength grating period. The spatial frequency mixing between the object and the SIs converts evanescent waves to propagating waves that can reach sensors in the far field. The image of the object is then reconstructed from multiple far-field measurements utilizing a joint-sparsity compressive sensing algorithm. Numerical simulations and experiments in kilohertz airborne sound are performed to validate the proposed system. A subwavelength imaging resolution of around 1/3 wavelength is achieved in experiments, and around 1/5 wavelength in simulations. The blind-SIM system largely reduces the imaging hardware complexity compared with existing acoustic subwavelength technologies that mostly rely on metamaterial lenses, time-reversal mirrors, or the precise localization of time-varying contrast agents. The combination of blind-SIM and the joint-sparsity compressive sensing algorithm for image reconstruction reduces the requirements for an object's sparsity level and enhances the system's robustness against noise. Our work has potential in improving the practicality of acoustic far-field subwavelength imaging in medical diagnoses, nondestructive testing, and underwater acoustic imaging and communications.

Utilizing the spatial frequency domain imaging to investigate change in optical parameters of skin exposed to thermal‐hydrotherapy: Ex‐vivo study

AbstractHydrotherapy is a traditional clinical practice that has been widely used for pain relief. However, immersion in hot water influences skin's moisture and morphology. In this study, we investigate the physiological changes of ex‐vivo chicken skin immersed in hot water (at 40 and 50°C) via monitoring the change in its absorption and scattering properties using the spatial frequency domain imaging (SFDI) technique. The procedure started by calculating the modulation transfer function of the proposed imaging system over a range of spatial frequencies using a high‐resolution test target. Thereafter, structured illumination patterns at different spatial frequencies were projected onto the examined samples via a reflective phase‐only spatial light modulator using transmission and reflection modes. The transmission and reflection images were recorded using a digital camera to reconstruct the optical absorption and scattering parameters. Such parameters were calculated using Kubelka–Munk and lookup table methods. For system validation, the optical properties of two kinds of milk (skimmed and full cream), as reference samples, were reconstructed using the proposed SFDI system in the reflection mode via lookup table method. The results revealed an increase in the scattering coefficient of the skin samples immersed in higher water temperature, while the absorption coefficient values were nearly the same. Furthermore, the obtained results were validated using Monte‐Carlo method showing absolute errors that range from 0.004 to 0.057. In conclusion, the proposed system was presented to investigate the changes in ex‐vivo skin properties under thermal‐hydrotherapy in the context of measured changes in optical parameters.

Real-time reconstruction using electro-optics modulator-based structured illumination microscopy.

Structured illumination microscopy (SIM), a super-resolution technology, has a wide range of applications in life sciences. In this study, we present an electro-optic high-speed phase-shift super-resolution microscopy imaging system including 2D SIM, total internal reflection fluorescence-SIM, and 3D SIM modes. This system uses galvanometers and an electro-optic modulator to flexibly and quickly control the phase and direction of structured illumination patterns. Moreover, its design consists of precise timing for improved acquisition speed and software architecture for real-time reconstruction. The highest acquisition rate achieved was 151 frames/s, while the highest real-time super-resolution reconstruction frame rate achieved was over 25 frames/s.

Spatially modulated illumination microscopy: application perspectives in nuclear nanostructure analysis

Thousands of genes and the complex biochemical networks for their transcription are packed in the micrometer sized cell nucleus. To control biochemical processes, spatial organization plays a key role. Hence the structure of the cell nucleus of higher organisms has emerged as a main topic of advanced light microscopy. So far, a variety of methods have been applied for this, including confocal laser scanning fluorescence microscopy, 4Pi-, STED- and localization microscopy approaches, as well as (laterally) structured illumination microscopy (SIM). Here, we summarize the state of the art and discuss application perspectives for nuclear nanostructure analysis of spatially modulated illumination (SMI). SMI is a widefield-based approach to using axially structured illumination patterns to determine the axial extension (size) of small, optically isolated fluorescent objects between less than or equal to 200 nm and greater than or equal to 40 nm diameter with a precision down to the few nm range; in addition, it allows the axial positioning of such structures down to the 1 nm scale. Combined with SIM, a three-dimensional localization precision of less than or equal to 1 nm is expected to become feasible using fluorescence yields typical for single molecule localization microscopy applications. Together with its nanosizing capability, this may eventually be used to analyse macromolecular complexes and other nanostructures with a topological resolution, further narrowing the gap to Cryoelectron microscopy. This article is part of the Theo Murphy meeting issue ‘Super-resolution structured illumination microscopy (part 2)’.

Improved resolution in 3D structured illumination microscopy using 3D model-based restoration with positivity-constraint

The performance of structured illumination microscopy (SIM) systems depends on the computational method used to process the raw data. In this paper, we present a regularized three-dimensional (3D) model-based (MB) restoration method with positivity constraint (PC) for 3D processing of data from 3D-SIM (or 3-beam interference SIM), in which the structured illumination pattern varies laterally and axially. The proposed 3D-MBPC method introduces positivity in the solution through the reconstruction of an auxiliary function using a conjugate-gradient method that minimizes the mean squared error between the data and the 3D imaging model. The 3D-MBPC method provides axial super resolution, which is not the same as improved optical sectioning demonstrated with model-based approaches based on the 2D-SIM (or 2-beam interference SIM) imaging model, for either 2D or 3D processing of a single plane from a 3D-SIM dataset. Results obtained with our 3D-MBPC method show improved 3D resolution over what is achieved by the standard generalized Wiener filter method, the first known method that performs 3D processing of 3D-SIM data. Noisy simulation results quantify the achieved 3D resolution, which is shown to match theoretical predictions. Experimental verification of the 3D-MBPC method with biological data demonstrates successful application to data volumes of different sizes.

Far-field acoustic subwavelength imaging with blind structured illumination

The blind structured illumination microscopy (blind-SIM) is an optical method to surpass the diffraction limit, a fundamental limitation to all wave-based imaging techniques. By illuminating the object with multiple structured illumination patterns, the spatial frequency mixing between the object and the illuminations converts evanescent waves to propagating waves that can be detected by sensors in the far field. In this work, we develop blind-SIM in the acoustic domain. The blind acoustic structured illuminations (ASI) are generated by randomly located scatterers with subwavelength spatial distributions. The image of the object can be reconstructed with a subwavelength resolution from multiple far-field measurements (each produced by an ASI pattern) utilizing a compressed sensing algorithm. We demonstrate subwavelength acoustic imaging resolution in both kilohertz airborne and megahertz underwater acoustic imaging systems. The developed blind-SIM method has great potential in medical imaging, non-destructive testing, and underwater acoustic imaging for improving the imaging resolution of objects far away from the sensors.