Single Optical Imaging Research Articles

Surface modification of Nd3+ activated gadolinium core-shell nanospheres for near-infrared and magnetic resonance dual functional bioimaging system

Visualization techniques are effective tools in diagnosis and therapy for biomedical applications. Due to the unavoidable shortcomings of single optical or magnetic resonance imaging, the introduction of dual functional imaging system is of great significance. Herein a new Gd1-xNdx(CO3)OH@SiO2 core–shell nanostructure were developed to fulfill the corresponding demands. Uniform nanospheres were acquired with surface modification and coated with thickness tunable silica shells. The samples presented strong luminescence penetrating biological tissue, due to the wide spectral region covering NIR I & II window of Nd3+ under 808 nm laser excitation. On the other hand, distinct contrast enhancement could be viewed for in vitro and in vivo MRI trials, which generates from high paramagnetic relaxation of Gd3+ and well dispersibility caused by silica shells. Along with the non-toxicity proved by the low concentration of leaching Gd3+ and nice biocompatibility of silica coated nanospheres, it is a superior candidate for NIR and MRI dual functional imaging platform in biological applications.

High-resolution mapping of forest structure from integrated SAR and optical images using an enhanced U-net method

Forest structure is an essential part of biodiversity and ecological analysis and provides crucial insights to address challenges in these areas. Modern sensor technologies unlock new possibilities for more advanced vegetation monitoring. This study examines the potential of single high resolution X-band synthetic aperture radar (SAR) and optical images for pixel-wise mapping of four forest structure attributes (height, average height, fractional cover, and density) at a striking 0.5 m resolution. The study site is situated in Western Norway, hosting trees from flatlands to elevated mountainous areas and in-between. The proposed model architecture, called PSE-UNet, is a modified UNet incorporating key components from state-of-the-art deep learning from the field of forest structure monitoring. A comparative analysis involving state-of-the-art models shows promising results with MAE% between 21.5 and 24.7, depending on the variable.

3D single shot lensless incoherent optical imaging using coded phase aperture system with point response of scattered airy beams

Interferenceless coded aperture correlation holography (I-COACH) techniques have revolutionized the field of incoherent imaging, offering multidimensional imaging capabilities with a high temporal resolution in a simple optical configuration and at a low cost. The I-COACH method uses phase modulators (PMs) between the object and the image sensor, which encode the 3D location information of a point into a unique spatial intensity distribution. The system usually requires a one-time calibration procedure in which the point spread functions (PSFs) at different depths and/or wavelengths are recorded. When an object is recorded under identical conditions as the PSF, the multidimensional image of the object is reconstructed by processing the object intensity with the PSFs. In the previous versions of I-COACH, the PM mapped every object point to a scattered intensity distribution or random dot array pattern. The scattered intensity distribution results in a low SNR compared to a direct imaging system due to optical power dilution. Due to the limited focal depth, the dot pattern reduces the imaging resolution beyond the depth of focus if further multiplexing of phase masks is not performed. In this study, I-COACH has been realized using a PM that maps every object point into a sparse random array of Airy beams. Airy beams during propagation exhibit a relatively high focal depth with sharp intensity maxima that shift laterally following a curved path in 3D space. Therefore, sparse, randomly distributed diverse Airy beams exhibit random shifts with respect to one another during propagation, generating unique intensity distributions at different distances while retaining optical power concentrations in small areas on the detector. The phase-only mask displayed on the modulator was designed by random phase multiplexing of Airy beam generators. The simulation and experimental results obtained for the proposed method are significantly better in SNR than in the previous versions of I-COACH.

Desorption electrospray ionization mass spectrometry imaging (DESI-MSI) in disease diagnosis: an overview.

Tissue analysis, which is essential to histology and is considered the benchmark for the diagnosis and prognosis of many illnesses, including cancer, is significant. During surgery, the surgical margin of the tumor is assessed using the labor-intensive, challenging, and commonly subjective technique known as frozen section histopathology. In the biopsy section, large numbers of molecules can now be visualized at once (ion images) following recent developments in [MSI] mass spectrometry imaging under atmospheric conditions. This is vastly superior to and different from the single optical tissue image processing used in traditional histopathology. This review article will focus on the advancement of desorption electrospray ionization mass spectrometry imaging [DESI-MSI] technique, which is label-free and requires little to no sample preparation. Since the proportion of molecular species in normal and abnormal tissues is different, DESI-MSI can capture ion images of the distributions of lipids and metabolites on biopsy sections, which can provide rich diagnostic information. This is not a systematic review but a summary of well-known, cutting-edge and recent DESI-MSI applications in cancer research between 2018 and 2023.

Distance measurements between 5 nanometer diamonds - single particle magnetic resonance or optical super-resolution imaging?

5 nanometer sized detonation nanodiamonds (DNDs) are studied as potential single-particle labels for distance measurements in biomolecules. Nitrogen-vacancy (NV) defects in the crystal lattice can be addressed through their fluorescence and optically-detected magnetic resonance (ODMR) of a single particle can be recorded. To achieve single-particle distance measurements, we propose two complementary approaches based on spin-spin coupling or optical super-resolution imaging. As a first approach, we try to measure the mutual magnetic dipole-dipole coupling between two NV centers in close DNDs using a pulse ODMR sequence (DEER). The electron spin coherence time, a key parameter to reach long distance DEER measurements, was prolonged using dynamical decoupling reaching T 2,DD ≈ 20 μs, extending the Hahn echo decay time T 2 by one order of magnitude. Nevertheless, an inter-particle NV-NV dipole coupling could not be measured. As a second approach, we successfully localize the NV centers in DNDs using STORM super-resolution imaging, achieving a localization precision of down to 15 nm, enabling optical nanometer-scale single-particle distance measurements.

A Novel Dense-Attention Network for Thick Cloud Removal by Reconstructing Semantic Information

The presence of thick clouds in single optical images shows the contamination of interesting objects. Besides, the difficulty of thick cloud removal is mainly the restoration of the weak boundary information from cloud-contaminated areas. Recently, many deep-learning-based frameworks were applied for cloud removal by obtain the related semantic information from the weak boundary information. However, the large-size cloud-contaminated areas lead to the artificial textures in the resulting images. Thus, obtaining the optimal semantic information from finite boundary information is the key to solve this problem. In this work, we design a deep-learning framework for cloud removal, especially large-size clouds removal (i.e., more than 30% coverage of the whole image). First, we design a cloud location model (CLM) which adopted the fully convolutional network to locate the cloud. Second, desired by theory of the coarse-to-fine restoration, we build a dense-attention network (termed as DANet) for restoring cloud contaminated areas. In the DANet, we design a dense block into the coarse network for training the features of restoring directions of each pixel from the weak boundary information. Furthermore, a contextual attention module is built into refinement network for restoring contaminated areas relying on the semantic relationship between the background and foreground information. Compared with the state-of-the-art methods, the proposed DANet achieved greater removal performances and reconstruct more natural image textures.

Non-equilibrium ordering of liquid crystalline (LC) films driven by external gradients in surfactant concentration

HypothesisGradients in the concentration of amphiphiles play an important role in many non-equilibrium processes involving complex fluids. Here we explore if non-equilibrium interfacial behaviors of thermotropic (oily) liquid crystals (LCs) can amplify microscopic gradients in surfactant concentration into macroscopic optical signals. ExperimentsWe use a milli-fluidic system to generate gradients in aqueous sodium dodecyl sulfate (SDS) concentration and optically quantify the dynamic ordering of micrometer-thick nematic LC films that contact the gradients. FindingsWe find that the reordering of the LCs is dominated by interfacial shearing by Marangoni flows, thus providing simple methods for rapid mapping of interfacial velocities from a single optical image and investigating the effects of confinement of surfactant-driven interfacial flows. Additionally, we establish that surface advection and surfactant desorption are the two key processes that regulate the interfacial flows, revealing that the dynamic response of the LC can provide rapid and potentially high throughput approaches to measurement of non-equilibrium interfacial properties of amphiphiles. We also observe flow-induced assemblies of microparticles to form at the LC interface, hinting at new non-equilibrium approaches to microparticle assembly. We conclude that dynamic states adopted by LCs in the presence of surfactant concentration gradients provide new opportunities for engineering complex fluids beyond equilibrium.

The Significance of Organometallic Complexes in the Photochemical Reduction of Cu2+ Onto Semiconductor Scaffolds

Metal-metal oxide nanoparticles (MMO-NP) show promise as an alternative to diffusive topical antibiotics as they are capable of slowly releasing metal ions upon contact-activated oxidation which promotes toxicity to prokaryotes. Modification of semiconducting nanoparticles with metals also has great relevance in terms of increasing the photocatalytic activity of the particles, which are inherently limited in practicality due to their large band gap energies. The use of semiconductor nanoparticles for the reduction of metal ions has been widely explored, however, more work needs to be done to make the reduction of metal ions like Cu2+ more facile. This work will focus on the mechanism and efficacy of incorporating Cu2+ into organometallic complexes for the photoreduction of Cu2+ on the surface of TiO2 and ZnO. The mechanistic contribution of bidentate complexes of Cu2+ in the complete reduction of Cu2+ will be explored. The implementation of L-serine (SER), L-arginine (ARG), and glucono-delta-lactone (GDL) as ligands will be observed. Initial results show that the coordination of Cu2+ into an organometallic complex provides an effective medium for the delivery of Cu2+ to the surface of TiO2. Spectroscopic analysis will be used to determine if the ligands contribute to the two-electron reduction via ligand to metal charge transfer (LMCT) or via non-covalent interactions that increase the effective concentration of Cu2+ at the nanoparticle surface. The surface modification of TiO2 will be observed post-reaction with scanning electron microscopy (SEM) coupled with energy-dispersive X-ray spectroscopy (EDS) and atomic absorption spectroscopy (AAS) to measure copper content on the surface. Antimicrobial effectiveness will be characterized via Kirby Bauer testing as well as single cell optical imaging, with E. coli being the model microbe. Photocatalytic activity will be observed via the degradation kinetics of model pollutants. Preliminary results indicate that the addition of ligands greatly increased the yield of reduced copper at low pH and that there is some extent of control over the copper ion leaching as Kirby Bauer tests showed no zones of inhibition.

Optical Microscope Based Universal Parameter for Identifying Layer Number in Two-Dimensional Materials.

Optical contrast is the most common preliminary method to identify layer number of two-dimensional (2D) materials, but it is seldom used as a confirmatory technique. We explain the reason for variation of optical contrast between imaging systems, motivating system-independent measurement of optical contrast as a critical need. We describe a universal method to quantify the layer number using the RGB (red-green-blue) and RAW optical images. For RGB images, the slope of 2D flake (MoS2, WSe2, graphene) intensity vs substrate intensity is extracted from optical images with varying lamp power. The intensity slope identifies layer number and is system independent. For RAW images, intensity slopes and intensity ratios are completely system and intensity independent. Intensity slope (for RGB) and intensity ratio (for RAW) are thus universal parameters for identifying layer number. The RAW format is not present in all imaging systems, but it can confirm layer number using a single optical image, making it a rapid and system-independent universal method. A Fresnel-reflectance-based optical model provides an excellent match with experiments. Furthermore, we have created a MATLAB-based graphical user interface that can identify layer number rapidly. This technique is expected to accelerate the preparation of heterostructures and to fulfill a prolonged need for universal optical contrast method.

Multi-Path Interactive Network for Aircraft Identification with Optical and SAR Images

Aircraft identification has been a research hotspot in remote-sensing fields. However, due to the presence of clouds in satellite-borne optical imagery, it is difficult to identify aircraft using a single optical image. In this paper, a Multi-path Interactive Network (MIN) is proposed to fuse Optical and Synthetic Aperture Radar (SAR) images for aircraft identification on cloudy days. First, features are extracted from optical and SAR images separately by convolution backbones of ResNet-34. Second, a piecewise residual fusion strategy is proposed to reduce the effect of clouds. A plug-and-play Interactive Attention Sum-Max fusion module (IASM), is thus constructed to interact with features from multi-modal images. Moreover, multi-path IASM is designed to mix multi-modal features from backbones. Finally, the fused features are sent to the neck and head of MIN for regression and classification. Extensive experiments are carried out on the Fused Cloudy Aircraft Detection (FCAD) dataset that is constructed, and the results show the efficiency of MIN in identifying aircraft under clouds with different thicknesses.Compared with the single-source model, the multi-source fusion model MIN is improved by more than 20%, and the proposed method outperforms the state-of-the-art approaches.

Correction: Almar et al. Sea State from Single Optical Images: A Methodology to Derive Wind-Generated Ocean Waves from Cameras, Drones and Satellites. Remote Sens. 2021, 13, 679

There was an error in the original article [...]

Mapping riparian zone macro litter abundance using combination of optical and thermal sensor

A significant increase in the world's population will lead to an increase in consumption and, therefore, an increase in global waste. Various attempts have been made to monitor and map waste, but the proposed approaches are difficult and complicated, and they incur high costs. In this study, to overcome limitations in monitoring and mapping plastic waste, using combined optical and thermal sensors installed on drones is proposed. The study area is the riparian zone, or the zone around the river, where the accumulation of plastic waste at the mouth of the river eventually reaches the sea. The image data obtained were processed using machine learning methods to produce high accuracy and precision. To determine the effectiveness of the proposed method, an accuracy assessment was conducted. The results of this study indicate that the combination of optical and thermal sensors provides the best accuracy compared to using only single optical or thermal image data.

A New Pattern for Detection of Streak-Like Space Target From Single Optical Images

In recent years, the continuously increasing number of space objects has caught great attention and triggered the need for space surveillance for the prevention of potential space collisions. Moving objects may appear as streaks in optical images in some situations. However, most of them are usually unknown so that they lack sufficient prior information, which brings great difficulty to detection. Motivated by the fact that streaks have their unique image characteristics, this article presents a novel detection pattern for streak-like targets in single optical images. First, a space target model from point-like target to streak-like target is studied to obtain characteristics. Next, the background removal method used in software named SExtractor is applied to find target candidates. Then, an operator is used to find local maximum points in each target candidate, and the points in the same candidate are clustered according to three domains including space, intensity, and distribution, to identify whether it contains a streak. Finally, the trajectory line of each cluster is extracted. The proposed method was tested on multiple sequences of collected real optical images, which contain targets of different features and found to obtain superior performance in the probability of detection, probability of false alarm, and receiver operating characteristic (ROC), in comparison to baseline methods. In particular, the setting of thresholds is intuitive because of the designed parameters with concrete meaning in images and the normalization process.

Dim Space Target Detection via Convolutional Neural Network in Single Optical Image

Real-time dim space target detection is a significant challenge in space situation awareness. This paper proposes a single-frame space object segmentation and detection method based on deep learning. Firstly, Channel and Space Attention U-net (CSAU-Net) is presented based on space image features. We remove unnecessary feature layers and add attention modules in the traditional encoder and decoder structure to enhance feature fusion and better use original feature layers. The proposed network structure can achieve accurate segmentation of space objects with fewer data training. At the same time, we construct a space target dataset for training, which contains targets with different signal-to-noise ratios to enhance the generalization of convolutional neural networks. After obtaining the segmentation masks, a simple connected component labeling method is applied to extract the centroid of the space target. Experiments show that our approach can achieve an ideal segmentation effect when the signal-to-noise ratio of the space target in the simulated dataset is 0.3. In addition, the proposed algorithm can realize fast segmentation and achieve an accuracy of 98.5%, which is similar to the traditional multi-frame space target detection method in real space image detection.

Self-Supervised Monocular Depth Estimation With Multiscale Perception

Extracting 3D information from a single optical image is very attractive. Recently emerging self-supervised methods can learn depth representations without using ground truth depth maps as training data by transforming the depth prediction task into an image synthesis task. However, existing methods rely on a differentiable bilinear sampler for image synthesis, which results in each pixel in a synthetic image being derived from only four pixels in the source image and causes each pixel in the depth map to perceive only a few pixels in the source image. In addition, when calculating the photometric error between a synthetic image and its corresponding target image, existing methods only consider the photometric error within a small neighborhood of each single pixel and therefore ignore correlations between larger areas, which causes the model to tend to fall into the local optima for small patches. In order to extend the perceptual area of the depth map over the source image, we propose a novel multi-scale method that downsamples the predicted depth map and performs image synthesis at different resolutions, which enables each pixel in the depth map to perceive more pixels in the source image and improves the performance of the model. As for the locality of photometric error, we propose a structural similarity (SSIM) pyramid loss to allow the model to sense the difference between images in multiple areas of different sizes. Experimental results show that our method achieves superior performance on both outdoor and indoor benchmarks.

Large Area High-Resolution 3D Mapping of Oxia Planum: The Landing Site for the ExoMars Rosalind Franklin Rover

We demonstrate an end-to-end application of the in-house deep learning-based surface modelling system, called MADNet, to produce three large area 3D mapping products from single images taken from the ESA Mars Express’s High Resolution Stereo Camera (HRSC), the NASA Mars Reconnaissance Orbiter’s Context Camera (CTX), and the High Resolution Imaging Science Experiment (HiRISE) imaging data over the ExoMars 2022 Rosalind Franklin rover’s landing site at Oxia Planum on Mars. MADNet takes a single orbital optical image as input, provides pixelwise height predictions, and uses a separate coarse Digital Terrain Model (DTM) as reference, to produce a DTM product from the given input image. Initially, we demonstrate the resultant 25 m/pixel HRSC DTM mosaic covering an area of 197 km × 182 km, providing fine-scale details to the 50 m/pixel HRSC MC-11 level-5 DTM mosaic. Secondly, we demonstrate the resultant 12 m/pixel CTX MADNet DTM mosaic covering a 114 km × 117 km area, showing much more detail in comparison to photogrammetric DTMs produced using the open source in-house developed CASP-GO system. Finally, we demonstrate the resultant 50 cm/pixel HiRISE MADNet DTM mosaic, produced for the first time, covering a 74.3 km × 86.3 km area of the 3-sigma landing ellipse and partially the ExoMars team’s geological characterisation area. The resultant MADNet HiRISE DTM mosaic shows fine-scale details superior to existing Planetary Data System (PDS) HiRISE DTMs and covers a larger area that is considered difficult for existing photogrammetry and photoclinometry pipelines to achieve, especially given the current limitations of stereo HiRISE coverage. All of the resultant DTM mosaics are co-aligned with each other, and ultimately with the Mars Global Surveyor’s Mars Orbiter Laser Altimeter (MOLA) DTM, providing high spatial and vertical congruence. In this paper, technical details are presented, issues that arose are discussed, along with a visual evaluation and quantitative assessments of the resultant DTM mosaic products.

U-IMG2DSM: Unpaired Simulation of Digital Surface Models With Generative Adversarial Networks

High-resolution digital surface models (DSMs) provide valuable height information about the Earth's surface, which can be successfully combined with other types of remotely sensed data in a wide range of applications. However, the acquisition of DSMs with high spatial resolution is extremely time-consuming and expensive with their estimation from a single optical image being an ill-possed problem. To overcome these limitations, this letter presents a new unpaired approach to obtain DSMs from optical images using deep learning techniques. Specifically, our new deep neural model is based on variational autoencoders (VAEs) and generative adversarial networks (GANs) to perform image-to-image translation, obtaining DSMs from optical images. Our newly proposed method has been tested in terms of photographic interpretation, reconstruction error, and classification accuracy using three well-known remotely sensed data sets with very high spatial resolution (obtained over Potsdam, Vaihingen, and Stockholm). Our experimental results demonstrate that the proposed approach obtains satisfactory reconstruction rates that allow enhancing the classification results for these images. The source code of our method is available from: https://github.com/mhaut/UIMG2DSM.

Single Exposure Optical Image Watermarking Using a cGAN Network

A single exposure optical image watermarking framework based on deep learning (DL) is proposed in this paper, and original watermark image information can be reconstructed from only single-frame watermarked hologram by using an end-to-end network with high-quality. First, the single exposure watermarked hologram is acquired with our presented phase-shifted interferometry based optical image watermarking (PSOIW) frame, and then all holograms and corresponding watermark images are constructed to the train datasets for the learning of an end-to-end conditional generative adversarial network (cGAN), finally retrieved the watermark image well with the trained cGAN network using only one hologram. This DL-based method greatly reduces the recording or transmitting data burden by 1/4 compared with our presented PSOIW technique, and may provide a new way for the real-time 3D image/video security applications. The feasibility and security of the proposed method are demonstrated by the optical experiment results.

Sea State from Single Optical Images: A Methodology to Derive Wind-Generated Ocean Waves from Cameras, Drones and Satellites

Sea state is a key variable in ocean and coastal dynamics. The sea state is either sparsely measured by wave buoys and satellites or modelled over large scales. Only a few attempts have been devoted to sea state measurements covering a large domain; in particular its estimation from optical images. With optical technologies becoming omnipresent, optical images offer incomparable spatial resolution from diverse sensors such as shore-based cameras, airborne drones (unmanned aerial vehicles/UAVs), or satellites. Here, we present a standalone methodology to derive the water surface elevation anomaly induced by wind-generated ocean waves from optical imagery. The methodology was tested on drone and satellite images and compared against ground truth. The results show a clear dependence on the relative azimuth view angle in relation to the wave crest. A simple correction is proposed to overcome this bias. Overall, the presented methodology offers a practical way of estimating ocean waves for a wide range of applications.

Spatial clustering of orientation preference in primary visual cortex of the large rodent agouti.

SummaryAll rodents investigated so far possess orientation-selective neurons in the primary visual cortex (V1) but – in contrast to carnivores and primates – no evidence of periodic maps with pinwheel-like structures. Theoretical studies debating whether phylogeny or universal principles determine development of pinwheels point to V1 size as a critical constraint. Thus, we set out to study maps of agouti, a big diurnal rodent with a V1 size comparable to cats'. In electrophysiology, we detected interspersed orientation and direction-selective neurons with a bias for horizontal contours, corroborated by homogeneous activation in optical imaging. Compatible with spatial clustering at short distance, nearby neurons tended to exhibit similar orientation preference. Our results argue against V1 size as a key parameter in determining the presence of periodic orientation maps. They are consistent with a phylogenetic influence on the map layout and development, potentially reflecting distinct retinal traits or interspecies differences in cortical circuitry.