Super-resolution Feature Research Articles

Optimization of Super-Resolution Features via Deep Learning Techniques

Optimization of Super-Resolution Features via Deep Learning Techniques

Super-Resolution Imaging of Sub-diffraction-Limited Pattern with Superlens Based on Deep Learning

The development of super-resolution imaging techniques has revolutionized our ability to study the nano-scale world, where objects are often smaller than the diffraction limit of traditional optical microscopes. Super-resolution superlenses have been proposed to solve this problem by manipulating the light wave in the near field. A superlens is a kind of metamaterial-based lens that can enhance the evanescent waves generated by nano-scale objects, utilizing the surface plasmon phenomenon. The superlens allows for the imaging of nano-scale objects that would otherwise be impossible to resolve using traditional lenses. Previous research has shown that nanostructures can be imaged using superlenses, but the exact shape of the superlens must be known in advance, and an analytical calculation is needed to reconstruct the image. Localized plasmon structured illumination microscopy is an approach to achieve super-resolution by imaging the superlens-enhanced evanescent wave with illumination shifts. This study proposes a new approach utilizing a conditional generative adversarial network to obtain super-resolution images of arbitrary nano-scale patterns. To test the efficacy of this approach, finite-difference time-domain simulation was utilized to obtain superlens imaging results. The data from the simulation were then used for deep learning to develop the model. With the help of deep learning, the inverse calculation of complex sub-diffraction-limited patterns can be achieved. The super-resolution feature of the superlens based on deep learning is investigated. The findings of this study have significant implications for the field of nano-scale imaging, where the ability to resolve arbitrary nano-scale patterns will be crucial for advances in nanotechnology and materials science.

Süper Çözünürlük Yönteminin Uydu İmgelerinin Sınıflandırma Performansına Etkisi

The high resolution of the image is very important for applications. Publicly available satellite images generally have low resolutions. Since low resolution causes loss of information, the desired performance cannot be achieved depending on the type of problem studied in the field of remote sensing. In such a case, super resolution algorithms are used to render low resolution images high resolution. Super resolution algorithms are used to obtain high resolution images from low resolution images. In studies with satellite images, the use of images enhanced with super resolution is important. Since the resolution of satellite images is low, the success rate in the classification process is low. In this study, super resolution method is proposed to increase the classification performance of satellite images. The attributes of satellite images were extracted using AlexNet, ResNet50, Vgg19 from deep learning architecture. Then the extracted features were then classified into 6 classes by giving input to AlexNet-Softmax, ResNet50-Softmax, Vgg19-Softmax, Support Vector Machine, K-Nearest Neighbor, decision trees and Naive Bayes classification algorithms. Without super resolution and with super resolution feature extraction and classification processes were performed separately. Classification results without super resolution and with super resolution were compared. Improvement in classification performance was observed using super resolution.

High-precision real-time UAV target recognition based on improved YOLOv4

In recent years, unmanned aerial vehicles (UAVs) have gained widespread use in both military and civilian fields with the advancement of aviation technology and improved communication capabilities. However, the phenomenon of unauthorized UAV flights, or “black flying”, poses a serious threat to the safe flight of aircraft in airspace and public safety. To effectively interfere with and attack UAV targets, it is crucial to enhance the detection and identification of “low, slow and small” UAVs. This study focuses on achieving high-precision and lightweight detection and identification of four-rotor, six-rotor, and fixed-wing UAVs in low-altitude complex environments. By combining deep learning target detection with superresolution feature enhancement, a lightweight UAV detection model is designed and field-tested for verification. To address the challenge of detecting small UAV targets with limited information, the feature fusion network is enhanced based on the traditional YOLOv4 algorithm to improve the detection ability of small targets via small target enhancement and candidate box adjustment. The feasibility of the improved network is quantitatively and qualitatively analyzed. Channel pruning and layer pruning are then applied to the network, significantly reducing its depth and width and realizing a lightweight network. Finally, reasoning quantification is conducted on the embedded platform to enable end-side deployment of the target detection algorithm.

Single-Pixel Photoacoustic Microscopy with Speckle Illumination

Wide-field optical-resolution microscopy with structured illumination and single-pixel detection has been the topic of a number of research investigations. Its advantages over point scanning approaches are many and include a faster acquisition rate for sparse samples, sectioning, and super-resolution features. Initially introduced for fluorescence imaging, structured illumination approaches have been adapted and developed for many other imaging modalities. In this paper, we illustrate how speckle illumination, as a particular type of structured illumination, can be exploited to perform optical-resolution photoacoustic microscopy with a single-pixel imaging approach. We first introduce the principle of single-pixel detection applied to photoacoustic imaging and then illustrate in 2 different situations how photoacoustic images may be computationally reconstructed from speckle illumination: In the first situation where the speckle patterns are known through a prior calibration, various reconstruction approaches may be implemented, which are demonstrated experimentally through both scattering layers and multimode optical fibers; in the second situation where the speckle patterns are unknown (blind structured illumination), the so-called memory effect can be harnessed to produce calibration-free photoacoustic images, following the approach initially proposed for fluorescence imaging through thin scattering layers.

Large-scale expanded sample imaging with tiling lattice lightsheet microscopy.

The ability to observe biological nanostructures forms a vital step in understanding their functions. Thanks to the invention of expansion microscopy (ExM) technology, super-resolution features of biological samples can now be easily visualized with conventional light microscopies. However, when the sample is physically expanded, the demand for deep and precise 3D imaging increases. Lattice lightsheet microscopy (LLSM), which utilizes a planar illumination that is confined within the imaging depth of high numerical aperture (NA=1.1) detection objective, fulfils such requirements. In addition, optical tiling could be implemented to increase the field of view (FoV) by moving the lightsheet without mechanically moving the samples or the objective for high-precision 3D imaging. In this review article, we will explain the principle of the tiling lattice lightsheet microscopy (tLLSM), which combines optical tiling and lattice lightsheet, and discuss the applications of tLLSM in ExM.

Semantic Segmentation Algorithm of Rice Small Target Based on Deep Learning

At present, rice is generally in a state of dense adhesion and small granular volume during processing, resulting in no effective semantic segmentation method for rice to extract complete rice. Aiming at the above problems, this paper designs a small object semantic segmentation network model based on multi-view feature fusion. The overall structure of the network is divided into a multi-view feature extraction module, a super-resolution feature building module and a semantic segmentation module. The extraction ability of small target features is improved by super-resolution construction of small target detail features, and the learning ability of the network for small target features is enhanced and expanded through multi-view. At the same time, a dataset of quality inspection during rice processing was constructed. We train and test the model on this dataset. The results show that the average segmentation accuracy of the semantic segmentation model in this paper reaches 87.89%. Compared with the semantic segmentation models such as SegNet, CBAM, RefineNet, DeepLabv3+ and G-FRNet, it has obvious advantages in various indicators, which can provide rice quality detection and an efficient method of rice grain extraction.

Small Ship Detection Based on Hybrid Anchor Structure and Feature Super-Resolution

Small ships in remote sensing images have blurred details and are difficult to detect. Existing algorithms usually detect small ships based on predefined anchors with different sizes. However, limited by the number of different sizes, it is difficult for anchor-based methods to match small ships of different sizes and structures during training, as they can easily cause misdetections. In this paper, we propose a hybrid anchor structure to generate region proposals for small ships, so as to take full advantage of both anchor-based methods with high localization accuracy and anchor-free methods with fewer misdetections. To unify the output evaluation and obtain the best output, a label reassignment strategy is proposed, which reassigns the sample labels according to the harmonic intersection-over-union (IoU) before and after regression. In addition, an adaptive feature pyramid structure is proposed to enhance the features of important locations on the feature map, so that the features of small ship targets are more prominent and easier to identify. Moreover, feature super-resolution technology is introduced for the region of interest (RoI) features of small ships to generate super-resolution feature representations with a small computational cost, as well as generative adversarial training to improve the realism of super-resolution features. Based on the super-resolution feature, ship proposals are further classified and regressed by using super-resolution features to obtain more accurate detection results. Detailed ablation and comparison experiments demonstrate the effectiveness of the proposed method.

Micro-stepping extended focus reduces photobleaching and preserves structured illumination super-resolution features.

Despite progress made in confocal microscopy, even fast systems still have insufficient temporal resolution for detailed live-cell volume imaging, such as tracking rapid movement of membrane vesicles in three-dimensional space. Depending on the shortfall, this may result in undersampling and/or motion artifacts that ultimately limit the quality of the imaging data. By sacrificing detailed information in the Z-direction, we propose a new imaging modality that involves capturing fast 'projections' from the field of depth and shortens imaging time by approximately an order of magnitude as compared to standard volumetric confocal imaging. With faster imaging, radiation exposure to the sample is reduced, resulting in less fluorophore photobleaching and potential photodamage. The implementation minimally requires two synchronized control signals that drive a piezo stage and trigger the camera exposure. The device generating the signals has been tested on spinning disk confocal and instant structured-illumination-microscopy (iSIM) microscopes. Our calibration images show that the approach provides highly repeatable and stable imaging conditions that enable photometric measurements of the acquired data, in both standard live imaging and super-resolution modes.This article has an associated First Person interview with the first author of the paper.

First person – Xian Hu

ABSTRACT First Person is a series of interviews with the first authors of a selection of papers published in Journal of Cell Science, helping early-career researchers promote themselves alongside their papers. Xian Hu is first author on ‘Micro-stepping extended focus reduces photobleaching and preserves structured illumination super-resolution features’, published in JCS. Xian is a Postdoc in the lab of Oddmund Bakke at the Department of Biosciences University of Oslo, Norway, investigating the link between intracellular vesicular trafficking and mechano-sensing by using state-of-the-art bioimaging and image-processing technologies.

Gated Fusion Network for Degraded Image Super Resolution

Single image super resolution aims to enhance image quality with respect to spatial content, which is a fundamental task in computer vision. In this work, we address the task of single frame super resolution with the presence of image degradation, e.g., blur, haze, or rain streaks. Due to the limitations of frame capturing and formation processes, image degradation is inevitable, and the artifacts would be exacerbated by super resolution methods. To address this problem, we propose a dual-branch convolutional neural network to extract base features and recovered features separately. The base features contain local and global information of the input image. On the other hand, the recovered features focus on the degraded regions and are used to remove the degradation. Those features are then fused through a recursive gate module to obtain sharp features for super resolution. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process by avoiding learning the mixed degradation all-in-one and thus enhance the final high-resolution prediction results. We evaluate the proposed method in three degradation scenarios. Experiments on these scenarios demonstrate that the proposed method performs more efficiently and favorably against the state-of-the-art approaches on benchmark datasets.

Statistical Compressive Sensing and Feature Extraction of Time-Frequency Spectrum From Narrowband Radar

Aiming at the signal reconstruction problem for the conventional narrowband radar system, we propose a new statistical compressive sensing (SCS) method to achieve the reconstruction of superresolution time-frequency spectrum from the corrupted time-domain measurement. The proposed method assumes that the signal obeys complex Gaussian distribution and develops a hierarchical Bayesian model. Variational Bayesian expectation maximization (VBEM) is used to perform inference for the posterior distributions of the model parameters. In order to fully exploit the superresolution characteristics of reconstructed spectrum, a novel superresolution time-frequency feature vector is extracted for subsequent classification of ground moving targets, i.e., walking person and a moving wheeled vehicle. Experimental results on measured data show that the proposed reconstruction method can obtain good reconstruction results and the superresolution feature has good classification performance for human and vehicle targets.

Detection of Traffic Signs Based on Combination of GAN and Faster-RCNN

Object detection has been widely researched and a series of algorithms have been proposed. These algorithms also have satisfactory verification results in various public datasets. However, the detection effect of small objects is still not satisfactory. Existing object detection algorithms usually achieves the detection of small objects by learning Multi-scale features, but this brings a large amount of calculation. In this paper, we solve the problem that is difficult to detect with small objects by mapping the non-obvious features of the small object into large object Features with the same feature distribution. Then the super-resolution feature obtained by mapping can be used to significant improve the detection performance in the process of training small objects detection.

Superresolution far-field imaging of complex objects using reduced superoscillating ripples

Superoscillation is a phenomenon where a wave oscillates locally faster than its highest Fourier component. While previous reports have shown attractive possibilities for a superoscillation-based far-field superresolution imaging device, it has also been recognized that a high-energy “sideband” region coexists with the superresolution features. This sideband causes strong restrictions and necessitates trade-offs in achievable resolution, viewing area, and sensitivity of the imaging device. In this work, we introduce a new class of superoscillation waveform—which consists of a diffraction-limited hotspot surrounded by low-energy superoscillating sidelobe ripples. This waveform alleviates the aforementioned trade-off and enables superresolution imaging for complex objects over a larger viewing area while maintaining a practical level of sensitivity. Using this waveform as the point spread function of an imaging system, we demonstrate the successful superresolution of Latin letters without performing scanning and/or post-processing operations.

Axial super-resolution evanescent wave tomography.

Optical tomographic reconstruction of a three-dimensional (3D) nanoscale specimen is hindered by the axial diffraction limit, which is 2-3 times worse than the focal plane resolution. We propose and experimentally demonstrate an axial super-resolution evanescent wave tomography method that enables the use of regular evanescent wave microscopes like the total internal reflection fluorescence microscope beyond surface imaging and achieve a tomographic reconstruction with axial super-resolution. Our proposed method based on Fourier reconstruction achieves axial super-resolution by extracting information from multiple sets of 3D fluorescence images when the sample is illuminated by an evanescent wave. We propose a procedure to extract super-resolution features from the incremental penetration of an evanescent wave and support our theory by one-dimensional (along the optical axis) and 3D simulations. We validate our claims by experimentally demonstrating tomographic reconstruction of microtubules in HeLa cells with an axial resolution of ∼130 nm. Our method does not require any additional optical components or sample preparation. The proposed method can be combined with focal plane super-resolution techniques like stochastic optical reconstruction microscopy and can also be adapted for THz and microwave near-field tomography.

High-photosensitive resin for super-resolution direct-laser-writing based on photoinhibited polymerization

An ethoxylated bis-phenol-A dimethacrylate based photoresin BPE-100 of relatively high photosensitivity and modulus is used for the creation of sub-50 nm features. This is achieved by using the direct laser writing technique based on the single-photon photoinhibited polymerization. The super-resolution feature is realized by overlapping two laser beams of different wavelengths to enable the wavelength-controlled activation of photoinitiating and photoinhibiting processes in the polymerization. The increased photosensitivity of the photoresin promotes a fast curing speed and enhances the photopolymerization efficiency. Using the photoresin BPE-100, we achieve 40 nm dots for the first time in the super-resolution fabrication technique based on the photoinhibited polymerization, and a minimum linewidth of 130 nm. The influence of the power of the inhibiting laser and the exposure time on the feature size is studied and the results agree well with the prediction obtained from a simulation based on a non-steady-state kinetic model.

Super resolution feature extraction of moving targets

High range resolution (HRR) moving target indicator (MTI) is becoming increasingly important for many military and civilian applications such as the detection and classification of moving targets in strong clutter background. We consider the problem of extracting the HRR features of moving targets with very closely spaced scatterers in the presence of strong stationary clutter, where the range migration and Doppler frequency are taken into account. A relaxation-based algorithm, which is robust and computationally simple, is proposed to deal with the above problem. Numerical results have shown that the proposed algorithm exhibits super resolution and excellent estimation performance.