Scale Recurrent Network Research Articles

Enhancing Total Optical Throughput of Microscopy with Deep Learning for Intravital Observation.

The significance of performing large-depth dynamic microscopic imaging in vivo for life science research cannot be overstated. However, the optical throughput of the microscope limits the available information per unit of time, i.e., it is difficult to obtain both high spatial and temporal resolution at once. Here, a method is proposed to construct a kind of intravital microscopy with high optical throughput, by making near-infrared-II (NIR-II, 900-1880nm) wide-field fluorescence microscopy learn from two-photon fluorescence microscopy based on a scale-recurrent network. Using this upgraded NIR-II fluorescence microscope, vessels in the opaque brain of a rodent are reconstructed three-dimensionally. Five-fold axial and thirteen-fold lateral resolution improvements are achieved without sacrificing temporal resolution and light utilization. Also, tiny cerebral vessel dilatations in early acute respiratory failure mice are observed, with this high optical throughput NIR-II microscope at an imaging speed of 30fps.

Film and Video Quality Optimization Using Attention Mechanism-Embedded Lightweight Neural Network Model.

In filming, the collected video may be blurred due to camera shake and object movement, causing the target edge to be unclear or deforming the targets. In order to solve these problems and deeply optimize the quality of movie videos, this work proposes a video deblurring (VD) algorithm based on neural network (NN) model and attention mechanism (AM). Based on the scale recurrent network, Haar planar wavelet transform (WT) is introduced to preprocess the video image and to deblur the video image in the wavelet domain. Additionally, the spatial and channel AMs are fused into the overall network framework to improve the feature expression ability. Further, the residual inception spatial-channel attention (RISCA) mechanism is introduced to extract the multiscale feature information from video images. Meanwhile, skip spatial-channel attention (SSCA) accelerates the network training time to achieve a better VD effect. Finally, relevant experiments are designed, factoring in peak signal-to-noise ratio (PSNR) and structural similarity (SSI). The experimental findings corroborate that the proposed Haar and attention video deblurring (HAVD) outperforms multisize network Haar (MSNH) in PSNR and structural similarity (SSIM), improved by 0.10 dB and 0.005, respectively. Therefore, embedding the dual AMs can improve the model performance and optimize the video quality. This work provides technical support for solving the video distortion problems.

An Imaging Enhancement Method for a Terahertz Rotation Mirror Imaging System Based on a Scale-Recurrent Network

The terahertz (THz) rotation mirror imaging system is an alternative to the THz array imaging system. A THz rotation mirror imaging system costs less than a THz array imaging system, while the imaging speed of a THz rotation mirror imaging system is much higher than the imaging speed of a THz raster-scan imaging system under the same hardware conditions. However, there is some distortion in the THz image from the THz rotation mirror imaging system. The distortion, which makes images from the THz rotation mirror imaging system difficult to identify, results from the imaging principle of the THz rotation mirror imaging system. In this article, a method based on the scale-recurrent network (SRN) is put in place to correct the distortion. A comparison between distorted THz images and corrected images shows that the proposed method significantly increases the structural similarity between the THz images and the samples.

Stack-based Scale-recurrent Network for Face Image Deblurring

Stack-based Scale-recurrent Network for Face Image Deblurring

Beyond Camera Motion Blur Removing: How to Handle Outliers in Deblurring

Camera motion deblurring is an important low-level vision task for achieving better imaging quality. When a scene has outliers such as saturated pixels, the captured blurred image becomes more difficult to restore. In this paper, we propose a novel method to handle camera motion blur with outliers. We first propose an edge-aware scale-recurrent network (EASRN) to conduct deblurring. EASRN has a separate deblurring module that removes blur at multiple scales and an upsampling module that fuses different input scales. Then a salient edge detection network is proposed to supervise the training process and constraint the edges restoration. By simulating camera motion and adding various light sources, we can generate blurred images with saturation cutoff. Using the proposed data generation method, our network can learn to deal with outliers effectively. We evaluate our method on public test datasets including the GoPro dataset, Kohler's dataset and Lai's dataset. Both objective evaluation indexes and subjective visualization show that our method results in better deblurring quality than other state-of-the-art approaches.

A Phase Filtering Method with Scale Recurrent Networks for InSAR

Phase filtering is a key issue in interferometric synthetic aperture radar (InSAR) applications, such as deformation monitoring and topographic mapping. The accuracy of the deformation and terrain height is highly dependent on the quality of phase filtering. Researchers are committed to continuously improving the accuracy and efficiency of phase filtering. Inspired by the successful application of neural networks in SAR image denoising, in this paper we propose a phase filtering method that is based on deep learning to efficiently filter out the noise in the interferometric phase. In this method, the real and imaginary parts of the interferometric phase are filtered while using a scale recurrent network, which includes three single scale subnetworks based on the encoder-decoder architecture. The network can utilize the global structural phase information contained in the different-scaled feature maps, because RNN units are used to connect the three different-scaled subnetworks and transmit current state information among different subnetworks. The encoder part is used for extracting the phase features, and the decoder part restores detailed information from the encoded feature maps and makes the size of the output image the same as that of the input image. Experiments on simulated and real InSAR data prove that the proposed method is superior to three widely-used phase filtering methods by qualitative and quantitative comparisons. In addition, on the same simulated data set, the overall performance of the proposed method is better than another deep learning-based method (DeepInSAR). The runtime of the proposed method is only about 0.043s for an image with a size of 1024×1024 pixels, which has the significant advantage of computational efficiency in practical applications that require real-time processing.

Image Deblurring Using Multi-Stream Bottom-Top-Bottom Attention Network and Global Information-Based Fusion and Reconstruction Network.

Image deblurring has been a challenging ill-posed problem in computer vision. Gaussian blur is a common model for image and signal degradation. The deep learning-based deblurring methods have attracted much attention due to their advantages over the traditional methods relying on hand-designed features. However, the existing deep learning-based deblurring techniques still cannot perform well in restoring the fine details and reconstructing the sharp edges. To address this issue, we have designed an effective end-to-end deep learning-based non-blind image deblurring algorithm. In the proposed method, a multi-stream bottom-top-bottom attention network (MBANet) with the encoder-to-decoder structure is designed to integrate low-level cues and high-level semantic information, which can facilitate extracting image features more effectively and improve the computational efficiency of the network. Moreover, the MBANet adopts a coarse-to-fine multi-scale strategy to process the input images to improve image deblurring performance. Furthermore, the global information-based fusion and reconstruction network is proposed to fuse multi-scale output maps to improve the global spatial information and recurrently refine the output deblurred image. The experiments were done on the public GoPro dataset and the realistic and dynamic scenes (REDS) dataset to evaluate the effectiveness and robustness of the proposed method. The experimental results show that the proposed method generally outperforms some traditional deburring methods and deep learning-based state-of-the-art deblurring methods such as scale-recurrent network (SRN) and denoising prior driven deep neural network (DPDNN) in terms of such quantitative indexes as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) and human vision.

Automatic Catheter and Tube Detection in Pediatric X-ray Images Using a Scale-Recurrent Network and Synthetic Data.

Catheters are commonly inserted life supporting devices. Because serious complications can arise from malpositioned catheters, X-ray images are used to assess the position of a catheter immediately after placement. Previous computer vision approaches to detect catheters on X-ray images were either rule-based or only capable of processing a limited number or type of catheters projecting over the chest. With the resurgence of deep learning, supervised training approaches are beginning to show promising results. However, dense annotation maps are required, and the work of a human annotator is difficult to scale. In this work, we propose an automatic approach for detection of catheters and tubes on pediatric X-ray images. We propose a simple way of synthesizing catheters on X-ray images to generate a training dataset by exploiting the fact that catheters are essentially tubular structures with various cross sectional profiles. Further, we develop a UNet-style segmentation network with a recurrent module that can process inputs at multiple scales and iteratively refine the detection result. By training on adult chest X-rays, the proposed network exhibits promising detection results on pediatric chest/abdomen X-rays in terms of both precision and recall, with Fβ = 0.8. The approach described in this work may contribute to the development of clinical systems to detect and assess the placement of catheters on X-ray images. This may provide a solution to triage and prioritize X-ray images with potentially malpositioned catheters for a radiologist's urgent review and help automate radiology reporting.

High-capacity embedding of synfire chains in a cortical network model

Synfire chains, sequences of pools linked by feedforward connections, support the propagation of precisely timed spike sequences, or synfire waves. An important question remains, how synfire chains can efficiently be embedded in cortical architecture. We present a model of synfire chain embedding in a cortical scale recurrent network using conductance-based synapses, balanced chains, and variable transmission delays. The network attains substantially higher embedding capacities than previous spiking neuron models and allows all its connections to be used for embedding. The number of waves in the model is regulated by recurrent background noise. We computationally explore the embedding capacity limit, and use a mean field analysis to describe the equilibrium state. Simulations confirm the mean field analysis over broad ranges of pool sizes and connectivity levels; the number of pools embedded in the system trades off against the firing rate and the number of waves. An optimal inhibition level balances the conflicting requirements of stable synfire propagation and limited response to background noise. A simplified analysis shows that the present conductance-based synapses achieve higher contrast between the responses to synfire input and background noise compared to current-based synapses, while regulation of wave numbers is traced to the use of variable transmission delays.