Sequence Of Low-resolution Images Research Articles

Automatic segmentation of Caenorhabditis elegans skeletons in worm aggregations using improved U-Net in low-resolution image sequences

Pose estimation of C. elegans in image sequences is challenging and even more difficult in low-resolution images. Problems range from occlusions, loss of worm identity, and overlaps to aggregations that are too complex or difficult to resolve, even for the human eye. Neural networks, on the other hand, have shown good results in both low-resolution and high-resolution images. However, training in a neural network model requires a very large and balanced dataset, which is sometimes impossible or too expensive to obtain.In this article, a novel method for predicting C. elegans poses in cases of multi-worm aggregation and aggregation with noise is proposed. To solve this problem we use an improved U-Net model capable of obtaining images of the next aggregated worm posture. This neural network model was trained/validated using a custom-generated dataset with a synthetic image simulator. Subsequently, tested with a dataset of real images. The results obtained were greater than 75% in precision and 0.65 with Intersection over Union (IoU) values.

MULS-Net: A Multilevel Supervised Network for Ship Tracking From Low-Resolution Remote-Sensing Image Sequences

MULS-Net: A Multilevel Supervised Network for Ship Tracking From Low-Resolution Remote-Sensing Image Sequences

Optimal guidance for lunar soft landing with dynamic low-resolution image sequences

To relieve the burden of obtaining high-precision lunar surface information during lunar soft landing, a new guidance algorithm based on dynamic low-resolution image sequences is proposed. The integrated procedure of dynamic descent stage considering obstacle-avoiding requirement without the hover measurement is designed, which enables the spacecraft to optimize the final landing site by rolling in orbit. A small number of points are used to obtain the variance, slope and roughness of the field of vision subarea, and the shortest distance spiral search algorithm is used to determine the safest landing point. In order to reduce unnecessary control, an acceptance domain algorithm of alternative landing sites is designed to judge whether or not to accept the change of landing sites in dynamic low-precision image sequences. Assuming that the control acceleration is adjustable, the non-singular fast terminal sliding mode (NFTSM) controller based on terminal attractor is adopted and improved, and the stability of the closed-loop system is also proved. The validity, reliability and effectiveness of the proposed guidance algorithm are verified by numerical simulation on the case with disturbance and large sample data. The results demonstrate that the image information required for stable lunar soft landing is far less than that of the traditional methods, which can greatly reduce the burden of the onboard computer.

High-performance heterogeneous FPGA data-flow architecture for Fourier ptychographic microscopy.

Fourier ptychographic microscopy (FPM) is a recently developed computational imaging technique that can achieve both high-resolution and a wide field-of-view via a sequence of low-resolution images. FPM is a complex iterative process, and it is difficult to meet the needs of rapid reconstruction imaging with the conventional FPM deployed on general purpose processors. In this paper, we propose a high-performance heterogeneous field-programmable gate array (FPGA) architecture based on the principle of full pipeline and the data-flow structure for the iterative reconstruction procedure of FPM. By optimizing the architecture network at gate-level logic circuits, the running time of the FPGA-based FPM reconstruction procedure is nearly 20 times faster than conventional methods. Our proposed architecture can be used to develop FPM imaging equipment that meets resource and performance requirements.

Algorithms for multi-frame image super-resolution under applicative noise based on deep neural networks

The article describes algorithms for multi-frame image super-resolution, which recover high-resolution images from a sequence of low-resolution images of the same scene under applicative noise. Applicative noise generates local regions of outlying observations in each image and reduces the image resolution. So far, little attention has been paid to this problem. At the same time, the use of deep neural networks is considered to be a promising method of image processing, including multi-frame image super-resolution. The article considers the existing solutions to the problem and suggests a new approach based on using several pre-trained convolutional neural networks and directed acyclic graph neural networks trained by the authors. The developed approach and the algorithms based on this approach involve iterative processing of the input sequence of low-resolution images using different neural networks at different processing stages. The stages include registration of low-resolution images, their segmentation performed in order to determine regions damaged by applicative noise, and transformation performed in order to increase the resolution. The approach combines the strengths of the existing solutions while lacking their drawbacks resulting from the use of approximate mathematical data models required for the synthesis of the image processing algorithms within the statistical theory of solutions. The experimental studies demonstrated that the suggested algorithm is fully functional and allows more accurate recovery of high-resolution images than the existing analogues.

Automatic Segmentation of Caenorhabditis Elegans Skeletons in Worm Aggregations Using Improved U-Net in Low-Resolution Image Sequences

Automatic Segmentation of Caenorhabditis Elegans Skeletons in Worm Aggregations Using Improved U-Net in Low-Resolution Image Sequences

Phase-Based GLRT Detection of Moving Targets with Pixel Tracking in Low-Resolution SAR Image Sequences

Spaceborne synthetic aperture radar (SAR) can provide ground area monitoring with large coverage. However, achieving a wide observation scope comes at the cost of resolution reduction owing to the trade-off between these parameters in conventional SAR. In low-resolution imaging, the moving target appears unresolved, weakly scattered, and slow moving in the image sequence, which can be generated by the subaperture technique. This article proposes a novel moving target detection method. First, interferometric phase statistics are combined with the generalized likelihood ratio test detector. A pixel tracking strategy is further exploited to determine whether a motion signal is present. These methods rely on the approximation of both clutter and noise statistics using Gaussian distributions in a low-resolution scenario. In addition, the motion signals are imaged with a subpixel offset. The proposed method is primarily validated using four real image sequences from TerraSAR-X data, which represent two types of homogeneous areas. The results reveal that moving targets can be detected in nearby areas using this strategy. The method is compared with the stack averaged coherence change detection and particle-filter-based tracking strategies.

Towards Lifespan Automation for Caenorhabditis elegans Based on Deep Learning: Analysing Convolutional and Recurrent Neural Networks for Dead or Live Classification.

The automation of lifespan assays with C. elegans in standard Petri dishes is a challenging problem because there are several problems hindering detection such as occlusions at the plate edges, dirt accumulation, and worm aggregations. Moreover, determining whether a worm is alive or dead can be complex as they barely move during the last few days of their lives. This paper proposes a method combining traditional computer vision techniques with a live/dead C. elegans classifier based on convolutional and recurrent neural networks from low-resolution image sequences. In addition to proposing a new method to automate lifespan, the use of data augmentation techniques is proposed to train the network in the absence of large numbers of samples. The proposed method achieved small error rates (3.54% ± 1.30% per plate) with respect to the manual curve, demonstrating its feasibility.

Single-molecule localization microscopy.

Single-molecule localization microscopy (SMLM) describes a family of powerful imaging techniques that dramatically improve spatial resolution over standard, diffraction-limited microscopy techniques and can image biological structures at the molecular scale. In SMLM, individual fluorescent molecules are computationally localized from diffraction-limited image sequences and the localizations are used to generate a super-resolution image or a time course of super-resolution images, or to define molecular trajectories. In this Primer, we introduce the basic principles of SMLM techniques before describing the main experimental considerations when performing SMLM, including fluorescent labelling, sample preparation, hardware requirements and image acquisition in fixed and live cells. We then explain how low-resolution image sequences are computationally processed to reconstruct super-resolution images and/or extract quantitative information, and highlight a selection of biological discoveries enabled by SMLM and closely related methods. We discuss some of the main limitations and potential artefacts of SMLM, as well as ways to alleviate them. Finally, we present an outlook on advanced techniques and promising new developments in the fast-evolving field of SMLM. We hope that this Primer will be a useful reference for both newcomers and practitioners of SMLM.

Human Skeleton Estimation Method Focusing on the Difference of Human Position and Action from an Extremely Low-Resolution FIR Image Sequence

Elderly monitoring systems are gaining attention in the modern aging society. For the purpose, Far-InfraRed (FIR) sensors are often used, because they can avoid privacy concerns and are robust to environmental lightings. The authors have previously proposed several methods for human skeleton estimation from an extremely low-resolution FIR image sequence whose resolution is 16 × 16 pixels. For more accurate estimation, this paper proposes a method that is robust to variations of human positions and actions in the FIR sequences. Specifically, to extract features robust to the human positions from the images by using a Convolutional Neural Network (CNN), a global max-pooling layer is inserted into the last layer instead of multiple pooling layers which are not suitable for low-resolution inputs. Also, a network with two branches is introduced that focuses on capturing spatial and temporal information respectively. Moreover, the network has a weighted sum mechanism of their outputs, which depends on the human actions. For evaluation, a dataset was created by capturing action sequences of a human at various positions in the FIR images. Through an experiment, we confirmed that the human motion can be smoothly estimated and that the estimation accuracy is improved by the proposed method.

Survey of Learning Based Single Image Super-Resolution Reconstruction Technology

With the development of information technology, there is a high demand for high-resolution images. Image super-resolution reconstruction technology is to estimate a high-resolution image with better quality from one or a sequence of low-resolution images, with the help of signal processing technology. The core idea is to integrate useful information with strong correlations and complementarities from single image or multiple images as desired. Learning based single image super-resolution reconstruction technology is the current research hotspot. The paper systematically overviews this technology and discuss some main categories of it, such as super-resolution reconstruction based on neighbors, based on sparse representation, based on deep learning. At the end of the paper, challenge issues and future research directions for super-resolution image reconstruction are put forward.

Color Fourier ptychographic microscopy based on symmetrical illumination and wavelength multiplexing

Fourier ptychographic microscopy (FPM), by using a LED array as its light source and each LED being lit up sequentially, can recover a high resolution (HR) and large field of view image from a sequence of low resolution (LR) images. It is difficult to obtain a perfect high resolution color image by a monochrome camera because of non-overlapping focal planes of different color channels resulting from the residual chromatic aberration of the microscopic objective. In this paper, a kind of method, based on symmetrical LED illumination and wavelength multiplexing, is applied in a FPM system to collect LR images and recover HR color images. Compared with single LED illumination, symmetrical illumination has higher defocus tolerance. It is proven by simulated and experimental results that higher-quality HR color images can be obtained based on wavelength multiplexing and symmetrical illumination by a monochrome camera when defocus or chromatic aberration exist. It is demonstrated that symmetrical illumination is helpful in chromatic aberration correction in color FPM imaging.

Diffraction tomography with a deep image prior.

We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy.

Enhanced spatial resolution for snapshot hyperspectral imaging of blood perfusion and melanin information within human tissue.

We report a reconstruction method to achieve high spatial resolution for hyperspectral imaging of chromophore features in skin in vivo. The method utilizes an established structure-adaptive normalized convolution algorithm to reconstruct high spatial resolution of hyperspectral images from snapshot low-resolution hyperspectral image sequences captured by a snapshot spectral camera. The reconstructed images at chromophore-sensitive wavebands are used to map the skin features of interest. We demonstrate the method experimentally by mapping the blood perfusion and melanin features (moles) on the facial skin. The method relaxes the constrains of the relatively low spatial resolution in the snapshot hyperspectral camera, making it more usable in imaging applications.

Super-resolution optical mapping of floating macroalgae from geostationary orbit.

The spatial resolution of an observation from a geostationary orbiting satellite is usually too coarse to track small scale macroalgae blooms. For macroalgae mapping to benefit from a geostationary orbit's staring monitoring and frequent revisit intervals, we introduced a super-resolution method that reconstructs a high-resolution (HR) image of a region from a sequence of raw geostationary low-resolution images of the same region. We tested our method with GF-4 images at 50 m spatial resolution and demonstrated that the spatial resolution increased to 25 m. In addition, the derived HR image had better image quality characterized by a higher signal-to-noise ratio, clarity, and contrast. The increased spatial resolution and improved image quality improved our ability to distinguish macroalgae patches from the surrounding waters, especially tiny patches of macroalgae, and to precisely delineate the patch boundaries. Lastly, we more accurately estimated the areal coverage of the patches by reducing underestimation of the coverage of tiny patches and overestimation of the coverage of large patches.

Performance of super-resolution algorithms under applicative noise

The paper considers the problem of multi-frame super-resolution under applicative noise which generates distributed regions of outlying observations in low resolution images. The analysis of existing solutions is performed. They include algorithms based on spin-glass models and Markov random fields used to remove applicative noise. The authors suggest their own approach, which involves using a recurrent algorithm of quasi-linear optimal filtering of a sequence of low resolution images together with superpixel segmentation performed in order to determine the regions damaged by applicative noise. The considered algorithms are compared as applied to a set of test images. The results of the experiment demonstrate that the suggested approach allows for more accurate recovery of HR images than the existing analogues.

Error Improvement in Visual Odometry Using Super-resolution

Visual odometry (VO), a method that estimates odometry using visual sensors, is hard to operate successfully with the low-resolution and noisy image sequences. To address this problem, a super-resolution technique is applied to input data before performing VO. Since most conventional super-resolution literature mainly deals with the resolution increment, we present a novel deep neural super-resolution network, which can remove noises as well. The execution time is also taken into account by adjusting the number of CNN layers for a real-time VO. By applying the proposed super-resolution approach, the resolution increases and noises disappear with a suitable speed, hence VO can be performed successfully. Experimental results show that the proposed method improves the VO performance compared with the conventional VO which uses low-resolution and noisy image sequences.

Spatio-temporal silhouette sequence reconstruction for gait recognition against occlusion

Gait-based features provide the potential for a subject to be recognized even from a low-resolution image sequence, and they can be captured at a distance without the subject’s cooperation. Person recognition using gait-based features (gait recognition) is a promising real-life application. However, several body parts of the subjects are often occluded because of beams, pillars, cars and trees, or another walking person. Therefore, gait-based features are not applicable to approaches that require an unoccluded gait image sequence. Occlusion handling is a challenging but important issue for gait recognition. In this paper, we propose silhouette sequence reconstruction from an occluded sequence (sVideo) based on a conditional deep generative adversarial network (GAN). From the reconstructed sequence, we estimate the gait cycle and extract the gait features from a one gait cycle image sequence. To regularize the training of the proposed generative network, we use adversarial loss based on triplet hinge loss incorporating Wasserstein GAN (WGAN-hinge). To the best of our knowledge, WGAN-hinge is the first adversarial loss that supervises the generator network during training by incorporating pairwise similarity ranking information. The proposed approach was evaluated on multiple challenging occlusion patterns. The experimental results demonstrate that the proposed approach outperforms the existing state-of-the-art benchmarks.

The Spatial Resolution Enhancement for a Thermogram Enabled by Controlled Subpixel Movements

The measurement accuracy and reliability of thermography are largely limited by a relatively low spatial resolution of the thermal imager. Using a high-end camera to achieve high spatial resolution can be expensive or infeasible as a high sample rate is required. Furthermore, system miniaturization becomes an inevitable trend with the continuous development of the Internet of Things (IoT) and their suitability to in situ inspection scenarios. However, a miniaturized sensor usually suffers from low spatial resolution. Addressing this challenge, this article reports a novel spatial resolution enhancement for a thermogram (SRE4T) system to significantly improve the spatial resolution without upgrading the sensor. A high-resolution (HR) thermal image is reconstructed by fusing a sequence of low-resolution (LR) images with subpixel movements. To achieve the best image quality, instead of benefiting from natural movements of the existing studies, this article proposes to use an HR xy translation stage to produce a sequence of controlled subpixel movements. The performance of the proposed system was tested on both high-end and low-end thermal imagers. Both visual and quantitative results successfully demonstrated the considerable improvement of the quality of thermal images [up to 30.5% improvement of the peak signal-to-noise ratio (PSNR)]. This technique allows improving the measurement accuracy of thermography inspection without upgrading sensors. It also has potential to be applied on other imaging systems.

A novel method for super resolution image reconstruction

The paper proposes a technique for reconstruction of high resolution images based on surveying adjustment. The concept proposed in this technique is, an observation model defined for the sequence of low resolution images based on the observation equation proposed for the super-resolution image reconstruction. The observation equations are applied by the surveying adjustments in order to predict the grey function. The quantitative analysis of proposed technique is performed on assumed image data and real time image data. Efficiency assessment parameters like peak signal to noise ratio and sharpness index are used to analyse the performance of proposed technique with other latest techniques. In both the cases of experiments, i.e., assumed data and real time data the proposed reconstruction technique of surveying adjustment is found to be highly effective which is needed for various applications like satellite imaging, medical imaging diagnosis, military surveillance etc.