Low Resolution Image Data Research Articles

DA-YOLOv7: A Deep Learning-Driven High-Performance Underwater Sonar Image Target Recognition Model

Affected by the complex underwater environment and the limitations of low-resolution sonar image data and small sample sizes, traditional image recognition algorithms have difficulties achieving accurate sonar image recognition. The research builds on YOLOv7 and devises an innovative fast recognition model designed explicitly for sonar images, namely the Dual Attention Mechanism YOLOv7 model (DA-YOLOv7), to tackle such challenges. New modules such as the Omni-Directional Convolution Channel Prior Convolutional Attention Efficient Layer Aggregation Network (OA-ELAN), Spatial Pyramid Pooling Channel Shuffling and Pixel-level Convolution Bilat-eral-branch Transformer (SPPCSPCBiFormer), and Ghost-Shuffle Convolution Enhanced Layer Aggregation Network-High performance (G-ELAN-H) are central to its design, which reduce the computational burden and enhance the accuracy in detecting small targets and capturing local features and crucial information. The study adopts transfer learning to deal with the lack of sonar image samples. By pre-training the large-scale Underwater Acoustic Target Detection Dataset (UATD dataset), DA-YOLOV7 obtains initial weights, fine-tuned on the smaller Smaller Common Sonar Target Detection Dataset (SCTD dataset), thereby reducing the risk of overfitting which is commonly encountered in small datasets. The experimental results on the UATD, the Underwater Optical Target Detection Intelligent Algorithm Competition 2021 Dataset (URPC), and SCTD datasets show that DA-YOLOV7 exhibits outstanding performance, with mAP@0.5 scores reaching 89.4%, 89.9%, and 99.15%, respectively. In addition, the model maintains real-time speed while having superior accuracy and recall rates compared to existing mainstream target recognition models. These findings establish the superiority of DA-YOLOV7 in sonar image analysis tasks.

Statistical segmentation model for accurate electrode positioning in Parkinson's deep brain stimulation based on clinical low-resolution image data and electrophysiology.

Deep Brain Stimulation (DBS), applying chronic electrical stimulation of subcortical structures, is a clinical intervention applied in major neurologic disorders. In order to achieve a good clinical effect, accurate electrode placement is necessary. The primary localisation is typically based on presurgical MRI imaging, often followed by intra-operative electrophysiology recording to increase the accuracy and to compensate for brain shift, especially in cases where the surgical target is small, and there is low contrast: e.g., in Parkinson's disease (PD) and in its common target, the subthalamic nucleus (STN). We propose a novel, fully automatic method for intra-operative surgical navigation. First, the surgical target is segmented in presurgical MRI images using a statistical shape-intensity model. Next, automated alignment with intra-operatively recorded microelectrode recordings is performed using a probabilistic model of STN electrophysiology. We apply the method to a dataset of 120 PD patients with clinical T2 1.5T images, of which 48 also had available microelectrode recordings (MER). The proposed segmentation method achieved STN segmentation accuracy around dice = 0.60 compared to manual segmentation. This is comparable to the state-of-the-art on low-resolution clinical MRI data. When combined with electrophysiology-based alignment, we achieved an accuracy of 0.85 for correctly including recording sites of STN-labelled MERs in the final STN volume. The proposed method combines image-based segmentation of the subthalamic nucleus with microelectrode recordings to estimate their mutual location during the surgery in a fully automated process. Apart from its potential use in clinical targeting, the method can be used to map electrophysiological properties to specific parts of the basal ganglia structures and their vicinity.

DiffraGAN: a conditional generative adversarial network for phasing single molecule diffraction data to atomic resolution.

Proteins that adopt multiple conformations pose significant challenges in structural biology research and pharmaceutical development, as structure determination via single particle cryo-electron microscopy (cryo-EM) is often impeded by data heterogeneity. In this context, the enhanced signal-to-noise ratio of single molecule cryo-electron diffraction (simED) offers a promising alternative. However, a significant challenge in diffraction methods is the loss of phase information, which is crucial for accurate structure determination. Here, we present DiffraGAN, a conditional generative adversarial network (cGAN) that estimates the missing phases at high resolution from a combination of single particle high-resolution diffraction data and low-resolution image data. For simulated datasets, DiffraGAN allows effectively determining protein structures at atomic resolution from diffraction patterns and noisy low-resolution images. Our findings suggest that combining single particle cryo-electron diffraction with advanced generative modeling, as in DiffraGAN, could revolutionize the way protein structures are determined, offering an alternative and complementary approach to existing methods.

CHALLENGES IN PREPARING DATASETS FOR SUPER-RESOLUTION ON THE EXAMPLE OF SENTINEL-2 AND PLANET SCOPE IMAGES

Abstract. Benchmark datasets is an significant aspect in in many areas such as computer vision, deep learning, geospatial data as they serve as standardized test sets for evaluating the performance of models. Among many techniques of image processing, there is super-resolution (SR) which is aimed at reconstructing a low-resolution (LR) image into a high-resolution (HR) image. For training and validation SR models as a dataset the pairs of HR and LR images are needed, which should be the same apart from resolution. There is a lot of benchmark datasets for super-resolution methods, but they usually include conventional photographs of an common objects, while remote sensing data have different characteristic in general. This paper focuses on the process of preparing datasets for super-resolution in satellite images, where high-resolution and low-resolution image data come from different sources. The case of the single-image super-resolution method was considered. The experiment was performed on Sentinel-2 and PlanetScope data, but the assumptions can also be transferred to data obtained from other satellites. The procedure on how to make the pairs of HR and LR images consistent in terms of time, location and spectral values was proposed. The impact of the processes carried out was measured using image similarity measurement methods such as PSNR, SSIM and SCC.

Demographic Bias in Low-Resolution Deep Face Recognition in the Wild

Face biometrics play a primary role in smart cities, from consumer- to organizational-level applications. This class of technologies has been recently shown to emphasize performance disparities across gender and ethnic groups. Prior work on demographic bias in deep face recognition has tended to focus exclusively on high-resolution images, leaving out low-resolution images captured, for example, by a certain surveillance camera at a distance. Notable reasons for this focus include the lack of low-resolution face image data sets that report user's demographic attributes. Therefore, demographic bias in low-resolution deep face recognition in the wild still remains under-explored. In this paper, we propose a framework for exploring demographic disparities in low-resolution face recognition at a distance. To this end, we devised a deep generative approach to turn high-resolution face images to their low-resolution counterpart. We then trained state-of-the-art face recognition models on different combinations of high- and low-resolution images. Finally, we assessed the model demographic disparities on artificially degraded images from five data sets. Our results show that face images artificially degraded through our generative approach are more realistic than those obtained with other statistical changes. Key disparities across gender and ethnic groups exist and urge timely interventions. Source code: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://cutt.ly/FChLfOC</uri>

Semi-Supervised Implicit Neural Representation for Polarimetric ISAR Image Super-Resolution

Compared with the optical imaging system, polarimetric inverse synthetic aperture radar (ISAR) can work all-day and all-weather, which plays an important role in space surveillance. However, high-resolution (HR) ISAR images usually require large bandwidth and coherent integration angle, which is limited by the equipment’s physical conditions. In this vein, the super-resolution (SR) of ISAR images is of vital importance. At present, supervised learning methods are often used in image SR of computer vision. By constructing low-resolution (LR) and HR data pairs, the neural network can learn the mapping relationship between them. However, the low-frequency information in LR image data is less considered. In addition, to obtain different scales of SR reconstruction results, multiple network training repetitions are usually needed, which consumes time and hardware resources. Based on the idea of implicit neural representation, this paper constructs an implicit neural network representation framework for polarimetric ISAR image SR, which can obtain multiscale SR results through one training. A semi-supervised module is also constructed to make the network have the ability of supervised and unsupervised learning, which is conducive to mine and make better use of LR images. A polarimetric ISAR image SR dataset is constructed for satellite targets while four indexes are adopted for quantitative evaluation in global and local aspects. Experiments demonstrate that the proposed approach achieves better SR performance, where the PSNR index can be increased at least by 0.93dB.

ResDeepD: A residual super-resolution network for deep downscaling of daily precipitation over India

Abstract In the twenty-first century, machine learning and deep learning have been successfully used to find hidden information from coarse-grained data in various domains. In Computer Vision, scientists have used neural networks to identify hidden pixel-level information from low-resolution (LR) image data. This approach of estimating high-resolution (HR) information from LR data is called the super-resolution (SR) approach. This approach has been borrowed by climate scientists to downscale coarse-level measurements of climate variables to obtain their local-scale projections. Climate variables are spatial in nature and can be represented as images where each pixel denotes a grid point where the variables can be measured. We can apply the deep learning-based SR techniques on such “images” for statistical downscaling of such variables. This approach of downscaling can be termed as deep downscaling. In this work, we have tried to make HR projection of the Indian summer monsoon rainfall by using a novel deep residual network called ResDeepD. The aim is to downscale the 10 × 10 low LR precipitation data to get the values at 0.250 × 0.250 resolution. The proposed model uses a series of skip connections across residual blocks to give better results as compared to the existing models like super-resolution convolutional neural network, DeepSD, and Nest-UNet that have been used previously for this task. We have also examined the model’s performance for downscaling rainfall during some extreme climatic events like cyclonic storms and deep depression and found that the model performs better than the existing models.

Ultrasound Image Super-Resolution with Two-Stage Zero-Shot CycleGAN

Medical ultrasound imaging is widely used in clinical diagnosis because of its non-invasive, convenient and quick characteristics. However, due to its low image contrast, multiple artifacts, noise and lack of paired high-resolution and low-resolution image data sets, the task of super-resolution reconstruction of medical ultrasound images is more challenging. In this paper, the Two-Stage GAN network model was adjusted by CycleGAN generation and unsupervised learning methods, and the Two-Stage ZSSR (“Zero-Shot” Super-Resolution) CycleGAN network was proposed. The objective evaluation indexes PSNR and SSIM were raised to 40.8079 and 0.9953. The visual effect was also significantly improved.

Remote sensing image super‐resolution based on convolutional blind denoising adaptive dense connection

The current super-resolution (SR) deep network is mainly applied to the common image and pays little attention to the image with noise. The remote sensing image contains much noise, so that the SR reconstruction effect is not satisfactory. Therefore, a convolution blind denoising adaptive dense connection SR (CBD-ADCSR) network for the remote sensing image is proposed in this paper. The whole model is divided into a convolution blind denoising (CBD) network for denoising and an ADCSR network for reconstruction. Firstly, the components of the network are given in detail and are analysed. Secondly, a data set making method is designed combining motion blur, defocusing blur and Gaussian noise, which is used to generate low-resolution image data sets with complex degradation for the model training. Finally, through the detailed comparative experiment, it is proved that the reconstruction effect of the CBD-ADCSR model is better than that of the most state-of-the-art algorithms in objective criteria. In addition, compared with the original ADCSR network, CBD-ADCSR has a stronger ability for noise suppression.

Multivariate image fusion: A pipeline for hyperspectral data enhancement

Hyperspectral cameras provide high spectral resolution data, but their usual low spatial resolution when compared to color (RGB) instruments is still a limitation for more detailed studies. This article presents a simple yet powerful method for fusing co-registered high spatial and low spectral resolution image data – e.g. RGB – with low spatial and high spectral resolution data – Hyperspectral. The proposed method exploits the overlap in observed phenomena by the two cameras to create a model through least square projections. This yields two images: 1) A high-resolution image spatially correlated with the input RGB image but with more spectral information than just the 3 RGB bands. 2) A low-resolution image showing the spectral information what is spatially uncorrelated with the RGB image. We show results for semi-artificial benchmark datasets and a real-world application. Performance metrics indicate the method is well suited for data enhancement.

Development and evaluation of inexpensive automated deep learning-based imaging systems for embryology.

Embryo assessment and selection is a critical step in an in vitro fertilization (IVF) procedure. Current embryo assessment approaches such as manual microscopy analysis done by embryologists or semi-automated time-lapse imaging systems are highly subjective, time-consuming, or expensive. Availability of cost-effective and easy-to-use hardware and software for embryo image data acquisition and analysis can significantly empower embryologists towards more efficient clinical decisions both in resource-limited and resource-rich settings. Here, we report the development of two inexpensive (<$100 and <$5) and automated imaging platforms that utilize advances in artificial intelligence (AI) for rapid, reliable, and accurate evaluations of embryo morphological qualities. Using a layered learning approach, we have shown that network models pre-trained with high quality embryo image data can be re-trained using data recorded on such low-cost, portable optical systems for embryo assessment and classification when relatively low-resolution image data are used. Using two test sets of 272 and 319 embryo images recorded on the reported stand-alone and smartphone optical systems, we were able to classify embryos based on their cell morphology with >90% accuracy.

HASAN: Highly accurate sensitivity for auto-contrast-corrected pMRI reconstruction

A novel method for highly accurate coil sensitivity-map estimation, based on a constrained image-domain multi-channel LMS (c-iMCLMS) algorithm, is proposed for image reconstruction using self-calibrating SENSE. The sensitivity information is extracted by developing an image-domain cross-relation equation using the low-resolution images constructed from the fully sampled central region of the variable density MR data. Then this formulation is solved in an iterative way using a novel sum-of-squares (SOS) constraint. The improvement of the convergence speed of the c-iMCLMS algorithm is accomplished by SOS normalization of the low resolution image data and using a variable step-size in the update equation. The salient feature of the proposed technique is that it does not require any prior selection of the basis function and/or simultaneous estimation of the object image and the coil sensitivity-map. Only the low resolution images are re-filtered for the compensation of the data truncation effect to improve the consistency of the estimated coil maps. Besides, the application of the novel SOS-constraint, estimated using the pixel position-wise variance of the coil maps, gives closest to the true sensitivity-map. As a result, true object image with auto-corrected contrast is reconstructed without adopting any traditional post-contrast correction techniques. For minimization of the process noise, regularized conjugate gradient (CG) based SENSE reconstruction algorithm is used for image reconstruction using the estimated coil sensitivity-map. The proposed technique is tested on various simulation, synthetic and in-vivo datasets and significant signal-to-artifact-noise-ratio (SANR) improvement closest to the theoretical limit set by coil geometric factor is obtained as compared to some noted techniques in the literature both visually and numerically.

Selective Feature Generation Method for Classification of Low-Dimensional Data

We propose a method that generates input features to effectively classify low-dimensional data. To do this, we first generate high-order terms for the input features of the original low-dimensional data to form a candidate set of new input features. Then, the discrimination power of the candidate input features is quantitatively evaluated by calculating the ‘discrimination distance’ for each candidate feature. As a result, only candidates with a large amount of discriminative information are selected to create a new input feature vector, and the discriminant features that are to be used as input to the classifier are extracted from the new input feature vectors by using a subspace discriminant analysis. Experiments on low-dimensional data sets in the UCI machine learning repository and several kinds of low-resolution facial image data show that the proposed method improves the classification performance of low-dimensional data by generating features.

A SPIRAL-BASED DOWNSCALING METHOD FOR GENERATING 30 M TIME SERIES IMAGE DATA

Abstract. The spatial detail and updating frequency of land cover data are important factors influencing land surface dynamic monitoring applications in high spatial resolution scale. However, the fragmentized patches and seasonal variable of some land cover types (e. g. small crop field, wetland) make it labor-intensive and difficult in the generation of land cover data. Utilizing the high spatial resolution multi-temporal image data is a possible solution. Unfortunately, the spatial and temporal resolution of available remote sensing data like Landsat or MODIS datasets can hardly satisfy the minimum mapping unit and frequency of current land cover mapping / updating at the same time. The generation of high resolution time series may be a compromise to cover the shortage in land cover updating process. One of popular way is to downscale multi-temporal MODIS data with other high spatial resolution auxiliary data like Landsat. But the usual manner of downscaling pixel based on a window may lead to the underdetermined problem in heterogeneous area, result in the uncertainty of some high spatial resolution pixels. Therefore, the downscaled multi-temporal data can hardly reach high spatial resolution as Landsat data. A spiral based method was introduced to downscale low spatial and high temporal resolution image data to high spatial and high temporal resolution image data. By the way of searching the similar pixels around the adjacent region based on the spiral, the pixel set was made up in the adjacent region pixel by pixel. The underdetermined problem is prevented to a large extent from solving the linear system when adopting the pixel set constructed. With the help of ordinary least squares, the method inverted the endmember values of linear system. The high spatial resolution image was reconstructed on the basis of high spatial resolution class map and the endmember values band by band. Then, the high spatial resolution time series was formed with these high spatial resolution images image by image. Simulated experiment and remote sensing image downscaling experiment were conducted. In simulated experiment, the 30 meters class map dataset Globeland30 was adopted to investigate the effect on avoid the underdetermined problem in downscaling procedure and a comparison between spiral and window was conducted. Further, the MODIS NDVI and Landsat image data was adopted to generate the 30m time series NDVI in remote sensing image downscaling experiment. Simulated experiment results showed that the proposed method had a robust performance in downscaling pixel in heterogeneous region and indicated that it was superior to the traditional window-based methods. The high resolution time series generated may be a benefit to the mapping and updating of land cover data.

MRI image enhancement using Biot--Savart law at 3 tesla

Coil sensitivity is considered as the most valuable data in the parallel reconstruction of magnetic resonance imaging (MRI). In this study, a novel coil sensitivity map extraction method is introduced for spatially fixed phased array coils. The proposed technique uses the Biot--Savart law with coil internal shape information and low-resolution phase image data to form sensitivity maps. The performance of this method is tested in a parallel image reconstruction task, using the sensitivity-encoding (SENSE) technique. Under the quasi-static assumption and using the duality principle, we computed the sensitivity maps of a phased-array head coil and reconstructed full FOV images. The experiments show that the resulting image quality is higher in terms of sharpness, artifact level, homogeneity, etc. than existing methods that use sensitivity maps calculated only from low-resolution image data. Moreover, several simulations were conducted using an electromagnetic simulation software tool to theoretically prove the success of the proposed technique. Our simulation results indicate that the success of the method depends heavily on the size of the coil elements. A new method for calculation of sensitivity maps is briefly introduced, which increases image quality and homogeneity while reducing the artifacts.

Statistical estimation of femur micro-architecture using optimal shape and density predictors

The personalization of trabecular micro-architecture has been recently shown to be important in patient-specific biomechanical models of the femur. However, high-resolution in vivo imaging of bone micro-architecture using existing modalities is still infeasible in practice due to the associated acquisition times, costs, and X-ray radiation exposure. In this study, we describe a statistical approach for the prediction of the femur micro-architecture based on the more easily extracted subject-specific bone shape and mineral density information. To this end, a training sample of ex vivo micro-CT images is used to learn the existing statistical relationships within the low and high resolution image data. More specifically, optimal bone shape and mineral density features are selected based on their predictive power and used within a partial least square regression model to estimate the unknown trabecular micro-architecture within the anatomical models of new subjects. The experimental results demonstrate the accuracy of the proposed approach, with average errors of 0.07 for both the degree of anisotropy and tensor norms.

Visualizing the temporal development of thermo-radiative features on ground-based thermographs

In urban microclimate research, ground-based thermography is used to gain insight into the spatial distribution of surface temperatures of various materials. Taking snapshots over a certain time span helps experts to observe the temporal thermo-radiative behavior of the monitored surface elements and therefore supports decisions on possible optimizations, e.g., improving the thermal comfort in a neighborhood. Appropriate visualization techniques facilitate decision-making and are thus crucial in the optimization process. In this study, we present a tool that eases the extraction of thermo-radiative features from multi-temporal thermographs taken from a monitored scene. Assisted by our tool, users can identify, choose, and register thermo-radiative features for each time step according to their individual research needs. The features’ temporal development is then visualized using a directed graph that encodes topological events as well as each feature’s size and summarizing statistics. To enhance this summary, a comprehensive animated sequence emphasizes the spatiotemporal behavior of the most significant thermo-radiative features. Salient developments are visually embedded and highlighted in the original infrared images, which are blended in an animation from time step to time step. Since we enable the user to interact with the data in a flexible way, noisy and low resolution image data sets can also be processed.

Spectral modeling based on ground measurements for mine tailing mapping with Landsat ETM+ imagery

In the north of Tunisia, mine tailings cause multiple environmental consequences in the Mejerda watershed. In this paper, we are interested in mine tailing mapping around the Jebel (Hill) Hallouf-Bouaouane mine. Several ways to estimate the spectral reflectance for end-members in a mixture problem have been proposed in the literature such as pure pixels, spectral library, and field measurements. One of the most common approaches consists in the use of spectral library. This approach presents the advantage of using pure end-members and replacing field measurements. However, it is generally not possible in the case of low spectral resolution image data, due to the large spectral range covered by mineral reflectances. In this paper, a methodology aiming to overcome this limitation is proposed. The proposed approach makes use of the spectral linear mixing model. In the proposed methodology, the spectral component of tailing end-member is estimated using a fusion of available mineral spectra. This spectral modeling approach is based on laboratory analysis as well as on the use of spectral library data. The performance of this method is tested through the comparison of resulting tailing maps using both modeled and Landsat ETM+-derived end-members. The best results were obtained with the JPL spectral library data.

3-D-Skeleton-Based Head Detection and Tracking Using Range Images

Vision-based 3-D head detection and tracking systems have been studied in several applications like video surveillance, face-detection systems, and occupant posture analysis. In this paper, we present the development of a topology-based framework using a 3-D skeletal model for the robust detection and tracking of a vehicle occupant's head position from low-resolution range image data for a passive safety system. Unlike previous approaches to head detection, the proposed approach explores the topology information of a scene to detect the position of the head. Among the different available topology representations, the Reeb graph technique is chosen and is adapted to low-resolution 3-D range images. Invariance of the graph under rotations is achieved by using a Morse radial distance function. To cope with the particular challenges such as the noise and the large variations in the density of the data, a voxel neighborhood connectivity notion is proposed. A multiple-hypothesis tracker (MHT) with nearest-neighbor data association and Kalman filter prediction is applied on the endpoints of the Reeb graph to select and filter the correct head candidate out of Reeb graph endpoints. A systematic evaluation of the head detection framework is carried out on full-scale experimental 3-D range images and compared with the ground truth. It is shown that the Reeb graph topology algorithm developed herein allows the correct detection of the head of the occupant with only two head candidates as input to the MHT. Results of the experiments demonstrate that the proposed framework is robust under the large variations of the scene. The processing requirements of the proposed approach are discussed. It is shown that the number of operations is rather low and that real-time processing requirements can be met with the proposed method.

Predicting Water-Surface Fluctuation of Continental Lakes: A RS and GIS Based Approach in Central Mexico

Changes in the water-surface area occupied by the Cuitzeo Lake, Mexico, during the 1974–2001 period are analysed in this study. The research is based on remote sensing and geographic information techniques, as well as statistical analysis. High-resolution satellite image data were used to analyse the 1974–2000 period, and very low-resolution satellite image data were used for the 1997–2001 period. The long-term analysis (1974–2000) indicated that there were temporal changes in the surface area of the Cuitzeo Lake and that these changes were related to precipitation and temperatures that occurred in the previous year. Short-term monitoring (1997–2001) showed that the Cuitzeo Lake surface is lowering. Field observations demonstrated also that yearly desiccation is recurrent, particularly, in the western section of the lake. Results suggested that this behaviour was probably due to a drought period in the basin that began in the mid 1990s. Regression models constructed from long-term data showed that fluctuations of lake level can be estimated by monthly mean precipitation and temperatures of the previous year.