Generation Of High-resolution Images Research Articles

Unveiling Details in the Dark: Simultaneous Brightening and Zooming for Low-Light Image Enhancement

Existing super-resolution methods exhibit limitations when applied to nighttime scenes, primarily due to their lack of adaptation to low-pair dynamic range and noise-heavy dark-light images. In response, this research introduces an innovative customized framework to simultaneously Brighten and Zoom in low-resolution images captured in low-light conditions, dubbed BrZoNet. The core method begins by feeding low-light, low-resolution images, and their corresponding ground truths into the Retinex-induced siamese decoupling network. This process yields distinct reflectance maps and illuminance maps, guided by supervision from the ground truth’s decomposition maps. Subsequently, these reflectance and illuminance maps transition into an intricate super-resolution sub-network. This sub-network employs a meticulously designed cross-layer content-aware interactor - Illumination-aware Interaction Unit(IaIU), elegantly endowed with a gating mechanism. The IaIU facilitates meaningful feature interaction between illuminance and reflectance features while effectively reducing unwanted noise. An intricate super-resolution cage is also constructed to comprehensively integrate information, ultimately resulting in the generation of high-resolution images featuring intricate details. Thorough and diverse experiments validate the superiority of the proposed BrZoNet, surpassing contemporary cutting-edge technologies by proficiently augmenting brightness and intricately recovering complex details, showcasing advancements of 7.1% in PSNR, 2.4% in SSIM, and an impressive 36.8% in LPIPS metrics.

이미지 자동 생성 인공지능의 원리와 현황

This study identifies the principles of image generation artificial intelligence, examines its performance and limitations, and examines the innovative changes that have occurred in image production methods. Specifically, we reveal the structure of the Generative Adversarial Network (GAN), trace its technical improvement process in detail, and classify the types of subsequent models developed later. The tasks performed by GAN’s successor models are diverse. GAN is steadily evolving, completing amazing tasks that were impossible with existing generative models, such as high-resolution image generation, image substitution such as drawings and photos, and face synthesis. The principles of GAN are, first, data learning through adversarial competition between generator and discriminator, second, calculation of accurate probability distribution through Maximum Likelihood Estimation, and third, calculation of fine features of images through deep convolution structure. The most important characteristic of GAN as a generative model is its ability to create realistic images similar to photographs. Humans have been producing images using hands and tools, but with the invention of photography, humans began producing images using camera devices. Now, generative artificial intelligence is becoming another means of image production as technology continues to improve rapidly.

Appearance and Pose-guided Human Generation: A Survey

Appearance and pose-guided human generation is a burgeoning field that has captured significant attention. This subject’s primary objective is to transfer pose information from a target source to a reference image, enabling the generation of high-resolution images or videos that seamlessly link the virtual and real worlds, leading to novel trends and applications. This survey thoroughly illustrates the task of appearance and pose-guided human generation and comprehensively reviews mainstream methods. Specifically, it systematically discusses prior information, pose-based transformation modules, and generators, offering a comprehensive understanding and discussion of each mainstream pose transformation and generation process. Furthermore, the survey explores current applications and future challenges in the domain. Its ultimate goal is to serve as quick guidelines, providing practical assistance in human generation and its diverse applications.

Efficient Convolutional Dataflows on Low-Power Neural Network Accelerators

Dilated and transposed convolutions are widely used in modern convolutional neural networks (CNNs). These kernels are used extensively during CNN training and inference of applications such as image segmentation and high-resolution image generation. We find that commonly-used low-power CNN inference accelerators are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">not</i> optimized for both these convolutional kernels. Dilated and transposed convolutions introduce significant zero padding when mapped to the underlying spatial architecture, significantly degrading performance and energy efficiency. Existing approaches that address this issue require significant design changes to the otherwise simple, efficient, and well-adopted architectures used to compute direct convolutions. To address this challenge, we propose EcoFlow, a new set of dataflows and mapping algorithms for dilated and transposed convolutions. These algorithms are tailored to execute efficiently on existing low-cost, small-scale spatial architectures and requires minimal changes to existing accelerators. At its core, EcoFlow eliminates zero padding through careful dataflow orchestration and data mapping tailored to the spatial architecture. We evaluate EcoFlow on CNN training workloads and Generative Adversarial Network (GAN) workloads. Experiments in our new cycle-accurate simulator show that, using a common CNN inference accelerator, EcoFlow 1) reduces end-to-end CNN training time between 7-85%, and 2) improves end-to-end GAN training performance between 29-42%, compared to state-of-the-art CNN dataflows.

Kiosk 9R-TC-02 - Enhancing Disease Classification in Cardiac Magnetic Resonance Imaging Through High-resolution Image Generation with Deep Learning

Kiosk 9R-TC-02 - Enhancing Disease Classification in Cardiac Magnetic Resonance Imaging Through High-resolution Image Generation with Deep Learning

Mitigating Label Noise in GANs via Enhanced Spectral Normalization

Label noise is a ubiquitous issue in GANs, which degrades the generalization ability of the discriminator and usually leads to instability when training GANs. This issue stems from both real data and generated data. Previous works either only consider one of these two sources, or are not robust enough to noisy labels. In this paper, we revisit spectral normalization in robust learning with noisy labels. Based on its pros and cons, we propose to combine spectral normalization and weight decay to regularize the discriminator, which enjoys a more robust training process. To extend to conditional GANs, we propose to balance the relative importance of marginal matching and conditional matching in the projection discriminator. The proposed Enhanced Spectral Normalization for Generative Adversarial Networks (ESNGAN) can be easily integrated into various existing GANs frameworks without excessive additional cost. The effectiveness of the proposed method is validated on the CIFAR10, LSUN Church, CelebA, and ImageNet datasets, including the unconditional image generation task and the class-conditional image generation task. We also show that the proposed method can further improve the performance of the high-resolution image generation task.

Single Image based Super Resolution Ultrasound Imaging Using Residual Learning of Wavelet Features.

The generation of super resolution ultrasound images from the low-resolution (LR) brightness mode (B-mode) images acquired by the portable point of care ultrasound systems has been of sufficient interest in the recent past. With the advancements in deep learning, there have been numerous attempts in this direction. However, all the approaches have been concentrated on employing the direct image as the input to the neural network. In this work, a stationary wavelet (SWT) decomposition is employed to extract the features from the input LR image which is passed through a modified residual network and the learned features are combined using the inverse SWT to reconstruct the high resolution (HR) image at a 4× scale factor. The proposed approach when compared to the state-of-the art approaches, results in an improved high resolution reconstruction.Clinical relevance- The proposed approach will enable the generation of high-resolution images from portable ultrasound systems, allowing for easier interpretation and faster diagnostics in primary care settings.

Exploring the Effectiveness of Mask-Guided Feature Modulation as a Mechanism for Localized Style Editing of Real Images (Student Abstract)

The success of Deep Generative Models at high-resolution image generation has led to their extensive utilization for style editing of real images. Most existing methods work on the principle of inverting real images onto their latent space, followed by determining controllable directions. Both inversion of real images and determination of controllable latent directions are computationally expensive operations. Moreover, the determination of controllable latent directions requires additional human supervision. This work aims to explore the efficacy of mask-guided feature modulation in the latent space of a Deep Generative Model as a solution to these bottlenecks. To this end, we present the SemanticStyle Autoencoder (SSAE), a deep Generative Autoencoder model that leverages semantic mask-guided latent space manipulation for highly localized photorealistic style editing of real images. We present qualitative and quantitative results for the same and their analysis. This work shall serve as a guiding primer for future work.

UAV Aerial Image Generation of Crucial Components of High-Voltage Transmission Lines Based on Multi-Level Generative Adversarial Network

With the aim of improving the image quality of the crucial components of transmission lines taken by unmanned aerial vehicles (UAV), a priori work on the defective fault location of high-voltage transmission lines has attracted great attention from researchers in the UAV field. In recent years, generative adversarial nets (GAN) have achieved good results in image generation tasks. However, the generation of high-resolution images with rich semantic details from complex backgrounds is still challenging. Therefore, we propose a novel GANs-based image generation model to be used for the critical components of power lines. However, to solve the problems related to image backgrounds in public data sets, considering that the image background of the common data set CPLID (Chinese Power Line Insulator Dataset) is simple. However, it cannot fully reflect the complex environments of transmission line images; therefore, we established an image data set named “KCIGD” (The Key Component Image Generation Dataset), which can be used for model training. CFM-GAN (GAN networks based on coarse–fine-grained generators and multiscale discriminators) can generate the images of the critical components of transmission lines with rich semantic details and high resolutions. CFM-GAN can provide high-quality image inputs for transmission line fault detection and line inspection models to guarantee the safe operation of power systems. Additionally, we can use these high-quality images to expand the data set. In addition, CFM-GAN consists of two generators and multiple discriminators, which can be flexibly applied to image generation tasks in other scenarios. We introduce a penalty mechanism-related Monte Carlo search (MCS) approach in the CFM-GAN model to introduce more semantic details in the generated images. Moreover, we presented a multiscale discriminator structure according to the multitask learning mechanisms to effectively enhance the quality of the generated images. Eventually, the experiments using the CFM-GAN model on the KCIGD dataset and the publicly available CPLID indicated that the model used in this work outperformed existing mainstream models in improving image resolution and quality.

Hybrid Quantum–Classical Generative Adversarial Network for High-Resolution Image Generation

Hybrid Quantum–Classical Generative Adversarial Network for High-Resolution Image Generation

High Resolution Solar Image Generation Using Generative Adversarial Networks

We applied Deep Learning algorithm known as Generative Adversarial Networks (GANs) to perform solar image-to-image translation. That is, from Solar Dynamics Observatory (SDO)/Helioseismic and Magnetic Imager (HMI) line of sight magnetogram images to SDO/Atmospheric Imaging Assembly (AIA) 0304-Å images. The Ultraviolet (UV)/Extreme Ultraviolet observations like the SDO/AIA 0304-Å images were only made available to scientists in the late 1990s even though the magnetic field observations like the SDO/HMI have been available since the 1970s. Therefore, by leveraging Deep Learning algorithms like GANs we can give scientists access to complete datasets for analysis. For generating high resolution solar images, we use the Pix2PixHD and Pix2Pix algorithms. The Pix2PixHD algorithm was specifically designed for high resolution image generation tasks, and the Pix2Pix algorithm is by far the most widely used image to image translation algorithm. For training and testing we used the data for the year 2012, 2013 and 2014. After model training, we evaluated the model on the test data. The results show that our deep learning models are capable of generating high resolution (1024 × 1024 pixels) SDO/AIA0304 images from SDO/HMI line of sight magnetograms. Specifically, the pixel-to-pixel Pearson Correlation Coefficient of the images generated by Pix2PixHD and original images is as high as 0.99. The number is 0.962 if Pix2Pix is used to generate images. The results we get for our Pix2PixHD model is better than the results obtained by previous works done by others to generate SDO/AIA 0304 images. Thus, we can use these models to generate AIA0304 images when the AIA0304 data is not available which can be used for understanding space weather and giving researchers the capability to predict solar events such as Solar Flares and Coronal Mass Ejections. As far as we know, our work is the first attempt to leverage Pix2PixHD algorithm for SDO/HMI to SDO/AIA0304 image-to-image translation.

Efficient Simulation for Visual Comfort Evaluations

This paper provides a validation of a novel sampling, storage, and evaluation method named raytraverse that can quickly and accurately compute glare and visual comfort metrics including vertical illuminance (Ev), Daylight Glare Probability (DGP), and Unified Glare Probability (UGP). The motivation is to provide a path towards understanding the spatial and temporal distribution of daylight conditions in an architectural space. Current spatial temporal simulation workflows are constrained by the trade-offs between simulation time, accuracy, generality, and storage requirements. Raytraverse provides a bridge between illuminance sensor calculations, which are fast to calculate but provide limited information, and high-resolution image generation, which provide more information but have long simulation times. To make this bridging possible, it relies on a pair of strategies that yields both high accuracy and high information data. First, an iteratively guided sampling approach based on the discrete wavelet transformation greatly reduces the number of view rays submitted to the renderer. Second, rather than collecting returned values in a raster image or dense matrix, a spatial data structure is used to both sparsely store and re-weight the samples according to their effective solid angle, allowing for the direct integration of any view based lighting metric and the synthesis of interpretable high dynamic range images (HDRi). These strategies can be coupled with existing rendering and climate based daylight modeling (CBDM) methods. Through a comparison with high-quality reference simulations and a best practice CBDM method based on Radiance, the raytraverse methods are shown to significantly reduce the simulation time needed to accurately simulate saturation, contrast, and combined visual comfort metrics for a complete set of annual hourly sky conditions from a range of locations within an office floor plan. The stored simulation data can be quickly re-analyzed for different view directions, metrics or images, and sky conditions.

GL-GAN: Adaptive global and local bilevel optimization for generative adversarial network

Although Generative Adversarial Networks (GAN) have shown remarkable performance in image generation, there exist some challenges in instability and convergence speed. During the training, the results of some models display the imbalances of quality within a generated image, in which some defective parts appear compared with other regions. Different from general single global optimization methods, we introduce an adaptive global and local bilevel optimization model (GL-GAN). The model achieves the generation of high-resolution images in a complementary and promoting way, where global optimization is to optimize the whole images and local is only to optimize the low-quality areas. Based on DCGAN, GL-GAN is able to effectively avoid the nature of imbalance by local bilevel optimization, which is accomplished by first locating low-quality areas and then optimizing them. Moreover, through feature map cues from discriminator output, we propose the adaptive local and global optimization method (Ada-OP) for interactive optimization and observe that it boosts the convergence speed. Compared with the current GAN methods, our model has shown impressive performance on CelebA, Oxford Flowers, CelebA-HQ and LSUN datasets.

An Automated Approach to Improve the Quantification of Pericytes and Microglia in Whole Mouse Brain Sections.

Whole slide scanning technology has enabled the generation of high-resolution images from complete tissue sections. However, commonly used analysis software is often unable to handle the large data files produced. Here, we present a method using the open-source software QuPath to detect, classify and quantify fluorescently-labeled cells (microglia and pericytes) in whole coronal brain tissue sections. Whole-brain sections from both male and female NG2DsRed x CX3CR1+/GFP mice were analyzed. Small regions of interest were selected and manual counts were compared with counts generated from an automated approach, across a range of detection parameters. The optimal parameters for detecting cells and classifying them as microglia or pericytes in each brain region were determined and applied to annotations corresponding to the entire somatosensory and motor cortices, hippocampus, thalamus, and hypothalamus in each section. 3.74% of all detected cells were classified as pericytes; however, this proportion was significantly higher in the thalamus (6.20%) than in other regions. In contrast, microglia (4.51% of total cells) were more abundant in the cortex (5.54%). No differences were detected between male and female mice. In conclusion, QuPath offers a user-friendly solution to whole-slide image analysis which could lead to important new discoveries in both health and disease.

Small UAV-based SAR system using low-cost radar, position, and attitude sensors with onboard imaging capability

AbstractThis paper describes a small unmanned aerial vehicle (UAV)-based synthetic aperture radar (SAR) system using low-cost radar (5–6 GHz), position (GNSS/RTK) and attitude (IMU) sensors for the generation of high-resolution images. Measurements using straight as well as highly curved flight trajectories and varying flight speeds are presented, showing range and cross-range lobe-widths close to the theoretical limits. An analysis of the improvements obtained by the use of attitude angles (roll, pitch, and yaw), to correct for the relative offsets in antenna positions as the UAV moves, is included. A capability to generate SAR images onboard with the back-projection algorithm has been implemented using a GPU accelerated single-board computer. Generated images are transmitted to ground using a Wi-Fi data link.

Cooperation: A new force for boosting generative adversarial nets with dual‐networkstructure

The principle of generative adversarial net is to fit the given data distribution by combining a generative model and discriminative model. There are two major challenges to conventional systems - they are difficult to train and they easily fall into `mode collapse'. To improve it, this study describes a novel network structure with dual generators. A `cooperation' mechanism is introduced to help the generators work together. During training, generators not only learn from discriminative feedback but also from each other (like a study group). Compared with a single-generator network, a dual-generator network could capture many more `modes' and eventually reduce the impact of `mode collapse.' Dual networks also require extra computational resources. However, our experiment shows that even with network parameters of similar size, dual networks still achieved better results. Additionally, a dual-generator structure could be extended to multiple generators. The proposed network structure is also very robust and flexible. It can be adapted to various application scenarios, such as high-resolution image generation, domain adaptation and 3D model generation. The experimental results showed that with the same computing resources, multiple generators can generate better quality synthetic data, including 2D images, 3D objects, style transferring etc.

SubSpace Capsule Network

Convolutional neural networks (CNNs) have become a key asset to most of fields in AI. Despite their successful performance, CNNs suffer from a major drawback. They fail to capture the hierarchy of spatial relation among different parts of an entity. As a remedy to this problem, the idea of capsules was proposed by Hinton. In this paper, we propose the SubSpace Capsule Network (SCN) that exploits the idea of capsule networks to model possible variations in the appearance or implicitly-defined properties of an entity through a group of capsule subspaces instead of simply grouping neurons to create capsules. A capsule is created by projecting an input feature vector from a lower layer onto the capsule subspace using a learnable transformation. This transformation finds the degree of alignment of the input with the properties modeled by the capsule subspace.We show that SCN is a general capsule network that can successfully be applied to both discriminative and generative models without incurring computational overhead compared to CNN during test time. Effectiveness of SCN is evaluated through a comprehensive set of experiments on supervised image classification, semi-supervised image classification and high-resolution image generation tasks using the generative adversarial network (GAN) framework. SCN significantly improves the performance of the baseline models in all 3 tasks.

Deep learning approach to classification of lung cytological images: Two-step training using actual and synthesized images by progressive growing of generative adversarial networks.

Cytology is the first pathological examination performed in the diagnosis of lung cancer. In our previous study, we introduced a deep convolutional neural network (DCNN) to automatically classify cytological images as images with benign or malignant features and achieved an accuracy of 81.0%. To further improve the DCNN’s performance, it is necessary to train the network using more images. However, it is difficult to acquire cell images which contain a various cytological features with the use of many manual operations with a microscope. Therefore, in this study, we aim to improve the classification accuracy of a DCNN with the use of actual and synthesized cytological images with a generative adversarial network (GAN). Based on the proposed method, patch images were obtained from a microscopy image. Accordingly, these generated many additional similar images using a GAN. In this study, we introduce progressive growing of GANs (PGGAN), which enables the generation of high-resolution images. The use of these images allowed us to pretrain a DCNN. The DCNN was then fine-tuned using actual patch images. To confirm the effectiveness of the proposed method, we first evaluated the quality of the images which were generated by PGGAN and by a conventional deep convolutional GAN. We then evaluated the classification performance of benign and malignant cells, and confirmed that the generated images had characteristics similar to those of the actual images. Accordingly, we determined that the overall classification accuracy of lung cells was 85.3% which was improved by approximately 4.3% compared to a previously conducted study without pretraining using GAN-generated images. Based on these results, we confirmed that our proposed method will be effective for the classification of cytological images in cases at which only limited data are acquired.

HMCES Functions in the Alternative End-Joining Pathway of the DNA DSB Repair during Class Switch Recombination in B Cells

(Molecular Cell 77, 384–394.e1–e4; January 16, 2020) The authors report two minor errors in Figure 3G of the originally published article. In this panel, there appeared a letter “E” that was not relevant to the figure and may have been confusing to readers. Also, in the same panel, the authors report having inadvertently copied an older version of the bar graph, with fewer replicates, which is different from the one provided to reviewers in the revised version of the manuscript. These errors were made during the generation of high-resolution images toward the end of the production process and have since been corrected online. The errors do not change the message of the figure, nor do they impact the overall conclusions of the article. The authors apologize for any inconvenience.Figure 3GHMCES Deficiency Leads to a Defect in CSR in the CH12 B Cell Line (original)View Large Image Figure ViewerDownload Hi-res image Download (PPT) HMCES Functions in the Alternative End-Joining Pathway of the DNA DSB Repair during Class Switch Recombination in B CellsShukla et al.Molecular CellDecember 2, 2019In BriefHMCES (5hmC binding, embryonic stem cell-specific protein), originally found to bind 5-hydroxymethylcytosine (5hmC), was recently reported to covalently crosslink to DNA at abasic sites. Shukla et al. show that HMCES specifically enables DNA double-strand break repair through the microhomology-mediated alternative-end-joining (Alt-EJ) pathway during class switch recombination (CSR) in B cells. Full-Text PDF Open Archive

Instance Mask Embedding and Attribute-Adaptive Generative Adversarial Network for Text-to-Image Synthesis

Existing image generation models have achieved the synthesis of reasonable individuals and complex but low-resolution images. Directly from complicated text to high-resolution image generation still remains a challenge. To this end, we propose the instance mask embedding and attribute-adaptive generative adversarial network (IMEAA-GAN). Firstly, we use the box regression network to compute a global layout containing the class labels and locations for each instance. Then the global generator encodes the layout, combines the whole text embedding and noise to preliminarily generate a low-resolution image; the instance embedding mechanism is used firstly to guide local refinement generators obtain fine-grained local features and generate a more realistic image. Finally, in order to synthesize the exact visual attributes, we introduce the multi-scale attribute-adaptive discriminator, which provides local refinement generators with the specific training signals to explicitly generate instance-level features. Extensive experiments based on the MS-COCO dataset and the Caltech-UCSD Birds-200-2011 dataset show that our model can obtain globally consistent attributes and generate complex images with local texture details.