Original Image Content Research Articles

A unified noise and watermark removal from information bottleneck-based modeling

Both image denoising and watermark removal aim to restore a clean image from an observed noisy or watermarked one. The past research consists of the non-learning type with limited effectiveness or the learning types with limited interpretability. To address these issues simultaneously, we propose a method to deal with both the image-denoising and watermark removal tasks in a unified approach. The noises and watermarks are both considered to have different nuisance patterns from the original image content, therefore should be detected by robust image analysis. The unified detection method is based on the well-known information bottleneck (IB) theory and the proposed SIB-GAN where image content and nuisance patterns are well separated by a supervised approach. The IB theory guides us to keep the valuable content such as the original image by a controlled compression on the input (the noisy or watermark-included image) and then only the content without the nuisances can go through the network for effective noise or watermark removal. Additionally, we adjust the compression parameter in IB theory to learn a representation that approaches the minimal sufficient representation of the image content. In particular, to deal with the non-blind noises, an appropriate amount of compression can be estimated from the solid theory foundation. Working on the denoising task given the unseen data with blind noises also shows the model’s generalization power. All of the above shows the interpretability of the proposed method. Overall, the proposed method has achieved promising results across three tasks: image denoising, watermark removal, and mixed noise and watermark removal, obtaining resultant images very close to the original image content and owning superior performance to almost all state-of-the-art approaches that deal with the same tasks.

GAN‐Based Multi‐Decomposition Photo Cartoonization

AbstractBackgroundCartoon images play a vital role in film production, scientific and educational animation, video games, and other fields, and are one of the key visual expressions of artistic creation. However, since hand‐crafted cartoon images often require a great deal of time and effort on the part of professional artists, it is necessary to be able to automatically transform real‐world images into different styles of cartoon images. Although cartoon images vary from artist to artist, cartoon images generally have the unique characteristics of being highly simplified and abstract, with clear edges, smooth color shading, and relatively simple textures. However, existing image cartoonization methods tend to create a number of problems when performing style transfer, which mainly include: (1) the resulting generated images do not have obvious cartoon‐style textures; and (2) the generated images are prone to structural confusion, color artifacts, and loss of the original image content. Therefore, it is also a great challenge in the field of image cartoonization to be able to make a good balance between style transfer and content keeping.MethodsIn this paper, we propose a GAN‐based multi‐attention mechanism for image cartoonization to address the above issues. The method combines the residual block used to extract deep network features in the generator with the attention mechanism, and further strengthens the perceptual ability of the generative model to cartoon images through the adaptive feature correction of the attention module to improve the cartoon features of the generated images. At the same time, we also introduce the attention mechanism in the convolution block of the discriminator, which is used to further reduce the image visual quality problem caused by the style transfer process. By introducing the attention mechanism into the generator and discriminator models of the generative adversarial network, our method enables the generated images to have obvious cartoon‐style features while effectively improving the image's visual quality.ResultsA large number of quantitative, qualitative, and ablation experiments are conducted to demonstrate the advantages of our method in the field of image cartoonization and the role of each module in the method.

A Secret JPEG Image Sharing Method Over GF(2 M ) Galois Fields

With the rise of exchanges over the cloud and social networks, JPEG images have taken an important place in world image transmission and storage. In order to avoid security breaches and combat threats on the Internet, numerous JPEG image security methods have been proposed in both the academic and industrial communities. Encryption methods have been specifically designed to make JPEG images visually secure using so called crypto-compression methods. The drawback of crypto-compression is that it depends on only one secret key. Indeed, if this key is lost, so is the totality of the content of the secret original image. In this paper, we propose a secret JPEG image sharing approach. Shamir’s secret sharing scheme over Galois fields is used during the JPEG Huffman coding step, ensuring the visual security of the secret image in the compressed domain, while solving the issue of secret key loss. In addition, we also describe an eco-friendly scenario, dealing with a public shared JPEG image. In this scenario, we can obtain compressed shares because there is no need to duplicate the redundant information. According to our obtained results, our approach is fully format compliant and size preserving when compared to a standard JPEG compression of a secret original image, while ensuring the visual security of the content of the secret original image.

Research advanced in deep learning-based image restoration

In the computer vision community, which tries to address issues with image data change, loss, and missing that may arise in the process of image collecting, transmission, and preprocessing, image restoration has always been a hot study topic.Learning the contextual information of the visible regions in an image is the basic idea of image restoration algorithms, which perform noise removal, missing region repair and transformation on the regions to be restored to reproduce the original image content. Most of the early image restoration algorithms rely on the design of manual features, whose accuracy and generalization ability can not meet the practical application requirements. Due to the rapid advancement of CNN, in recent years, image repair accuracy and effect have made breakthroughs, and have been widely used in old photos, movies, videos and other repair scenes. In this paper, three restoration algorithms (sequence-based restoration, CNN-based restoration and GAN-based restoration) are comprehensively introduced while the research progress of image restoration based on depth learning is reviewed. Secondly, we quantitatively compare the performance of different repair algorithms and analyze their advantages and disadvantages. Finally, we summarize the challenges facing the area of image restoration research and evaluate its potential future directions.

An Adaptive Richardson-Lucy Algorithm for Medical Image Restoration

Image restoration is the process of estimating the original image content from a degraded picture. In this paper, the Richardson-Lucy iterative algorithm was developed to improve the quality of degraded medical images. It has been assumed that medical images are exposed to two types of degradation. The first type is the blur function in the Gaussian form with different widths, i.e. σ = 1 , 2, and 3. The second type of degradation was assumed to be of the independent white Gaussian noise type with different signal-to-noise ratio values: SNR = 10, 50 , and 100. The results obtained from the adaptive filter are compared, quantitatively, with different conventional filters: inverse, Wiener, and constraint least square, by applying different measures, such as: power signal to noise ratio (PSNR), structural similarity index (SSID), and root mean square error (RMSE). The comparison showed that the adaptive recovery filter achieves better results.

Detection of Recolored Image by Texture Features in Chrominance Components

Image recoloring is an emerging editing technique that can change the color style of an image by modifying pixel values without altering the original image content. With the rapid proliferation of social network and image editing techniques, recolored images (RIs) have raised new security issues in society. Existing detection methods have good performance in detecting RIs for certain categories of recoloring techniques. However, the performance on the handcrafted recoloring scenario is still poor due to the influence of human prior knowledge. To deal with this problem, we explore a solution from the perspective of chrominance texture artifacts to improve the generalization ability. The results of the analysis show that natural images (NIs) and RIs have textural disparities in different color components, especially in the chrominance components (i.e., Cb, Cr, and H). Based on such new prior knowledge of statistical discriminability, we propose a feature set to capture texture features in chrominance components for identifying RIs. Extensive experimental results show that the proposed method can accurately identify RIs with certain categories of recoloring techniques, and outperforms existing methods in the scenario of handcrafted recoloring.

Towards faithful neural fusion to infrared and visible images with a full-scale connected network

The aim of image fusion is to merge image information from multiple source images into one image. The fused image can contain more complete and clear scene information without introducing artifacts. The results of image fusion are required to be faithful to the original image contents. However, we notice that the unfaithful fusions often contain hallucinated content, which cannot be aligned with any input images. For the first time, we propose a full-scale connected-based fusion network (FSCF-Net) for infrared and visible image fusion to achieve the goal. To make full use of the multiscale features of the image, the full-scale skip connection is used in the decoder network of FSCF-NET to fuse the full-scale deep features extracted by the encoder. To enforce faithfulness, the FSCF-Net adopts a hybrid loss function consisting of detail-preserving loss and structure-preserving loss. To effectively fuse the features of different source images, a spatial-channel attention-based feature fusion strategy (SCAF) is proposed. The SCAF fusion strategy measures the importance of features from both spatial and channel dimensions. The detailed analysis is provided about the proposed model in our experiments that are performed on publicly available datasets. We adopt subjective evaluation and 6 objective metrics to compare the proposed FSCF-Net with 10 state-of-the-art fusion algorithms. The experiments demonstrate that our method achieves the best comprehensive performance on the overall metrics.

CT Image Denoising Model Using Image Segmentation for Image Quality Enhancement for Liver Tumor Detection Using CNN

Image denoising is an important concept in image processing for improving the image quality. It is difficult to remove noise from images because of the various causes of noise. Imaging noise is made up of many different types of noise, including Gaussian, impulse, salt, pepper, and speckle noise. Increasing emphasis has been paid to Convolution Neural Networks (CNNs) in image denoising. Image denoising has been researched using a variety of CNN approaches. For the evaluation of these methods, various datasets were utilized. Liver Tumor is the leading cause of cancer-related death worldwide. By using Computed Tomography (CT) to detect liver tumor early, millions of patients could be spared from death each year. Denoising a picture means cleaning up an image that has been corrupted by unwanted noise. Due to the fact that noise, edge, and texture are all high frequency components, denoising can be tricky, and the resulting images may be missing some finer features. Applications where recovering the original image content is vital for good performance benefit greatly from image denoising, including image reconstruction, activity recognition, image restoration, segmentation techniques, and image classification. Tumors of this type are difficult to detect and are almost always discovered at an advanced stage, posing a serious threat to the patient's life. As a result, finding a tumour at an early stage is critical. Tumors can be detected non-invasively using medical image processing. There is a pressing need for software that can automatically read, detect, and evaluate CT scans by removing noise from the images. As a result, any system must deal with a bottleneck in liver segmentation and extraction from CT scans. To segment and classify liver CT images after denoising images, a deep CNN technique is proposed in this research. An Image Quality Enhancement model with Image Denoising and Edge based Segmentation (IQE-ID-EbS) is proposed in this research that effectively reduces noise levels in the image and then performs edge based segmentation for feature extraction from the CT images. The proposed model is compared with the traditional models and the results represent that the proposed model performance is better.

Stylization of a Seismic Image Profile Based on a Convolutional Neural Network

Seismic data are widely used in oil, gas, and other kinds of mineral exploration and development. However, due to low artificial interpretation accuracy and small sample sizes, seismic data may not meet the needs of convolutional neural network training. There are major differences between optical image and seismic data, making it difficult for a model to learn seismic data characteristics. Therefore, a style transfer network is necessary to make the styles of optical image and seismic data more similar. Since the stylization effect of a seismic section is similar to that of most art styles, based on an in-depth study of image style transfer, this paper compared the effects of various style transfer models, and selected a Laplacian pyramid network to carry out a study of seismic section stylization. It transmits low-resolution global style patterns through a drafting network, revises high-resolution local details through correction networks, and aggregates all pyramid layers to output final stylized images of seismic profiles. Experiments show that this method can effectively convey the whole style pattern without losing the original image content. This style transfer method, based on the Laplacian pyramid network, provides theoretical guidance for the fast and objective application of the model to seismic data features.

Image Enhancement based on the Histogram Equalization and Multiresolution Discrete Stationary Wavelet Transforms

In recent years, due to the tremendous development that took place on the Internet and its applications in the aspects of human life, the demands of using digital images have also increased dramatically, and that opens up horizons for scientific research in the field of improving the quality of the digital images by removing the noise that caused by to processes that are applied within network transmissions like performing storage, retrieval, and encryption to preserve privacy. All these effects are yielded to reduce image quality and loss of visual information. To surmount this problem, image enhancement methods are used to eliminate the noise while preserving supreme exact details and essential characteristics as much as possible in the digital image. The wavelet image enhancement technique played a critical role in this field by attempts to reduce noise in the image while retaining the vital features of the image due to the capability to separate the image into sub-bands (sub-images) and influence the frequencies of each sub-band separately, where acquiring the original image content is essential to obtaining reliable performance.
 Different enhancement techniques have been realized by many researchers so far. Each technique has its own privileges and shortcomings. In this work, a proposed procedure is presented and effectuated to the image modified by Additive White Gaussian Noise (AWGN). The proposed enhancement operation was achieved using the combination of Histogram Equalization with a two-dimensional stationary discrete wavelet transform (2D-SWT) as a multi-resolution analysis technique in image processing at three levels of decomposition to obtain revised results of the method of noise removing. To distinguish and eliminate noise from affected pixel points in the wavelet domain the 2D-SWT is used based on the hard and soft threshold systems on both high and low frequencies to decrease noise from the noisy image. Then, the multi-level 2D inverse wavelet transform (2D-IWT) is applied to eliminate noise and complete the synthesis of the image by the proposed image enhancement techniques.
 In the end, the performance of the proposed methods has been evaluated by using the Peak Signal to Noise Ratio (PSNR). Experimental measurements determine that the results of the proposed techniques enhanced the PSNR by about (16.16%) with respect to the results of the related works, and the structure of the image quality has also been improved in terms of edges retaining and greater noise elimination.

Research on image to illustration translation method based on CycleGAN

Aiming at the problem that balance between abstract style and original painting content is not enough in traditional image style migration, this paper puts forward an improved method that based on the traditional Cyclic-Consistent Generation Adversarial Networks (CycleGAN), which enables the generator to subsample the feature map of each residual layer, and uses skip link and upsampling to merge the low-level feature and the advanced feature. Then the image averaging operation is used to enhance contrast of the generated images, and weighted mean filter is used to filter out redundant details. Finally, the grey level of the image is reduced in order to improve its abstraction. The experimental results show that with CycleGAN, DualGAN, CartoonGAN compared three methods of migration image style, this model not only improves the image style of abstract degree, also a better retain the original image content, significantly improve the processing and transfer the balance between the content of the original painting abstract style.

Perceptual Image Hashing Based on Multitask Neural Network

With the advent of the era of multimedia and in-depth development, the whole human society has been produced and spread a huge amount of image data, but at the same time, in view of the digital image and tamper with the attack of piracy phenomenon also more and more serious, malicious attacks will produce serious social, military, and political influence, therefore, to protect the authenticity of the original image content, which is also more and more important. In order to further improve the performance of image hashing and enhance the protection of image data, we proposed an end-to-end dual-branch multitask neural network based on VGG-19 to produce a perceptual hash sequence and used prepart of network of pretrained VGG-19 model to extract image features, and then, the image features are transformed into a hash sequence through a convolutional and fully connected network. At the same time, in order to enhance the function of the network and improve the adaptability of the proposed network to using scenarios, the rest part of the network layer of the VGG-19 model was used as another branch for image classification, so as to realize the multitask characteristics of the network. Through the experiment of the testing set, the network can not only resist many kinds of attack operations (content retention operations), but also realize accurate classification about the image, and has a satisfactory tampering detection ability.

Secure reversible data hiding in encrypted images based on adaptive prediction-error labeling

Reversible data hiding ensures that the legitimate recipient can completely recover the original image and additional data. However, in order to hide the original image content, the content-owner might encrypt the image locally first. Thus reversible data hiding in encrypted images (RDHEI) has attracted wide attention, which can be used for cloud storage and privacy protection. In this paper, a high-capacity and secure RDHEI method based on adaptive prediction-error labeling (APL) is proposed, which takes advantage of the Laplacian-like distribution of prediction errors. To increase the reserved room before encryption, we present two approaches that combine APL and encryption. One pessimistically exposes the shuffled labels in encrypted images, while the other optimistically performs the encryption on the labels. Dependent of both approaches, the original image and additional data can be completely and separately recovered if the recipient has the secret key. More importantly, security of RDHEI is emphasized, not only the invisibility of image content, but also the undetectability of additional data. Experimental results show that our method is able to achieve higher embedding capacity and relevant security compared with prior arts.

Reversible data hiding in encrypted images with separability and high embedding capacity

In this paper, a high-capacity reversible data hiding (RDH) scheme for encrypted images with separability is proposed. The image is first divided into non-overlapping blocks, and each block is encrypted with the same random value. The advantage is that the correlation between adjacent pixels can be preserved. Utilizing the preserved correlation, the prediction difference in encrypted domain is exactly the same as that of plaintext domain, so that the separability can be achieved. Without accessing the original image content, the data-hider can embed additional data into encrypted image through histogram shifting and difference expansion. At the receiving end, the embedded additional data and the original image can be recovered without any error in separable manner. Experimental results are presented to demonstrate the feasibility and efficiency of the proposed scheme.

High-Capacity Reversible Data Hiding in Encrypted Images Based on Hierarchical Quad-Tree Coding and Multi-MSB Prediction

Nowadays, more and more researchers are interested in reversible data hiding in encrypted images (RDHEI), which can be applied in privacy protection and cloud storage. In this paper, a new RDHEI method on the basis of hierarchical quad-tree coding and multi-MSB (most significant bit) prediction is proposed. The content owner performs pixel prediction to obtain a prediction error image and explores the maximum embedding capacity of the prediction error image by hierarchical quad-tree coding before image encryption. According to the marked bits of vacated room capacity, the data hider can embed additional data into the room-vacated image without knowing the content of original image. Through the data hiding key and the encryption key, the legal receiver is able to conduct data extraction and image recovery separately. Experimental results show that the average embedding rates of the proposed method can separately reach 3.504 bpp (bits per pixel), 3.394 bpp, and 2.746 bpp on three well-known databases, BOSSBase, BOWS-2, and UCID, which are higher than some state-of-the-art methods.

A Review on Neural Style Transfer

Image style transfer is a method that can output styled images, which can both retain the original image content and add new artistic style. When using neural network, this method is referred as Neural Style Transfer (NST), which is a hot topic in the field of image processing and video processing. This article will provide a comprehensive overview of the current NST methods. Firstly, we introduce the current progress of NST from two aspects: the image-optimisation-based method and model-optimisation-based method. Then we compare and summarize different types of the NST algorithms. The review concludes with a discussion of applications of NST and some proposals for future research.

High capacity reversible data hiding in encrypted image based on image encoding and POB

Reversible data-hiding in encrypted image (RDH-EI) can be used for protection of original image by image owner; data hider can embed secret data in the encrypted image without necessity of knowing the original image content. In order to enhance the capacity of embedding in encrypted image, a novel RDH-EI framework based on JPEG-LS and permutation ordered binary (POB) number system is proposed. In the proposed scheme, the original image is first encoded by JPEG-LS, and then the encoded image is encrypted by bit planes. In the phase of data hiding, the encrypted image is first shuffled, and then non-encoded encrypted pixels are totally replaced by the secret data. Lastly, the above generated image is divided into some non-overlapping 2 × 2 blocks, the block feature of 3-bits which is calculated from the pixels within the block and 5-bits secret data are embedded into the every pixel of the block, and then the pixels with 10-bits data in the block are transformed into the POB value. Experimental results show that the proposed scheme obtains the higher embedding rate compared with some of the state-of-the-art schemes, the weak robustness and performance of verification for tampered parts in the marked image are also achieved for the proposed scheme.

Reversible Data Hiding in Encrypted Images using Chunk Encryption and Redundancy Matrix Representation

For reversible data hiding in encrypted images (RDHEI), the private information in the original image content is protected and the embedded secret data can be used to manage the encrypted image. Up to now, many RDHEI algorithms have been reported, however, these reported RDHEI algorithms cannot achieve a high embedding capacity. Therefore, this paper proposes a new RDHEI scheme, which will have improved performance. The main contributions of the proposed scheme include two points: chunk encryption (CE) and redundancy matrix representation (RMR). First, different from the encryption method of pixel-by-pixel encryption used in the previous RDHEI algorithms, CE is the method that uses the traditional exclusive-or (XOR) method to encrypt the original image in chunks. By using the CE method, redundancy will be retained in the encrypted chunks. Second, the RMR method is designed to generate available room for accommodating secret data in redundancy matrices, which are existed in some encrypted chunks. Experimental results show that the proposed scheme achieves a high embedding capacity and the directly decrypted images retain high quality.

Structural Image De-Identification for Privacy-Preserving Deep Learning

Due to the risk of data leakage while training deep learning models in a shared environment, we propose a new privacy-preserving deep learning (PPDL) method using a structural image de-identification approach for object classification. The proposed structural image de-identification approach is designed based on the fact that the degree of structural distortion of an image object has the greatest impact on human’s perceptual system. Thus, by modifying only the structural parts of the original one using order preserving encryption(OPE), the proposed structural image de-identification approach decreases only the recognition rate by human. From the experimental results using different standard datasets, we show that the object classification accuracy of the proposed structural image de-identification method is almost the same as the deep learning performance for non-encrypted images, without revealing the original image contents including sensitive information. Also, by handling the trade-off between object classification accuracy and privacy protection for the de-identified image, we experimentally find the optimal size of input image for the proposed structural image de-identification approach.

Adversarial Dual Network Learning With Randomized Image Transform for Restoring Attacked Images

We develop a new method for defending deep neural networks against attacks using adversarial dual network learning with randomized nonlinear image transform. We introduce a randomized nonlinear transform to disturb and partially destroy the sophisticated pattern of attack noise. We then design a generative cleaning network to recover the original image content damaged by this nonlinear transform and remove residual attack noise. We also construct a detector network which serves as the dual network for the target classifier to be defended, being able to detect patterns of attack noise. The generative cleaning network and detector network are jointly trained using adversarial learning, fighting against each other to minimize both perceptual loss and adversarial loss. Our extensive experimental results demonstrate that our approach improves the state-of-art by large margins in both white-box and black-box attacks. It significantly improves the classification accuracy for white-box attacks upon the second best method by more than 30% on the SVHN dataset and more than 14% on the challenging CIFAR-10 dataset.