Noise Compression Research Articles

Generative adversarial networks with deep blind degradation powered terahertz ptychography

Ptychography is an imaging technique that uses the redundancy of information generated by the overlapping of adjacent light regions to calculate the relative phase of adjacent regions and reconstruct the image. In the terahertz domain, in order to make the ptychography technology better serve engineering applications, we propose a set of deep learning terahertz ptychography system that is easier to realize in engineering and plays an outstanding role. To address this issue, we propose to use a powerful deep blind degradation model which uses isotropic and anisotropic Gaussian kernels for random blurring, chooses the downsampling modes from nearest interpolation, bilinear interpolation, bicubic interpolation and down-up-sampling method, and introduces Gaussian noise, JPEG compression noise, and processed detector noise. Additionally, a random shuffle strategy is used to further expand the degradation space of the image. Using paired low/high resolution images generated by the deep blind degradation model, we trained a multi-layer residual network with residual scaling parameters and dense connection structure to achieve the neural network super-resolution of terahertz ptychography for the first time. We use two representative neural networks, SwinIR and RealESRGAN, to compare with our model. Experimental result shows that the proposed method achieved better accuracy and visual improvement than other terahertz ptychographic image super-resolution algorithms. Further quantitative calculation proved that our method has significant advantages in terahertz ptychographic image super-resolution, achieving a resolution of 33.09 dB on the peak signal-to-noise ratio (PSNR) index and 3.05 on the naturalness image quality estimator (NIQE) index. This efficient and engineered approach fills the gap in the improvement of terahertz ptychography by using neural networks.

On a Closer Look of a Doppler Tolerant Noise Radar Waveform in Surveillance Applications.

The prevalence of Low Probability of Interception (LPI) and Low Probability of Exploitation (LPE) radars in contemporary Electronic Warfare (EW) presents an ongoing challenge to defense mechanisms, compelling constant advances in protective strategies. Noise radars are examples of LPI and LPE systems that gained substantial prominence in the past decade despite exhibiting a common drawback of limited Doppler tolerance. The Advanced Pulse Compression Noise (APCN) waveform is a stochastic radar signal proposed to amalgamate the LPI and LPE attributes of a random waveform with the Doppler tolerance feature inherent to a linear frequency modulation. In the present work, we derive closed-form expressions describing the APCN signal's ambiguity function and spectral containment that allow for a proper analysis of its detection performance and ability to remove range ambiguities as a function of its stochastic parameters. This paper also presents a more detailed address of the LPI/LPE characteristic of APCN signals claimed in previous works. We show that sophisticated Electronic Intelligence (ELINT) systems that employ Time Frequency Analysis (TFA) and image processing methods may intercept APCN and estimate important parameters of APCN waveforms, such as bandwidth, operating frequency, time duration, and pulse repetition interval. We also present a method designed to intercept and exploit the unique characteristics of the APCN waveform. Its performance is evaluated based on the probability of such an ELINT system detecting an APCN radar signal as a function of the Signal-to-Noise Ratio (SNR) in the ELINT system. We evaluated the accuracy and precision of the random variables characterizing the proposed estimators as a function of the SNR. Results indicate a probability of detection close to 1 and show good performance, even for scenarios with a SNR slightly less than -10 dB. The contributions in this work offer enhancements to noise radar capabilities while facilitating improvements in ESM systems.

Compressed Gate Characterization for Quantum Devices with Time-Correlated Noise

As quantum devices make steady progress towards intermediate scale and fault-tolerant quantum computing, it is essential to develop rigorous and efficient measurement protocols that account for known sources of noise. Most existing quantum characterization protocols such as gate-set tomography and randomized benchmarking assume the noise acting on the qubits is Markovian. However, this assumption is often not valid, as for the case of 1/f charge noise or hyperfine nuclear spin noise. Here, we present a general framework for quantum process tomography (QPT) in the presence of time-correlated noise. We further introduce fidelity benchmarks that quantify the relative strength of different sources of Markovian and non-Markovian noise. As an application of our method, we perform a comparative theoretical and experimental analysis of silicon spin qubits. We first develop a detailed noise model that accounts for the dominant sources of noise and validate the model against experimental data. Applying our framework for time-correlated QPT, we find that the number of independent parameters needed to characterize one- and two-qubit gates can be compressed by 10× and 100×, respectively, when compared to the fully generic case. These compressions reduce the amount of tomographic measurements needed in experiment, while also significantly speeding up numerical simulations of noisy quantum circuit dynamics compared to time-dependent Hamiltonian simulation. Using this compressed noise model, we find good agreement between our theoretically predicted process fidelities and two-qubit interleaved randomized benchmarking fidelities of 99.8% measured in recent experiments on silicon spin qubits. More broadly, our formalism can be directly extended to develop efficient and scalable tuning protocols for high-fidelity control of large arrays of quantum devices with non-Markovian noise. Published by the American Physical Society 2024

Pragmatic degradation learning for scene text image super-resolution with data-training strategy

Super-resolution of scene text images represents a formidable computational problem, marred by a myriad of intricate challenges. This paper focuses on the specific hurdles that have impeded significant advancements in this domain, and introduces the Higher-Order Degradation-Based Super-Resolution Network (HDSN) as a novel solution to address these intricate issues. The challenges in super-resolving scene text images are manifold. Firstly, the semantic ambiguity inherent to text in natural scenes often leads to degraded results, as standard super-resolution techniques struggle to preserve meaningful textual content. Additionally, the uncertainty surrounding font variability exacerbates this issue, as different fonts require distinct treatment for optimal super-resolution. Furthermore, scene text images often exhibit long trailing shadows, artifacts, and strong noise, rendering conventional methods inadequate in producing satisfactory results. To tackle these intricate challenges, we propose a pragmatic higher-order degradation modeling process. This process takes into account the nuanced characteristics of scene text images, including the diverse forms of noise such as Gaussian, Poisson, speckle, and JPEG compression noise, as well as varying levels of blurring. By meticulously considering these real-world scenarios, our approach significantly enhances the robustness and adaptability of super-resolution for scene text images. In addition to addressing these challenges, we recognize the issues arising from sparse datasets and the lack of corresponding paired images for training. To surmount this limitation, we introduce a text image pre-training strategy, which proves to be highly effective in improving recognition accuracy. The experimental results on TextZoom affirm the effectiveness of our approach, demonstrating substantial improvements over existing methods. Notably, our HDSN achieves average recognition rates of 67.2% on ASTER, 63.2% on MORAN, and 58.0% on CRNN, surpassing the performance of available approaches. Our source code is available at https://github.com/syyang2022/HDSN.

Neural Adaptive Loop Filtering for Video Coding: Exploring Multi-Hypothesis Sample Refinement

Adaptive loop filtering (ALF) is extensively investigated for lossy video coding to mitigate compression noise. Numerous learning-based ALFs have emerged recently and improved the coding efficiency significantly through the use of complexity-intensive, large-scale models trained on excessive samples, making it impractical for real-life applications. By contrast, lightweight, small-scale ALF models cannot promise convincing performance and model generalization. In principle, the ALF estimates the sample distortion for restoration. Instead of directly approximating the distortion as in existing solutions, we reformulate it as a Multi-hypothesis Sample Refinement (MSR) problem. To this end, we first generate multiple distortion hypotheses through a deep neural network (DNN) model. Then, these hypotheses are linearly superimposed to approximate the final distortion through the minimization of mean square error (MMSE) between the filtered reconstruction and its original, uncompressed input. Finally, the linear superimposition coefficients are explicitly signaled in the compressed bitstream. As seen, the superimposition coefficients inherently generalize the MSR to various content. And using DNNs to generate distortion hypotheses essentially models the spatial priors of a local block and its underlying compression error distribution. As a result, the MSR using a small-scale convolutional neural network (CNN) model with only 5k parameters and 5 KMACs/pixel achieves 4.35% (Intra) and 2.49% (Inter) BD-Rate (Bjontegaard Delta Rate) gains over the AV1 anchor. Ablation studies further demonstrate that the MSR can be generalized to diverse small-scale network structures, different standards (e.g., H.265/HEVC and H.266/VVC), and diverse video content (e.g., screen videos).

Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis

While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved. In this paper, we attempt to solve this problem from the perspective of network architecture design and training data synthesis. Specifically, for the network architecture design, we propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block, and then plug it as the main building block into the widely-used image-to-image translation UNet architecture. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise (including Gaussian, Poisson, speckle, JPEG compression, and processed camera sensor noises) and resizing, and also involves a random shuffle strategy and a double degradation strategy. Extensive experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance and the new degradation model can help to significantly improve the practicability. We believe our work can provide useful insights into current denoising research. The source code is available at https://github.com/cszn/SCUNet.

Byzantine-Robust Distributed Learning With Compression

Communication between workers and the master node to collect local stochastic gradients is a key bottleneck in a large-scale distributed learning system. Various recent works have proposed to compress the local stochastic gradients to mitigate the communication overhead. However, robustness to malicious attacks is rarely considered in such a setting. In this work, we investigate the problem of Byzantine-robust compressed distributed learning, where the attacks from Byzantine workers can be arbitrarily malicious. We theoretically point out that different to the attacks-free compressed stochastic gradient descent (SGD), its vanilla combination with geometric median-based robust aggregation seriously suffers from the compression noise in the presence of Byzantine attacks. In light of this observation, we propose to reduce the compression noise with gradient difference compression so as to improve the Byzantine-robustness. We also observe the impact of the intrinsic stochastic noise caused by selecting random samples, and adopt the stochastic average gradient algorithm (SAGA) to gradually eliminate the inner variations of regular workers. We theoretically prove that the proposed algorithm reaches a neighborhood of the optimal solution at a linear convergence rate, and the asymptotic learning error is in the same order as that of the state-of-the-art uncompressed method. Finally, numerical experiments demonstrate the effectiveness of the proposed method.

A quality enhancement network with coding priors for constant bit rate video coding

Video coding can effectively compress data, while the introduction of compression artifacts degrades the visual quality and the performance of artificial intelligence video applications. Video quality enhancement (VQE) methods can improve compressed video quality without modifying the coding standard’s main modules. The existing VQE works mainly aim to improve video quality in the constant quantization parameter (CQP) coding mode. However, constant bit rate (CBR) coding mode is widely used in some streaming playback video applications, and VQE at CBR is more challenging than that at CQP. This article presents a novel Constant bit rate Video quality enhancement Network combined with Coding priors (CVCN). The proposed CVCN can be inserted into High Efficiency Video Coding (HEVC) codec as a CNN-based in-loop filter (LF) module or a post-processing module out of the codec. Moreover, we design a two-pass training strategy for the LF module to overcome multiple filtering. To adapt the QP diversity of CBR videos, we utilize the coding unit (CU)-wise QP prior by constructing a CU-wise QP adaptive module (CQAM) and a QP adaptive multi-scale residual block (QAMSRB) based on CQAM. We construct the CU-partition prior (CPP) by exploiting the relationship between the compression noise and CU partition information. A novel CPP spatial-attention block is proposed to combine the CPP with the self-attention module. Furthermore, the proposed CVCN can effectively enhance CBR videos at different bits per pixel (BPP) via a single model. Extensive experimental results show the superiority of the CVCN over state-of-the-art VQE approaches for CBR videos.

Cell-Free mmWave Massive MIMO Systems With Low-Capacity Fronthaul Links and Low-Resolution ADC/DACs

In this paper, we consider the uplink channel estimation phase and downlink data transmission phase of cell-free millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems with low-capacity fronthaul links and low-resolution analog-to-digital converters/digital-to-analog converters (ADC/DACs). In cell-free massive MIMO, a control unit dictates the baseband processing at a geographical scale, while the base stations communicate with the control unit through fronthaul links. Unlike most of previous works in cell-free massive MIMO with finite-capacity fronthaul links, we consider the general case where the fronthaul capacity and ADC/DAC resolution are not necessarily the same. In particular, the fronthaul compression and ADC/DAC quantization occur independently where each one is modeled based on the information theoretic argument and additive quantization noise model (AQNM). Then, we address the codebook design problem that aims to minimize the channel estimation error for the independent and identically distributed (i.i.d.) and colored compression noise cases. Also, we propose an alternating optimization (AO) method to tackle the max-min fairness problem. In essence, the AO method alternates between two subproblems that correspond to the power allocation and codebook design problems. The AO method proposed for the zero-forcing (ZF) precoder is guaranteed to converge, whereas the one for the maximum ratio transmission (MRT) precoder has no such guarantee. Finally, the performance of the proposed schemes is evaluated by the simulation results in terms of both energy and spectral efficiency. The numerical results show that the proposed scheme for the ZF precoder yields spectral and energy efficiency 28% and 15% higher than that of the best baseline.

A Novel Reduced Reference Image Quality Assessment Based on Formal Concept Analysis

Abstract The assessment of image quality provides the confidentiality, safety, quality and transparency of the obtained images. At the same time, there are many existing quality metric approaches that belong to the group of modification alteration procedures (pixel-based metric). These techniques have not been well-matched with perceptual image quality. The Perceptual Human Visual System (HVS) aspect drives our approach; The human visual system (HVS) is the best judge of image quality. This paper presents a new reduced reference image quality metric in the spatial domain that reduces the complexity of the image quality assessment. From the reference and distortion images, we extract four intrinsic features, namely, contrast, entropy, histogram and standard deviation. Then, we build the formal concept analysis matrix for reference and distortion images. Finally, we compare the obtained matrixes to evaluate the image quality. The performance of the proposed technique is assessed using LIVE, TID2013 and CSIQ datasets, and the obtained results are compared in terms of PSNR, SSIM and NCC metrics. Also, a comparison with more recent and relevant approaches is performed to highlight the superior performance of our proposed approach, the experimental results indicated that the proposed approach provides efficient performance among the compression, Gaussian blur, Contrast and add noise distortion types.

A nonlocal HEVC in-loop filter using CNN-based compression noise estimation

A nonlocal HEVC in-loop filter using CNN-based compression noise estimation

Detecting Compressed Deepfake Videos in Social Networks Using Frame-Temporality Two-Stream Convolutional Network

The development of technologies that can generate Deepfake videos is expanding rapidly. These videos are easily synthesized without leaving obvious traces of manipulation. Though forensically detection in high-definition video datasets has achieved remarkable results, the forensics of compressed videos is worth further exploring. In fact, compressed videos are common in social networks, such as videos from Instagram, Wechat, and Tiktok. Therefore, how to identify compressed Deepfake videos becomes a fundamental issue. In this paper, we propose a two-stream method by analyzing the frame-level and temporality-level of compressed Deepfake videos. Since the video compression brings lots of redundant information to frames, the proposed frame-level stream gradually prunes the network to prevent the model from fitting the compression noise. Aiming at the problem that the temporal consistency in Deepfake videos might be ignored, we apply a temporality-level stream to extract temporal correlation features. When combined with scores from the two streams, our proposed method performs better than the state-of-the-art methods in compressed Deepfake videos detection.

Parameter estimation, data compression and stochastic noise elimination in robotics: a wavelet domain-based integrated approach

Parameter estimation, data compression and stochastic noise elimination in robotics: a wavelet domain-based integrated approach

Differential real-time single-pixel imaging with Fourier domain regularization: applications to VIS-IR imaging and polarization imaging.

The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product so the cost of the reconstruction method scales proportionally with the number of measured samples. A differential operator is included in the reconstruction and following the method is based on finding the generalized inversion of the modified measurement matrix with regularization in the Fourier domain. We demonstrate 256 × 256 SPI at up to 17 Hz at visible and near-infrared wavelength ranges using 2 polarization or spectral channels. A low bit-resolution data acquisition device with alternating-current-coupling can be used in the measurement indicating that the proposed method combines improved noise robustness with a differential removal of the direct current component of the signal.

Emergency Medicine Cases in Underwater and Hyperbaric Environments: The Use of in situ Simulation as a Learning Technique.

IntroductionHyperbaric chambers and underwater environments are challenging and at risk of serious accidents. Personnel aiming to assist patients and subjects should be appropriately trained, and several courses have been established all over the world. In healthcare, simulation is an effective learning technique. However, there have been few peer-reviewed articles published in the medical literature describing its use in diving and hyperbaric medicine.MethodsWe implemented the curriculum of the Master’s degree in hyperbaric and diving medicine held at the University of Padova with emergency medicine seminars created by the faculty and validated by external experts. These seminars integrated traditional lectures and eight in situ simulation scenarios.ResultsFor the hyperbaric medicine seminar, simulations were carried out inside a real hyperbaric chamber at the ATIP Hyperbaric Treatment Centre, only using air and reproducing compression noise without pressurization to avoid damages to the manikins. The four scenarios consisted of hyperoxic seizures, pneumothorax, hypoglycemia, and sudden cardiac arrest. Furthermore, we added a hands-on session to instruct participants to prepare an intubated patient undergoing hyperbaric oxygen treatment with a checklist and simulating the patient transfer inside and outside the hyperbaric chamber. The diving medicine seminar was held at the Y-40 The Deep Joy pool in Montegrotto Terme (Italy), also involving SCUBA/breath-hold diving (BHD) instructors to rescue subjects from the water. These diving medicine scenarios consisted of neurologic syndrome (“taravana/samba”) in BHD, drowning of a breath-hold diver, pulmonary barotrauma in BHD, and decompression illness in a SCUBA diver.ConclusionWith this experience, we report the integration of simulation in the curriculum of a teaching course in diving and hyperbaric medicine. Future studies should be performed to investigate learning advantages, concept retention, and satisfaction of participants.

Human Detection via Image Denoising for 5G‐Enabled Intelligent Applications

5G technology strongly supports the development of various intelligent applications, such as intelligent video surveillance and autonomous driving. And the human detection technology in intelligent video surveillance has also ushered in new challenges. A number of video images will be compressed for efficient transmission; the resulting incomplete feature representation of images will drop the human detection performance. Therefore, in this work, we propose a new human detection method based on compressed denoising. We exploit the quality factor in the compressed image and incorporate the pixel_shuffle inverse transform based on FFDNet to effectively improve the performance of image compression denoising, then HRNet and HRFPN are used to extract and fuse high‐resolution features of denoised images, respectively, to obtain high‐quality feature representation, and finally, a cascaded object detector is used for classification and bounding box regression to further improve object detection performance. At last, the experimental results on PASCAL VOC show that the proposed method effectively removes the compression noise and further detects human objects with multiple scales and different postures. Compared with the state‐of‐the‐art methods, our method achieved better detection performance and is, therefore, more suited for human detection tasks.

Hybrid beamforming designs for 5G new radio with fronthaul compression and functional splits

In this study, the authors investigate the intra-physical functional splits of 5G new radio protocol stack proposed by different groups. Based on the location of the digital beamforming block, the radio units (RUs) are divided into two categories: Category A and Category B. Two implementation modes of hybrid beamforming at the physical layer are considered, in which digital beamforming is performed either at the distributed unit (DU), as in the Category A RU based hybrid beamforming (HBF-A) scheme, or at the RUs, as in the Category B RU based hybrid beamforming (HBF-B) scheme. To maximise the weighted sum rate, the authors formulate the problems of jointly designing hybrid beamforming, analogue combining and fronthaul compression strategies for both HBF-A and HBF-B. The formulated problems are simplified by adopting the codebook-based design, and further tackled by leveraging the majorisation–minimisation algorithm. Finally, numerical results confirm that the HBF-A scheme outperforms the HBF-B scheme in the large power regime. Compared with the HBF-A method, the HBF-B method is more sensitive to changes in system parameters, such as the compression noise and the number of receive antennas, in the large power regime, while it is less sensitive in the small power regime.

Mononizing binocular videos

This paper presents the idea of mono-nizing binocular videos and a framework to effectively realize it. Mono-nize means we purposely convert a binocular video into a regular monocular video with the stereo information implicitly encoded in a visual but nearly-imperceptible form. Hence, we can impartially distribute and show the mononized video as an ordinary monocular video. Unlike ordinary monocular videos, we can restore from it the original binocular video and show it on a stereoscopic display. To start, we formulate an encoding-and-decoding framework with the pyramidal deformable fusion module to exploit long-range correspondences between the left and right views, a quantization layer to suppress the restoring artifacts, and the compression noise simulation module to resist the compression noise introduced by modern video codecs. Our framework is self-supervised, as we articulate our objective function with loss terms defined on the input: a monocular term for creating the mononized video, an invertibility term for restoring the original video, and a temporal term for frame-to-frame coherence. Further, we conducted extensive experiments to evaluate our generated mononized videos and restored binocular videos for diverse types of images and 3D movies. Quantitative results on both standard metrics and user perception studies show the effectiveness of our method.

A quality enhancement framework with noise distribution characteristics for high efficiency video coding

Video coding effectively reduces the amount of video data while unavoidably producing compression noise. Compression noise can cause significant artifacts in compressed video, such as blocking, ringing, and blurring, which seriously affects the visual quality of videos and the value of videos for content analysis. In compressed video quality enhancement, few methods based on deep learning fully consider the relationship between video content and compression noise or the possibility of uniting the encoder or the decoder to enhance the quality of compressed video. In an approach different from existing methods, we propose a video quality enhancement framework based on the distribution characteristics of compression noise. The proposed framework consists of two parts: at the encoder, we propose a convolutional neural network (CNN)-based in-loop filtering network combined with noise distribution (IFN-ND) characteristics for the I frame instead of high efficiency video coding (HEVC) standard in-loop filters; at the decoder, we propose a CNN-based quality enhancement network combined with the noise distribution characteristics (PQEN-ND) for the P frames. The noise characteristics are extracted from the code stream to further improve the performance of the proposed networks. The experiments show that the proposed method can significantly improve the quality of HEVC compressed video, achieving an average 12.84% reduction in the BD rate and up to a 1.0476 dB increase in the peak signal-to-noise ratio (PSNR).

Adaptive image coding efficiency enhancement using deep convolutional neural networks

In recent years, the application of systems for the storage and transmission of images has faced severe challenges owing to the significant increase in the number of images being generated and sent. Therefore, it is necessary to enhance the efficiency of existing coding approaches while maintaining compatibility. This paper proposes an adaptive image coding efficiency enhancement framework that innovatively integrates deep network-based super-resolution (SR) and soft-decoding (SD) into image coding. To address the problem of insufficient number of coding bits at high compression ratios, the given image is downsampled prior to encoding, and the decoded image is super-resolved in the proposed low-resolution coding mode. For the full-resolution coding mode developed for medium or low compression ratios, the given image is directly encoded and the decoded image is enhanced by suppressing the compression noise. To efficiently determine the optimal coding mode of a given image, an adaptive mode decision strategy is proposed by modeling the relationship between the critical coding quality parameter and the distortion caused by successive downsampling and upsampling processes. Experimental results show that the proposed algorithm significantly improves the performance of the widely used image coding standard JPEG, demonstrating the effectiveness of the SR and SD-enabled coding framework.