Photo-response Non-uniformity Research Articles

Generalized and robust model for GAN-generated image detection

Generative adversarial network (GAN)-generated image detection is a crucial yet challenging problem. The advancement of diverse generative models makes it simpler to build realistic synthetic images, but it also increases the likelihood of using images maliciously. Recent techniques of fake image detection often overfit to specific GANs, incapable of generalizing to other GANs. Each GAN leaves its distinct fingerprint on the images it generates. We propose a simple and compact attention network that exploits high-frequency and noise residual patterns of images. We design two-channel image representations in the preprocessing phase, with one channel using a bilateral high pass filter (BiHPF) for high-frequency patterns and the other using photo response non-uniformity (PRNU) for noise residuals capturing from GAN-generated and real images. The feature maps pass through the attention network with a dense layer for classification tasks. We assess the generalizability and robustness of the model through extensive experiments. The model is evaluated on a benchmark dataset in various settings, including cross-category, cross-GAN models with varying class settings and different color manipulations. This study demonstrates that the proposed model outperforms state-of-the-art (SOTA) methods by approximately 5% in different experiment settings.

Video source camera identification using fusion of texture features and noise fingerprint

In Video forensics, the objective of Source Camera Identification (SCI) is to identify and verify the origin of a video that is under investigation. This aids the investigator to trace the video to its owner or narrow down the search space for identifying the offender. Nowadays, it is easy to record and share videos via internet or social media with smartphones. The availability of sophisticated video editing tools and software allow offenders to modify video's context. Thus, identifying the right source camera that was used to capture the video becomes complicated and strenuous. Existing methods based on video metadata information are no longer reliable as it could be modified or stripped off. Better forensic procedures are therefore required to prove the authenticity and integrity of the video that will be used as evidence in court of law. Certain inherent camera sensor properties such as, subtle traces of Photo Response Non-Uniformity (PRNU) are present in all captured videos due to unnoticeable defect during the manufacture of camera's sensor. These properties are used in SCI to classify devices or models as they are unique. In this work, we focus on SCI from videos or Video Source Camera Identification (VSCI) to verify the authenticity of videos. PRNU can be affected by highly textured content or post-processing when computed from a set of flat field images. To mitigate these effects, Higher Order Wavelet Statistics (HOWS) information from PRNU of a video I-frame is combined with information from two other texture features i.e., Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM). The extracted feature vector is fused via concatenation and fed to Support Vector Machine (SVM) classifier to perform training and testing for VSCI. Experimental evaluation of our proposed method on videos from different publicly available datasets show the effectiveness of our method in terms of accuracy, resource efficiency, and complexity.

Investigation of Passivation Pretreatment Processes for Mid‐/Long‐Wavelength Dual‐Band Infrared Focal Plane Arrays Based on Type‐II InAs/GaSb Superlattices

Herein, through the analysis of devices with different passivation pretreatment processes, it is found that after surface corrosion, the optimized process of O3 treatment combined with hydrochloric acid and H3PO4/C6H8O7/H2O2/H2O treatments can significantly improve the electrical performance of the device. By analyzing the curves of dark current density varying with temperature of the mid‐wavelength (MW) and long‐wavelength (LW) dual‐color devices, it is verified that there are three leakage mechanisms in the device: diffusion current, trap‐assisted tunneling current, and generation–recombination current. Finally, a 640 × 512 MW/LW dual‐color superlattice infrared detector is fabricated with a pixel pitch of 20 μm. The cutoff wavelength of the MW channel is 5.04 μm. The cutoff wavelength of the LW channel is 9.51 μm. The noise equivalent temperature difference of MW is 16.5 mK and LW is 35.1 mK. The photoresponse nonuniformity is 6.21% for MW and 10.51% for LW. The blind pixel rate is 0.66% for MW and 8.56% for LW. The imaging demonstration is completed. The research has laid the foundation for the engineering application of dual‐color infrared detectors.

WhatsApp compressed video source camera identification based on photo response nonuniformity

WhatsApp compressed video source camera identification based on photo response nonuniformity

Detection of AI-Created Images Using Pixel-Wise Feature Extraction and Convolutional Neural Networks.

Generative AI has gained enormous interest nowadays due to new applications like ChatGPT, DALL E, Stable Diffusion, and Deepfake. In particular, DALL E, Stable Diffusion, and others (Adobe Firefly, ImagineArt, etc.) can create images from a text prompt and are even able to create photorealistic images. Due to this fact, intense research has been performed to create new image forensics applications able to distinguish between real captured images and videos and artificial ones. Detecting forgeries made with Deepfake is one of the most researched issues. This paper is about another kind of forgery detection. The purpose of this research is to detect photorealistic AI-created images versus real photos coming from a physical camera. Id est, making a binary decision over an image, asking whether it is artificially or naturally created. Artificial images do not need to try to represent any real object, person, or place. For this purpose, techniques that perform a pixel-level feature extraction are used. The first one is Photo Response Non-Uniformity (PRNU). PRNU is a special noise due to imperfections on the camera sensor that is used for source camera identification. The underlying idea is that AI images will have a different PRNU pattern. The second one is error level analysis (ELA). This is another type of feature extraction traditionally used for detecting image editing. ELA is being used nowadays by photographers for the manual detection of AI-created images. Both kinds of features are used to train convolutional neural networks to differentiate between AI images and real photographs. Good results are obtained, achieving accuracy rates of over 95%. Both extraction methods are carefully assessed by computing precision/recall and F1-score measurements.

L-PRNU: Low-Complexity Privacy-Preserving PRNU-Based Camera Attribution Scheme

A personal camera fingerprint can be created from images in social media by using Photo Response Non-Uniformity (PRNU) noise, which is used to identify whether an unknown picture belongs to them. Social media has become ubiquitous in recent years and many of us regularly share photos of our daily lives online. However, due to the ease of creating a PRNU-based camera fingerprint, the privacy leakage problem is taken more seriously. To address this issue, a security scheme based on Boneh–Goh–Nissim (BGN) encryption was proposed in 2021. While effective, the BGN encryption incurs a high run-time computational overhead due to its power computation. Therefore, we devised a new scheme to address this issue, employing polynomial encryption and pixel confusion methods, resulting in a computation time that is over ten times faster than BGN encryption. This eliminates the need to only send critical pixels to a Third-Party Expert in the previous method. Furthermore, our scheme does not require decryption, as polynomial encryption and pixel confusion do not alter the correlation value. Consequently, the scheme we presented surpasses previous methods in both theoretical analysis and experimental performance, being faster and more capable.

Hierarchical deep learning approach using fusion layer for Source Camera Model Identification based on video taken by smartphone

Over the last decade, videos uploaded and shared through web-based multimedia platforms and mobile applications have proliferated worldwide. This is because cloud-based applications such as iCloud, YouTube, Facebook, Twitter, and WhatsApp offer affordable and secure environments for video storage and sharing. However, new challenges have emerged alarming forensic analysts and investigators since videos can be used to commit heinous crimes such as blackmail, fraud, and forgery. Source Camera Identification (SCI) has become of paramount importance in the field of image and video forensics. Camera model identification can also help identify the perpetrators or narrow down the search and can be used to enhance SCI systems. In this context, existing approaches such as the Photo Response Non-Uniformity (PRNU) based methods and machine learning techniques such as the support vector machine (SVM) and deep learning models are commonly used solutions. This work exploits these two categories of methods by exploring a hierarchical deep learning model for camera model identification based on smartphone videos. The PRNU features are extracted by CNN-based structures during the training process. Proposed six-stream networks are leveraged to extract both low-level and high-level features through the network. A fusion layer is created based on joint sparse representation using forward and backward functions defined for fusing the proposed six streams. The proposed approach has been implemented and evaluated through intensive experiments, and results showed successful camera model identification with a performance at the frame level reaching an average accuracy of 69.9% for the Daxing dataset and 81.6% for the QUFVD dataset.

Source Camera Identification with a Robust Device Fingerprint: Evolution from Image-Based to Video-Based Approaches.

With the increasing prevalence of digital multimedia content, the need for reliable and accurate source camera identification has become crucial in applications such as digital forensics. While effective techniques exist for identifying the source camera of images, video-based source identification presents unique challenges due to disruptive effects introduced during video processing, such as compression artifacts and pixel misalignment caused by techniques like video coding and stabilization. These effects render existing approaches, which rely on high-frequency camera fingerprints like Photo Response Non-Uniformity (PRNU), inadequate for video-based identification. To address this challenge, we propose a novel approach that builds upon the image-based source identification technique. Leveraging a global stochastic fingerprint residing in the low- and mid-frequency bands, we exploit its resilience to disruptive effects in the high-frequency bands, envisioning its potential for video-based source identification. Through comprehensive evaluation on recent smartphones dataset, we establish new benchmarks for source camera model and individual device identification, surpassing state-of-the-art techniques. While conventional image-based methods struggle in video contexts, our approach unifies image and video source identification through a single framework powered by the novel non-PRNU device-specific fingerprint. This contribution expands the existing body of knowledge in the field of multimedia forensics.

IMGCAT: An approach to dismantle the anonymity of a source camera using correlative features and an integrated 1D convolutional neural network

With the proliferation of smartphones, digital data collection has become trivial. The ability to analyze images has increased, but source authentication has stagnated. Editing and tampering of images has become more common with advancements in signal processing technology. Recent developments have introduced the use of seam carving (insertion and deletion) techniques to disguise the identity of the camera, specifically in the child pornography market. In this article, we focus on the available features in the image based on PRNU (photo response nonuniformity). The forced-seam sculpting technique is a well-known method to create occlusion for camera attribution by injecting seams into each 50 × 50 pixel block. To counter this, we perform camera identification using a 1D CNN integrated with feature extractions on 20 × 20 pixel blocks. We achieve state-of-the-art performance for our proposed IMGCAT (image categorization) in three-class classification over the baselines (original, seam removed, seam inserted). Based on our experimental findings, our model is robust when dealing with blind facts related to the questionable camera.

Radon transform of image monotonic rearrangements as feature for noise sensor signature

Source camera identification represents a delicate, crucial but challenging task in digital forensics, especially when an image has to be used as a proof in a court of law. This paper investigates some properties of the Photo Response Non Uniformity (PRNU) pattern noise that represents the fingerprint of any acquisition sensor. The main goal is to define specific and distinctive features for this noise source that enable the identification of the acquisition sensor by simply analysing a few images. These features are required to be independent of image size, modifications, storage mode, etc. The discrimination power of the decreasing rearrangement of a function, combined with the Radon transform, has been investigated in this paper. Preliminary tests show that a proper rearrangement of PRNU image provides specific and device-dependent geometric structures that can be properly coded through the Radon transform. In particular, the empirical distribution of the Radon Transform of rearranged Flat Field images alone is capable to correctly characterize each device with high accuracy, showing robusteness to some standard image modifications, such as quantization and blurring; in addition, it guarantees independence of image size.

A Stress Test for Robustness of Photo Response Nonuniformity (Camera Sensor Fingerprint) Identification on Smartphones

In the field of forensic imaging, it is important to be able to extract a camera fingerprint from one or a small set of images known to have been taken by the same camera (or image sensor). Note that we are using the word fingerprint because it is a piece of information extracted from images that can be used to identify an individual source camera. This technique is very important for certain security and digital forensic situations. Camera fingerprint is based on a certain kind of random noise present in all image sensors that is due to manufacturing imperfections and is, thus, unique and impossible to avoid. Photo response nonuniformity (PRNU) has become the most widely used method for source camera identification (SCI). In this paper, a set of attacks is designed and applied to a PRNU-based SCI system, and the success of each method is systematically assessed both in the case of still images and in the case of video. An attack method is defined as any processing that minimally alters image quality and is designed to fool PRNU detectors or, in general, any camera fingerprint detector. The success of an attack is assessed as the increment in the error rate of the SCI system. The PRNU-based SCI system was taken from an outstanding reference that is publicly available. Among the results of this work, the following are remarkable: the use of a systematic and extensive procedure to test SCI methods, very thorough testing of PRNU with more than 2000 test images, and the finding of some very effective attacks on PRNU-based SCI.

Source camera attribution via PRNU emphasis: Towards a generalized multiplicative model

The photoresponse non-uniformity (PRNU) is a camera-specific pattern, which acts as unique fingerprint of any imaging sensor and thus is widely adopted to solve multimedia forensics problems such as device identification or forgery detection. Customarily, the theoretical analysis of this fingerprint relies on a multiplicative model for the denoising residuals. This setup assumes that the nonlinear mapping from scene irradiance to preprocessed luminance, that is, the composition of the Camera Response Function (CRF) with the digital preprocessing pipeline, is a gamma correction. However, this assumption seldom holds in practice. In this paper, we improve the multiplicative model by including the influence of this nonlinear mapping, termed PRNU emphasis, on the denoising residuals. On the theoretical side, we conduct first an exploratory analysis to show that the response of typical cameras deviates from a gamma correction. We also propose a regularized least squares estimator to measure this effect. On the practical side, we argue that the PRNU emphasis is especially beneficial for a source camera attribution problem with cropped images. We back our argument with an extensive empirical evaluation using different denoisers and both compressed and uncompressed images. This new model will pave the way to future PRNU estimators and detectors.

Wafer-Scale Fabrication of CMOS-Compatible Trapping-Mode Infrared Imagers with Colloidal Quantum Dots

Silicon-based complementary metal oxide semiconductor (CMOS) devices have dominated the technological revolution in the past decades. With increasing demands in machine vision, autonomous driving, and artificial intelligence, silicon CMOS imagers, as the major optical information input devices, face great challenges in spectral sensing ranges. In this paper, we demonstrate the development of CMOS-compatible infrared colloidal quantum-dot (CQD) imagers in the broadband short-wave and mid-wave infrared ranges (SWIR and MWIR, 1.5–5 μm). A new device architecture of trapping-mode detectors is proposed, fabricated, and demonstrated with lowered darkcurrents and improved responsivity. The CMOS-compatible fabrication process is completed with two-step sequential spin-coating processes of intrinsic and doped HgTe CQDs on an 8 in. CMOS readout wafer with photoresponse non-uniformity (PRNU) down to 4%, dead pixel rate of 0%, external quantum efficiency up to 175%, and detectivity as high as 2 × 1011 Jones for extended SWIR at 300 K and 8 × 1010 Jones for MWIR at 80 K. Both SWIR images and MWIR thermal images are demonstrated with great potential for semiconductor inspection, chemical identification, and temperature monitoring.

PRNU Anonymous Algorithm Used for Privacy Protection in Biometric Authentication Systems

The photo response non-uniformity (PRNU) is used to connect an image to its source sensor. In this paper, researchers propose a PRNU anonymity method based on image segmentation to cut the relationship between the image and its source camera. According to the distribution rule of PRNU in the high and low frequency band of the image, the high and low frequency information of the part is also processed differently, which ensures the quality of the output image to a large extent. Experiments on the datasets show that the proposed method can preserve the biometric characteristics of the device while maintaining the anonymity of the device. Comparing with prior art, peak signal to noise ratio (PSNR) and cosine similarity are improved by 1.9 dB and 0.02 points, respectively.

PRNU-based Image Forgery Localization with Deep Multi-scale Fusion

Photo-response non-uniformity (PRNU), as a class of device fingerprint, plays a key role in the forgery detection/localization for visual media. The state-of-the-art PRNU-based forensics methods generally rely on the multi-scale trace analysis and result fusion, with Markov random field model. However, such hand-crafted strategies are difficult to provide satisfactory multi-scale decision, exhibiting a high false-positive rate. Motivated by this, we propose an end-to-end multi-scale decision fusion strategy, where a mapping from multi-scale forgery probabilities to binary decision is achieved by a supervised deep fully connected neural network. As the first time, the deep learning technology is employed in PRNU-based forensics for more flexible and reliable integration of multi-scale information. The benchmark experiments exhibit the state-of-the-art accuracy performance of our method in both pixel-level and image-level, especially for false positives. Additional robustness experiments also demonstrate the benefits of the proposed method in resisting noise and compression attacks.

Unsupervised Learning-Based Framework for Deepfake Video Detection

With the continuous development of computer hardware equipment and deep learning technology, it is easier for people to swap faces in videos by currently-emerging multimedia tampering tools, such as the most popular deepfake. It would bring a series of new threats of security. Although many forensic researches have focused on this new type of manipulation and achieved high detection accuracy, most of which are based on supervised learning mechanism with requiring a large number of labeled samples for training. In this paper, we first develop a novel unsupervised detection manner for identifying deepfake videos. The main fundamental behind our proposed method is that the face region in the real video is taken by the camera while its counterpart in the deepfake video is usually generated by the computer; the provenance of two videos is totally different. Specifically, our method includes two clustering stages based on Photo-Response Non-Uniformity (PRNU) and noiseprint feature. Firstly, the PRNU fingerprint of each video frame is extracted, which is used to cluster the full-size identical source video (regardless of its real or fake). Secondly, we extract the noiseprint from the face region of the video, which is used to identify (re-cluster for the task of binary classification) the deepfake sample in each cluster. Numerical experiments verify our proposed unsupervised method performs very well on our own dataset and the benchmark FF++ dataset. More importantly, its performance rivals that of the supervised-based state-of-the-art detectors.

Uniformity Correction of CMOS Image Sensor Modules for Machine Vision Cameras.

Flat-field correction (FFC) is commonly used in image signal processing (ISP) to improve the uniformity of image sensor pixels. Image sensor nonuniformity and lens system characteristics have been known to be temperature-dependent. Some machine vision applications, such as visual odometry and single-pixel airborne object tracking, are extremely sensitive to pixel-to-pixel sensitivity variations. Numerous cameras, especially in the fields of infrared imaging and staring cameras, use multiple calibration images to correct for nonuniformities. This paper characterizes the temperature and analog gain dependence of the dark signal nonuniformity (DSNU) and photoresponse nonuniformity (PRNU) of two contemporary global shutter CMOS image sensors for machine vision applications. An optimized hardware architecture is proposed to compensate for nonuniformities, with optional parametric lens shading correction (LSC). Three different performance configurations are outlined for different application areas, costs, and power requirements. For most commercial applications, the correction of LSC suffices. For both DSNU and PRNU, compensation with one or multiple calibration images, captured at different gain and temperature settings are considered. For more demanding applications, the effectiveness, external memory bandwidth, power consumption, implementation, and calibration complexity, as well as the camera manufacturability of different nonuniformity correction approaches were compared.

Identification of Source Camera by Amalgamation of PRNU and Noise Print Using Dimensionality Expansive Residual Network

It might be challenging in the field of image forensics to identify the source camera of a picture. This researchproposes a noise adaptable convolutional neural network-based technique for camera identification. For cameraidentification, the suggested solution combines Photo Response Non-Uniformity (PRNU) noise and Noiseprint.Three parallel dimensionality expanded residual networks with convolutional layers of kernel size 1x1 were puttogether for enhanced feature extraction. The experiment mentioned above uses pictures from the ”Vision Dataset”as its subject matter. The experimental findings demonstrate the effectiveness of the suggested methodology inidentifying the source camera at the brand, model, and device levels. When two of the three networks were fedwith PRNU and one with noiseprint, the best performance was obtained.

Direct Optical Lithography Enabled Multispectral Colloidal Quantum-Dot Imagers from Ultraviolet to Short-Wave Infrared.

Complementary metal oxide semiconductor (CMOS) silicon sensors play a central role in optoelectronics with widespread applications from small cell phone cameras to large-format imagers for remote sensing. Despite numerous advantages, their sensing ranges are limited within the visible (0.4-0.7 μm) and near-infrared (0.8-1.1 μm) range , defined by their energy gaps (1.1 eV). However, below or above that spectral range, ultraviolet (UV) and short-wave infrared (SWIR) have been demonstrated in numerous applications such as fingerprint identification, night vision, and composition analysis. In this work, we demonstrate the implementation of multispectral broad-band CMOS-compatible imagers with UV-enhanced visible pixels and SWIR pixels by layer-by-layer direct optical lithography of colloidal quantum dots (CQDs). High-resolution single-color images and merged multispectral images were obtained by using one imager. The photoresponse nonuniformity (PRNU) is below 5% with a 0% dead pixel rate and room-temperature responsivities of 0.25 A/W at 300 nm, 0.4 A/W at 750 nm, and 0.25 A/W at 2.0 μm.

Beyond PRNU: Learning Robust Device-Specific Fingerprint for Source Camera Identification.

Source-camera identification tools assist image forensics investigators to associate an image with a camera. The Photo Response Non-Uniformity (PRNU) noise pattern caused by sensor imperfections has been proven to be an effective way to identify the source camera. However, the PRNU is susceptible to camera settings, scene details, image processing operations (e.g., simple low-pass filtering or JPEG compression), and counter-forensic attacks. A forensic investigator unaware of malicious counter-forensic attacks or incidental image manipulation is at risk of being misled. The spatial synchronization requirement during the matching of two PRNUs also represents a major limitation of the PRNU. To address the PRNU’s fragility issue, in recent years, deep learning-based data-driven approaches have been developed to identify source-camera models. However, the source information learned by existing deep learning models is not able to distinguish individual cameras of the same model. In light of the vulnerabilities of the PRNU fingerprint and data-driven techniques, in this paper, we bring to light the existence of a new robust data-driven device-specific fingerprint in digital images that is capable of identifying individual cameras of the same model in practical forensic scenarios. We discover that the new device fingerprint is location-independent, stochastic, and globally available, which resolves the spatial synchronization issue. Unlike the PRNU, which resides in the high-frequency band, the new device fingerprint is extracted from the low- and mid-frequency bands, which resolves the fragility issue that the PRNU is unable to contend with. Our experiments on various datasets also demonstrate that the new fingerprint is highly resilient to image manipulations such as rotation, gamma correction, and aggressive JPEG compression.