Global Image Statistics Research Articles

Jointly learning and training: using style diversification to improve domain generalization for deepfake detection

<abstract><p>Most existing deepfake detection methods often fail to maintain their performance when confronting new test domains. To address this issue, we propose a generalizable deepfake detection system to implement style diversification by alternately learning the domain generalization (DG)-based detector and the stylized fake face synthesizer (SFFS). For the DG-based detector, we first adopt instance normalization- and batch normalization-based structures to extract the local and global image statistics as the style and content features, which are then leveraged to obtain the more diverse feature space. Subsequently, contrastive learning is used to emphasize common style features while suppressing domain-specific ones, and adversarial learning is performed to obtain the domain-invariant features. These optimized features help the DG-based detector to learn generalized classification features and also encourage the SFFS to simulate possibly unseen domain data. In return, the samples generated by the SFFS would contribute to the detector's learning of more generalized features from augmented training data. Such a joint learning and training process enhances both the detector's and the synthesizer's feature representation capability for generalizable deepfake detection. Experimental results demonstrate that our method outperforms the state-of-the-art competitors not only in intra-domain tests but particularly in cross-domain tests.</p></abstract>

A Simple Denoising Algorithm for Real-World Noisy Camera Images.

The noise statistics of real-world camera images are challenging for any denoising algorithm. Here, I describe a modified version of a bionic algorithm that improves the quality of real-word noisy camera images from a publicly available image dataset. In the first step, an adaptive local averaging filter was executed for each pixel to remove moderate sensor noise while preserving fine image details and object contours. In the second step, image sharpness was enhanced by means of an unsharp mask filter to generate output images that are close to ground-truth images (multiple averages of static camera images). The performance of this denoising algorithm was compared with five popular denoising methods: bm3d, wavelet, non-local means (NL-means), total variation (TV) denoising and bilateral filter. Results show that the two-step filter had a performance that was similar to NL-means and TV filtering. Bm3d had the best denoising performance but sometimes led to blurry images. This novel two-step filter only depends on a single parameter that can be obtained from global image statistics. To reduce computation time, denoising was restricted to the Y channel of YUV-transformed images and four image segments were simultaneously processed in parallel on a multi-core processor.

Texture and material classification with multi-scale ternary and septenary patterns

This paper ideates, inspired by LBP and its variants, a novel local feature extraction operator for texture classification, referred to as Multi-scale Ternary and Septenary Pattern (MTSP). MTSP is a histogram-based feature reṕresentation that is composed of two single-scale STP and SSP (single-scale ternary and septenary patterns, respectively) encoders designed according to a novel set theory based pattern encoding scheme that integrates the concepts of both LQP’s and LTP’s operators. The essence of STP and SSP is to compute several virtual pixels based on various local and global image statistics and progressively encode interactions between local and non-local pixels by examining the directional information and differential excitation according to relationships between adjacent pixels rearranged in a variety of spatial arrangements. Unlike various parametric state-of-the-art texture operators that perform thresholding based on static thresholds, MTSP incorporates dynamic thresholds estimated automatically. MTSP descriptor has good ability as faithfully as possible to capture more detailed image information via complementary texture information generated from the fusion of both STP and SSP encoders. Experimental results show that MTSP ensures reliable performance stability over ten texture datasets and against several recent representative methods. In addition, the performance of MTSP is further proved statistically via the Wilcoxon signed rank test demonstrating thus that MTSP is a good candidate for texture modeling.

Photorealistic Reconstruction of Visual Texture From EEG Signals.

Recent advances in brain decoding have made it possible to classify image categories based on neural activity. Increasing numbers of studies have further attempted to reconstruct the image itself. However, because images of objects and scenes inherently involve spatial layout information, the reconstruction usually requires retinotopically organized neural data with high spatial resolution, such as fMRI signals. In contrast, spatial layout does not matter in the perception of “texture,” which is known to be represented as spatially global image statistics in the visual cortex. This property of “texture” enables us to reconstruct the perceived image from EEG signals, which have a low spatial resolution. Here, we propose an MVAE-based approach for reconstructing texture images from visual evoked potentials measured from observers viewing natural textures such as the textures of various surfaces and object ensembles. This approach allowed us to reconstruct images that perceptually resemble the original textures with a photographic appearance. The present approach can be used as a method for decoding the highly detailed “impression” of sensory stimuli from brain activity.

Steady-state visual evoked potential responses predict visual discomfort judgements.

It has been suggested that aesthetically pleasing stimuli are processed efficiently by the visual system, whereas uncomfortable stimuli are processed inefficiently. This study consists of a series of three experiments investigating this idea using a range of images of abstract artworks, photographs of natural scenes, and computer-generated stimuli previously shown to be uncomfortable. Subjective judgements and neural correlates were measured using electroencephalogram (EEG) (steady-state visual evoked potentials, SSVEPs). In addition, global image statistics (contrast, Fourier amplitude spectral slope and fractal dimension) were taken into account. When effects of physical image contrast were controlled, fractal dimension predicted discomfort judgements, suggesting the SSVEP response is more likely to be influenced by distribution of edges than the spectral slope. Importantly, when effects of physical contrast and fractal dimension were accounted for using linear mixed effects modelling, SSVEP responses predicted subjective judgements of images. Specifically, when stimuli were not matched for perceived contrast, there was a positive relationship between SSVEP responses and how pleasing a stimulus was judged to be, and conversely a negative relationship between discomfort and SSVEP response. This is significant as it shows that the neural responses in early visual areas contribute to the subjective (un)pleasantness of images, although the results of this study do not provide clear support for the theory of efficient coding as the cause of perceived pleasantness or discomfort of images, and so other explanations need to be considered.

A statistical active contour model for interactive clutter image segmentation using graph cut optimization

This paper presents a statistical region based Active Contour Model (ACM) considering the correlation between local and global image statistics to segment cluttered images. Generally, cluttered images do not have constant intensity distribution; rather, the intensity may follow near constant variation in different regions. To quantify this variation, we have considered the Coefficient of Variation (CoV) of the regions interior and exterior to the contour as global statistics and the CoV in the local patches as local statistics. Subsequently, the region energy term of the proposed ACM is designed such that it minimizes the difference between the local and global statistics i.e. it encourages CoV for all the local patches inside and outside of the final contour to be nearly homogeneous. Further, we have verified that the energy formulation can be efficiently discretized and solved using graph cut optimization. The main advantages of graph-based formulation over level set formulation are the existence of a global optimal solution and lesser sensitivity to contour initialization. Additionally, the former formulation is significantly faster being non-iterative or convergable with very few iterations. Experimental results demonstrate the superior performance of our approach against other state-of-the-art active contour approaches and also over its level set counterpart.

Localised edge‐region‐based active contour for medical image segmentation

Segmenting the region of interest (ROI) from medical images is a fundamental but challenging task due to the illumination change and imaging devices. Although many models based on the local region-based active contour model (LR-ACM) are proposed to deal with intensity inhomogeneity, it is still difficult for the global energy-based ACM with local image information to accurately extract the ROI from medical images. To solve this problem, this study proposes a novel localised ACM by constructing the gradient information based on the probability scores from the fuzzy k-nearest neighbour classifier. Different from the traditional LR-ACMs, our model is based on local rather than global image statistics, where a probability score-based edge detector is directly used for gradient information. The energy functional consists of localised region energy and an edge energy. By introducing a local characteristics function, the localised region energy with the probability-score-based edge information is formulated, which can make the evolution curve stop on the exact boundaries of ROI. The edge energy including the regularisation and the penalty terms is used to avoid the reinitialisation process and smooth the evolution curve during evolution. To maintain the signed distance property of the evolution curve, a novel potential function in the penalty term is designed, which can consistently control the diffusion direction of the evolution curve. Experiments on the medical images including the cardiac magnetic resonance imaging and the 3DIRCADb databases demonstrate that the proposed model is more robust and accurate to extract the ROI than the popular localised and region-based ACMs. The code is available at: https://github.com/HuaxiangLiu/Localized_ERAC/.

When Scenes Look Like Materials: René Magritte’s Reversible Figure–Ground Motif

We draw attention to a frequent motif in the work of the Belgian surrealist René Magritte (1898–1967). In the motif, a scene is depicted that contains a silhouette, which itself contains another depicted scene. The silhouette is bistable, appearing either as a figural region whose positive space is covered, or filled, with the interior scene texture, or as a ground region providing a window onto a more distant scene. We call this the ‘reversible figure–ground motif’. Because the stimulus does not change when our percept changes, the motif’s appearance at any particular moment cannot be explained by its local or global image statistics. Instead principles of perceptual organization, and in particular image segmentation and figure–ground assignment, appear crucial for determining whether the interior of the silhouette is processed as a material vs. a scene — which in turn reflects the fundamental role of visual segmentation in material and scene perception more generally.

Robust Texture Description Using Local Grouped Order Pattern and Non-Local Binary Pattern

Local binary pattern (LBP) and its many variants have shown effectiveness for texture classification. However, most of these LBP methods focus on encoding local intensity differences between a central pixel and its neighboring sampling points and consequently have two major problems: 1) they are unable to describe the intensity order relationships among neighboring sampling points, and 2) they fail to capture long-range pixel interactions that take place outside a compact neighborhood. In view of these problems, in this paper we propose two novel operators, called local grouped order pattern (LGOP) and non-local binary pattern (NLBP), for texture description. For the first problem, LGOP groups the neighboring sampling points by referring to a dominant direction and encodes the groupwise intensity order relationships. For the second problem, NLBP computes several anchors based on global image statistics and progressively encodes non-local intensity differences between the neighboring sampling points and anchors. Finally, we combine LGOP and NLBP via central pixel encoding to construct discriminative histogram features as texture descriptor LGONBP. Experiments on four texture benchmark databases (i.e., Outex, CUReT, UMD and KTH-TIPS) demonstrate the superiority of LGONBP over state-of-the-art LBP variants for texture classification under both noise-free and noisy conditions. The code is available at <uri>https://github.com/stc-cqupt/LGONBP</uri>.

Single Image Highlight Removal for Real-Time Image Processing Pipelines

This paper presents a fully automatic method for the separation of diffuse and specular reflection components from a single image. Overall, the mechanisms in which the available methods operate on are computationally costly and do not translate well to modern hardware-implemented image processing pipelines, such as the ones present in consumer electronics. Consequently, the objective of this article is to introduce a simple yet effective method for specular highlight removal. It is based on the dichromatic reflection model and operates through histogram matching in the YCbCr color space. The proposed method performs in real-time. It only uses global image statistics and point-wise intensity transformations. Experimental evaluation shows that the proposed approach has competitive results in comparison to state-of-the-art methods. Limitations of the proposed approach are seldom and are common to most methods available. The proposed method, however, achieves better quality results with much less computational cost, thus enabling feasibility in systems with limited processing power.

Hybrid SPF and KD Operator-Based Active Contour Model for Image Segmentation

Image segmentation is a crucial stage of image analysis systems because it detects and extracts regions of interest for further processing, such as image recognition and the image description. However, segmenting images is not always easy because segmentation accuracy depends significantly on image characteristics, such as color, texture, and intensity. Image inhomogeneity profoundly degrades the segmentation performance of segmentation models. This article contributes to image segmentation literature by presenting a hybrid Active Contour Model (ACM) based on a Signed Pressure Force (SPF) function parameterized with a Kernel Difference (KD) operator. An SPF function includes information from both the local and global regions, making the proposed model independent of the initial contour position. The proposed model uses an optimal KD operator parameterized with weight coefficients to capture weak and blurred boundaries of inhomogeneous objects in images. Combined global and local image statistics were computed and added to the proposed energy function to increase the proposed model's sensitivity. The segmentation time complexity of the proposed model was calculated and compared with previous state-of-the-art active contour methods. The results demonstrated the significant superiority of the proposed model over other methods. Furthermore, a quantitative analysis was performed using the mini-MIAS database. Despite the presence of complex inhomogeneity, the proposed model demonstrated the highest segmentation accuracy when compared to other methods.

Assessing Neural Network Scene Classification from Degraded Images

Scene recognition is an essential component of both machine and biological vision. Recent advances in computer vision using deep convolutional neural networks (CNNs) have demonstrated impressive sophistication in scene recognition, through training on large datasets of labeled scene images (Zhou et al. 2018, 2014). One criticism of CNN-based approaches is that performance may not generalize well beyond the training image set (Torralba and Efros 2011), and may be hampered by minor image modifications, which in some cases are barely perceptible to the human eye (Goodfellow et al. 2015; Szegedy et al. 2013). While these “adversarial examples” may be unlikely in natural contexts, during many real-world visual tasks scene information can be degraded or limited due to defocus blur, camera motion, sensor noise, or occluding objects. Here, we quantify the impact of several image degradations (some common, and some more exotic) on indoor/outdoor scene classification using CNNs. For comparison, we use human observers as a benchmark, and also evaluate performance against classifiers using limited, manually selected descriptors. While the CNNs outperformed the other classifiers and rivaled human accuracy for intact images, our results show that their classification accuracy is more affected by image degradations than human observers. On a practical level, however, accuracy of the CNNs remained well above chance for a wide range of image manipulations that disrupted both local and global image statistics. We also examine the level of image-by-image agreement with human observers, and find that the CNNs’ agreement with observers varied as a function of the nature of image manipulation. In many cases, this agreement was not substantially different from the level one would expect to observe for two independent classifiers. Together, these results suggest that CNN-based scene classification techniques are relatively robust to several image degradations. However, the pattern of classifications obtained for ambiguous images does not appear to closely reflect the strategies employed by human observers.

River channel extraction from synthetic aperture radar images based on region-based active contour model

River channel extraction is an active and challenging task in many applications, for improving the accuracy and efficiency of river channel extraction, and the paper presents a region-based active contour model, which is driven by the logarithm kernel-based energy function. Firstly, the regional energy function is defined by the logarithm kernel distance which is robust and non-Euclidean. Secondly, the local median information of image is considered rather than global image statistics to reduce the influence of intensity inhomogeneity presented in each region. Furthermore, the interior and exterior energies are calculated by the variation which integrates each pixel with fitting center. Finally, the distance regularized term is introduced to avoid re-initialization of the level set function. Experimental results show that, comparing with other state-of-the-art models, the proposed model can be applied to synthetic aperture radar river channel extraction with desirable results.

Increasing the Complexity of the Illumination May Reduce Gloss Constancy.

We examined in which way gradual changes in the geometric structure of the illumination affect the perceived glossiness of a surface. The test stimuli were computer-generated three-dimensional scenes with a single test object that was illuminated by three point light sources, whose relative positions in space were systematically varied. In the first experiment, the subjects were asked to adjust the microscale smoothness of a match object illuminated by a single light source such that it has the same perceived glossiness as the test stimulus. We found that small changes in the structure of the light field can induce dramatic changes in perceived glossiness and that this effect is modulated by the microscale smoothness of the test object. The results of a second experiment indicate that the degree of overlap of nearby highlights plays a major role in this effect: Whenever the degree of overlap in a group of highlights is so large that they perceptually merge into a single highlight, the glossiness of the surface is systematically underestimated. In addition, we examined the predictability of the smoothness settings by a linear model that is based on a set of four different global image statistics.

Design and Implementation of Gain-Offset Correction Algorithm Hardware Architecture for Grayscale and Color Images Contrast Enhancement

This paper describes the design and implementation of a novel, high-speed hardware (HW) architecture for the gain-offset correction (GOC) image contrast enhancement algorithm on an FPGA fabric. The design is extremely fast and has been shown to process megapixel image frames at frame rates greatly exceeding real-time requirements. The design is small and compact enough to fit on small FPGAs to form a low-cost image processing solution. The automated nature of the contrast enhancement operation is due to the computation of global image statistics for each image. The architecture does not store any image frames to perform this task, heavily reducing memory requirements.

A robust region-based active contour for object segmentation in heterogeneous case

In this paper, we propose a new approach using a local version of the region-based active contour for object segmentation in images presenting heterogeneity in both the object of interest and the background. The local version was recently developed to deal with heterogeneous appearances by relying on extracting the local instead of global image statistics where the local extracted area is defined by using a disk, sketched at each point along the active contour, with a constant radius size inside and outside the contour. However, the use of a constant radius may prevent the active contour getting more information from its neighborhood and thus being trapped by undesired boundaries. To avoid this error segmentation, the local extracted area using our proposed approach is determined based on using two different radii to extract separately the interior and the exterior local information of the active contour. Using synthetic and real images, our approach shows an improvement in term of computation time and outperforms the conventional methods in noisy images presenting heterogeneity in both the object of interest and the background and using inadequate contour initialization.

Robust Model for Segmenting Images With/Without Intensity Inhomogeneities

Intensity inhomogeneities and different types/levels of image noise are the two major obstacles to accurate image segmentation by region-based level set models. To provide a more general solution to these challenges, we propose a novel segmentation model that considers global and local image statistics to eliminate the influence of image noise and to compensate for intensity inhomogeneities. In our model, the global energy derived from a Gaussian model estimates the intensity distribution of the target object and background; the local energy derived from the mutual influences of neighboring pixels can eliminate the impact of image noise and intensity inhomogeneities. The robustness of our method is validated on segmenting synthetic images with/without intensity inhomogeneities, and with different types/levels of noise, including Gaussian noise, speckle noise, and salt and pepper noise, as well as images from different medical imaging modalities. Quantitative experimental comparisons demonstrate that our method is more robust and more accurate in segmenting the images with intensity inhomogeneities than the local binary fitting technique and its more recent systematic model. Our technique also outperformed the region-based Chan–Vese model when dealing with images without intensity inhomogeneities and produce better segmentation results than the graph-based algorithms including graph-cuts and random walker when segmenting noisy images.

Memory modulated saliency: A computational model of the incremental learning of target locations in visual search

The top-down guidance of visual attention is one of the main factors allowing humans to effectively process vast amounts of incoming visual information. Nevertheless we still lack a full understanding of the visual, semantic, and memory processes governing visual attention. In this paper, we present a computational model of visual search capable of predicting the most likely positions of target objects. The model does not require a separate training phase, but learns likely target positions in an incremental fashion based on a memory of previous fixations. We evaluate the model on two search tasks and show that it outperforms saliency alone and comes close to the maximal performance of the Contextual Guidance Model (CGM; Torralba, Oliva, Castelhano, & Henderson, 2006; Ehinger, Hidalgo-Sotelo, Torralba, & Oliva, 2009), even though our model does not perform scene recognition or compute global image statistics. The search performance of our model can be further improved by combining it with the CGM.

Tera-scale astronomical data analysis and visualization

We present a high-performance, graphics processing unit (GPU)-based framework for the efficient analysis and visualization of (nearly) terabyte (TB)-sized 3-dimensional images. Using a cluster of 96 GPUs, we demonstrate for a 0.5 TB image: (1) volume rendering using an arbitrary transfer function at 7--10 frames per second; (2) computation of basic global image statistics such as the mean intensity and standard deviation in 1.7 s; (3) evaluation of the image histogram in 4 s; and (4) evaluation of the global image median intensity in just 45 s. Our measured results correspond to a raw computational throughput approaching one teravoxel per second, and are 10--100 times faster than the best possible performance with traditional single-node, multi-core CPU implementations. A scalability analysis shows the framework will scale well to images sized 1 TB and beyond. Other parallel data analysis algorithms can be added to the framework with relative ease, and accordingly, we present our framework as a possible solution to the image analysis and visualization requirements of next-generation telescopes, including the forthcoming Square Kilometre Array pathfinder radiotelescopes.

IMAGE COSEGMENTATION BASED ON LOCAL AND GLOBAL LEVEL SET METHODS

In this paper, we propose a novel cosegmentation algorithm based on active contour model which utilizes local and global image statistics. Many localized region-based active contour models have been proposed to solve a challenging problem of the property (such as intensity, color, texture, etc.) inhomogeneities that often occurs in real images, but these models usually cannot reasonably evolve the curve in this situation that some center points along the curve are in homogeneous regions and their local regions are far away from the object. In order to overcome the difficulties we selectively enlarge the driven force of some points and introduce the edge indicator function to avoid the curve over-shrinking or over-expanding on the salient boundaries. In addition, we introduce global image statistics to better the curve evolution and try to avoid the given energy functional converging to a local minimum. Practical experiments show that our algorithm can obtain better segmentation results.