Joint Histogram Research Articles

Design of a hybrid measure for image similarity: a statistical, algebraic, and information-theoretic approach

ABSTRACTImage similarity or distortion assessment is fundamental to a wide range of applications throughout the field of image processing and computer vision. Many image similarity measures have been proposed to treat specific types of image distortions. Most of these measures are based on statistical approaches, such as the classic SSIM. In this paper, we present a different approach by interpolating the information theory with the statistic, because the information theory has a high capability to predict the relationship among image intensity values. Our unique hybrid approach incorporates information theory (Shannon entropy) with a statistic (SSIM), as well as a distinctive structural feature provided by edge detection (Canny). Correlative and algebraic structures have also been utilized. This approach combines the best features of Shannon entropy and a joint histogram of the two images under test, and SSIM with edge detection as a structural feature. The proposed method (ISSM) has been tested versus SSIM and FSIM under Gaussian noise, where good results have been obtained even under a wide range of PSNR. Simulation results using the IVC and TID2008 image databases show that the proposed approach outperforms the SSIM and FSIM approaches in similarity and recognition of the image.

Dynamic Texture Classification Using Unsupervised 3D Filter Learning and Local Binary Encoding

Local binary descriptors, such as local binary pattern (LBP) and its various variants, have been studied extensively in texture and dynamic texture analysis due to their outstanding characteristics, such as grayscale invariance, low computational complexity and good discriminability. Most existing local binary feature extraction methods extract spatio-temporal features from three orthogonal planes of a spatio-temporal volume by viewing a dynamic texture in 3D space. For a given pixel in a video, only a proportion of its surrounding pixels is incorporated in the local binary feature extraction process. We argue that the ignored pixels contain discriminative information that should be explored. To fully utilize the information conveyed by all the pixels in a local neighborhood, we propose extracting local binary features from the spatio-temporal domain with 3D filters that are learned in an unsupervised manner so that the discriminative features along both the spatial and temporal dimensions are captured simultaneously. The proposed approach consists of three components: 1) 3D filtering; 2) binary hashing; and 3) joint histogramming. Densely sampled 3D blocks of a dynamic texture are first normalized to have zero mean and are then filtered by 3D filters that are learned in advance. To preserve more of the structure information, the filter response vectors are decomposed into two complementary components, namely, the signs and the magnitudes, which are further encoded separately into binary codes. The local mean pixels of the 3D blocks are also converted into binary codes. Finally, three types of binary codes are combined via joint or hybrid histograms for the final feature representation. Extensive experiments are conducted on three commonly used dynamic texture databases: 1) UCLA; 2) DynTex; and 3) YUVL. The proposed method provides comparable results to, and even outperforms, many state-of-the-art methods.

Establishing Pose Based Features Using Histograms for the Detection of Abnormal Infant Movements.

The pursuit of early diagnosis of cerebral palsy has been an active research area with some very promising results using tools such as the General Movements Assessment (GMA). In this paper, we conducted a pilot study on extracting important information from video sequences to classify the body movement into two categories, normal and abnormal, and compared the results provided by an independent expert reviewer based on GMA. We present two new pose-based features, Histograms of Joint Orientation 2D (HOJO2D) and Histograms of Joint Displacement 2D (HOJD2D), for the pose-based analysis and classification of infant body movement from video footage. We extract the 2D skeletal joint locations from 2D RGB images using Cao et al.'s method [1]. Using the MINI-RGBD dataset [2], we further segment the body into local regions to extract part specific features. As a result, the pose and the degree of displacement are represented by histograms of normalised data. To demonstrate the effectiveness of the proposed features, we trained several classifiers using combinations of HOJO2D and HOJD2D features and conducted a series of experiments to classify the body movement into categories. The classification algorithms used included k-Nearest Neighbour (kNN, k=1 and k=3), Linear Discriminant Analysis (LDA) and the Ensemble classifier. Encouraging results were attained, with high accuracy (91.67%) obtained using the Ensemble classifier.

A novel joint histogram equalization based image contrast enhancement

The limitation to the most commonly used histogram equalization (HE) technique is the inconsideration of the neighborhood info near each pixel for contrast enhancement. This gives rise to noise in the output image. To overcome this effect, a novel joint histogram equalization (JHE) based technique is suggested. The focus is to utilize the information among each pixel and its neighbors, which improves the contrast of an image. The suggested method is developed in a truly two-dimensional domain. The joint histogram is constructed using the original image and its average image. Further, it does not require a target uniform distribution for generating the output. The two-dimensional cumulative distribution function (CDF) is utilized as a mapping function to get the output pixel intensity. Extensive experiments are performed using 300 test images from BSD database. The experimental analysis indicates that the procedure produces better results than the state-of-the-art HE based contrast enhancement algorithms. More significantly, it produces the best results even for images having a narrow dynamic range. The implementation simplicity of the proposed algorithm may attract researchers to explore the idea for new applications in image processing.

The colours of concrete as seen by X-rays and neutrons

Both x-ray tomography and neutron tomography give very detailed insight in the microstructure of concrete. However, their different contrasts, due to different compositional sensitivities, make one modality more relevant for some features. The present study shows that both types of images acquired on the same specimen may be registered onto each other, after the statistical joint distribution of absorption coefficients has been learned. A Gaussian mixture model has been used to identify up to five different phases having different signatures. A staggered algorithm consisting in i) adjusting the joint histogram to fit phases and their variances and ii) registering the two 3D images onto each other, within a multi-scale algorithm is presented in details. The analysed experimental data illustrates the benefit of using jointly both modalities as compared to their parallel usage.

A Reconfigurable Memory-Based Fast VLSI Architecture for Computation of the Histogram

Histogram computation is a fundamental task often encountered in image processing systems. This plays an important role in various applications like image registration, show-through correction method in scanned image of duplex mode printed document, etc. Mutual information (MI) is one of the best metric for intensity-based multimodality image registration. Similarity measurement between the images consumes significant amount of the execution time in image registration. Computation of MI requires to obtain individual and joint histogram of two images. Typical joint histogram sizes range between 322 and 2562. The demand of hardware resource for computation of histogram increases substantially with increasing of histogram size. The array-based method may not be a suitable candidate in this application, because of large histogram size. Computation of histogram is inherently sequential in nature. But a parallel computation of histogram would reduce the processing time, which is going to help various imaging systems. In this paper, a memory-based parallel algorithm for histogram computation and its possible VLSI architecture have been presented. The architecture is mapped in field programmable gate array. The proposed architecture utilizes 99.66% less of the hardware compared to latest available architecture in the literature and consumes 8.78 mW power.

An Intelligent Sample Selection Method for Space-Time Adaptive Processing in Heterogeneous Environment

In the heterogeneous interference and target-rich environment, the clutter-plus-noise covariance matrix (CCM) estimate with a good match for the cell under test is significant for space–time adaptive processing. In this paper, an intelligent sample selection method is proposed to estimate the CCM and, thus, to suppress the clutter in the heterogeneous background. First, the proposed method applies the sample amplitude and the spatial relationship between the sample and the antenna beam center to describe the characteristics of samples, which preliminarily selects the crucial homogeneous and heterogeneous samples based on the joint histogram property. Then, the generalized inner product value and the spatial included angle are used to work as two representative features for the training samples as well as the testing samples. Based on the feature vector, the multi-dimensional feature projection based on the kernel function is carried out to achieve the offline training classifier, and the testing samples are classified with intelligence in the multi-dimensional feature space by utilizing the trained support vector machine. Due to the utilization of space–time property in representative feature extraction and intelligent strategy for the decision-making process, the proposed method acquires more reliable and efficient sample selection results compared with most of conventional sample selection methods. The experimental results based on the simulated and measured data show the effectiveness of the proposed method.

MRI-based attenuation correction for brain PET/MRI based on anatomic signature and machine learning

Deriving accurate attenuation maps for PET/MRI remains a challenging problem because MRI voxel intensities are not related to properties of photon attenuation and bone/air interfaces have similarly low signal. This work presents a learning-based method to derive patient-specific computed tomography (CT) maps from routine T1-weighted MRI in their native space for attenuation correction of brain PET. We developed a machine-learning-based method using a sequence of alternating random forests under the framework of an iterative refinement model. Anatomical feature selection is included in both training and predication stages to achieve optimal performance. To evaluate its accuracy, we retrospectively investigated 17 patients, each of which has been scanned by PET/CT and MR for brain. The PET images were corrected for attenuation on CT images as ground truth, as well as on pseudo CT (PCT) images generated from MR images. The PCT images showed mean average error of 66.1 ± 8.5 HU, average correlation coefficient of 0.974 ± 0.018 and average Dice similarity coefficient (DSC) larger than 0.85 for air, bone and soft tissue. The side-by-side image comparisons and joint histograms demonstrated very good agreement of PET images after correction by PCT and CT. The mean differences of voxel values in selected VOIs were less than 4%, the mean absolute difference of all active area is around 2.5%, and the mean linear correlation coefficient is 0.989 ± 0.017 between PET images corrected by CT and PCT. This work demonstrates a novel learning-based approach to automatically generate CT images from routine T1-weighted MR images based on a random forest regression with patch-based anatomical signatures to effectively capture the relationship between the CT and MR images. Reconstructed PET images using the PCT exhibit errors well below accepted test/retest reliability of PET/CT indicating high quantitative equivalence.

Texture Representation Through Overlapped Multi-Oriented Tri-Scale Local Binary Pattern

This paper ideates a novel texture descriptor that retains its classification accuracy under varying conditions of image orientation, scale, and illumination. The proposed Overlapped Multi-oriented Tri-scale Local Binary Pattern (OMTLBP) texture descriptor also remains insensitive to additive white Gaussian noise. The wavelet decomposition stage of the OMTLBP provides robustness to photometric variations, while the two subsequent stages - overlapped multi-oriented fusion and multi-scale fusion - provide resilience against geometric transformations within an image. Isolated encoding of constituent pixels along each scale in the joint histogram enables the proposed descriptor to capture both micro and macro structures within the texture. Performance of the OMTLBP is evaluated by classifying a variety of textured images belonging to Outex, KTH-TIPS, Brodatz, CUReT, and UIUC datasets. The experimental results validate the superiority of the proposed method in terms of classification accuracy when compared with the state-of-the-art texture descriptors for noisy images.

A Comparison of Cloud Classification Methodologies: Differences Between Cloud and Dynamical Regimes

AbstractClassifications of cloud data into Cloud Regimes (CRs) and compositing based on meteorological parameters, Dynamic Regimes (DRs), are often used in the analysis of clouds. We compare CR and DR classifications to understand the relative merits of these approaches and develop a comparison methodology for future studies. We apply the Self‐Organizing Map technique to International Satellite Cloud Climatology Project (ISCCP) D1 joint histograms to produce a CR and ERA‐Interim pressure vertical velocity output to produce a DR. The CR created improves the separation between high‐level CRs compared to previous work. Composites of ISCCP joint histogram data using the DR produce coherent groupings similar to those in the CR scheme particularly in regions of ascent. Both classifications display coherent geographical patterns and reproduce relationships between vertical velocity and cloud properties. However, the CR produces more coherent clusters with higher intracluster similarity and a greater range of independent cloud classes. Independent tests of composites using ISCCP FD output show that the regional variability of longwave cloud radiative effect for particular nodes are significantly higher in the DR than the CR scheme suggesting a poorer classification. Composite mean CloudSat reflectivity‐altitude joint histograms represent all major cloud types in the CR scheme, while the current DR grouping is less coherent and misses classes. This suggests that the CR scheme is a more useful classification than the DR scheme based solely on vertical velocity data. Contingency table analysis indicates a low association between these classifications, suggesting combining these schemes would be valuable.

An Improved Workflow for Permeability Estimation from Image Logs with Uncertainty Quantification

Production forecasts are extremely sensitive to the spatial distribution of certain reservoir parameters, especially permeability, which controls fluid flow and efficiency of many recovery methods. Modeling the heterogeneity in permeability distribution is challenging due to various reasons. First, proper sampling is difficult due to biased sampling, dilation effects and sample disturbance. Second, calibration of core-plug measurements with well test data is problematic because of their vast differences in scale; in the case of heavy oil reservoirs, conventional well tests are not feasible due to bitumen’s ultra-low mobility at initial conditions. Therefore, in addition to flow-based measurements, alternative static data including image logs and core photographs, which capture detailed high-resolution information regarding heterogeneities at the millimeter scale, may be integrated for reliable permeability estimation along the wellbore. This paper examines the existing technique for permeability prediction from image logs (or core photographs) and proposes a set of improvements. For example, automatic variogram modeling is performed over a set of moving windows to simulate the sand–shale indicators along the well trajectory, capturing the locally varying anisotropy in variogram model parameters. In addition, it presents a unified approach to properly quantify the uncertainty in the histogram of core measurements and well log data: Joint histogram uncertainty considering the correlations between variables is incorporated into the existing methodology. The millimeter-scale model is constructed and upscaled to the centimeter scale via steady-state flow-based upscaling. This step is repeated with the centimeter-scale model until the final flow-simulation scale is attained. The entire methodology is applied within individual facies with distinct spatial correlations. A field case study involving data from the McMurray Formation (Alberta, Canada) is presented, where the permeability distributions in five different facies are modeled. The results indicate that considering automatic variogram modeling may lead to more realistic representation of the orientation of certain heterogeneous features that are strongly influencing the permeability to flow; furthermore, incorporating histogram uncertainty enables the modeling of porosity and permeability with improved uncertainty.

A hierarchical CBIR framework using adaptive tetrolet transform and novel histograms from color and shape features

Content-Based Image Retrieval (CBIR) systems retrieve the most analogous images from the image database with respect to a given query image based on the texture, shape, and/or color image features. These three image features can be used alone for the image retrieval or also can be used together for the retrieval purpose. In hierarchical CBIR system, three image features are extracted in proper order to discard the irrelevant images in each hierarchy level for reducing the image search space. In this paper, the authors have proposed a three-level hierarchical CBIR system/framework where, each level of the hierarchy uses either texture, shape or color image features to reduce the size of the image database by discarding the irrelevant images and at final level of the hierarchy, it will extract the most analogous images from the reduced image database. We have used adaptive tetrolet transform to extract the texture features from the regions of interest of the images. To extract the shape features of the image, a novel edge joint histogram has been proposed which uses the orientation of the edge pixels and their distance from the origin together to create a novel joint histogram. For color feature extraction, another color channel correlation histogram has been introduced. The order of the three different feature extraction processes on each level of the hierarchy is not rigid because it is difficult to predict the proper order for the highest retrieval. In the experiment, we have considered all possible order of the texture, shape and color features for image retrieval process. The retrieval experiments have been carried out in six different types of standard image databases and results show that the performance of proposed CBIR system has been increased significantly as compared to the other state-of-arts CBIR systems.

An analysis of the cloud environment over the Ross Sea and Ross Ice Shelf using CloudSat/CALIPSO satellite observations: the importance of synoptic forcing

Abstract. We use the 2B-GEOPROF-LIDAR R04 (2BGL4) and R05 (2BGL5) products and the 2B-CLDCLASS-LIDAR R04 (2BCL4) product, all generated by combining CloudSat radar and CALIPSO lidar satellite measurements with auxiliary data, to examine the vertical distribution of cloud occurrence around the Ross Ice Shelf (RIS) and Ross Sea region. We find that the 2BGL4 product, used in previous studies in this region, displays a discontinuity at 8.2 km which is not observable in the other products. This artefact appears to correspond to a change in the horizontal and vertical resolution of the CALIPSO dataset used above this level. We then use the 2BCL4 product to examine the vertical distribution of cloud occurrence, phase, and type over the RIS and Ross Sea. In particular we examine how synoptic conditions in the region, derived using a previously developed synoptic classification, impact the cloud environment and the contrasting response in the two regions. We observe large differences between the cloud occurrence as a function of altitude for synoptic regimes relative to those for seasonal variations. A stronger variation in the occurrence of clear skies and multi-layer cloud and in all cloud type occurrences over both the Ross Sea and RIS is associated more with synoptic type than seasonal composites. In addition, anomalies from the mean joint histogram of cloud top height against thickness display significant differences over the Ross Sea and RIS sectors as a function of synoptic regime, but are near identical over these two regions when a seasonal analysis is completed. However, the frequency of particular phases of cloud, notably mixed phase and water, is much more strongly modulated by seasonal than synoptic regime compositing, which suggests that temperature is still the most important control on cloud phase in the region.

An Entropy-Histogram Approach for Image Similarity and Face Recognition

Image similarity and image recognition are modern and rapidly growing technologies because of their wide use in the field of digital image processing. It is possible to recognize the face image of a specific person by finding the similarity between the images of the same person face and this is what we will address in detail in this paper. In this paper, we designed two new measures for image similarity and image recognition simultaneously. The proposed measures are based mainly on a combination of information theory and joint histogram. Information theory has a high capability to predict the relationship between image intensity values. The joint histogram is based mainly on selecting a set of local pixel features to construct a multidimensional histogram. The proposed approach incorporates the concepts of entropy and a modified 1D version of the 2D joint histogram of the two images under test. Two entropy measures were considered, Shannon and Renyi, giving a rise to two joint histogram-based, information-theoretic similarity measures: SHS and RSM. The proposed methods have been tested against powerful Zernike-moments approach with Euclidean and Minkowski distance metrics for image recognition and well-known statistical approaches for image similarity such as structural similarity index measure (SSIM), feature similarity index measure (FSIM) and feature-based structural measure (FSM). A comparison with a recent information-theoretic measure (ISSIM) has also been considered. A measure of recognition confidence is introduced in this work based on similarity distance between the best match and the second-best match in the face database during the face recognition process. Simulation results using AT&T and FEI face databases show that the proposed approaches outperform existing image recognition methods in terms of recognition confidence. TID2008 and IVC image databases show that SHS and RSM outperform existing similarity methods in terms of similarity confidence.

Dynamic Texture Recognition Using Time-Causal and Time-Recursive Spatio-Temporal Receptive Fields

This work presents a first evaluation of using spatio-temporal receptive fields from a recently proposed time-causal spatio-temporal scale-space framework as primitives for video analysis. We propose a new family of video descriptors based on regional statistics of spatio-temporal receptive field responses and evaluate this approach on the problem of dynamic texture recognition. Our approach generalises a previously used method, based on joint histograms of receptive field responses, from the spatial to the spatio-temporal domain and from object recognition to dynamic texture recognition. The time-recursive formulation enables computationally efficient time-causal recognition. The experimental evaluation demonstrates competitive performance compared to state of the art. In particular, it is shown that binary versions of our dynamic texture descriptors achieve improved performance compared to a large range of similar methods using different primitives either handcrafted or learned from data. Further, our qualitative and quantitative investigation into parameter choices and the use of different sets of receptive fields highlights the robustness and flexibility of our approach. Together, these results support the descriptive power of this family of time-causal spatio-temporal receptive fields, validate our approach for dynamic texture recognition and point towards the possibility of designing a range of video analysis methods based on these new time-causal spatio-temporal primitives.

Mixed co-occurrence of local binary patterns and Hamming-distance-based local binary patterns

Local binary patterns (LBP) have powerful discriminative capabilities. However, traditional methods with LBP histograms cannot capture spatial structures of LBP codes. To extract the spatial structures of an LBP code map, we compute and encode the Hamming distances between LBP codes of a center point and its neighbors on the LBP code map to generate a new code, which is called Hamming-distance-based local binary patterns (HDLBP). Then, we calculate a joint histogram of LBP and HDLBP to represent the LBP co-occurrence with HDLBP (LBPCoHDLBP). Circular bit-wise shift techniques are used to align HDLBP with LBP for rotation invariance. To achieve scale invariance, we extract the feature of LBPCoHDLBP from each scale and concatenate all features of different scales. Finally, we use the sum of absolute differences (SAD) between the intensities of the center point and its neighbors to weight LBPCoHDLBP for further improvement. Extensive experiments show that our method achieves better performance for smoke detection, texture classification and material recognition than most existing methods and is more computationally efficient.

Regional Regime‐Based Evaluation of Present‐Day General Circulation Model Cloud Simulations Using Self‐Organizing Maps

AbstractGlobal clusters are derived by applying the self‐organizing map technique to the Moderate Resolution Imaging Spectroradiometer cloud top pressure‐cloud optical thickness joint histograms. These cloud clusters are then used to classify Cloud Feedback Model Intercomparison Project Observation Simulator Package output from the HadGEM3 (Global Atmosphere version 7) atmosphere‐only climate model. Discrepancies in the Global Atmosphere version 7 representation of particular clusters can be established by examining the two sets of cluster's occurrence rate and radiative effect. The overall differences in the occurrence rates show major discrepancies in several of the clusters, resulting in a shift from five dominant clusters in Moderate Resolution Imaging Spectroradiometer (above 10% occurrence rate) to two dominant clusters in the model. A comparison of the geographic distributions of occurrence rate shows that the differences are strongly regional and unique to each cluster. While comparisons of the global mean longwave and shortwave cloud radiative effect (CRE) show strong agreement, examination of the CRE of individual cloud types reveals larger errors that highlight the role of compensating errors in masking model deficiencies. CRE data for each of the clusters is further partitioned into regions. This establishes that the bias associated with a cluster is highly variable globally, with no clusters showing consistent biases across all regions. Therefore, regional level phenomena likely play an important role in the creation of these errors.

Applying stochastic second-order entropy images to multi-modal image registration

Which metric to use for multi-modal image registration is still a nontrivial research problem. Recently, some methods have used structural representations of images to address this problem. Efficiency and optimization simplicity are the advantages of these algorithms; however, they have some limitations. First, structural representation based registrations often fail when there are intensity variations in the patch (local block) of the image. Second, structural representation based methods are not as accurate as mutual information based methods. In this article, the shortcomings of structural image representation are overcome by devising a new similarity metric called Stochastic Second-Order Entropy Image (SSOEI). We interpolate the neighbourhood intensity information of random pixels in each patch to estimate the histogram of the intensity distribution. Then, entropy of a patch can be computed by this joint histogram. An entropy image is a collection of the entropy value of every image patch. Then, the sum of squared difference from the entropy image can be utilized as the metric of the registration framework. The robustness and accuracy of SSOEI were tested on both synthetic and clinical data, and the results showed the advantages of SSOEI over the state-of-art methods.

Image Retrieval System based on Color Global and Local Features Combined with GLCM for Texture Features

In CBIR (content-based image retrieval) features are extracted based on color, texture, and shape. There are many factors affecting the accuracy (precision) of retrieval such as number of features, type of features (local or global), color model, and distance measure. In this paper, a two phases approach to retrieve similar images from data set based on color and texture is proposed. In the first phase, global color histogram is utilized with HSV (hue, saturation, and value) color model and an automatic cropping technique is proposed to accelerate the process of features extraction and enhances the accuracy of retrieval. Joint histogram and GLCM (gray-level co-occurrence matric) are deployed in phase two. In this phase, color features and texture features are combined to enhance the accuracy of retrieval. Finally, a new way of using K-means as clustering algorithm is proposed to classify and retrieve images. Two experiments are conducted using WANG database. WANG database consists of 10 different classes each with 100 images. Results of comparing the proposed approach with the most relevant approaches are promising.

Multimodal Medical Image Registration Based on an Information-Theory Measure with Histogram Estimation of Continuous Image Representation

This work presents a novel method for multimodal medical registration based on histogram estimation of continuous image representation. The proposed method, regarded as “fast continuous histogram estimation,” employs continuous image representation to estimate the joint histogram of two images to be registered. The Jensen–Arimoto (JA) divergence is a similarity measure to measure the statistical dependence between medical images from different modalities. The estimated joint histogram is exploited to calculate the JA divergence in multimodal medical image registration. In addition, to reduce the grid effect caused by the grid-aligning transformations between two images and improve the implementation speed of the registration method, random samples instead of all pixels are extracted from the images to be registered. The goal of the registration is to optimize the JA divergence, which would be maximal when two misregistered images are perfectly aligned using the downhill simplex method, and thus to get the optimal geometric transformation. Experiments are conducted on an affine registration of 2D and 3D medical images. Results demonstrate the superior performance of the proposed method compared to standard histogram, Parzen window estimations, particle filter, and histogram estimation based on continuous image representation without random sampling.