Local Binary Pattern Operator Research Articles

APPLICATION OF NEW RETRIEVAL METHODS IN MICROSCOPIC MEDICAL IMAGES AND CT IMAGES

Medical image retrieval is of great significance for forming correct medical judgments during the diagnosis and treatment process. In response to the fact that there are rich types of features in medical images, this paper constructs a multi-feature fusion model and applies it to medical image retrieval. This multi-feature fusion model utilizes three features. Color information is characterized using three different moments, texture information is obtained using the LBP (Local Binary Pattern) operator, and shape information is described using Hu moments. After collecting three features of medical images, they are fused into a similarity measure model through a self-set weight structure for comparison in medical image retrieval. Experiments show that our method has satisfactory image retrieval result for medical images such as endoscopic images and computed tomography (CT) images. Despite the continuous improvement of recall indicators, the proposed method still has high precision.

Camera model identification based on dual-path enhanced ConvNeXt network and patches selected by uniform local binary pattern

With the rapid advancement of multimedia technologies, there is a growing demand for reliable methods to verify image integrity. Camera model identification, a passive approach aiming to determine the specific capturing device model, has garnered considerable attention in the field of source camera forensics. In this paper, we first propose a novel patch selection method that enhances the diversity of training data by utilizing the uniform local binary pattern operator to reveal spatial textual information. Secondly, we introduce a complex dual-path enhanced ConvNeXt network for camera model identification, effectively leveraging the multi-frequency information present in the image. Notably, our network demonstrates the ability to learn camera model-related features without relying on a residual prediction module. Finally, extensive experimental results on both Dresden and Vision datasets shown that the proposed network outperforms several state-of-the-art methods in both teams of identification accuracy and computational efficiency.

A deformation measurement method based on surface texture information of rocks and its application

Users of the digital image correlation method are faced with the problem of poor operability, low repeatability, and lack of standardized specifications for spraying speckles. To solve the problem, the research proposed a rock deformation measurement method that obviates the need to spray speckles. A local binary model was established by using the local binary pattern (LBP) operator based on deep texture features on rock surfaces. The resulting LBP digital speckle pattern can substitute artificial speckle patterns and demonstrates high quality and strong applicability. Based on the LBP digital speckle pattern, the target tracking algorithm was employed to achieve non-contact measurement of the dynamic displacements of rocks. The feasibility and effectiveness of the algorithm in practical application were verified by conducting shear tests on granite and siltstone. Test results show that the deformation characteristics in the displacement nephograms are in line with the measured data pertaining to rock fracturing and conform to the basic characteristics of the shear failure of rocks. The deformation measurement method based on surface texture information can realize non-contact displacement measurement of rocks under conditions without speckles: this obviates the influence of the quality of sprayed speckles on the accuracy of the measurement of deformation.

No-Reference Light Field Image Quality Assessment Exploiting Saliency

In the near future, the broadcasting scenario will be characterized by immersive content. One of the systems for capturing the 3D content of a scene is the Light Field imaging. The huge amount of data and the specific transmission scenario impose strong constraints on services and applications. Among others, the evaluation of the quality of the received media cannot rely on the original signal but should be based only on the received data. In this direction, we propose a no-reference quality metric for light field images which is based on spatial and angular characteristics. In more details, the estimated saliency and cyclopean maps of light field images are exploited to extract the spatial features. The angular consistency features are, instead, measured with the use of the Global Luminance Distribution knowledge and the Weighted Local Binary Patterns operator on Epipolar Plane Images. The effectiveness of the proposed metric is assessed by comparing its performance with state-of-the-art quality metrics using 4 datasets: SMART, Win5-LID, VALID 10-bit, and VALID 8-bit. Furthermore, the performance is analyzed in cross-datasets, with different distortions, and for different saliency maps. The achieved results show that the performance of the proposed model outperforms state-of-the-art approaches and perform well for different distortion types and with various saliency models.

Semi-supervised cooperative regression model for small sample estimation of citrus leaf nitrogen content with UAV images

ABSTRACT Nitrogen is an essential nutrient element for the growth of citrus tree. The accurate estimation of leaf nitrogen content (LNC) is important to guarantee fruit quality and yield. The unmanned aerial vehicle (UAV) remote sensing has shown great potential for monitoring the LNC of crops. However, the number of training samples is always limited due to the high cost and time consumption of acquiring LNC samples. Obtaining satisfactory results is difficult for most machine-learning models with insufficient samples. Thus, a semi-supervised regression model is designed in this paper to improve the performance of estimating citrus LNC from UAV images under the condition of small samples. First, the local binary pattern (LBP) operator is employed to fully extract textural features in UAV images. Second, semi-supervised cooperative regression models based on ridge regression (Ridge), support vector regression (SVR), and random forest (RF) are constructed by combining spectral and textural features. Finally, the best learning model is used to realize the accurate limited sample LNC estimation from the available training samples. The experiments confirm that the semi-supervised cooperative regression model has better and promising results than other machine-learning methods under the condition of small samples. The RF-based cooperative regression model (CoRF) performs best among other models, with a determination coefficient (R2) of 0.716 and a root mean square error (RMSE) of 0.620 g·kg−1. The CoRF model also achieves superior performance when combining LBP textural features. The results estimated by the semi-supervised cooperative regression model is capable of providing instructions for the production and management of citrus orchard.

A multi-channel framework based Local Binary Pattern with two novel local feature descriptors for texture classification

Local binary pattern (LBP) has been widely used in various application fields, including object detection, texture analysis, and remote sensing. To improve discrimination performance, many LBP-based methods have been proposed for extracting different local feature information for texture classification. However, current LBP-based algorithms typically describe local features at a single sampling scale, but some significant and discriminative texture feature information is contained between different scales. Therefore, the lack of capability to extract cross-scale features can lead to losing critical texture features. Moreover, low-frequency texture information should be accorded great importance in texture classification. Additionally, if the diversity and validity of local feature extraction are lacking, the effectiveness of classification capability suffers. To address these issues, (1) we propose a completed cross-scale Local binary pattern (ccsLBP) operator to extract cross-scale texture features. (2) A mean-filtered Local binary pattern (LBPmf) operator is presented to highlight the important low-frequency texture information of texture images. (3) We build a high-performing multi-channel framework based Local binary pattern (MC-LBP) for texture classification, which combines complementary features extracted by LBP, ccsLBP, and LBPmf hybridly to form a final feature vector of the texture image. The effectiveness of the proposed MC-LBP framework is verified on 6 representative texture databases, and the experimental results demonstrate its state-of-art texture classification performance.

A Novel ST-ViBe Algorithm for Satellite Fog Detection at Dawn and Dusk

Satellite remote sensing provides a potential technology for detecting fog at dawn and dusk on a large scale. However, the spectral characteristics of fog at dawn and dusk are similar to those of the ground surface, which makes satellite-based fog detection difficult. With the aid of time-series datasets from the Himawari-8 (H8)/AHI, this study proposed a novel algorithm of the self-adaptive threshold of visual background extractor (ST-ViBe) model for satellite fog detection at dawn and dusk. Methodologically, the background model was first built using the difference between MIR and TIR (BTD) and the local binary similarity patterns (LBSP) operator. Second, BTD and scale invariant local ternary pattern (SILTP) texture features were coupled to form scene factors, and the detection threshold of each pixel was determined adaptively to eliminate the influence of the solar zenith angles. The background model was updated rapidly by accelerating the updating rate and increasing the updating quantity. Finally, the residual clouds were removed with the traditional cloud removal method to achieve accurate detection of fog at dawn and dusk over a large area. The validation results demonstrated that the ST-ViBe algorithm could detect fog at dawn and dusk precisely, and on a large scale. The probability of detection, false alarm ratio, and critical success index were 72.5%, 18.5%, 62.4% at dawn (8:00) and 70.6%, 33.6%, 52.3% at dusk (17:00), respectively. Meanwhile, the algorithm mitigated the limitations of the traditional algorithms, such as illumination mutation, missing detection, and residual shadow. The results of this study could guide satellite fog detection at dawn and dusk and improve the detection of similar targets.

SD-CapsNet: A Siamese Dense Capsule Network for SAR Image Registration with Complex Scenes

SAR image registration is the basis for applications such as change detection, image fusion, and three-dimensional reconstruction. Although CNN-based SAR image registration methods have achieved competitive results, they are insensitive to small displacement errors in matched point pairs and do not provide a comprehensive description of keypoint information in complex scenes. In addition, existing keypoint detectors are unable to obtain a uniform distribution of keypoints in SAR images with complex scenes. In this paper, we propose a texture constraint-based phase congruency (TCPC) keypoint detector that uses a rotation-invariant local binary pattern operator (RI-LBP) to remove keypoints that may be located at overlay or shadow locations. Then, we propose a Siamese dense capsule network (SD-CapsNet) to extract more accurate feature descriptors. Then, we define and verify that the feature descriptors in capsule form contain intensity, texture, orientation, and structure information that is useful for SAR image registration. In addition, we define a novel distance metric for the feature descriptors in capsule form and feed it into the Hard L2 loss function for model training. Experimental results for six pairs of SAR images demonstrate that, compared to other state-of-the-art methods, our proposed method achieves more robust results in complex scenes, with the number of correctly matched keypoint pairs (NCM) at least 2 to 3 times higher than the comparison methods, a root mean square error (RMSE) at most 0.27 lower than the compared methods.

Hyperspectral video target tracking based on pixel-wise spectral matching reduction and deep spectral cascading texture features

Illumination variation (IV) is a challenging issue in hyperspectral video target tracking (HVT). In order to resolve this issue, this paper proposes a novel HVT algorithm based on Pixel-wise Spectral Matching Reduction (PSMR) and Deep Spectral Cascading Texture (Deep-SCT) features. The PSMR is a novel dimensionality reduction method that approximately segments the target and background while compressing the hyperspectral image data. The Deep-SCT features are composed of spectral cascading texture (SCT) features and deep features. The local binary pattern operator and illumination invariance along with the Deep-SCT features overcome the interference caused by IV. In addition, we propose a feature fusion method, called group-pixel joint convolution, which fuses SCT features and deep features. Moreover, the segmentation results, produced by the dimensionality reduction process, are used to generate a coarse location mask. This mask is then used to predict the location of target and also to suppress the texture features of the background. Finally, after localizing the target using the Deep-SCT features, the proposed algorithm uses a step-by-step estimation strategy to adjust the size of the target bounding box. Experiments, over benchmark datasets, illustrate the superior performance of the proposed tracker in comparison with the state-of-the-art approaches. It is evident that the proposed strategy is highly robust against IV.

No-reference Quality Assessment for Contrast-distorted Images Based on Gray and Color-gray-difference Space

No-reference image quality assessment is a basic and challenging problem in the field of image processing. Among them, contrast distortion has a great impact on the perception of image quality. However, there are relatively few studies on no-reference quality assessment of contrast-distorted images. This article proposes a no-reference quality assessment algorithm for contrast-distorted images based on gray and color-gray-difference (CGD) space. In terms of gray space, we consider the local and global aspects, and use the distribution characteristics of the grayscale histogram to represent global features, while local features are described by the fusion of Local Binary Pattern (LBP) operator and gradient. In terms of CGD space, we first randomly extract patches from the entire image and then extract appropriate quality perception features in the patch’s CGD histogram. Finally, the AdaBoosting back propagation (BP) neural network is used to train the prediction model to predict the quality of the contrast-distorted image. Extensive analysis and cross-validation are carried out on five contrast-related image databases, and the experimental results have proved the superiority of this method compared with recent related algorithms.

Blind omnidirectional image quality assessment with representative features and viewport oriented statistical features

With the development of information technologies, various types of streaming images are generated, such as videos, graphics, Virtual Reality (VR)/omnidirectional images (OIs), etc. Among them, the OIs usually have a broader view and a higher resolution, which provides human an immersive visual experience in a head-mounted display. However, the current image quality assessment works cannot achieve good performance without considering representative human visual features and visual viewing characteristics of OIs, which limited OIs’ further development. Motivated by the above problem, this work proposes a blind omnidirectional image quality assessment (BOIQA) model based on representative features and viewport oriented statistical features. Specifically, we apply the local binary pattern operator to encoder the cross-channel color information, and apply the weighted LBP to extract the structural features. Then the local natural scene statistics (NSS) features are extracted by using the viewport sampling to boost the performance. Finally, we apply support vector regression to predict the OIs’ quality score, and experimental results on CVIQD2018 and OIQA2018 Databases prove that the proposed model achieves better performance than state-of-the-art OIQA models.

Learning Streamed Attention Network from Descriptor Images for Cross-Resolution 3D Face Recognition

In this article, we propose a hybrid framework for cross-resolution 3D face recognition which utilizes a Streamed Attention Network (SAN) that combines handcrafted features with Convolutional Neural Networks (CNNs). It consists of two main stages: first, we process the depth images to extract low-level surface descriptors and derive the corresponding Descriptor Images (DIs), represented as four-channel images. To build the DIs, we propose a variation of the 3D Local Binary Pattern (3DLBP) operator that encodes depth differences using a sigmoid function. Then, we design a CNN that learns from these DIs. The peculiarity of our solution consists in processing each channel of the input image separately, and fusing the contribution of each channel by means of both self- and cross-attention mechanisms. This strategy showed two main advantages over the direct application of Deep-CNN to depth images of the face; on the one hand, the DIs can reduce the diversity between high- and low-resolution data by encoding surface properties that are robust to resolution differences. On the other, it allows a better exploitation of the richer information provided by low-level features, resulting in improved recognition. We evaluated the proposed architecture in a challenging cross-dataset, cross-resolution scenario. To this aim, we first train the network on scanner-resolution 3D data. Next, we utilize the pre-trained network as feature extractor on low-resolution data, where the output of the last fully connected layer is used as face descriptor. Other than standard benchmarks, we also perform experiments on a newly collected dataset of paired high- and low-resolution 3D faces. We use the high-resolution data as gallery, while low-resolution faces are used as probe, allowing us to assess the real gap existing between these two types of data. Extensive experiments on low-resolution 3D face benchmarks show promising results with respect to state-of-the-art methods.

HVS-Based Perception-Driven No-Reference Omnidirectional Image Quality Assessment

Evaluating the quality of panoramic images has gradually become a hot research topic with the development of virtual reality (VR) technology. Therefore, a novel method is proposed to assess the quality of omnidirectional images without any reference information. Inspired by the characteristics of the human visual system (HVS) and visual attention mechanism, the proposed model is composed of the structure feature, statistical feature, and saliency feature to measure the panoramic image quality, in which structure information is expressed by combining the local Taylor series with the local binary pattern (LBP) operator, gradient-based statistical information of panoramic images are summarized comprehensively from three levels: the gradient measure, the relative gradient magnitude, and the relative gradient orientation, and the saliency detection by combining simple priors (SDSP)-based saliency information is extracted in this article to enrich perception feature of our model and improve the visibility of the saliency region in the omnidirectional image. Finally, according to the subjective scores provided and the above features, we use support vector regression (SVR) to predict the objective scores. The experiments indicate that our model has more substantial competitiveness and stability than other state-of-the-art methods on two reliable databases.

LG-IQA: Integration of local and global features for no-reference image quality assessment

The visual physiology research shows that the human visual system (HVS) first rapidly and unconsciously produces a global perception, and then gradually focuses on specific local areas for the perception of image quality. Therefore, in order to better simulate the perception of image quality by the HVS, we need to consider both local and global information when evaluating image quality. Based on above analysis, this paper proposes a new method by integrating local and global features for no-reference image quality assessment (IQA), which is named local and global IQA (LG-IQA). Specifically, a test image is transformed by the dual-tree complex wavelet transform (DTCWT) to get the wavelet coefficients, and then the local binary pattern (LBP) operator is applied on the reconstructed image of high-frequency wavelet sub-bands. The obtained results are used as local features. After the DTCWT, we estimate parameters using an asymmetric generalized Gaussian distribution (AGGD) model by fitting the wavelet coefficients, and take the results as global features. Finally, the extracted local and global features are combined together, and the relationship between image features and image quality scores will be learned by an AdaBoosting back-propagation (BP) neural network. It is worth mentioning that this is the first attempt to integrate the LBP and DTCWT together in the IQA research based on neural network. Experimental results demonstrate that the predicted scores of our proposed LG-IQA correlate well with the subjective quality scores.

DEEP LEARNING APPROACHES FOR ANOMALIES DETECTION OF BLADDER CT CONTOURS IN PROSTATE CANCER PATIENTS

In radiation therapy, poor contour delineations of Region of Interests (ROI) may result in incorrect radiation dosing and increase the chance of radiation-induced toxicities. To better recognize these contours, we used a feature fusion-based 3-dimensional convolution neural network (3D-CNN), multi-view convolution neural network (MV-CNN), and 3D point cloud to retrospectively detect abnormal bladder contours among prostate cancer patients. We defined three patterns of abnormal bladder contours: (1) discontinuities: missing slices and disparate volumes, (2) surface aberrations: extended or contracted slices, (3) small volume: bladder volume [Formula: see text][Formula: see text]ml. We created virtual samples for discontinuities and surface aberrations by randomly adding pixel to one or more slices outside the original contours and randomly extending or shrinking one or more slices. For both 3D-CNN and MV-CNN methods, we utilized the Local Binary Pattern (LBP) and Canny operators for feature extraction and compared several feature fusion-based approaches, including Original image + LBP processed image, Original image + Canny processed image, Original image + LBP processed image + Canny processed image. For the 3D point cloud, we applied the 3D point cloud segmentation method (Rand LA-Net) and the 3D point cloud classifying method (DGCNN). We had 344 samples available (219 real contours and 125 virtual contours) altogether. All the algorithms were trained on 200 bladder contours, validated on 100 bladders, then tested on 44 real annotated contours. Among all the approaches, the “Original + LBP”-based MV-CNN model achieved the highest accuracy: 100% for predicting discontinuity and small volume, 91.3% for predicting normal contours, and 90.9% for predicting surface aberrations. MV-CNN performed more accurately and efficiently across all the feature fusion methods. In particular, “Original + LBP”-based MV-CNN achieved the highest test accuracy. Our tools can detect anomalies of bladder contours in clinical settings and may improve the process of contour quality assurance (QA).

Ghost-Free Multi-Exposure Image Fusion Technology Based on the Multi-Scale Block LBP Operator

This paper proposes a ghost-free multi-exposure image fusion technique based on the multi-scale block LBP (local binary pattern) operator. The method mainly includes two steps: first, the texture variation, brightness, and spatial consistency weight maps of the image are computed, and then these three image features are used to construct the initial weight map. Finally, the multi-resolution method is used to fuse the images to obtain the resulting image. The main advantage of this technique lies in the step of extracting the details of the source image based on the multi-scale block LBP operator, which is used to preserve the details of the brightest and darkest areas in high dynamic range scenes and preserve the texture features of the source image. Another advantage is that a new LBP operator-based motion detection method is proposed for fusing multi-exposure images in dynamic scenes containing moving objects. In addition, this paper also studies two spatially consistent weight distribution methods and compares and discusses the effects of these two methods on the results of dynamic image fusion. Through a large number of experimental comparisons, the superiority and feasibility of this method are proved.

Mineral Texture Identification Using Local Binary Patterns Equipped with a Classification and Recognition Updating System (CARUS)

In this paper, a rotation-invariant local binary pattern operator equipped with a local contrast measure (riLBPc) is employed to characterize the type of mineral twinning by inspecting the texture properties of crystals. The proposed method uses photomicrographs of minerals and produces LBP histograms, which might be compared with those included in a predefined database using the Kullback–Leibler divergence-based metric. The paper proposes a new LBP-based scheme for concurrent classification and recognition tasks, followed by a novel online updating routine to enhance the locally developed mineral LBP database. The discriminatory power of the proposed Classification and Recognition Updating System (CARUS) for texture identification scheme is verified for plagioclase, orthoclase, microcline, and quartz minerals with sensitivity (TPR) near 99.9%, 87%, 99.9%, and 96%, and accuracy (ACC) equal to about 99%, 97%, 99%, and 99%, respectively. According to the results, the introduced CARUS system is a promising approach that can be applied in a variety of different fields dealing with classification and feature recognition tasks.

Face Recognition Attendance System Method Based on Fusion of LBP and HOG

As one of the hot topics in the field of computer vision research, face recognition technology has received significant attention due to its potentiality for a wide range of applications in government as well as commercial purposes. In practical applications, although several existing face recognition methods have achieved good performances in specific scenes, they easily suffer from a sharp decline in recognition rate if affected by different conditions of light, expression, posture and occlusion. Among many factors, influences of complex illuminations on face recognition are particularly significant. To further improve the performance of the existing local binary pattern (LBP) operator, neighbourhood weighted average LBP (NWALBP) is first proposed for fully considering the strong correlations between pixel pairs in the neighbourhood, which extends the traditional LBP uni-layer neighbourhood template window to the bi-layer neighbourhood template window and calculates the weighted average of bi-layer neighbourhood pixels in each direction. Then, inspired by centre symmetric LBP (CS-LBP), centre symmetric NWALBP (CS-NWALBP) is further proposed, which can effectively reduce computation complexity by only comparing the weighted average values of the neighbourhood pixels that are symmetric about the centre pixel. Finally, by combining the merit of histogram of oriented gradient (HOG), a feature fusion algorithm named CS-NWALBP+HOG is suggested. Several experiments have eventually demonstrated that our proposed algorithms have more robust performance under complex illumination conditions if compared with many other latest algorithms.

Texture image classification based on a pseudo-parabolic diffusion model.

This work proposes a novel method based on a pseudo-parabolic diffusion process to be employed for texture recognition. The proposed operator is applied over a range of time scales giving rise to a family of images transformed by nonlinear filters. Therefore each of those images are encoded by a local descriptor (we use local binary patterns for that purpose) and they are summarized by a simple histogram, yielding in this way the image feature vector. Three main novelties are presented in this manuscript: (1) The introduction of a pseudo-parabolic model associated with the signal component of binary patterns to the process of texture recognition and a real-world application to the problem of identifying plant species based on the leaf surface image. (2) We also introduce a simple and efficient discrete pseudo-parabolic differential operator based on finite differences as texture descriptors. While the work in [26] uses complete local binary patterns, here we use the original version of the local binary pattern operator. (3) We also discuss, in more general terms, the possibilities of exploring pseudo-parabolic models for image analysis as they balance two types of processing that are fundamental for pattern recognition, i.e., they smooth undesirable details (possibly noise) at the same time that highlight relevant borders and discontinuities anisotropically. Besides the practical application, the proposed approach is also tested on the classification of well established benchmark texture databases. In both cases, it is compared with several state-of-the-art methodologies employed for texture recognition. Our proposal outperforms those methods in terms of classification accuracy, confirming its competitiveness. The good performance can be justified to a large extent by the ability of the pseudo-parabolic operator to smooth possibly noisy details inside homogeneous regions of the image at the same time that it preserves discontinuities that convey critical information for the object description. Such results also confirm that model-based approaches like the proposed one can still be competitive with the omnipresent learning-based approaches, especially when the user does not have access to a powerful computational structure and a large amount of labeled data for training.

Recognition of Skin Lesions in Dermoscopic Images using Local Binary Patterns and Multinomial Logistic Regression

Skin cancer, such as melanoma, affects more than 13 million people annually and causes more than 65,000 deaths. As with almost all types of cancer, it can be treated effectively if it's diagnosed in it's early stages. The development of computer-aided diagnostic tools is, therefore, a valuable and urgent need to suport the decision of dermatologists, minimize human error, and accelerate the time to diagnosis and the prognosis of patients. The present work proposes the automatic recognition of three skin lesions, namely Common Nevus, Atypical Nevus, and Melanoma, from dermoscopic images. First, a feature vector representing texture is extracted from the dermoscopic images using the local binary pattern operator (LBP). Then, a multinomial logistic regression (MRL) model is used to exploit the feature vector to perform the multiclass classification. The hyperparameters of the LBP operator, namely the radius and the pixel neighborhoods, were modified to provide the feature set to the MRL model that maximizes the classification performance. Moreover, to assure the generalizability of the model and the reproducibility of the results, a 10-iteration Monte Carlo cross-validation (MCCV) was employed, by dividing the dataset into 70% for training and 30% for testing, with different random seeds in each iteration. The optimal LBP-MRL classification model was finally embeddeb in a graphical user interface (GUI) to facilitate the interaction between the user and the classification system. An average accuracy of 90%, averge recall of 89.23%, 84.75% and 100%, and average precision of 89.95%, 88.12% and 100% were obtained, to identify Common Nevus, Atypical Nevus and Melanoma, respectively, in the MCCV strategy. It should be highlighted that the proposed classification system can perfectly differentiate melanoma from the other two skin lesions.