Multi-block Local Binary Pattern Research Articles

Face presentation attack detection performances of facial regions with multi-block LBP features.

Biometric recognition systems are frequently used in daily life although they are vulnerable to attacks. Today, especially the increasing use of face authentication systems has made these systems the target of face presentation attacks (FPA). This has increased the need for sensitive systems detecting the FPAs. Recently surgical masks, frequently used due to the pandemic, directly affect the performance of face recognition systems. Researchers design face recognition systems only from the eye region. This motivated us to evaluate the FPA detection performance of the eye region. Based on this, in cases where the whole face is not visible, the FPA detection performance of other parts of the face has also been examined. Therefore, in this study, FPA detection performances of facial regions of wide face, cropped face, eyes, nose, and mouth was investigated. For this purpose, the facial regions were determined and normalized, and texture features were extracted using powerful texture descriptor local binary patterns (LBP) due to its easy computability and low processing complexity. Multi-block LBP features are used to obtain more detailed texture information. Generally uniform LBP patterns are used for feature extraction in the literature. In this study, the FPA detection performances of both uniform LBP patterns and all LBP patterns were investigated. The size of feature vector is reduced by principal component analysis, and real/fake classification is performed with support vector machines. Experimental results on NUAA, CASIA, REPLAY-ATTACK and OULU-NPU datasets show that the use of all patterns increased the performance of FPA detection.

An efficient hand gesture recognition based on optimal deep embedded hybrid convolutional neural network‐long short term memory network model

AbstractHand gestures are the nonverbal communication done by individuals who cannot represent their thoughts in form of words. It is mainly used during human‐computer interaction (HCI), deaf and mute people interaction, and other robotic interface applications. Gesture recognition is a field of computer science mainly focused on improving the HCI via touch screens, cameras, and kinetic devices. The state‐of‐art systems mainly used computer vision‐based techniques that utilize both the motion sensor and camera to capture the hand gestures in real‐time and interprets them via the usage of the machine learning algorithms. Conventional machine learning algorithms often suffer from the different complexities present in the visible hand gesture images such as skin color, distance, light, hand direction, position, and background. In this article, an adaptive weighted multi‐scale resolution (AWMSR) network with a deep embedded hybrid convolutional neural network and long short term memory network (hybrid CNN‐LSTM) is proposed for identifying the different hand gesture signs with higher recognition accuracy. The proposed methodology is formulated using three steps: input preprocessing, feature extraction, and classification. To improve the complex visual effects present in the input images, a histogram equalization technique is used which improves the size of the gray level pixel in the image and also their occurrence probability. The multi‐block local binary pattern (MB‐LBP) algorithm is employed for feature extraction which extracts the crucial features present in the image such as hand shape structure feature, curvature feature, and invariant movements. The AWMSR with the deep embedded hybrid CNN–LSTM network is applied in the two‐benchmark datasets namely Jochen Triesch static hand posture and NUS hand posture dataset‐II to detect its stability in identifying different hand gestures. The weight function of the deep embedded CNN‐LSTM architecture is optimized using the puzzle optimization algorithm. The efficiency of the proposed methodology is verified in terms of different performance evaluation metrics such as accuracy, loss, confusion matrix, Intersection over the union, and execution time. The proposed methodology offers recognition accuracy of 97.86% and 98.32% for both datasets.

Secure Rotation Invariant Face Detection System for Authentication

Biometric applications widely use the face as a component for recognition and automatic detection. Face rotation is a variable component and makes face detection a complex and challenging task with varied angles and rotation. This problem has been investigated, and a novice algorithm, namely RIFDS (Rotation Invariant Face Detection System), has been devised. The objective of the paper is to implement a robust method for face detection taken at various angle. Further to achieve better results than known algorithms for face detection. In RIFDS Polar Harmonic Transforms (PHT) technique is combined with Multi-Block Local Binary Pattern (MBLBP) in a hybrid manner. The MBLBP is used to extract texture patterns from the digital image, and the PHT is used to manage invariant rotation characteristics. In this manner, RIFDS can detect human faces at different rotations and with different facial expressions. The RIFDS performance is validated on different face databases like LFW, ORL, CMU, MIT-CBCL, JAFFF Face Databases, and Lena images. The results show that the RIFDS algorithm can detect faces at varying angles and at different image resolutions and with an accuracy of 99.9%. The RIFDS algorithm outperforms previous methods like Viola-Jones, Multi-block Local Binary Pattern (MBLBP), and Polar Harmonic Transforms (PHTs). The RIFDS approach has a further scope with a genetic algorithm to detect faces (approximation) even from shadows.

Robust Face Recognition Under Partial Occlusion Based on Local Generic Features

Face recognition has drawn significant attention due to its potential use in biometric authentication, surveillance, security, robotics, and so on. It is a challenging task in the field of computer vision. Although the various state-of-the-art methods of face recognition in constrained environments have achieved satisfactory results, there are still many issues which are untouched in unconstrained environments, such as partial occlusions, large pose variations, etc. In this paper, the authors have proposed an approach which utilized the local generic feature (LGF) to recognize the face in the partial occlusion by fusing features scale invariant feature transform (SIFT) and multi-block local binary pattern (MB-LBP). It also utilizes robust kernel method for classification of the query image. They have validated the effectiveness of the proposed approach on the benchmark AR face database. The experimental outcomes illustrate that the proposed approach outperformed the state-of-art methods for robust face recognition.

Intensity prediction of tropical cyclone using multilayer multi-block local binary pattern and tree-based classifiers over North Indian Ocean

Local binary pattern (LBP) is an extensively used method in image analysis and pattern recognition related works across various domains. In this paper, we explore cyclone cloud evolution patterns using a modified LBP. The multilayer multi-block local binary pattern (MMLBP) is an extended version of Completed LBP (CLBP) with multiple blocks and many layers. A tropical cyclone (TC) image is converted into 3 × 3 identical blocks to generate central pixels using CLBP magnitude descriptor block-wise (CLBP_MB). These central pixels are used to find the next layer, which is further divided into 3 × 3 blocks to generate central pixels by CLBP_MB descriptor. This technique will iterate until the dimension of the final layer reached to 1 × 1. These central pixels are extracted from each layer and gather in a vector called a feature vector. The proposed method is applied to 600 tropical cyclone (TC) images and each image generates one feature vector. Hence, 600 vectors are passed through tree-based classifiers to classify cyclone images of various classes. The proposed method of feature extraction and classification reaches a maximum of 84.66% accuracy using Random Forest (RF) classifier. The root mean squared error (RMSE) of the MMLBP method is 9.27 kt, which is better than the LBP and MBLBP approaches. Finally, the size of the original feature vector is reduced to 97.4% using correlation based feature subset selection method.

Automatic Retrieval of Shoeprints Using Modified Multi-Block Local Binary Pattern

A shoeprint is a valuable clue found at a crime scene and plays a significant role in forensic investigations. In this paper, in order to maintain the local features of a shoeprint image and place a pattern in a block, a novel automatic method was proposed, referred to as Modified Multi-Block Local Binary Pattern (MMB-LBP). In this method, shoeprint images are divided into blocks according to two different models. The histograms of all blocks of the first and second models are separately measured and stored in the first and second feature matrices, respectively. The performance evaluations of the proposed method were carried out by comparing with state-of-the-art methods. The evaluation criteria are the successful retrieval rates obtained using the best match score at rank one and cumulative match score for the first five matches. The comparison results indicated that the proposed method performs better than other methods, in terms of retrieval of complete and incomplete shoeprints. That is, the proposed method was able to retrieve 97.63% of complete shoeprints, 96.5% of incomplete toe shoeprints, and 91.18% of incomplete heel shoeprints. Moreover, the experiments showed that the proposed method is significantly resistant to the rotation, salt and pepper noise, and Gaussian white noise distortions in comparison with the other methods.

A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function

Extensive use of surveillance cameras for human tracking and observation have been fostering the research on face recognition technique for individual identification in an unconstrained environment. However, face recognition is a challenging task in an unconstrained environment, where the captured images are affected by illumination effect, varying poses, noise and occlusion. The main objective of this research is to improve the accuracy and processing time in extracting facial features by using the fusion of deep learning and handcrafted architecture for recognizing individuals in unconstrained conditions, thereby providing accurate information about the individuals to security systems. The proposed system consists of Multi-Block Local Binary Pattern (MB-LBP) modules for extracting the handcrafted features and Convolutional Neural Network (CNN) for extracting the high-level distinctive features. The features from both modules are fused and passed through fully connected layer with Softmax classifier to identify individuals. The results show that the enhanced algorithm based on Softmax loss function aided classifier with regularization improves the accuracy and processing time for face recognition. The proposed model improves accuracy by 94.37% against 90.01% for the state-of-the-art solution. In addition to that, it improves the processing time of 307 ms against 357 ms. The proposed system focuses on fusing hand-crafted and deep learned features to extract face features accurately and thus improving the accuracy and overall performance of the proposed system in an unconstrained environment.

Invariant Feature-Based Darknet Architecture for Moving Object Classification

Object detection and classification is important for video surveillance applications. Counting vehicles like cars, truck and vans is useful for intelligent transportation systems to identify dense and sparse roads, track loaded vehicles at the country borders. Even though many solutions such as appearance-based (Multi-block Local Binary Pattern) and model-based ((DATMO) algorithm) are proposed to classify the moving objects within the satellite images using machine learning and deep learning techniques, they either have over fitting problems or low performance. Hence these challenges have to be addressed during detecting and classifying the objects. Instead of training the classifiers with hand-crafted features, this paper uses neural network based object detection and classification to achieve promising accuracy better than the humans. Invariant feature concept is added to the existing Darknet Architecture of You Only Look Once (YOLO) and is combined with Faster Region-Based Convolutional Neural Networks (Faster R-CNN) to count the number of vehicles with different spatial locations. This combined model improves feature extraction step and vehicle classification process. The proposed system is tested on two benchmark datasets Cars Overhead with Context (COWC) and Vehicle Detection in Aerial Imagery (VEDAI) for counting the cars and trucks. Experimental results prove that the proposed system is better by 9% in detecting smaller objects than existing works.

Segmentation of River Scenes Based on Water Surface Reflection Mechanism

Segmentation of a river scene is a representative case of complex image segmentation. Different from road segmentation, river scenes often have unstructured boundaries and contain complex light and shadow on the water’s surface. According to the imaging mechanism of water pixels, this paper designed a water description feature based on a multi-block local binary pattern (MB-LBP) and Hue variance in HSI color space to detect the water region in the image. The improved Local Binary Pattern (LBP) feature was used to recognize the water region and the local texture descriptor in HSI color space using Hue variance was used to detect the shadow area of the river surface. Tested on two data sets including simple and complex river scenes, the proposed method has better segmentation performance and consumes less time than those of two other widely used methods.

Adaboost Cascade Classifier for Classification and Identification of Wild Animals using Movidius Neural Compute Stick

Deep learning has gone deeper and has been utilized in almost all the applications like object recognition, image classification, speech recognition and much more. Most of the real-time applications rely on deep learning for accurate results. But one downside to the deep learning is its demand for GPUs (Graphical Processing Unit) or TPUs (Tensor Processing Units) for faster execution. There was no one-stop-shop hardware and software for deep learning applications, until recently Intel launched the Movidius Neural Compute Stick (NCS). This sleek device provides the power of GPU in a CPU based system. In this work, we have modeled an animal detection system using NCS and AdaBoost classifier powered by Multi-Block Local Binary Pattern (MB-LBP) features. The model has been built upon AlexNet and has achieved an average accuracy of 96.8% and a false rate 2.3% in classifying the animals as wild and non-wild. Furthermore, the model has a speedup of 467% when compared to the execution in the CPU based system.

Driver fatigue detection based on deeply-learned facial expression representation

Driver fatigue detection is a significant application in smart cars. In order to improve the accuracy and timeliness of driver fatigue detection, a fatigue detection algorithm based on deeply-learned facial expression analysis is proposed. Specifically, the face key point detection model is first trained by multi block local binary patterns (MB-LBP) and Adaboost classifier. Subsequently, the eyes and mouth state are detected by using the trained model to detect the 24 facial features. Afterwards, we calculate the number of two parameters that can describe the driver's fatigue state and the proportion of the closed eye time within the unit time (PERCLOS) and yawning frequency. Finally, the fuzzy inference system is utilized to deduce the driver's fatigue state (normal, slight fatigue, severe fatigue). Experimental results show that the proposed algorithm can detect driver fatigue degree quickly and accurately.

A highly secure Multi- Factor authentication system using biometrics to enhance privacy in Internet of Things (IOT)

Authentication is becoming critical in the Internet of Things (IoT) environment because of its many applications and services have been emerging in the areas such as smart city, healthcare, industry etc. Security and privacy plays a vital role in IoT because their services can be accessed through smart device applications by the user from everywhere and at any time. Hence a multi-factor based authentication can provide high security in IOT environment. This security system incorporates most of the valuable methods such as cryptography, steganography and pattern recognition for authentication process. Among various biometric traits, palm vein is more efficient because it has essential sufficient features points for individual unique identification. The system employs registration phase and authentication phase. The registration phase enrolls person privacy data with their biometric and the obtained data’s are encrypted with the help of Elliptical Curve Cryptography (ECC) and this confidential information is embedded into person palm print image using bits substitution procedure. In authentication phase, recognition will be performed through three levels such as password, palm print and One Time Password (OTP). Using these three levels the matching can be done. The texture features can be obtained by using Multi Block Local Binary Pattern (MB-LBP) and Gabor filter. To afford high authentication, OTP method is also appended. This system provides better information security and texture analysis rather than previous approaches. Thus this multiple level approach ensures a fool proof and a reliable way for data access. Results are in terms of some validation parameters like false acceptance ratio, false rejection ratio and recognition rate. Observing from results, it is clear that the proposed approach outperform many existing methods. As a result, the proposed scheme has strong security, reliability and enhanced computational efficiency.

An Improved MB-LBP Defect Recognition Approach for the Surface of Steel Plates

The detection of surface defects is very important for the quality improvement of steel plates. In actual production, as the steel plate production line runs faster, the steel surface defect detection algorithm is required to meet the requirements of real-time detection (less than 100 ms/image), and the detection accuracy is improved (at least 90%). In this paper, an improved multi-block local binary pattern (LBP) algorithm is proposed. This algorithm not only has the simplicity and efficiency of the LBP algorithm, but also finds a suitable scale to describe the defect features by changing the block sizes, thus ensuring high recognition accuracy. The experiment proves that the method satisfies the requirements of online real-time detection in terms of speed (63 ms/image), and surpasses the widely-used scale invariant feature transform (SIFT), speeded up robust features (SURF), gray-level co-occurrence matrix (GLCM), and LBP algorithms in recognition accuracy (94.30%), which prove that the MB-LBP has practical application value in an online real-time detection system.

Enhanced Face Detection Based on Haar-Like and MB-LBP Features

The effective real-time face detection framework proposed by Viola and Jones gained much popularity due its computational efficiency and its simplicity. A notable variant replaces the original Haar-like features with MB-LBP (Multi-Block Local Binary Pattern) which are defined by the local binary pattern operator, both detector types are integrated into the OpenCV library. However, each descriptor and its evaluation method has its own set of strengths and setbacks. In this paper, an enhanced two-layer face detector composed of both Haar-like and MB-LBP features is presented. Haar-like features are employed as a coarse filter but with a new evaluation involving dual threshold. The already established MB-LBPs are arranged as the fine filter of the detector. The Gentle AdaBoost learning algorithm is deployed for the training of the proposed detector to reach the classification and performance potential. Experiments show that in the early stages of classification, Haar features with dual threshold are more discriminative than MB-LBP and original Haar-like features with respect to number of features required and computation. Benchmarking the proposed detector demonstrate overall 12% higher detection rate at 17% false alarm over using MB-LBP features singly while performing with ×3 speedup.

Abnormal crowd density estimation in aerial images

The unpreceded growth of intelligent surveillance systems has resulted in an urgent need for automatic analysis of the captured scenes. Automatic detection of an abnormal crowd in aerial images can provide useful information to prevent disasters. In fact, aerial images have the advantage of covering a very large view of the people distributed over a scene. Accordingly, we propose a high density crowd detection method in aerial images. In this method, we adapted the bag of words technique using the multiblock local binary pattern as a texture descriptor to extract low-level features. In addition, we also used a three-level classification strategy to reduce confusion between crowd density classes. The experimental results reveal the performance of our method while estimating the crowd density in a challenging context.

Stride towards aging problem in face recognition by applying hybrid local feature descriptors

In our proposed research, we developed a discriminative model for addressing the challenge of face recognition with respect to age variation by utilizing periocular region. Initially, we represent each face by using a densely sampled local feature description technique, in which multi-block local binary pattern (MB-LBP) and weber local descriptor (WLD) are adopted as the local descriptors. In this approach, extract the feature of entire facial image by the aid of both local descriptors. Since the feature space of both the MBLBP and WLD is vast, we develop an advanced multi-feature discriminant analysis to process these two local feature spaces in a unified framework. By random sampling, the training set as well as the feature space, multiple linear discriminant analysis (LDA) based K-nearest neighbor (K-NN) classifiers are constructed and then combined to generate a robust decision via a fusion rule. In the proposed method, the MORPH and FG-NET datasets are utilized for the experimental evaluation. In the MORPH dataset, our proposed method is compared with existing methods such as SIFT, Multi scale method based on morphology, signed grey level difference (SD), LBP, scale-invariant classification, gabor, hidden factor analysis (HFA), modified HFA (MHFA) and learning discriminant face descriptors (LFD). The accuracy of face and periocular region for our method is 98.83% and 98.32%. The existing method such as sparse null linear discriminate analysis (SNLDA), direct LDA, principle component analysis + LDA, cross-age reference coding (CARC), HFA and sparse representation classification (SRC) are compared with the proposed method and obtained the recognition rate is 96.74% and 97.89%. In parameter evaluation we get following values: recall is 93.34% and 96.56%, precision is 95.49% and 94.76%, F-score is 94.40% and 92.17% for face and periocular region. In the FG-NET dataset, our proposed method is compared with existing methods such as SIFT, LBP, MLBP, MWLD and multi-feature discriminant analysis (MFDA). The average accuracy in face region and the periocular region is 65.32% and 68.92%. Our proposed method produce recognition rate as 98.11% and 97.87% for face region and periocular region when compared with SNLDA, DLDA, SRC, PCA + LDA and Coupled Auto-encoder Networks (CAN). In parameter evaluation we get following values: recall is 90.85% and 91.65%, precision is 92.35% and 94.56%, F-score is 91.59% and 90.84% for face and periocular region. Our Method has error rate as 1.7% and our error rate is less when contrasted with Agarwal et al. (21%), Wallis et al. (11.5%), Wang et al. (4.515%) and Leibe et al. (2.5%) techniques.

Multimodal Biometrics for Person Identification Using Ear and Palm Print Features

ABSTRACT Biometrics are automated methods of recognising a person based on physiological or behavioural characteristics. To discriminate individuals, multimodal biometrics has already proven as an effective strategy. Biometric features can be broadly classified as physiological features and behavioural features. Ear, face, and palm come under physiological features. Gait and signature verification come under behavioural features. Combining multiple human trait features for biometric identification is multimodal biometric identification. Here, ear and palm print are the two biometric modalities used for person identification fused at feature level. To extract the features for person identification, Multiblock Local Binary Pattern and Binarised Statistical Image Features are used. Required intrusive means for acquiring the information can be a common drawback when using biometric features such as iris pattern, facial traits, etc. To overcome the drawbacks, ear can be used as a biometric feature; it also has an advantage of no changes over time and not influenced by facial expressions.

Local Gradient Hexa Pattern: A Descriptor for Face Recognition and Retrieval

Local descriptors used in face recognition are robust in a sense that these descriptors perform well in varying pose, illumination, and lighting conditions. The accuracy of these descriptors depends on the precision of mapping the relationship that exists in the local neighborhood of a facial image into microstructures. In this paper, a local gradient hexa pattern is proposed that identifies the relationship among the reference pixel and its neighboring pixels at different distances across different derivative directions. Discriminative information exists in the local neighborhood as well as in different derivative directions. The proposed descriptor effectively transforms these relationships into binary micropatterns discriminating inter-class facial images with optimal precision. The recognition and retrieval performance of the proposed descriptor has been compared with state-of-the-art descriptors, namely, local derivative pattern, local tetra pattern, multiblock local binary pattern, and local vector pattern over the most challenging and benchmark facial image databases, i.e., Cropped Extended Yale B, CMU-PIE, color-FERET, LFW, and Ghallager database. The proposed descriptor has better recognition as well as retrieval rates compared with state-of-the-art descriptors.

Adapting Local Features for Face Detection in Thermal Image.

A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. This similarity in face appearances is advantageous for face detection. To detect faces in thermal images, cascade classifiers with Haar-like features are generally used. However, there are few studies exploring the local features for face detection in thermal images. In this paper, we introduce two approaches relying on local features for face detection in thermal images. First, we create new feature types by extending Multi-Block LBP. We consider a margin around the reference and the generally constant distribution of facial temperature. In this way, we make the features more robust to image noise and more effective for face detection in thermal images. Second, we propose an AdaBoost-based training method to get cascade classifiers with multiple types of local features. These feature types have different advantages. In this way we enhance the description power of local features. We did a hold-out validation experiment and a field experiment. In the hold-out validation experiment, we captured a dataset from 20 participants, comprising 14 males and 6 females. For each participant, we captured 420 images with 10 variations in camera distance, 21 poses, and 2 appearances (participant with/without glasses). We compared the performance of cascade classifiers trained by different sets of the features. The experiment results showed that the proposed approaches effectively improve the performance of face detection in thermal images. In the field experiment, we compared the face detection performance in realistic scenes using thermal and RGB images, and gave discussion based on the results.

Combining random forest with multi-block local binary pattern feature selection for multiclass head pose estimation.

A new head pose estimation technique based on Random Forest (RF) and texture features for facial image analysis using a monocular camera is proposed in this paper, especially about how to efficiently combine the random forest and the features. In the proposed technique a randomized tree with useful attributes is trained to improve estimation accuracy and tolerance of occlusions and illumination. Specifically, a number of features including Multi-scale Block Local Block Pattern (MB-LBP) are extracted from an image, and random features such as the MB-LBP scale parameters, a block coordinate, and a layer of an image pyramid in the feature pool are used for training the tree. The randomized tree aims to maximize the information gain at each node while random samples traverse the nodes in the tree. To this aim, a split function considering the uniform property of the LBP feature is developed to move sample blocks to the left or the right children nodes. The trees are independently trained with random inputs, yet they are grouped to form a random forest so that the results collected from the trees are used for make the final decision. Precisely, we use a Maximum-A-Posteriori criterion in the decision. It is demonstrated with experimental results that the proposed technique provides significantly enhanced classification performance in the head pose estimation in various conditions of illumination, poses, expressions, and facial occlusions.