Face Retrieval Research Articles

Retraction Note: BPSO based neural network approach for content-based face retrieval

Retraction Note: BPSO based neural network approach for content-based face retrieval

A face retrieval technique combining large models and artificial neural networks

SummaryMultimodal sentiment analysis is a popular research direction in the field of affective computing. It extends unimodal‐based sentiment analysis to the environment based on multimodal information exchange. Face retrieval is the most critical technology in the current multimodal sentiment analysis field. Traditional methods for face retrieval rely on large amounts of data to train the matching relationship between text and face images. This approach suffers from several issues, including high data acquisition costs, insufficient or poor quality training data, and high computational costs for training models. To address the aforementioned issues, this paper proposes a face retrieval technique that merges a large language model with a visual base model. This technique explores the use of zero samples for training, eliminating the need to collect data and train the model to achieve face retrieval. As a result, it significantly reduces the cost requirements of data and computational power. First, an accurate natural text of face description is generated using a large language model based on cue engineering based on face‐independent features. Second, the text and images are encoded using a visual base model and mapped into the same multidimensional space for matching. The experimental results indicate that this method performs well on the CelebA dataset. Extracting 31 different sets of face feature descriptions from the CelebA dataset for retrieval achieves 72% on top1 accuracy and 93% on top3 accuracy. Compared to traditional methods, the method proposed in this paper achieves considerable face retrieval accuracy without collecting text image pairs of data and without training the model.

Zero‐sample face retrieval combining large language model and visual base model for IoT

AbstractThis paper presents a novel approach to face retrieval that leverages the capabilities of large language models and visual base models, marking a significant departure from traditional IoT text retrieval methods that depend on extensive data collection and model training. By eliminating the need for text‐image pair data collection and model training, our method not only dramatically reduces the data and computational costs associated with IoT applications but also achieves high accuracy in face retrieval, as demonstrated by a 72% top‐1 accuracy and 93% top‐3 accuracy on the Celeb‐A dataset. This substantial improvement in efficiency and performance has profound implications for the future of IoT systems, potentially revolutionizing face recognition technology by enabling more scalable, cost‐effective, and accurate solutions. The successful application of zero‐sample face retrieval illustrates the transformative impact that advanced AI models can have on real‐world applications and opens new avenues for research and development in the realm of intelligent IoT devices.

ALMEGA-VIR: face video retrieval system

ABSTRACT The limitations of Content Based Video Retrieval (CBVR), such as large pools of video data, selection of features, limited processing capacity, and content-related issues, can be overcome by Deep Belief Neural Networks (DBNs). The search engine does the processing, and the results are effectively returned to the users. The deep learning model also has some comparable challenges to be solved at various levels. Typically, the selection of frames is one of the most essential tasks for a face retrieval system which is efficiently accomplished by using deep learning models. Also, the performance of recognition models highly depends on the selection of frames that include both low-level and high-level features from the given video sequence. This paper proposes ALPHA-TO-OMEGA (ALMEGA-VIR) Visual Information Retrieval scheme that analyzes the contents of the video file for a particular face and retrieves the user's relevant videos. VIR aims to solve the challenge of locating relevant pictures and videos based on a query. VIR can be based on text-based search, content-based search, and Mixed based search. VIR here focuses on content-based search approach. The YouTube Celebrities (YTC) dataset, the Youtube Face (YTF) dataset, and Honda / UCSD dataset are used in this work. The used dataset addresses issues such as illumination changes, occlusion, rotation, and scale. We propose an ALMEGA-VIR framework for visual novelty face identification for specific investigation and evaluation. The classification accuracy of the proposed ALMEGA-VIR model shows that 4% is more effective than traditional classifier algorithms such as SVM, Naïve Bayesian classifier, MLP and Random Forest classifier. A comparative study is also implemented with other existing works to check the superiority of the proposed method.

Suspect face retrieval system using multicriteria decision process and deep learning

Suspect face retrieval system using multicriteria decision process and deep learning

Facial attribute classification by deep mining inter‐attribute correlations

AbstractFace attribute classification (FAC) has received considerable attention due to its excellent application value in bio‐metric verification and face retrieval. Current FAC methods suffer two typical challenges: complex inter‐attribute correlations and imbalanced learning. Aims at the challenges, presents an end‐to‐end FAC framework with integrated use of multiple strategies, which consists of a convolutional neural network (CNN) and a graph convolutional network (GCN). The GCN is used to model the semantic correlations among attributes and capture inter‐dependency among them. The correlation information learnt via the GCN is used to guide the learning of the inter‐dependent classification features of the FAC network. An adaptive thresholding strategy and a boosting scheme are adopted to alleviate the effect of the class‐imbalance. To deal with the task imbalance problem, a new dynamic weighting scheme is proposed to update the weight of each attribute classification task in the training process. We apply four evaluation metrics to evaluate the proposed method. Experimental results show all the proposed strategies are effective, and our approach outperforms state‐of‐the‐art FAC methods on two challenging datasets CelebA and LFWA.

Knowledge Distillation Hashing for Occluded Face Retrieval

Deep hashing has proven to be efficient and effective for large-scale face retrieval. However, existing hashing methods are designed for normal face images only. They fail to consider the fact that face images may be occluded because of wearing masks, hats, glasses, etc. Retrieval performance of existing face retrieval methods is much worse when dealing with occluded face images. In this work, we propose the knowledge distillation hashing (KDH) to deal with occluded face images. The KDH is a two-stage learning approach with teacher-student model distillation. We first train a teacher hashing network using normal face images and then the knowledge from teacher model is used to guide the optimization of the student model using occluded face images as input only. With knowledge distillation, we build a connection between imperfect face information and the optimal hash codes. Experimental results show that the KDH yields significant improvements and better retrieval performance in comparison to existing state-of-the-art deep hashing retrieval methods under six different face occlusion situations.

Generative face inpainting hashing for occluded face retrieval.

COVID-19 has resulted in a significant impact on individual lives, bringing a unique challenge for face retrieval under occlusion. In this paper, an occluded face retrieval method which consists of generator, discriminator, and deep hashing retrieval network is proposed for face retrieval in a large-scale face image dataset under variety of occlusion situations. In the proposed method, occluded face images are firstly reconstructed using a face inpainting model, in which the adversarial loss, reconstruction loss and hash bits loss are combined for training. With the trained model, hash codes of real face images and corresponding reconstructed face images are aimed to be as similar as possible. Then, a deep hashing retrieval network is used to generate compact similarity-preserving hashing codes using reconstructed face images for a better retrieval performance. Experimental results show that the proposed method can successfully generate the reconstructed face images under occlusion. Meanwhile, the proposed deep hashing retrieval network achieves better retrieval performance for occluded face retrieval than existing state-of-the-art deep hashing retrieval methods.

Analysis and control of the mechanism of coal pillar sloughing in the shallow buried thick coal seam

AbstractIn northern Shaanxi province, the coal seam deposit conditions are generally characterized by large thickness and shallow depth of burial, the phenomenon of coal pillar sloughing is serious in the process of working face retrieval, which brings great potential danger to mine safety production. In this paper, by monitoring the stress and damage of the coal pillar in the field section, the stress changes and damage degree of the working face in different mine stages are determined. A new mechanical model of coal pillar in a thick coal seam is constructed and the stress changes and structural breakage law of each section are obtained, the stress transfers mechanism of overburden structure and the stress damage to coal pillar during the mining process of upper and lower section comprehensive mining working face is analyzed. The stress, microscopic damage, and the degree of sloughing in the coal pillar were significantly reduced by monitoring through the top plate hydraulic cracking pressure relief control program. The results of the study show that the extent of sloughing coal pillar and internal damage is closely related to the length of key block B of the masonry beam formed by the overlying rock layer and the position of the break line, by changing the orientation of the overlying masonry beam rock layer and the position and length of the tendency to break line, the sloughing coal pillar is effectively controlled and its stress and damage are reduced. We have achieved good results in control of the sloughing in the Hanjiawan coal mine and similar mines in northern Shaanxi.

RETRACTED ARTICLE: BPSO based neural network approach for content-based face retrieval

RETRACTED ARTICLE: BPSO based neural network approach for content-based face retrieval

Neighborhood-Exact Nearest Neighbor Search for face retrieval

Features extracted by deep convolutional neural network perform well for face recognition. The crux of large-scale face retrieval using such features stems from the trade-off between speed and accuracy: search efficiently while preserving acceptable accuracy. Traditional methods perform Nearest Neighbor Search (NNS) over the entire gallery set based on feature matching in the continuous space, which is able to achieve exact (optimized) retrieval results for given features. However, it is not feasible for large-scale face retrieval due to unaffordable time consumption. A typical way of speeding up the face retrieval is to conduct Approximate Nearest Neighbor Search (ANNS) at the expense of accuracy without guarantee of search quality. To circumvent this dilemma, we define a new problem w.r.t. large-scale face retrieval, which is referred to as Neighborhood-Exact Nearest Neighbor Search (NENNS). NENNS demands efficient search while guaranteeing the search exactness within a specified neighborhood around the query. Further, we propose a heuristic method for face retrieval, which is able to perform NENNS efficiently. Specifically, it first discretizes the continuous features of samples into binary codes by our designed Angular Binary-Encoding Mechanism. Then it performs fast yet precise exclusion of bad candidates in the binary feature space based on our defined metric (binarized Cosine similarity) to reduce search space significantly. Finally the accurate searching results conforming to NENNS are achieved by conducting precise search in continuous feature space. We perform theoretical proof and extensive experiments to validate the correctness and effectiveness of the proposed method.

Image and video face retrieval with query image using convolutional neural network features

This paper addresses the issue of image and video face retrieval. The aim of this work is to be able to retrieve images and/or videos of specific person from a dataset of images and videos if we have a query image of that person. The methods proposed so far either focus on images or videos and use hand crafted features. In this work we built an end-to-end pipeline for both image and video face retrieval where we use convolutional neural network (CNN) features from an off-line feature extractor. And we exploit the object proposals learned by a region proposal network (RPN) in the online filtering and re-ranking steps. Moreover, we study the impact of finetuning the networks, the impact of sum-pooling and max-pooling, and the impact of different similarity metrics. The results that we were able to achieve are very promising.

Optimisation research on a thin coal seam isolated island coal face mining back channel layout and its support parameters

The large deformation and serious failure of the anchor bolt (cable) of a thin coal seam island coal face retrieval mining roadway is a very difficult technical problem to solve. This article is based on the background of the Jizhong Energy Company Handan Mining Group Guo Erzhuang mine 22204 face mining. Through research, theoretical analysis, numerical simulation and experimental methods that revealed the characteristics of the perimeter rock of the roadway and the destruction mechanism, the composite support scheme of 'anchor + anchor cable + lining' was designed to calculate the reliability of the ultimate bearing capacity of the scheme. The simulation analysis of the design with numerical simulation software was used to verify the rationality of the support scheme. Finally, the composite support was successfully applied in the Guo Erzhuang mine. The onsite monitoring results showed a 64% reduction in the roof subsidence and a 70% reduction in the deformation of two gangway. The support scheme effectively controlled the convergent deformation of the retrieval roadway and achieved good industrial testing expectations. The results of the research could provide a reference for the control technology of a thin coal seam island coal face retrieval roadway under similar geological conditions. [Received: October 30, 2020; Accepted: February 03, 2022]

High-Capacity Real-Time Face Retrieval Recognition Algorithm Based on Task Scheduling Model for the Treatment Area of Hospital

This paper presents an in-depth study of face detection, face feature extraction, and face classification from three important components of a high-capacity face recognition system for the treatment area of hospital and a study of a high-capacity real-time face retrieval and recognition algorithm for the treatment area of hospital based on a task scheduling model. Considering the real-time nature of our system, our face feature extraction network is modeled by DeepID, and the network is slightly improved by introducing a central loss verification signal to train a DeepID-like network model using central loss and use it to extract face features. To further investigate and optimize the schedulability analysis problem of the directed graph real-time task model, this paper proposes a rigorous and approximate response time analysis method for the directed graph real-time task model with an arbitrary time frame. Based on the theoretical results of the greatly additive algebra, it is shown that the coherent qualifying function is linearly periodic, i.e., the function can be represented by a finite nonperiodic part and an infinitely repeated periodic part, thus calculating the coherent qualifying function independent of the magnitude of the interval time. The algorithm for high-capacity real-time face retrieval and recognition in the treatment area of hospital based on the task scheduling model is further investigated, and a face database is established by using the PCA dimensionality reduction technique. Based on the internal architecture of the processor, image preprocessing and IP core packaging are implemented, and the hardware engineering of the high-capacity real-time face recognition system for hospital visits is built using the IP-based design concept. The performance tests of the face detection model and feature extraction network show that the face detection model has a significant reduction in false-positive rate, better fitting of border regression, and improved time performance. The face feature extraction network has no overfitting, and the features are highly discriminative with small feature extraction time consumption. The high-capacity real-time face recognition system for the treatment area of hospital combined with the optimized directed graph task scheduling model can approach 25 fps, which meets the real-time requirements, and the face recognition rate surpasses that of real people. It realizes the intelligence, self-help, and autonomy of medical services and satisfies the medical needs of users in all aspects.

Rapid Face Image Retrieval for Large Scale Based on Spark and Machine Learning

This paper proposed a distributed computing environment large-scale face image retrieval method. Based on Spark platform, a face retrieval method based on machine learning is established, which takes advantage of parallel computing to improve the efficiency of face image retrieval. In the method, SIFT algorithm is used for feature coding, PCA dimension reduction processing, HBase database is used for data storage, and KD-Tree query algorithm is used to match images similar to the query images. Meanwhile, large-scale computing engine uses Spark to process data to improve the retrieval efficiency. The CelebA dataset is selected to test the method, and the experimental results show the effectiveness of the method.

An Optimal Feature Selection Method for Automatic Face Retrieval Using Enhanced Grasshopper Optimization Algorithm

Background: Retrieval of facial images based on its contents is one of the main areas of research. However, images contain high dimensional feature vectors and it is a challenging task to select the relevant features due to the variations available in the images of similar objects. Therefore, the selection of relevant features is an important step to make the facial retrieval system computationally efficient and more accurate. Objective: The main aim of this paper is to design and develop an efficient feature selection method for obtaining relevant and non-redundant features from the face images so that the accuracy and computational cost of a face retrieval system can be improved. Methods: The proposed feature selection method uses a new enhanced grasshopper optimization algorithm to obtain the significant features from the high dimensional features vector of face images. The proposed algorithm modifies the target vector by considering more than one best solution which maintain the elitism property and save the search from local optimum. Furthermore, it has been utilized to select the prominent features from the high dimensional facial features vector. Results: The performance of the proposed feature selection method has been tested on Oracle Research Laboratory face database. The proposed method eliminates 89% features which are minimum among the other methods and increases the accuracy of face retrieval system to 96.5%. Conclusion: The enhanced grasshopper optimization algorithm-based feature selection method for face retrieval system outperforms the existing methods in terms of accuracy and computational cost.

High-order histogram-based local clustering patterns in polar coordinate for facial recognition and retrieval

Local feature patterns are conspicuous and are widely used in computer vision, especially in face recognition and retrieval. However, a statistical descriptor that can be used in various scenarios and effectively present the detailed local discrimination information of face images is a challenging and exploring task even if deep learning technology is widelyspread. In this study, we propose a novel local pattern descriptor called the Local Clustering Pattern (LCP) in high-order derivative space for facial recognition and retrieval. Unlike prior methods, LCP exploits the concept of clustering to analyze the relationship of intra- and inter-classes of the referenced pixel and its adjacent pixels to encode the local descriptor for facial recognition. There are three tasks (1) Local Clustering Pattern (LCP), (2) Clustering Coding Scheme, (3) High-order Local Clustering Pattern. To generate local clustering pattern, the local derivative variations with multi-direction are considered and that are integrated on rectangular coordinate system with the pairwise combinatorial direction. Moreover, to generate the discriminative local pattern, the features of local derivative variations are transformed from the rectangular coordinate system into the polar coordinate system to generate the characteristics of magnitude (m) and orientation ( $$\theta $$ ). Then, we shift and project the features (m and $$\theta $$ ), which are scattered in the four quadrants of polar coordinate system, into the first quadrant of polar coordinates to strengthen the relationship of intra- and inter-classes of the referenced pixel and its adjacent pixels. To encode the local pattern, we consider the spatial relationship between reference and its adjacent pixels and fuse the clustering algorithm into the coding scheme by utilizing the relationship of intra- and inter-classes in a local patch. In addition, we extend the LCP from low- into high-order derivative space to extract the detailed and abundant information for facial description. LCP efficiently encodes the feature of a local region that is discriminative the inter-classes and robust the intra-class of the related pixels to describe a face image. This study has three main contributions: (1) we generate the novel features with magnitude (m) and orientation ( $$\theta $$ ) based on the pairs of the derivative variations to describe the characteristics of each pixel, (2) we shift and project the features from four quadrants of polar coordinate system into the first quadrant of polar coordinates to strengthen the relationship of intra- and inter-classes between pixels in a local patch, (3) we exploit the concept of clustering, which considers the relationship of intra- and inter-classes of the referenced pixel and its adjacent pixels, to encode the local descriptor in a polar coordinate system for facial recognition and retrieval. Experimental results show that LCP outperforms the existing descriptors (LBP, ELBP LDP, LTrP, LVP, LDZP, LGHP) on six public datasets (ORL, Extend Yale B, CAS PEAL, and LFW, CMU-PIE and FERET) for both face recognition and retrieval tasks. Moreover, we further compare the proposed facial descriptor with the popular deep convolutional neural networks to demonstrate the discrimination of the extracted features and applicability of our approach.

Face retrieval system based on elastic web crawler over cloud computing

The web pages are considered as the main source of the available and provided information that is characterized by variation in its content. The facial recognition plays a key role in knowledge management and identity authentication systems. Although the rapid advance of the web technologies and face recognition systems, the improvement of real-time performance is still the bottleneck. The main objective of this study is to propose a real-time face retrieval system as a service over cloud computing based on a web face crawler. The proposed architecture ensures that the total response time is reduced and the resource utilization is optimized. The web crawlers fetch web pages and extract images in elastic storage over the cloud. Then the collected images are used to extract human faces and to prepare the faces images by succeeding phases to be ready for recognition and identifying the matched face of the collection. The proposed service depends on Principle Component Analysis (PCA) algorithm for feature extraction and dimensionality reduction. Furthermore, K-Nearest Neighbors (KNN) is used to classify the crawled facial images over cloud resources. The experimental results investigated that an enhancement of crawling speed is achieved by increasing the crawler instances. Moreover, the accuracy is enhanced in the face recognition based on the Euclidean over other metrics such as Manhattan and Cosine dissimilarity.

Average biased ReLU based CNN descriptor for improved face retrieval

The convolutional neural networks (CNN), including AlexNet, GoogleNet, VGGNet, etc. extract features for many computer vision problems which are very discriminative. The trained CNN model over one dataset performs reasonably well whereas on another dataset of similar type the hand-designed feature descriptor outperforms the same trained CNN model. The Rectified Linear Unit (ReLU) layer discards some values in order to introduce the non-linearity. In this paper, it is proposed that the discriminative ability of deep image representation using trained model can be improved by Average Biased ReLU (AB-ReLU) at the last few layers. Basically, AB-ReLU improves the discriminative ability in two ways: 1) it exploits some of the discriminative and discarded negative information of ReLU and 2) it also neglects the irrelevant and positive information used in ReLU. The VGGFace model trained in MatConvNet over the VGG-Face dataset is used as the feature descriptor for face retrieval over other face datasets. The proposed approach is tested over six challenging, unconstrained and robust face datasets (PubFig, LFW, PaSC, AR, FERET and ExtYale) and also on a large scale face dataset (PolyUNIR) in retrieval framework. It is observed that the AB-ReLU outperforms the ReLU when used with a pre-trained VGGFace model over the face datasets. The validation error by training the network after replacing all ReLUs with AB-ReLUs is also observed to be favorable over each dataset. The AB-ReLU even outperforms the state-of-the-art activation functions, such as Sigmoid, ReLU, Leaky ReLU and Flexible ReLU over all seven face datasets.

Attribute Prototype Learning for Interactive Face Retrieval

Interactive face retrieval aims at finding target subjects in face databases through human and machine interaction, which involves user feedback based on human perception and machine similarity measure in feature spaces. In this article, we propose an attribute prototype learning method to tackle the semantic gap between human and machine in face perception for fast interactive face retrieval. We reformulate the theoretical explanation of the interactive retrieval model and develop the algorithm of the heuristic solution of the model. Each module of the prototype model is learned with a set of identity-related facial attributes. The outputs of the prototype modules form the semantic representation. To adapt the prototype models across different databases, we propose a transfer selection algorithm based on the coherence measurements in interactive face retrieval. Coherence analysis proves that the proposed attribute prototype representation can effectively narrow down the semantic gap even in the case of cross-database transfer learning. The prototype representation can effectively reduce the feature dimension in the retrieval process. Real user retrieval with the Bayesian relevance feedback model shows that attribute prototype space is superior to low-level feature space and proves that interactive retrieval with attribute prototype representation can converge fast in large face databases.