Face Detection Task Research Articles

Exploring Distributed Deep Learning Inference Using Raspberry Pi Spark Cluster

Raspberry Pi (Pi) is a versatile general-purpose embedded computing device that can be used for both machine learning (ML) and deep learning (DL) inference applications such as face detection. This study trials the use of a Pi Spark cluster for distributed inference in TensorFlow. Specifically, it investigates the performance difference between a 2-node Pi 4B Spark cluster and other systems, including a single Pi 4B and a mid-end desktop computer. Enhancements for the Pi 4B were studied and compared against the Spark cluster to identify the more effective method in increasing the Pi 4B’s DL performance. Three experiments involving DL inference, which in turn involve image classification and face detection tasks, were carried out. Results showed that enhancing the Pi 4B was faster than using a cluster as there was no significant performance difference between using the cluster and a single Pi 4B. The difference between the mid-end computer and a single Pi 4B was between 6 and 15 times in the experiments. In the meantime, enhancing the Pi 4B is the more effective approach for increasing the DL performance, and more work needs to be done for scalable distributed DL inference to eventuate.

An Efficient Anchor-Free Face Detector with Attention Mechanisms

Enormous progress has been made in face detection tasks due to the rapid development of deep learning techniques. Meanwhile, debates arise on whether face detection should be treated as a generic object detection task or considered differently. In this paper, we design an efficient anchor-free face detector that focuses on a low flops regime and combines recent advances in generic object detection with the methods for detecting tiny faces. Specifically, we adopt the anchor-free Fully Convolutional One-Stage (FCOS) method with a recently developed Visual Attention Network (VAN) as the base detector. In accordance with the characteristics of the face dataset, we reallocate the computation across the network components by adjusting the network configurations of the base detector. Then we redesign the criteria for marking positive samples to realize a balanced distribution in pixel maps, and we also adopt the quadruple pixel prediction, which enables more positive samples matched with the model outputs. Under VGA resolution, our face detector achieves 70.5% in AP on the hard subset of the WIDER FACE dataset, while the computational cost is only 1.05 Gflops. This accuracy efficiency trade-off is comparable to state-of-the-art results.

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

Identity documents recognition is an important sub-field of document analysis, which deals with tasks of robust document detection, type identification, text fields recognition, as well as identity fraud prevention and document authenticity validation given photos, scans, or video frames of an identity document capture. Significant amount of research has been published on this topic in recent years, however a chief difficulty for such research is scarcity of datasets, due to the subject matter being protected by security requirements. A few datasets of identity documents which are available lack diversity of document types, capturing conditions, or variability of document field values. In this paper, we present a dataset MIDV-2020 which consists of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock identity documents, each with unique text field values and unique artificially generated faces, with rich annotation. The dataset contains 72409 annotated images in total, making it the largest publicly available identity document dataset to the date of publication. We describe the structure of the dataset, its content and annotations, and present baseline experimental results to serve as a basis for future research. For the task of document location and identification content-independent, feature-based, and semantic segmentation-based methods were evaluated. For the task of document text field recognition, the Tesseract system was evaluated on field and character levels with grouping by field alphabets and document types. For the task of face detection, the performance of Multi Task Cascaded Convolutional Neural Networks-based method was evaluated separately for different types of image input modes. The baseline evaluations show that the existing methods of identity document analysis have a lot of room for improvement given modern challenges. We believe that the proposed dataset will prove invaluable for advancement of the field of document analysis and recognition.

Spatial attention shifting to emotional faces is contingent on awareness and task relevancy

The relationship between visual attention and visual awareness has long been hotly debated. There has been limited evidence on whether the neural marker of spatial attention precedes or succeeds that of visual awareness in the processing of emotional faces. The current study aims to investigate the temporal sequence between the electrophysiological signatures of visual awareness (the visual awareness negativity – VAN) and spatial attention (the N2pc), in contexts where emotional faces are task-relevant (Experiment 1) or task-irrelevant (Experiment 2). Fifty-six healthy participants were presented with fearful and neutral faces under different levels of visibility using backward masking. They either performed a face detection task (Experiment 1) or a contrast detection task while ignoring the faces (Experiment 2). Compared to subliminal stimuli, supraliminal stimuli produced more negative ERPs at 170–270 msec and 210–310 msec in Experiments 1 and 2, respectively, identified as the VAN. The P3, a component also frequently considered to reflect awareness, produced a similar effect with larger amplitudes for supraliminal than subliminal stimuli in both experiments. With respect to spatial attention, a significant N2pc was observed in response to fearful faces but only in the supraliminal viewing condition of Experiment 1, in which faces were task-relevant. Crucially, the VAN was found to precede the N2pc in this case. Our results suggest that spatial attention as indexed by the N2pc, is oriented towards fearful faces when they are relevant to participants’ task and are consciously processed. Moreover, an early phenomenal stage of awareness, reflected by the VAN, precedes spatial attention shifting to fearful faces.

Research on Face Local Attribute Detection Method Based on Improved SSD Network Structure

The existing face detection methods usually had the problem of low accuracy of face recognition in the environment of occlusion interference, which was limited when applied to the face detection task in complex scenes. Therefore, in order to realize high precision and real-time local face recognition in a complex environment, a face local attribute detection method based on improved SSD network structure was proposed. Based on the analysis of the face local attribute detection task, SSD was used as the basic detection network structure, and the VGG16 feature extraction model was used as the framework of face local detection. On this basis, by organically connecting different layers of the SSD network and integrating convolution block attention module, the improved SSD network structure was used to realize face local attribute detection. The proposed model was trained and tested using typical public datasets such as Wider Face, MAFA, and COFW. Experimental results showed that this method had high recognition accuracy, can better detect local features of the human face than other models, and can provide some support for local face attribute detection. This method would provide a theoretical basis and technical support for local face attribute detection in complex scenes.

A New Pointwise Convolution in Deep Neural Networks Through Extremely Fast and Non Parametric Transforms

Some conventional transforms such as Discrete Walsh-Hadamard Transform (DWHT) and Discrete Cosine Transform (DCT) have been widely used as feature extractors in image processing but rarely applied in neural networks. However, we found that these conventional transforms can serve as a powerful feature extractor in channel dimension without any learnable parameters in deep neural networks. This paper firstly proposes to apply conventional transforms on pointwise convolution, showing that such transforms can significantly reduce the computational complexity of neural networks without accuracy degradation on various classification tasks and even on face detection task. Our comprehensive experiments show that the proposed DWHT-based model gained 1.49% accuracy increase with 79.4% reduced parameters and 49.4% reduced FLOPs compared with its baseline model on the CIFAR 100 dataset while achieving comparable accuracy under the condition that 81.4% of parameters and 49.4% of FLOPs reduced on SVHN dataset. Additionally, our DWHT-based model showed comparable accuracy with 89.2% reduced parameters and 26.5% reduced FLOPs compared to the baseline models on WIDER FACE and FDDB datasets.

D2BGAN: A Dark to Bright Image Conversion Model for Quality Enhancement and Analysis Tasks Without Paired Supervision

This paper presents an image enhancement model, D2BGAN (Dark to Bright Generative Adversarial Network), to translate low light images to bright images without a paired supervision. We introduce the use of geometric and lighting consistency along with a contextual loss criterion. These when combined with multiscale color, texture and edge discriminators prove to provide competitive results. We performed extensive experiments using benchmark datasets to visually and objectively compare our results. We observed the performance of D2BGAN on real-time driving datasets that are subject to motion blur, noise, and other artifacts. We further demonstrated that our enhanced images can be profitably used in image-understanding tasks. Images processed using our technique obtain the best or second best average scores for three different image quality evaluation methods on the Naturalness Preserved Enhancement (NPE), Low Light Image Enhancement (LIME), Multi-Exposure Image Fusion (MEF) benchmark datasets. Best scores are also obtained on the LOw-Light (LOL) test set and on Berkeley Driving Dataset (BDD) images processed with D2BGAN. Face detection tasks on the DarkFace benchmark dataset show an mAP (mean Average Precision) improvement from 0.209 to 0.301 when images are processed using D2BGAN. mAP further improves to 0.525 when finetuning techniques are adopted.

Atypically slow processing of faces and non-faces in older autistic adults.

Face recognition is a fundamental function that requires holistic processing. Differences in face processing have been consistently identified in autistic children, but it is unknown whether these differences persist across the adult lifespan. Using event-related functional magnetic resonance imaging, we measured holistic face processing with a rapid Mooney faces task in 50 autistic and 49 non-autistic participants (30–74 years). Behavioral tasks included a self-paced version of the same paradigm and a global–local processing task (Navon). Reduced detection rates for faces, but not non-faces, were found in autistic adults, including slower responses on all conditions. Without time constraints, differences in accuracy disappeared between groups, although reaction times in correctly identifying faces remained higher in autistic adults. The functional magnetic resonance imaging results showed lower activation in the left and right superior frontal gyrus in the autism group but no age-related differences. Overall, our findings point toward slower information processing speed rather than a face recognition deficit in autistic adults. This suggests that face-processing differences are not a core feature of autism across the adult lifespan.Lay abstractSome theories suggested that social difficulties in autism arise from differences in the processing of faces. If face-processing difficulties are central to autism, then they should be as persistent as social difficulties across the lifespan. We tested this by asking autistic and neurotypical participants between 30 and 75 years to complete face detection tasks. Both autistic and neurotypical adults responded more slowly with age. When participants had to respond quickly, autistic adults made more errors in face detection regardless of their age. However, when the time constraint was removed, autistic adults performed as well as the neurotypical group. Across tasks, autistic adults responded more slowly when asked to detect both face and non-face stimuli. We also investigated brain activation differences in the face detection task with functional magnetic resonance imaging. The results indicated lower activation in the autism group in the left and right superior frontal gyrus. The superior frontal gyrus is not typically implicated in face processing but in more general processing, for example, keeping instructions in mind and following them. Together with the behavioral results, this suggests that there is no specific deficit in face processing in autistic adults between 30 and 75 years. Instead, the results suggest differences in general processing, particularly in the speed of processing. However, this needs to be investigated further with methods that are more sensitive to the timing of brain activation.

Face detection in untrained deep neural networks

Face-selective neurons are observed in the primate visual pathway and are considered as the basis of face detection in the brain. However, it has been debated as to whether this neuronal selectivity can arise innately or whether it requires training from visual experience. Here, using a hierarchical deep neural network model of the ventral visual stream, we suggest a mechanism in which face-selectivity arises in the complete absence of training. We found that units selective to faces emerge robustly in randomly initialized networks and that these units reproduce many characteristics observed in monkeys. This innate selectivity also enables the untrained network to perform face-detection tasks. Intriguingly, we observed that units selective to various non-face objects can also arise innately in untrained networks. Our results imply that the random feedforward connections in early, untrained deep neural networks may be sufficient for initializing primitive visual selectivity.

Efficient Face Recognition System for Operating in Unconstrained Environments.

Facial recognition is fundamental for a wide variety of security systems operating in real-time applications. Recently, several deep neural networks algorithms have been developed to achieve state-of-the-art performance on this task. The present work was conceived due to the need for an efficient and low-cost processing system, so a real-time facial recognition system was proposed using a combination of deep learning algorithms like FaceNet and some traditional classifiers like SVM, KNN, and RF using moderate hardware to operate in an unconstrained environment. Generally, a facial recognition system involves two main tasks: face detection and recognition. The proposed scheme uses the YOLO-Face method for the face detection task which is a high-speed real-time detector based on YOLOv3, while, for the recognition stage, a combination of FaceNet with a supervised learning algorithm, such as the support vector machine (SVM), is proposed for classification. Extensive experiments on unconstrained datasets demonstrate that YOLO-Face provides better performance when the face under an analysis presents partial occlusion and pose variations; besides that, it can detect small faces. The face detector was able to achieve an accuracy of over 89.6% using the Honda/UCSD dataset which runs at 26 FPS with darknet-53 to VGA-resolution images for classification tasks. The experimental results have demonstrated that the FaceNet+SVM model was able to achieve an accuracy of 99.7% using the LFW dataset. On the same dataset, FaceNet+KNN and FaceNet+RF achieve 99.5% and 85.1%, respectively; on the other hand, the FaceNet was able to achieve 99.6%. Finally, the proposed system provides a recognition accuracy of 99.1% and 49 ms runtime when both the face detection and classifications stages operate together.

"I Spy with my Little Eye, Something that is a Face…": A Brain Network for Illusory Face Detection.

The most basic aspect of face perception is simply detecting the presence of a face, which requires the extraction of features that it has in common with other faces. Putatively, it is caused by matching high-dimensional sensory input with internal face templates, achieved through a top-down mediated coupling between prefrontal regions and brain areas in the occipito-temporal cortex ("core system of face perception"). Illusory face detection tasks can be used to study these top-down influences. In the present functional magnetic resonance imaging study, we showed that illusory face perception activated just as real faces the core system, albeit with atypical left-lateralization of the occipital face area. The core system was coupled with two distinct brain regions in the lateral prefrontal (inferior frontal gyrus, IFG) and orbitofrontal cortex (OFC). A dynamic causal modeling (DCM) analysis revealed that activity in the core system during illusory face detection was upregulated by a modulatory face-specific influence of the IFG, not as previously assumed by the OFC. Based on these findings, we were able to develop the most comprehensive neuroanatomical framework of illusory face detection until now.

Mask Detection and Tracing System

Covid19 has given a new identity for wearing a mask. It is meaningful when these masked faces are detected accurately and efficiently. As a unique face detection task, face mask detection is much more difficult because of extreme occlusions which leads to the loss of face details. Besides, there is almost no existing large-scale accurately labelled masked face dataset, which increase the difficulty of face mask detection. The system encourages to use CNN-based deep learning algorithms which has done vast progress towards researches in face detection In this paper, we propose novel CNN-based method which is formed of three convolutional neural networks to detect face mask. Besides, because of the shortage of face masked training samples, we propose a new dataset called” face mask dataset” to finetune our CNN models. We evaluate our proposed face mask detection algorithm on the face mask testing set, and it achieves satisfactory performance

Latent space visualization of half face and full face by generative model

Generally, most face detection and recognition tasks are based on the training of intact facial images and their corresponding labels. The training image is supposed to contain as much facial area as possible, and sometimes expanding the training image area to the upper body may also enhance the learning ability. However, we noticed that both the three-dimensional structure and two-dimensional appearance from the frontal view of human faces are bilaterally symmetrical. Few research makes use of this characteristics to simplify the learning process. We have proposed a flipping strategy to apply the facial symmetrical characteristic to transfer learning and proved training with half faces can also achieve equivalent performance in face recognition for a small group of individuals. This paper extend the transfer learning of cropped half face images for face recognition rather than flipping the half face. The facial symmetrical characteristics is utilized to improve face recognition through transfer learning of only a half of the common human face image. We also investigate and explain the reason why the half face area is enough to accurately classify small groups of individuals. A variational autoencoder network is utilized to impose the probability distribution on the facial latent space. Finally, the dimensions of the facial latent space are reduced to visualize the distributed perceptual manifold for face identity.

Emotion recognition at the edge with AI specific low power architectures

Nowadays Deep Learning is applied in almost every research field and helps getting amazing results in a great number of challenging tasks. The main problem is that this kind of learning and consequently Neural Networks that can be defined deep, are resource intensive. They need specialized hardware to perform computation in a reasonable time. Many tasks are mandatory to be as much real-time as possible . It is needed to optimize many components such as code, algorithms, numeric accuracy and hardware, to make them “efficient and usable”. All these optimizations can help us to produce incredibly accurate and fast learning models. The paper reports a study in this direction for the challenging face detection and emotion recognition tasks.

SINGLE-SHOT FACE DETECTION AND RECOGNITION BASED ON CNN

The paper proposes an architecture and training method of a deep convolutional neural network for simultaneous face detection and recognition. The implemented approach combines the ideas of SSD (Single Shot Detector) and Faster R-CNN (Region proposal Convolutional Neural Networks) algorithms. Face detection is performed similarly to single-stage detection algorithms, and then a biometric template is built by employing RoI (Region of Interest) pooling layers and using the separate branch of the neural network. Training process includes three stages: pretraining of thebasic CNN for face recognition on face images, fine-tuning by using RoI pooling on in painted face images, adding SSD layers and fine-tuning on face detection. Wherein, at the latter stage, training is performed by using shared layers technology for two databases simultaneously. The main feature of the algorithm is high processing speed, which does not depend on the number of faces in the input image. For example, in case of using ResNet-34 as the core architecture for the algorithm, the required time for detecting faces and building biometric templates on an image with 100 faces is less than 13 ms. For training purposes we use CASIA-WebFace for face recognition task and Wider Face for face detection task. Testing is performed on FDDB (Face Detection Dataset and Benchmark), since this database is closer to practical applications than Wider. As long as the main practical task the developed method is intended for is face reidentification, we use Fei Face DataBase for face recognition quality testing. We obtain TPR (True Positive Rate) = 0.928@1000 on FDDB Face DataBase and FAR (Face Acceptance Rate) = 0.03309@FRR (Face Rejection Rate) = 10–4. Therefore, the proposed algorithm allows solving face detection and reidentification tasks in real time with any number of faces on an input image.

Underexposed Image Correction via Hybrid Priors Navigated Deep Propagation.

Enhancing visual quality for underexposed images is an extensively concerning task that plays an important role in various areas of multimedia and computer vision. Most existing methods often fail to generate high-quality results with appropriate luminance and abundant details. To address these issues, we develop a novel framework, integrating both knowledge from physical principles and implicit distributions from data to address underexposed image correction. More concretely, we propose a new perspective to formulate this task as an energy-inspired model with advanced hybrid priors. A propagation procedure navigated by the hybrid priors is well designed for simultaneously propagating the reflectance and illumination toward desired results. We conduct extensive experiments to verify the necessity of integrating both underlying principles (i.e., with knowledge) and distributions (i.e., from data) as navigated deep propagation. Plenty of experimental results of underexposed image correction demonstrate that our proposed method performs favorably against the state-of-the-art methods on both subjective and objective assessments. In addition, we execute the task of face detection to further verify the naturalness and practical value of underexposed image correction. What is more, we apply our method to solve single-image haze removal whose experimental results further demonstrate our superiorities.

Benchmarking Low-Light Image Enhancement and Beyond

In this paper, we present a systematic review and evaluation of existing single-image low-light enhancement algorithms. Besides the commonly used low-level vision oriented evaluations, we additionally consider measuring machine vision performance in the low-light condition via face detection task to explore the potential of joint optimization of high-level and low-level vision enhancement. To this end, we first propose a large-scale low-light image dataset serving both low/high-level vision with diversified scenes and contents as well as complex degradation in real scenarios, called Vision Enhancement in the LOw-Light condition (VE-LOL). Beyond paired low/normal-light images without annotations, we additionally include the analysis resource related to human, i.e. face images in the low-light condition with annotated face bounding boxes. Then, efforts are made on benchmarking from the perspective of both human and machine visions. A rich variety of criteria is used for the low-level vision evaluation, including full-reference, no-reference, and semantic similarity metrics. We also measure the effects of the low-light enhancement on face detection in the low-light condition. State-of-the-art face detection methods are used in the evaluation. Furthermore, with the rich material of VE-LOL, we explore the novel problem of joint low-light enhancement and face detection. We develop an enhanced face detector to apply low-light enhancement and face detection jointly. The features extracted by the enhancement module are fed to the successive layer with the same resolution of the detection module. Thus, these features are intertwined together to unitedly learn useful information across two phases, i.e. enhancement and detection. Experiments on VE-LOL provide a comparison of state-of-the-art low-light enhancement algorithms, point out their limitations, and suggest promising future directions. Our dataset has supported the Track “Face Detection in Low Light Conditions” of CVPR UG2+ Challenge (2019–2020) ( http://cvpr2020.ug2challenge.org/ ).

Improving Performance and Adaptivity of Anchor-Based Detector Using Differentiable Anchoring With Efficient Target Generation.

Most anchor-based object detection methods have adopted predefined anchor boxes as regression references. However, the proper setting of anchor boxes may vary significantly across different datasets, improperly designed anchors severely limit the performances and adaptabilities of detectors. Recently, some works have tackled this problem by learning anchor shapes from datasets. However, all of these works explicitly or implicitly rely on predefined anchors, limiting universalities of detectors. In this paper, we propose a simple learning anchoring scheme with an effective target generation method to cast off predefined anchor dependencies. The proposed anchoring scheme, named as differentiable anchoring, simplifies learning anchor shape process by adding only one branch in parallel with the existing classification and bounding box regression branches. The proposed target generation method, including the Lp norm ball approximation and the optimization difficulty-based pyramid level assignment approach, generates positive samples for the new branch. Compared with existing learning anchoring-based approaches, the proposed method doesn't require any predefined anchors, while tremendously improving performances and adaptiveness of detectors. The proposed method can be seamlessly integrated to Faster RCNN, RetinaNet, and SSD, improving the detection mAP by 2.8%, 2.1% and 2.3% respectively on MS COCO 2017 test-dev set. Moreover, the differentiable anchoring-based detectors can be directly applied to specific scenarios without any modification of the hyperparameters or using a specialized optimization. Specifically, the differentiable anchoring-based RetinaNet achieves very competitive performances on tiny face detection and text detection tasks, which are not well handled by the conventional and guided anchoring based RetinaNets for the MS COCO dataset.

Spatial variability of low frequency brain signal differentiates brain states.

Temporal variability of the neural signal has been demonstrated to be closely related to healthy brain function. Meanwhile, the evolving brain functions are supported by dynamic relationships among brain regions. We hypothesized that the spatial variability of brain signal might provide important information about brain function. Here we used the spatial sample entropy (SSE) to investigate the spatial variability of neuroimaging signal during a steady-state presented face detection task. Lower SSE was found during task state than during resting state, associating with more repetitive functional interactions between brain regions. The standard deviation (SD) of SSE during the task was negatively related to the SD of reaction time, suggesting that the spatial pattern of neural activity is reorganized according to particular cognitive function and supporting the previous theory that greater variability is associated with better task performance. These results were replicated with reordered data, implying the reliability of SSE in measuring the spatial organization of neural activity. Overall, the present study extends the research scope of brain signal variability from the temporal dimension to the spatial dimension, improving our understanding of the spatiotemporal characteristics of brain activities and the theory of brain signal variability.

Application of Facial Symmetrical Characteristic to Transfer Learning

Most face detection and recognition tasks are based on the training of intact facial images and corresponding labels. Both the three‐dimensional (3D) structure and two‐dimensional (2D) appearance from the frontal view of human faces are bilaterally symmetrical in general. However, sometimes, illumination on the left and right halves of faces is uneven. In such cases, the symmetrical characteristic of human faces can facilitate expressing distinct identity information. This is because even if one side of the facial image is corrupted by noise, the opposite side can still be used for feature extraction. This paper proposes an automatic selection of the better half of the face using only a half‐face for identity recognition. Unlike the MegaFace challenge of recognizing millions of identities in the wild, this paper focuses on building recognition systems for a small number of people with fewer training images; the recognition system can, for example, build access control systems for research laboratory members or family members. This paper proposes an artificial face image construction method and a half‐face training strategy for transfer learning of pretrained conventional neural network models. Extensive experimental results show that the proposed method improves the performance of state‐of‐the‐art models by utilizing the symmetrical characteristics of human faces. © 2020 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.