Facial Image Processing Research Articles

Advanced emotion analysis: harnessing facial image processing and speech recognition through deep learning

The human face hides many secrets and is one of the most expressive human features. Human faces even contain hidden information about a person's personality. Considering the fundamental role of the human face, it is necessary to prepare appropriate deep-learning solutions that analyze human face data. This technology is becoming increasingly common in many industries, such as online retail, advertising testing, virtual makeovers, etc. For example, facial analysis technology now allows online shoppers to virtually apply makeup and try on jewelry or new glasses to get an accurate picture of what these products will look like. The human sense of hearing is a treasure trove of information about the current environment and the location and properties of sound-producing objects. For instance, we effortlessly absorb the sounds of birds singing outside the window, traffic passing in the distance, or the lyrics of a song on the radio. The human auditory system can process the intricate mix of sounds reaching our ears and create high-level abstractions of the environment by analyzing and grouping measured sensory signals. The process of obtaining segregation and identifying sources of a received complex acoustic signal, known as sound scene analysis, is a domain where the power of deep learning shines. The machine implementation of this functionality (separation and classification of sound sources) is pivotal in applications such as speech recognition in noise, automatic music transcription, searching and retrieving multimedia data, or recognizing emotions in statements.

A Study on Facial Landmark Detection and Image Processing

A Study on Facial Landmark Detection and Image Processing

Toward a Privacy-Preserving Face Recognition System: A Survey of Leakages and Solutions

Abstract Recent advancements in face recognition (FR) technology in surveillance systems make it possible to monitor a person as they move around. FR gathers a lot of information depending on the quantity and data sources. The most severe privacy concern with FR technology is its use to identify people in real-time public monitoring applications or via an aggregation of datasets without their consent. Due to the importance of private data leakage in the FR environment, academia and business have given it a lot of attention, leading to the creation of several research initiatives meant to solve the corresponding challenges. As a result, this study aims to look at privacy-preserving face recognition (PPFR) methods. We propose a detailed and systematic study of the PPFR based on our suggested six-level framework. Along with all the levels, more emphasis is given to the processing of face images as it is more crucial for FR technology. We explore the privacy leakage issues and offer an up-to-date and thorough summary of current research trends in the FR system from six perspectives. We also encourage additional research initiatives in this promising area for further investigation.

Retracted: English Web-Based Teaching Supervision Based on Intelligent Face Image Perception and Processing for IoT

Retracted: English Web-Based Teaching Supervision Based on Intelligent Face Image Perception and Processing for IoT

Modeling naturalistic face processing in humans with deep convolutional neural networks.

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

Application of Deep Learning in Super-resolution Processing of Face Images

The resolution of an image is generally defined as the number of pixels contained in a unit size. The higher the resolution, the more details it contains and the clearer the image is. Because the results of reconstructing high-resolution images from low-resolution images are not unique, the super-resolution of images is a morbid inverse problem, and it is also a challenging and open research topic in the field of computer vision. With the development of machine learning and deep neural network, this paper studies the application of DL (Deep Learning) in face image super-resolution processing. The results show that the algorithm in this paper combines detail enhancement module and synthesizes fine-grained structure from high-resolution example image, which can generate low-frequency details of the image and transmit high-frequency details from the example image to the basic image for enhancement. It can be seen that the algorithm in this paper is better than other methods in evaluation index and visual effect. However, there are still some shortcomings in this algorithm and experiment, which need further research and improvement. The two image super-resolution algorithms proposed in this paper are both aimed at improving the perceptual quality of reconstructed images.

Research on 3D reconstruction method of cattle face based on image

As the basis of intelligent breeding management and animal husbandry insurance, the identification of individual cattle is important in animal husbandry management. Given the difficulty of data acquisition caused by the non-rigid and lacking cooperation of cattle, this study proposes a method for cattle face image acquisition and processing that can efficiently adapt to the harsh environment of cattle barns. When processing the non-rigid cow face, the method of approximating the cow face to a rigid body is used to establish the cow face image data set., and the cattle face image data set is established. The Three Dimensional(3D) reconstruction method of cattle face uses a 3D image reconstruction method based on multiple perspectives. First, the scale-invariant feature transform algorithm is used to extract the image feature points. The fast library for approximate nearest neighbors algorithm is used to match feature points. The matching results are selected via random sampling consensus. Second, the structure of the motion method is used for the sparse reconstruction of point clouds, and the dense point cloud is then generated using the three-dimensional multi-view stereo vision algorithm. Finally, the Poisson surface reconstruction method is used for surface reconstruction. The results indicate that this method can effectively realize the three-dimensional reconstruction of cattle faces; the reconstructed images have obvious color, clear texture, and complete shape features.

Evaluation of edge detection algorithm of frontal image of facial contour in plastic surgery

With the improvement of medical levels and the continuous improvement of people’s living standards, the demand for beauty by the general public is increasing. The plastic surgery industry has also developed by leaps and bounds. People’s dissatisfaction with their own facial appearance, facial injuries and some other reasons have prompted people to carry out facial reconstruction, and facial plastic surgery has developed rapidly. However, in the current facial plastic surgery, the edge detection effect on the contour image is general. In order to improve the edge detection effect of facial contour lines in medical images, this paper proposed a facial contour line generation algorithm. First, the detection effects of four operators were compared. After comparing the effects, the Sobel operator was used as the input data to generate an edge detection algorithm. Then, the grayscale features of the tissue in the image and the symmetry of the image were used to perform bidirectional contour tracking on the detected image to extract facial contour lines. In addition, for facial contour features, the midpoint method can be used to generate auxiliary contours. The algorithm was verified by a set of facial CT (Computed Tomography) images in the experiment. The results showed that the new generation algorithm accelerated the edge detection speed, had good denoising performance, and enhanced the edge detection effect by about 12.05% compared with the traditional edge detection algorithm. The validity and practicability of facial edge detection were verified, and it provided a theoretical basis for further realizing the design of a facial contour digital image processing system.

An Analysis of Bias in Facial Image Processing: A Review of Datasets

Facial image processing is a major research area in digital signal processing. According to recent studies, most commercial facial image processing systems are prejudiced by bias towards specific races, ethnicities, cultures, ages, and genders. In some circumstances, bias may be traced back to the algorithms employed, while in others, bias can be elicited from the insufficient representations in datasets. This study tackles bias based on insufficient representations in datasets. To tackle this, the research undertakes an exploratory review in which the context of facial image dataset is analyzed to thoroughly examine the rate of bias. Facial image processing systems are developed using widely publicly available datasets since generating datasets are costly. However, these datasets are strongly biased towards Whites and Asians, and other geo-diversity such as indigenous Africans are underrepresented. In this study, 40 large publicly accessible facial image data sets were examined. The races of the datasets used for this study were visualized using the t-distributed Stochastic Neighbor Embedding (t-SNE) visualization method. Then, to measure the geo-diversity and rate of bias of the dataset, k-means clustering, principal component analysis (PCA) and the Oriented FAST and Rotated BRIEF (ORB) feature extraction techniques were used. The findings from this study indicate that these datasets seem to exhibit an obvious ethnicity representation bias, particularly for native African facial images; as a result, additional African indigenous datasets are required to reduce the bias currently present in the most publicly available facial image datasets.

Teknik Pengenalan Wajah Menggunakan Model Ekstraksi Fitur Citra Digital

The use of information technology has been widely encountered in our daily life. Not only to process data, to record tools but also to identify and recognize human characteristics. This is called biometric technology. This technology identifies the unique and permanent parts of the human body such as fingerprints, eyes, and the shape of the human face. To identify and recognize human faces, use facial image processing and analysis, such as determining the component regions of the human face and their characteristics. Splitting the face image into facial components, then extracting it into the features of the eyes, nose, mouth, and chin. The distance between each component is measured, then combined with other features to form facial semantics. The face can be categorized into the T Zone which consists of the forehead, eyes, nose and mouth. Eyes, nose, and mouth are the most unique facial components for facial recognition because they have unique facial recognition features. For the distance of the eye and mouth triangle feature, J1 – J3 shows that there are 140 unique data with the percentage value is 93.33%. The feature distance J4 – J6 also shows that there are 126 unique face images with a percentage value of 85%. Keyword: face image, eiginfaces, image extraction, face recognition.

Micro-expression spotting based on optical flow features

• The nose tip location-based image alignment method improves the performance of Micro-expression spotting. • The optical flow method can accurately capture micro-expression movements. • When a micro-expression occurs, a peak appears on the time-domain change curve. • Get complete micro-expression intervals after fusing local movements. Expression is the changes of facial organs due to facial muscle movement and an important way of human emotional interaction. Neurophysiological studies show that micro-expressions (MEs) are not controlled by subjective consciousness and reflect people's real emotions. That is why MEs have significant value in public security applications. This paper proposes an automatic ME spotting method of high accuracy and interpretability. Firstly, we design the nose tip location-based image alignment method to remove global displacement caused by head shaking. Secondly, according to the action unit definition in the face coding system (FACS), we select fourteen regions of interest (ROI) to capture subtle facial movements. The dense optical flow is introduced to estimate local movements and time-domain variations of the ROIs. Thirdly, we design a peak detection method on the time-domain variation curves to locate the movement intervals precisely. Lastly, we propose an overlapping index to measure the consistency of changes in different organs. Evaluation on the CAS(ME) 2 and SAMM Long Video database shows that our ME spotting method may achieve better accuracy with a relatively lower computation cost and can be applied to similar facial image processing applications.

Research on contour feature extraction method of multiple sports images based on nonlinear mechanics

Abstract This article solves the issue of long extraction time and low extraction accuracy in traditional moving image contour feature extraction methods. Here authors have explored deformable active contour model to research the image processing technology in scientific research and the application of multiple sports and the method. A B-spline active contour model based on dynamic programming method is proposed in this article. This article proposes a method of using it to face image processing and extracting computed tomography (CT) image data to establish a three-dimensional model. The Lyapunov exponent, correlation dimension and approximate entropy of the nonlinear dynamics algorithm were used to extract the features of eight types of motor imagination electroencephalogram (EEG) signals. The results show that the success rate of pose reconstruction is more than 97% when the contour extraction quality is relatively ideal. The method is also robust to image noise, and the success rate of pose reconstruction can reach 94% when the video image has large noise. The execution efficiency is sub-linear, which can basically meet the requirements of real-time processing in video-based human posture reconstruction. The proposed method has a low error rate in the calculation of curvature features, effectively reduces the time for extracting contour features of moving images, and improves the accuracy of feature information extraction.

The lower bow-shaped curve as a novel reference frame to determine the lateral limit of the maxillary anterior arch for smile esthetics

Smile analysis in horizontally angled views is indispensable for esthetic assessment and could reveal teeth-to-lip disharmony, which might escape discovery in the frontal and profile views. However, evidence is lacking on where the anterior teeth should be positioned for esthetics in angled smiles. Based on 3-dimensional facial image processing and geometric analysis, the lip edges were projected to the horizontal plane, and the horizontal teeth-to-lip relation was simplified and represented by the distances from dental landmarks to lower bow-shaped curves (LBSC), with the distance from facial-axis (FA) point of the canine to LBSC (FA-tangent line [TL] distance) identified as the key parameter. Using photographic modification and esthetic assessment, the effect of FA-TL distances on the attractiveness of 45° angled smiles was tested, with esthetic ranges identified. A simplified method was developed to obtain the estimative LBSC and FA-TL distances using 2-dimensional photographs and geometric analysis to facilitate clinical application. The FA-TL distance remarkably affected the esthetics of 45° angled smiles. Smiles were attractive when the FA-TL distance ranged from -1.0 to 1.5 mm perceived by orthodontists and from -1.5 to 1.5 mm perceived by laypersons. The 2-dimensional photograph-derived estimative FA-TL distance was not significantly different from that obtained in a 3-dimensional image, validating the simplified method. The LBSC could serve as a reference frame to determine the lateral limit of the maxillary anterior arch for the esthetics of 45° angled smiles. The FA-TL distance, which represented the spatial relation of the maxillary canine with the lower lip, was an esthetically essential parameter. For females aged 20-30 years, the FA point of the maxillary canines should be positioned no more than 1.5 mm labial or lingual to the LBSC.

A fusion representation for face learning by low-rank constrain and high-frequency texture components

• An effective technique for extracting the high-frequency texture component of image is designed. • A learning framework is proposed to reconcile the subspace learning procedure with texture information. • Experimental results show our approach can learn a better low-rank structure and achieve better clustering performance. Face image processing has always been an important field of artificial intelligence. In machine learning tasks, samples are vectorized into a data matrix. Hence, different faces lie in several respective low-rank subspaces , which can be effectively captured by Low-Rank Representation (LRR) learning. However, most technologies usually use the raw data directly but ignore the weight among different information of the image itself. Following the intuition that human beings distinguish different people through the lineaments and wrinkles of faces which are indeed the texture components of face images, it is believed that representation learning will benefit from introducing the texture components of samples into the information fusion. Inspired by Multi-view learning models which can reconcile the knowledge gained from different views, a fusion framework is proposed in this letter, named High-Frequency texture components Low-Rank Representation (HFLRR). In HFLRR, the high-frequency texture components of samples will be extracted by Fast Fourier Transformation and Butterworth filtering firstly. Subsequently, the texture information will be introduced into the subspace learning procedure. An effective optimization solution is presented for this framework and the experimental results show that the proposed algorithm outperforms the state-of-art methods on several real-world face datasets.

Development of a Method for Quantitative Evaluation of Facial Swelling in a Rat Model of Cerebral Ischemia by Facial Image Processing.

A quantitative method for the evaluation of facial swelling in rats with middle cerebral artery occlusion (MCAO) was established using a mathematical method for the first time. The rat model of MCAO was established via bilateral common carotid artery ligation. Three groups of rats with the same baseline were selected (model group, positive drug group, and control group) according to their behavioral score and body weight 24 h after surgery. Drug administration was initiated on post-MCAO day 8 and was continued for 28 days. Mobile phones were used to collect facial images at different time points after surgery. In facial image analysis, the outer canthi of both eyes were used as the facial dividing line, and the outer edge of the rat's face was framed using the marking method, and the framed part was regarded as the facial area (S) of the rats. The histogram created with Photoshop CS5 was used to measure the face area in pixels. The distance between the outer canthi of both eyes (Le) and vertical line from the tip of the nose to the line joining the eyes was recorded as H1, and the line from the tip of the nose to the midpoint of the line joining the eyes was recorded as H2. The facial area was calibrated based on the relationship between H1 and H2. The distance between the eyes was inversely proportional to the distance between the rats and mobile phone such that the face area was calibrated by unifying Le. The size of Le between the eyes was inversely proportional to the distance between the rats and mobile phone. This was used to calibrate the face area. When compared with the control group, the facial area of the model group gradually increased from postoperative day 1 to day 7, and there was a significant difference in the facial area of the model group on postoperative day 7. Hence, positive drugs exhibited the effect of improving facial swelling. H1 and H2 can reflect the state of turning the head and raising the head of the rats, respectively. Facial area was calibrated according to the relationship between H1 and H2, which had no obvious effect on the overall conclusion. Furthermore, mobile phone lens was used to capture the picture of rat face, and the distance between the eyes and H1 and H2 was used to calibrate the facial area. Hence, this method is convenient and can be used to evaluate subjective judgment of the human eyes via a quantitative method.

Blind Face Restoration via Multi-Prior Collaboration and Adaptive Feature Fusion.

Blind face restoration (BFR) from severely degraded face images is important in face image processing and has attracted increasing attention due to its wide applications. However, due to the complex unknown degradations in real-world scenarios, existing priors-based methods tend to restore faces with unstable quality. In this article, we propose a multi-prior collaboration network (MPCNet) to seamlessly integrate the advantages of generative priors and face-specific geometry priors. Specifically, we pretrain a high-quality (HQ) face synthesis generative adversarial network (GAN) and a parsing mask prediction network, and then embed them into a U-shaped deep neural network (DNN) as decoder priors to guide face restoration, during which the generative priors can provide adequate details and the parsing map priors provide geometry and semantic information. Furthermore, we design adaptive priors feature fusion (APFF) blocks to incorporate the prior features from pretrained face synthesis GAN and face parsing network in an adaptive and progressive manner, making our MPCNet exhibits good generalization in a real-world application. Experiments demonstrate the superiority of our MPCNet in comparison to state-of-the-arts and also show its potential in handling real-world low-quality (LQ) images from several practical applications.

Research on video image face detection and recognition technology based on improved MTCNN algorithm

With the development of modern computer technology and artificial intelligence, face image processing technology has been widely used in people's life and work. In order to realise face image detection and recognition in dynamic video, this paper proposes a face detection and recognition technology based on MTCNN algorithm. MTCNN algorithm includes R-Net, O-net and P-net deep network models, which can realise face image deep processing in dynamic video. In order to train MTCNN algorithm deeply, Wider_Face and CelebA database training sets were used to train the additional test tasks and regression key points of the model. After setting the main parameters of MTCNN algorithm, the algorithm is simulated and analysed. Through the comparative simulation analysis of traditional algorithm, SVM algorithm and 2DPCA algorithm, it can be seen that MTCNN algorithm has more excellent performance and can meet the needs of face image detection and recognition in dynamic video.

Artificial Intelligence (AI) at Schengen Borders: Automated Processing, Algorithmic Profiling and Facial Recognition in the Era of Techno-Solutionism

Abstract Since the past three decades, an elaborate legal framework on the operation of EU-Schengen information systems has been developed, whereby in the near future a series of personal data concerning almost all third-country nationals (TCN s) with an administrative or criminal law link with the EU/Schengen area will be monitored through at least one information system. This article provides a legal analysis on the embedment of Artificial Intelligence (AI) tools at the EU level in information systems for TCN s and critically examines the fundamental rights concerns that ensue from the use AI to manage and control migration. It discusses automated risk assessment and algorithmic profiling used to examine applications for travel authorisations and Schengen visas, the shift towards the processing of facial images of TCN s and the creation of future-proof information systems that anticipate the use of facial recognition technology. The contribution understands information systems as enabling the datafication of mobility and as security tools in an era whereby a foreigner is risky by default. It is argued that a violation of the right to respect for private life is merely the gateway for a series of other fundamental rights which are impacted, such as non-discrimination and right to effective remedies.

Retraction Note: Image detection of mountain soil loss and human avoidance trajectory based on cloud computing and machine learning

With the continuous progress and development of modern social science management and network security technology in China, the wide application of new-generation technologies such as intelligent robotics learning has made it more and more extensive in the field of analysis. At present, it has fully played an important technical leading role in many application fields, such as analysis methods, collection and sorting ability, analysis of various large-scale business computing application statistical data of enterprises, mining theoretical knowledge of natural environment science, image recognition, retrieval of global geographic location information and so on. Through the simulation of soil erosion in mountainous areas in my country, dynamic data analysis describes the basic situation of soil erosion in various mountainous areas in my country and the whole process of its evolution and development and comprehensively analyzes and studies various soil erosion events in mountainous areas in my country. The corresponding solutions can effectively and accurately respond to emergencies of soil erosion in mountainous areas and effectively reduce the loss of soil and water crops and economic losses in natural disasters. In recent years, information technology companies related to computer and machine learning capabilities, especially based on machine cloud computing, learning science and technology based on the theory of human machine artificial intelligence, and combining related professional technologies, have been used in various areas to prevent floods, droughts, risks, and rain in various areas. Disaster reduction and other non-natural engineering construction technology prevention measures have gradually been applied. The main research question of this paper is to design a related human body trajectory image detection system for new human facial image processing based on machine learning, and information collection and detection management system, and conduct in-depth research on its application prospects.

Research on face specular removal and intrinsic decomposition based on polarization characteristics.

It is well known that the specular component in the face image destroys the true informantion of the original image and is detrimental to the feature extraction and subsequent processing. However, in many face image processing tasks based on Deep Learning methods, the lack of effective datasets and methods has led researchers to routinely neglect the specular removal process. To solve this problem, we formed the first high-resolution Asian Face Specular-Diffuse-Image-Material (FaceSDIM) dataset based on polarization characterisitics, which consists of real human face specular images, diffuse images, and various corresponding material maps. Secondly, we proposed a joint specular removal and intrinsic decomposition multi-task GAN to generate a de-specular image, normal map, albedo map, residue map and visibility map from a single face image, and also further verified that the prediected de-specular images have a positive enhancement effect on face intrinsic decomposition. Compared with the SOTA algorithm, our method achieves optimal performance both in corrected linear images and in uncorrected wild images of faces.