Active Appearance Model Research Articles

3D bone shape from CT-scans provides an objective measure of osteoarthritis severity: Data from the IMI-APPROACH study

Decisions regarding total knee arthroplasty are usually made using a patient's own assessment of pain and the structural disposition of the joint as seen on plain film radiographs. Pain severity can fluctuate, and radiographs can be misleading, with the apparent joint status affected by anatomical orientation. An important component of the surgical management of knee osteoarthritis (OA) is the timing of surgical intervention: knee arthroplasty performed too early in the course of the disease may increase the need for revision surgery.Femoral 3D bone shape (B-score) from MR images is an objective measure of OA severity and has been correlated with current and future risk of pain. We aimed to derive the B-score from CT images and compare it against the B-score derived from MR images.We used baseline and 24-month image data from the IMI-APPROACH 2-year prospective cohort study, comprising pairs of CT and MR images taken for each subject-timepoint. The femur was automatically segmented in both CT and MR modalities using an active appearance model, a machine-learning method, to measure the B-score. Linear regression was used to test for correlation between measures. Limits of agreement and bias were tested using Bland-Altman analysis.CT-MR pairs of the same knee were available from 424 participants (78 % women). B-scores from CT and MR were strongly correlated (CCC = 0.980) with negligible bias of 0.0106 (95 % CI: −0. 0281, +0.0493).The strong correlation and small B-score bias suggests that B-scores may be measured reliably using CT images. Since CT images are used in planning robot-assisted knee arthroplasty, with further study B-scores derived from CT surgical planning images could in principle provide a useful objective input to deciding the appropriateness, timing and type of knee arthroplasty.

Numerical study of osteophyte effects on preoperative knee functionality in patients undergoing total knee arthroplasty.

Osteophytes are routinely removed during total knee arthroplasty, yet the preoperative planning currently relies on preoperative computed tomography (CT) scans of the patient's osteoarthritic knee, typically including osteophytic features. This complicates the surgeon's ability to anticipate the exact biomechanical effects of osteophytes and the consequences of their removal before the operation. The aim of this study was to investigate the effect of osteophytes on ligament strains and kinematics, and ascertain whether the osteophyte volume and location determine the extent of this effect. We segmented preoperative CT scans of 21 patients, featuring different osteophyte severity, using image-based active appearance models trained to identify the osteophytic and preosteophytic bone geometries and estimate the cartilage thickness in the segmented surfaces. The patients' morphologies were used to scale a template musculoskeletal knee model. Osteophytes induced clinically relevant changes to the knee's functional behavior, but these were variable and patient-specific. Generally, severe osteophytic knees significantly strained the oblique popliteal ligament (OPL) and posterior capsule (PC) relative to the preosteophytic state. Furthermore, there was a marked effect on the lateral collateral ligament and anterolateral ligament (ALL) strains compared to mild and moderate osteophytic knees, and concurrent alterations in the tibial lateral-medial translation and external-internal rotation. We found a strong correlation between the OPL, PC, and ALL strains and posterolateral condylar and tibial osteophytes, respectively. Our findings may have implications for the preoperative planning in total knee arthroplasty, toward reproducing the physiological knee biomechanics as close as feasibly possible.

CHEAT DETECTION IN ONLINE EXAMINATIONS USING ARTIFICIAL INTELLIGENCE

With increasing use of ICT and technical advancements in the education sector, distant and online education as well as examinations are being carried out frequently. However, online examination, as a method of assessment offers the risk of an unmonitored setting where students have full access to external resources. Online-proctored exams are the most efficient way for educational institutions to ensure academic honesty and ethics to counteract this. Typically, proctoring requires human assistance in the form of online proctors who remotely monitor students' performance. Yet, due to the rising demand for personnel and the intrusive nature of human proctoring, it is imperative to explore other areas. To tackle this pressing issue, this research work aims to devise a novel architecture that, through the development of a robust and automated Artificial Intelligence system, enables students to take exams remotely and reduces proctor involvement. The method overcomes the shortcomings of the previous automated proctoring system by combining important components of online exam cheating detection with cost-effective and efficient hardware. By proposing a Hybrid of FaceNet Model, Lucas Kanade Algorithm, and Active Appearance Model for Face Detection and Activity Monitoring of the student, the proposed system extracts semantic indicators to evaluate whether an applicant is cheating in an online examination. The proposed Cheat Detection system's experimental results measured via an F-score of 0.94 demonstrate its efficacy, and promising performance compared to the standard baseline techniques.

An AAM-Based Identification Method for Ear Acupoint Area.

Ear image segmentation and identification is for the "observation" of TCM (traditional Chinese medicine), because disease diagnoses and treatment are achieved through the massaging of or pressing on some corresponding ear acupoints. With the image processing of ear image positioning and regional segmentation, the diagnosis and treatment of intelligent traditional Chinese medicine ear acupoints is improved. In order to popularize ear acupoint therapy, image processing technology has been adopted to detect the ear acupoint areas and help to gradually replace well-trained, experienced doctors. Due to the small area of the ear and the numerous ear acupoints, it is difficult to locate these acupoints based on traditional image recognition methods. An AAM (active appearance model)-based method for ear acupoint segmentation was proposed. The segmentation was illustrated as 91 feature points of a human ear image. In this process, the recognition effects of the ear acupoints, including the helix, antihelix, cymba conchae, cavum conchae, fossae helicis, fossae triangularis auriculae, tragus, antitragus, and earlobe, were divided precisely. Besides these, specially appointed acupoints or acupoint areas could be prominent in ear images. This method made it possible to partition and recognize the ear's acupoints through computer image processing, and maybe own the same abilities as experienced doctors for observation. The method was proved to be effective and accurate in experiments and can be used for the intelligent diagnosis of diseases.

A New Hybrid Model to Predict Human Age Estimation from Face Images Based on Supervised Machine Learning Algorithms

Abstract Age estimation from face images is one of the significant topics in the field of machine vision, which is of great interest to controlling age access and targeted marketing. In this article, there are two main stages for human age estimation; the first stage consists of extracting features from the face areas by using Pseudo Zernike Moments (PZM), Active Appearance Model (AAM), and Bio-Inspired Features (BIF). In the second step, Support Vector Machine (SVM) and Support Vector Regression (SVR) algorithms are used to predict the age range of face images. The proposed method has been assessed utilizing the renowned databases of IMDB-WIKI and WIT-DB. In general, from all results obtained in the experiments, we have concluded that the proposed method can be chosen as the best method for Age estimation from face images.

An automatic facial landmarking for children with rare diseases.

Two to three thousand syndromes modify facial features: their screening requires the eye of an expert in dysmorphology. A widely used tool in shape characterization is geometric morphometrics based on landmarks, which are precise and reproducible anatomical points. Landmark positioning is user dependent and time consuming. Many automatic landmarking tools are currently available but do not work for children, because they have mainly been trained using photographic databases of healthy adults. Here, we developed a method for building an automatic landmarking pipeline for frontal and lateral facial photographs as well as photographs of external ears. We evaluated the algorithm on patients diagnosed with Treacher Collins (TC) syndrome as it is the most frequent mandibulofacial dysostosis in humans and is clinically recognizable although highly variable in severity. We extracted photographs from the photographic database of the maxillofacial surgery and plastic surgery department of Hôpital Necker-Enfants Malades in Paris, France with the diagnosis of TC syndrome. The control group was built from children admitted for craniofacial trauma or skin lesions. After testing two methods of object detection by bounding boxes, a Haar Cascade-based tool and a Faster Region-based Convolutional Neural Network (Faster R-CNN)-based tool, we evaluated three different automatic annotation algorithms: the patch-based active appearance model (AAM), the holistic AAM, and the constrained local model (CLM). The final error corresponding to the distance between the points placed by automatic annotation and those placed by manual annotation was reported. We included, respectively, 1664, 2044, and 1375 manually annotated frontal, profile, and ear photographs. Object recognition was optimized with the Faster R-CNN-based detector. The best annotation model was the patch-based AAM (p < 0.001 for frontal faces, p=0.082 for profile faces and p < 0.001 for ears). This automatic annotation model resulted in the same classification performance as manually annotated data. Pretraining on public photographs did not improve the performance of the model. We defined a pipeline to create automatic annotation models adapted to faces with congenital anomalies, an essential prerequisite for research in dysmorphology.

Comparison of 3D quantitative osteoarthritis imaging biomarkers from paired CT and MR images: data from the IMI-APPROACH study

IntroductionMRI bone surface area and femoral bone shape (B-score) measures have been employed as quantitative endpoints in DMOAD clinical trials. Computerized Tomography (CT) imaging is more commonly used for 3D visualization of bony anatomy due to its high bone-soft tissue contrast. We aimed to compare CT and MRI assessments of 3D imaging biomarkers.MethodsWe used baseline and 24-month image data from the IMI-APPROACH 2-year prospective cohort study. Femur and tibia were automatically segmented using active appearance models, a machine-learning method, to measure 3D bone shape, area and 3D joint space width (3DJSW). Linear regression was used to test for correlation between measures. Limits of agreement and bias were tested using Bland-Altman analysis.ResultsCT-MR pairs of the same knee were available from 434 participants (78% female). B-scores from CT and MR were strongly correlated (CCC = 0.967) with minimal bias of 0.1 (SDD = 0.227). Area measures were also correlated but showed a consistent bias (MR smaller). 3DJSW showed different biases (MR larger) in both lateral and medial compartments.DiscussionThe strong correlation and small B-score bias suggests that B-score may be measured reliably using either modality. It is likely that the bone surface identified using MR and CT will be at slightly different positions within the bone/cartilage boundary. The negative bone area bias suggests the MR bone boundary is inside the CT boundary producing smaller areas for MR, consistent with the positive 3DJSW bias. The lateral-medial 3DJSW difference is possibly due to a difference in knee pose during acquisition (extended for CT, flexed for MR).Trial registrationNCT03883568

Robust ACO-Based Landmark Matching and Maxillofacial Anomalies Classification

Imagery assessment is an efficient method for detecting craniofacial anomalies. A cephalometric landmark matching approach may help in orthodontic diagnosis, craniofacial growth assessment and treatment planning. Automatic landmark matching and anomalies detection helps face the manual labelling limitations and optimize preoperative planning of maxillofacial surgery. The aim of this study was to develop an accurate Cephalometric Landmark Matching method as well as an automatic system for anatomical anomalies classification. First, the Active Appearance Model (AAM) was used for the matching process. This process was achieved by the Ant Colony Optimization (ACO) algorithm enriched with proximity information. Then, the maxillofacial anomalies were classified using the Support Vector Machine (SVM). The experiments were conducted on X-ray cephalograms of 400 patients where the ground truth was produced by two experts. The frameworks achieved a landmark matching error (LE) of 0.50 ± 1.04 and a successful landmark matching of 89.47% in the 2 mm and 3 mm range and of 100% in the 4 mm range. The classification of anomalies achieved an accuracy of 98.75%. Compared to previous work, the proposed approach is simpler and has a comparable range of acceptable matching cost and anomaly classification. Results have also shown that it outperformed the K-nearest neighbors (KNN) classifier.

Analytical Study of Face Recognition Methods Using Machine Learning

Many facial recognition algorithms, as well as their adaptations, have previously been devised during the last decades. A number is indicated for standard algorithms. They are divided into two categories: appearance-based and model-based. Three distinct linear subspace analysis strategies for appearance-based processes are presented, along with a brief examination of numerous non-linear manifold analysis approaches for face recognition. Elastic Bunch Graph matching, Active Appearance Model, and 3D Model processes are discussed. A vast variety of publicly available face datasets and performance evaluation ratings are examined. Based on previous recognition outcomes, more study areas are recommended. Face recognition has piqued the interest of academics in biometrics, pattern recognition, and computer vision. Previous research in this field had a basic fault in that it was designed to recognize just single faces, resulting in a long detection time and additional time for annotation. The goal of this study is to create a face recognition system that makes use of a machine learning algorithm and principal component analysis (PCA). The study makes use of linear discriminant analysis, multilayer perceptrons, Naive Bayes, and support vector machines. Furthermore, it obtained 97% and 100% identification accuracy, respectively, using PCA and linear discriminant analysis. The extensive comprehension CNNs are the most frequent form of machine learning algorithm used for facial recognition. CNNs are a sort of artificial neural network that performs exceptionally well at image categorization. Face recognition is a popular topic in artificial intelligence. This program was widely used in our daily life. Several face recognition devices were installed on phones to protect private information and utilized on Facebook to quickly recognize when Facebook users appeared in photographs. Several face recognition algorithms have previously been developed; yet, it remains incredibly difficult in real-world situations. A basic technique for differentiating people is influenced by factors such as partial facial occlusion, illumination, and posture variance.

Recognition of human emotion transition from video sequence using triangulation induced various centre pairs distance signatures

Human emotion recognition manifests the concept of contextual understanding of human communication in nonverbal form. This fascinates us to monitor the dynamic information of an emotion. In this article, we present an automatic recognition system for observing gradual changes in human emotion available in the form of a video sequence. In our proposed system, we introduce a triangulation mechanism for geometric feature generation from salient landmarks on the face image. Here, well known active appearance model (AAM) is applied to consecutive image frames in a sequence to track all locations of landmark points. In this work, we explore four centre points (Incentre, Circumcentre, Barycentre, and Orthocentre) from all the triangles formed by salient facial points selected in a triplet. Six different geometric distance signatures viz. Incentre–Circumcentre, Incentre–Barycentre, Incentre–Orthocentre, Circumcentre–Barycentre, Circumcentre–Orthocentre, and Barycentre–Orthocentre are extracted accordingly by taking all possible pairs of those four centres into account to signify several ways of the geometric representation of an emotional image sequence. The discriminative merit of each distance signature is analysed independently by employing them separately as input feature sets into a multilayer perceptron (MLP) classifier for recognition of six basic emotions(anger, disgust, fear, happiness, sadness, and surprise). We verify the performance of our proposed system through the experimentation on several publically available benchmark video datasets such as Extended Cohn–Kanade (CK＋), M&M Initiative (MMI), and Multimedia Understanding Group (MUG). Experimental results indicate impressive accuracy of 98.47% for CK＋, 92.57% for MMI, and 98.62% for the MUG dataset on Incentre–Circumcentre pair distance signature. Moreover, to justify the effectiveness of our proposed method we compare our experimental results with other existing states of the art available in the literature.

A combined active shape and mean appearance model for the reconstruction of segmental bone loss.

This study investigates the novel combination of an active shape and mean appearance model to estimate missing bone geometry and density distribution from sparse inputs simulating segmental bone loss of the femoral diaphysis. An active shape Gaussian Process Morphable model was trained on healthy right femurs of South African males to model shape. The density distribution was approximated based on the mean appearance of computed tomography images from the training set. Estimations of diaphyseal resections were obtained by probabilistic fitting of the active shape model to sparse inputs consisting of proximal and distal femoral data on computed tomography images. The resulting shape estimates of the diaphyseal resections were then used to map the mean appearance model to the patients' missing bone geometry, constructing density estimations. In this way, resected bone surfaces were estimated with an average error of 2.24 (0.5)mm. Density distributions were approximated within 87 (0.7)% of the intensity of the original target images before the simulated segmental bone loss. These results fall within the acceptable tolerances required for surgical planning and reconstruction of long bone defects.

Facial Skin Texture and Distributed Dynamic Kernel Support Vector Machine (DDKSVM) Classifier for Age Estimation in Facial Wrinkles

Facial wrinkles are common aspects of ageing human skin that may be used in a variety of image-based ageing applications. Wrinkles on the face are 3D skin characteristics that appear as tiny discontinuities or fissures, as well as inconsistencies in the surrounding skin texture. Existing image-based techniques to ageing skin analysis focus on wrinkles as texture rather than curvilinear discontinuity/crack or irregularity characteristics. Picture processing methods are useful for reconstructing and manipulating an image in order to produce various images. The goal of this study is to create a FWDAE-IP (Face Wrinkles Detection and Age Estimate with Image Processing) system that can identify the exact position of wrinkle lines in relation to cracks/ discontinuities, abnormalities, and age estimation using skin texture data from one face image. This FWDAE-IP system is performed based on four major steps: Preprocessing, Skin texture feature extraction, wrinkles detection, and age estimations. In the first step, preprocessing is performed by rotation angle. In the second step, GFs (Gabor Filters), ALTPs (Adaptive Local Ternary Patterns), and AAMs (Active Appearance Models) are introduced against variation in pose or illumination. In the third step, GMM-EM (Gaussian Mixture Model- Expectation-Maximization) is introduced for detecting irregularities in the wrinkles, Gabor filters and image morphology is introduced for detecting discontinuity/crack in the wrinkles. In the fourth step, age estimation is performed based on the detected with wrinkles by DDKSVM (Distributed Dynamic Kernel Support Vector Machine) classifier. The results of age estimation methods are measured via the evaluation metrics such as MAEs (Mean Absolute Error), CSs (Cumulative Scores), accuracy, SSIMs (structural Similarity Indices), and JSIs (Jcard Similarity Indices). The experimentation work is performed based on FG-NET database.

Human face aging based on active appearance model using proper feature set

In this paper, a simulation of the human face evolution from youth to old age is presented. In this work, the goal is to maintain a person's appearance and accurately estimate the face at the target age; in such a way that the simulated face should be realistic and have the appearance and texture characteristics of the target age. To achieve this goal, the number of feature points and their arrangement on the face are very important. First, we suggest proper feature points. Then, using the active appearance model (AAM) method and the presented model, templates are obtained as representative of different age groups. To maintain the unique geometric characteristics of the input face and apply these characteristics to the target age template, the proposed feature point pattern and the MLS method have been used. Then, using the active appearance model method and having the target age template contain the geometric characteristics of the input face, and the steps of changing the input face age to reach the target age are presented. Finally, the results of the survey, using two methods of age recognition and real image recognition (separately for man and woman images), show an average of 80.77% (male images) and 81.36% (female images) correct answers of the participants in this Poll.

An activate appearance model‐based algorithm for ear characteristic points positioning

SummaryIn the field of ear acupoint therapy in traditional Chinese medicine, many diseases can be treated through ear acupoint acupuncture, buried needle, bleeding, ear acupoint compression, massage, electric stimulation, and other programs. To achieve this goal, the positioning of the earhole feature point region is required. Based on the active appearance model, this article proposes an automatic identification feature point scheme through 168 datasets of data (including data collected based on standard equipment and ordinary mobile phones) collected from Hangzhou Normal University, China. The experimental results show that the average distance error of 91 feature points is less than 6 pixels, which can replace the identification work of TCM ear acupoint doctors.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component-based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a "face space" based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

A PCA-Based Active Appearance Model for Characterising Modes of Spatiotemporal Variation in Dynamic Facial Behaviours.

Faces carry key personal information about individuals, including cues to their identity, social traits, and emotional state. Much research to date has employed static images of faces taken under tightly controlled conditions yet faces in the real world are dynamic and experienced under ambient conditions. A common approach to studying key dimensions of facial variation is the use of facial caricatures. However, such techniques have again typically relied on static images, and the few examples of dynamic caricatures have relied on animating graphical head models. Here, we present a principal component analysis (PCA)-based active appearance model for capturing patterns of spatiotemporal variation in videos of natural dynamic facial behaviours. We demonstrate how this technique can be applied to generate dynamic anti-caricatures of biological motion patterns in facial behaviours. This technique could be extended to caricaturing other facial dimensions, or to more general analyses of spatiotemporal variations in dynamic faces.

Manifold-enhanced CycleGAN for facial expression synthesis

ABSTRACT Facial expressions contain a great deal of information in inter-personal communication and social interaction and have a great impact on face recognition. This paper investigates expression synthesis methods based on deep learning and their performance in expression classification and face recognition. We propose a new synthesis method based on the active appearance model and CycleGAN. The eigendecomposition method is used to obtain the expression shape of a target subject through linear operations. An expression intensity coefficient is developed to control the dynamics of generated expressions. Subsequently, a generative model is introduced for the synthesis on textures. The identity loss is integrated into the traditional CycleGAN model, incorporated with the adversarial loss and the cycle consistency loss, to better preserve texture attributes and identity information of target subjects. Experiments and comparisons show that the proposed method offers marked improvements in verification of expressions and identities, as well as in generalization across databases.

COMPARISON OF 3D QUANTITATIVE OSTEOARTHRITIS IMAGING BIOMARKERS: A STUDY USING PAIRED CT AND MR IMAGES FROM THE IMI-APPROACH STUDY

MRI bone surface area and femoral bone shape (B-score) measures have been employed as quantitative endpoints in DMOAD clinical trials. The B-score (typical range -2 to +7) has been shown to be an objective, automated assessment of OA status, with clinical risk defined for current and future pain, functional limitation and total knee replacement. Computerized Tomography (CT) imaging is more commonly used for 3D visualization of bony anatomy due to its high bone-soft tissue contrast. Also, CT images are not subject to the geometric distortion in MR images due to field inhomogeneities, a problem addressed in DMOAD clinical trials by positioning the knee near the magnet isocenter. Measurement of 3D joint space width (3DJSW) has been previously reported using CT images but is possible from MR images. To test the robustness of the B-score, bone area and 3DJSW imaging endpoints to the choice of imaging modality, we compared the results of automated analysis of CT and MR image pairs acquired at baseline and 24-month timepoints. We used image data from the IMI-APPROACH exploratory, 2-year prospective cohort study. High resolution CT images were acquired at baseline and 24-months. These were matched with 3D T1w MR images with fat-sat or WE sequences. CT and MR images were acquired within a mean of 12 days of each other. Additional test-retest MR images with repositioning were available from 37 subjects. Kellgren-Lawrence Grades (KLG) were read centrally from screening weight-bearing x-rays. Femur and tibia bones were automatically segmented with sub-voxel accuracy from CT or MR images using active appearance models, a machine learning model trained to search images (Imorphics, Manchester, UK). Bone area measurement was produced for MF.tAB, LF.tAB, MT.tAB and LT.tAB regions. Average 3DJSW was measured in central regions of the medial or lateral tibial plateaus, normal to the tibial surface. Linear regression was used to test for correlation between measures. Limits of agreement and systematic bias were tested using Bland-Altman analysis. Smallest detectable difference (SDD) for MR B-score was computed from test-retest image pairs using Bland-Altman analysis. Images acquired from the same subject at both baseline and 24-month timepoints were treated as independent. Baseline and 24-month CT-MR pairs of the same knee were available from 231 and 203 subjects respectively. 338 were female knees (78%). KLG was KLG0 (19%); KLG1 (31%); KLG2 (30%); KLG3 (16%); KLG4 (3%) with 5 knees ungraded (1%). SDD of the B-score measure was 0.227. MRI B-score measures were highly correlated with CT (R2 = 0.97) and showed good agreement with a relatively small bias of 0.1. All 4 area measures were highly correlated (R2 = 0.96-0.97) and showed negative bias (MR smaller). Both 3DJSW measures showed a positive bias for MR (medial bias = 1.04mm, R2 = 0.58; lateral bias = 0.42mm, R2 = 0.47). The results suggest that B-score may be measured reliably using either modality. However, it is likely that the bone surface identified using MR and CT will be at slightly different positions within the bone/cartilage boundary. A consistent negative bias suggests the MR bone boundary is inside the CT boundary. This is consistent with the positive bias in favor of MR for 3DJSW. However, the MR-CT difference for medial 3DJSW is larger than for lateral 3DJSW, possibly due to a difference in knee pose during acquisition (fully extended for CT, flexed for MR) – the tibia is more externally rotated with respect to the femur in the fully extended knee. AB and MB are employees of Imorphics. This work has received support from the EU/EFPIA Innovative Medicines Initiative Joint Undertaking (APPROACH grant n° 115770). CORRESPONDENCE ADDRESS: alan.brett@stryker.com

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons

In order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream. To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age. Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells. Together our results imply that optimising the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain.

Real Time Pain Detection Using Facial Action Units in Telehealth System

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; During the Covid-19 pandemic, to reduce staff exposure to ill people, minimize the impact of patient surges on facilities, and preserve personal protective equipment, the recommendations are made by the World Health Organization to change the way that health care is delivered. Several telehealth systems are utilized including live audio-video interaction or real-time telephone typically with a patient using a computer, smartphone, or tablet. During these appointments, the doctors need to know the pain levels of the patient to be able to prescribe the correct medicine and diagnose the disease proficiently. In this paper, a real-time 4- pain levels recognition based on facial expression during telehealth is proposed. Generally, the pain is measured via verbal communication, normally the patient’s self-report. However, if the patient has a disability and unable to communicate with others due to being impaired mentally or having breathing problems or the child self-reporting may not be a perfect way to measure the pain. The proposed system consists of two methods to detect pain from a patient’s facial expressions. The AAM_Based method detects the face and facial landmarks from each video frame using Active Appearance Model AAM, these landmarks are used to compute the facial features. The AU_Based method uses Facial Action Units AU which objectively describes facial muscle activations that are considered as Region of Interest. Support Vector Machine classifier is utilized to detect the levels of pain. A labeled dataset such as Biovid is used to train test, and the AAM_based method, while and UNBC is used for the second method. The findings show that it is possible to depend on facial expression to detect pain level 1 and level 4 very accurately, while it is very tricky to detect pain level 2, and 3 because the AUs for them are similar for most of the patients.