AU Intensity Research Articles

Trend-Aware Supervision: On Learning Invariance for Semi-supervised Facial Action Unit Intensity Estimation

With the increasing need for facial behavior analysis, semi-supervised AU intensity estimation using only keyframe annotations has emerged as a practical and effective solution to relieve the burden of annotation. However, the lack of annotations makes the spurious correlation problem caused by AU co-occurrences and subject variation much more prominent, leading to non-robust intensity estimation that is entangled among AUs and biased among subjects. We observe that trend information inherent in keyframe annotations could act as extra supervision and raising the awareness of AU-specific facial appearance changing trends during training is the key to learning invariant AU-specific features. To this end, we propose Trend-AwareSupervision (TAS), which pursues three kinds of trend awareness, including intra-trend ranking awareness, intra-trend speed awareness, and inter-trend subject awareness. TAS alleviates the spurious correlation problem by raising trend awareness during training to learn AU-specific features that represent the corresponding facial appearance changes, to achieve intensity estimation invariance. Experiments conducted on two commonly used AU benchmark datasets, BP4D and DISFA, show the effectiveness of each kind of awareness. And under trend-aware supervision, the performance can be improved without extra computational or storage costs during inference.

Facial Expression Manipulation for Personalized Facial Action Estimation

Limited sizes of annotated video databases of spontaneous facial expression, imbalanced action unit labels, and domain shift are three main obstacles in training models to detect facial actions and estimate their intensity. To address these problems, we propose an approach that incorporates facial expression generation for facial action unit intensity estimation. Our approach reconstructs the 3D shape of the face from each video frame, aligns the 3D mesh to a canonical view, and trains a GAN-based network to synthesize novel images with facial action units of interest. We leverage the synthetic images to achieve two goals: 1) generating AU-balanced databases, and 2) tackling domain shift with personalized networks. To generate a balanced database, we synthesize expressions with varying AU intensities and perform semantic resampling. Our experimental results on FERA17 show that networks trained on synthesized facial expressions outperform those trained on actual facial expressions and surpass current state-of-the-art approaches. To tackle domain shift, we propose personalizing pretrained networks. We generate synthetic expressions of each target subject with varying AU intensity labels and use the person-specific synthetic images to fine-tune pretrained networks. To evaluate performance of the personalized networks, we use DISFA and PAIN databases. Personalized networks, which require only a single image from each target subject to generate synthetic images, achieved significant improvement in generalizing to unseen domains.

Standard-free single magnetic bead evaluation: a stable nanoplatform for prostate disease differentiation.

Explicit interpretation of heterogeneity between prostate-specific antigen (PSA) subtypes is essential for prostate cancer differentiation during different disease courses, whereas a universal protocol with uniform criteria is still lacking across the globe. In this work, a standard-free single magnetic bead (SMB) nanoplatform utilizing metal nanoparticles with optimal diameters was proposed for prostate disease differentiation in a 134-donor model. The inaccuracy of detection in absolute quantification was diminished via evaluations of metal intensities on the single magnetic bead. The intrinsic proportion of fPSA in tPSA was successfully evaluated by direct use of the Pt to Au intensity ratio (Pt/Au ratio), exhibiting better differentiation between healthy and unhealthy, benign prostatic hyperplasia (BPH) and cancer individuals compared with solo fPSA or tPSA. We generated thresholds respectively for prostate disease differentiation, envisioning that this standard-free SMB nanoplatform would establish a standardized methodology with uniform criteria worldwide in cancer diagnosis, staging, and postoperative assessments.

3D propagation of relativistic solar protons through interplanetary space

Context.Solar energetic particles (SEPs) with energy in the GeV range can propagate to Earth from their acceleration region near the Sun and produce ground level enhancements (GLEs). The traditional approach to interpreting and modelling GLE observations assumes particle propagation which is only parallel to the magnetic field lines of interplanetary space, that is, spatially 1D propagation. Recent measurements by PAMELA have characterised SEP properties at 1 AU for the ∼100 MeV–1 GeV range at high spectral resolution.Aims.We model the transport of GLE-energy solar protons using a 3D approach to assess the effect of the heliospheric current sheet (HCS) and drifts associated to the gradient and curvature of the Parker spiral. We derive 1 AU observables and compare the simulation results with data from PAMELA.Methods.We use a 3D test particle model including a HCS. Monoenergetic populations are studied first to obtain a qualitative picture of propagation patterns and numbers of crossings of the 1 AU sphere. Simulations for power law injection are used to derive intensity profiles and fluence spectra at 1 AU. A simulation for a specific event, GLE 71, is used for comparison purposes with PAMELA data.Results.Spatial patterns of 1 AU crossings and the average number of crossings per particle are strongly influenced by 3D effects, with significant differences between periods ofA+andA−polarities. The decay time constant of 1 AU intensity profiles varies depending on the position of the observer and it is not a simple function of the mean free path as in 1D models. Energy dependent leakage from the injection flux tube is particularly important for GLE energy particles, resulting in a rollover in the spectrum.

Laser-Induced Breakdown Spectroscopy and Principal Component Analysis for the Classification of Spectra from Gold-Bearing Ores.

Laser-induced breakdown spectroscopy (LIBS) and principal component analysis (PCA) were applied to the classification of LIBS spectra from gold ores prepared as pressed pellets from pulverized bulk samples. For each sample, 5000 single-shot LIBS spectra were obtained. Although the gold concentrations in the samples were as high as 7.7 µg/g, Au emission lines were not observed in most single-shot LIBS spectra, rendering the application of the usual ensemble-averaging approach for spectral processing to be infeasible. Instead, a PCA approach was utilized to analyze the collection of single-shot LIBS spectra. Two spectral ranges of 21 nm and 0.15 nm wide were considered, and LIBS variables (i.e., wavelengths) reduced to no more than three principal components. Single-shot spectra containing Au emission lines (positive spectra) were discriminated by PCA from those without the spectral feature (negative spectra) in a spectral range of less than 1 nm wide around the Au(I) 267.59 nm emission line. Assuming a discrete gold distribution at very low concentration, LIBS sampling of gold particles seemed unlikely; therefore, positive spectra were considered as data outliers. Detection of data outliers was possible using two PCA statistical parameters, i.e., sample residual and Mahalanobis distance. Results from such a classification were compared with a standard database created with positive spectra identified with a filtering algorithm that rejected spectra with an Au intensity below the smallest detectable analytical LIBS signal (i.e., below the LIBS limit of detection). The PCA approach successfully identified 100% of the data outliers when compared with the standard database. False identifications in the multivariate approach were attributed to variations in shot-to-shot intensity and the presence of interfering emission lines.

FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challenge.

The field of Automatic Facial Expression Analysis has grown rapidly in recent years. However, despite progress in new approaches as well as benchmarking efforts, most evaluations still focus on either posed expressions, near-frontal recordings, or both. This makes it hard to tell how existing expression recognition approaches perform under conditions where faces appear in a wide range of poses (or camera views), displaying ecologically valid expressions. The main obstacle for assessing this is the availability of suitable data, and the challenge proposed here addresses this limitation. The FG 2017 Facial Expression Recognition and Analysis challenge (FERA 2017) extends FERA 2015 to the estimation of Action Units occurrence and intensity under different camera views. In this paper we present the third challenge in automatic recognition of facial expressions, to be held in conjunction with the 12th IEEE conference on Face and Gesture Recognition, May 2017, in Washington, United States. Two sub-challenges are defined: the detection of AU occurrence, and the estimation of AU intensity. In this work we outline the evaluation protocol, the data used, and the results of a baseline method for both sub-challenges.

Regression-based intensity estimation of facial action units

Facial Action Coding System (FACS) is the de facto standard in the analysis of facial expressions. FACS describes expressions in terms of the configuration and strength of atomic units called Action Units: AUs. FACS defines 44 AUs and each AU intensity is defined on a nonlinear scale of five grades. There has been significant progress in the literature on the detection of AUs. However, the companion problem of estimating the AU strengths has not been much investigated. In this work we propose a novel AU intensity estimation scheme applied to 2D luminance and/or 3D surface geometry images. Our scheme is based on regression of selected image features. These features are either non-specific, that is, those inherited from the AU detection algorithm, or are specific in that they are selected for the sole purpose of intensity estimation. For thoroughness, various types of local 3D shape indicators have been considered, such as mean curvature, Gaussian curvature, shape index and curvedness, as well as their fusion. The feature selection from the initial plethora of Gabor moments is instrumented via a regression that optimizes the AU intensity predictions. Our AU intensity estimator is person-independent and when tested on 25 AUs that appear singly or in various combinations, it performs significantly better than the state-of-the-art method which is based on the margins of SVMs designed for AU detection. When evaluated comparatively, one can see that the 2D and 3D modalities have relative merits per upper face and lower face AUs, respectively, and that there is an overall improvement if 2D and 3D intensity estimations are used in fusion.

Power spectrum analysis of geomagnetic indices

Using the method of maximum entropy spectral analysis (MESA) by Burg and the least-squares linear prediction (lslp) (also calledFABNE) by Barrodale and Erickson, the spectra of geomagnetic indices Ap, AN, AS, AE, AU, AL, Dst and cosmic ray neutron intensity at Deep River are obtained for 1965 and 1969 from daily means and for longer periods from monthly means. A large number of peaks of periodicities from 2 days to several years is obtained, many of which are shown by all the indices. Some of these are probably harmonics of the 27 days solar rotation period and the 20–22 years double sunspot cycle period. Comparison is made with results of earlier workers who reported fewer peaks.