Divergence Cost Function Research Articles

A Second-Order Method for Fitting the Canonical Polyadic Decomposition With Non-Least-Squares Cost

The canonical polyadic decomposition (CPD) can be used to extract meaningful components from a tensor. Most existing optimization methods for fitting the CPD use as cost function the least-squares distance between the tensor and its CPD. While the minimum of this cost function coincides with the maximum likelihood estimator for data with additive i.i.d. Gaussian distributed noise, for other noise distributions, better-suited cost functions exist. For such cost functions, first-order, gradient-based optimization methods have been proposed. However, (approximate) second-order methods, which additionally use information from the Hessian of the cost function to achieve faster convergence, are still largely unexplored. In this paper, we generalize the Gauss–Newton nonlinear least-squares algorithm to twice differentiable entry-wise cost functions. The low-rank structure of the problem is exploited to keep the computational cost low. As a special case, $\beta$ -divergence cost functions are examined. We show that quadratic convergence can be obtained close to the solution with a reasonable extra cost in memory and computation time, making the proposed method particularly useful when high accuracy of the decomposition is desired.

Informed Weighted Non-Negative Matrix Factorization Using αβ-Divergence Applied to Source Apportionment.

In this paper, we propose informed weighted non-negative matrix factorization (NMF) methods using an -divergence cost function. The available information comes from the exact knowledge/boundedness of some components of the factorization—which are used to structure the NMF parameterization—together with the row sum-to-one property of one matrix factor. In this contribution, we extend our previous work which partly involved some of these aspects to -divergence cost functions. We derive new update rules which are extendthe previous ones and take into account the available information. Experiments conducted for several operating conditions on realistic simulated mixtures of particulate matter sources show the relevance of these approaches. Results from a real dataset campaign are also presented and validated with expert knowledge.

Retrieval of Maize Leaf Area Index Using Hyperspectral and Multispectral Data

Field spectra acquired from a handheld spectroradiometer and Sentinel-2 images spectra were used to investigate the applicability of hyperspectral and multispectral data in retrieving the maize leaf area index in low-input crop systems, with high spatial and intra-annual variability, and low yield, in southern Mozambique, during three years. Seventeen vegetation indices, comprising two and three band indices, and nine machine learning regression algorithms (MLRA) were tested for the statistical approach while five cost functions were tested in the look-up-table (LUT) inversion approach. The three band vegetation indices were selected, specifically the modified difference index (mDId: 725; 715; 565) for the hyperspectral dataset and the modified simple ratio (mSRc: 740; 705; 865) for the multispectral dataset of field spectra and the three band spectral index (TBSIb: 665; 865; 783) for the Sentinel-2 dataset. The relevant vector machine was the selected MLRA for the two datasets of field spectra (multispectral and hyperspectral) while the support vector machine was selected for the Sentinel-2 data. When using the LUT inversion technique, the minimum contrast estimation and the Bhattacharyya divergence cost functions were the best performing. The vegetation indices outperformed the other two approaches, with the TBSIb as the most accurate index (RMSE = 0.35). At the field scale, spectral data from Sentinel-2 can accurately retrieve the maize leaf area index in the study area.

Experimental demonstration of using divergence cost-function in SPGD algorithm for coherent beam combining with tip/tilt control

A novel approach of tip/tilt control by using divergence cost function in stochastic parallel gradient descent (SPGD) algorithm for coherent beam combining (CBC) is proposed and demonstrated experimentally in a seven-channel 2-W fiber amplifier array with both phase-locking and tip/tilt control, for the first time to our best knowledge. Compared with the conventional power-in-the-bucket (PIB) cost function for SPGD optimization, the tip/tilt control using divergence cost function ensures wider correction range, automatic switching control of program, and freedom of camera's intensity-saturation. Homemade piezoelectric-ring phase-modulator (PZT PM) and adaptive fiber-optics collimator (AFOC) are developed to correct piston- and tip/tilt-type aberrations, respectively. The PIB cost function is employed for phase-locking via maximization of SPGD optimization, while the divergence cost function is used for tip/tilt control via minimization. An average of 432-μrad of divergence metrics in open loop has decreased to 89-μrad when tip/tilt control implemented. In CBC, the power in the full width at half maximum (FWHM) of the main lobe increases by 32 times, and the phase residual error is less than λ/15.

Performance evaluation of nonnegative matrix factorization algorithms to estimate task-related neuronal activities from fMRI data

Nonnegative matrix factorization (NMF) is a blind source separation (BSS) algorithm which is based on the distinct constraint of nonnegativity of the estimated parameters as well as on the measured data. In this study, according to the potential feasibility of NMF for fMRI data, the four most popular NMF algorithms, corresponding to the following two types of (1) least-squares based update [i.e., alternating least-squares NMF (ALSNMF) and projected gradient descent NMF] and (2) multiplicative update (i.e., NMF based on Euclidean distance and NMF based on divergence cost function), were investigated by using them to estimate task-related neuronal activities. These algorithms were applied firstly to individual data from a single subject and, subsequently, to group data sets from multiple subjects. On the single-subject level, although all four algorithms detected task-related activation from simulated data, the performance of multiplicative update NMFs was significantly deteriorated when evaluated using visuomotor task fMRI data, for which they failed in estimating any task-related neuronal activities. In group-level analysis on both simulated data and real fMRI data, ALSNMF outperformed the other three algorithms. The presented findings may suggest that ALSNMF appears to be the most promising option among the tested NMF algorithms to extract task-related neuronal activities from fMRI data.