Pseudo-likelihood Method Research Articles

The Effect of Corruption on International Trade: A Case Study of Indonesian Trade to Nine Countries

Corruption is a form of abuse of ethical authority by public officials, which is divided into two parts: bribery and forced collection. The effect of corruption like bribes and illegal levies is widespread in the public sector. One interesting investigation is the effect of corruption on international trade. Corruption becomes a barrier in international trade, where corruption plays a role in the access of trade goods and services from within and abroad. Using the gravity model, the focus of this research was the effect of corruption on international trade by taking a case study of Indonesia’s bilateral trade with its nine largest export destination countries. Using panel data, analysis tools used in this research were common effect, fixed effect, random effect and poisson pseudo maximum likelihood (PPML). In this research, it was found that geographical distance variable in its fixed units caused the omitted variable so that the error term correlated with independent variables. In order to overcome the problem, poisson pseudo maximum likelihood method was used in performing regression gravity model with linear log form, so the omitted variable issue on the geographical distance can be eliminated. The results of this research concluded that corruption played a role in international trade through bureaucratic mechanisms of trade and investment licensing and the effect of corruption was more detrimental to exporters.Keywords: Gravity Model, Corruption, International Trade, Poisson Pseudo Maximum Likelihood (PPML).

Improved GGIW-PHD filter for maneuvering non-ellipsoidal extended targets or group targets tracking based on sub-random matrices.

For non-ellipsoidal extended targets and group targets tracking (NETT and NGTT), using an ellipsoid to approximate the target extension may not be accurate enough because of the lack of shape and orientation information. In consideration of this, we model a non-ellipsoidal extended target or target group as a combination of multiple ellipsoidal sub-objects, each represented by a random matrix. Based on these models, an improved gamma Gaussian inverse Wishart probability hypothesis density (GGIW-PHD) filter is proposed to estimate the measurement rates, kinematic states, and extension states of the sub-objects for each extended target or target group. For maneuvering NETT and NGTT, a multi-model (MM) approach based GGIW-PHD (MM-GGIW-PHD) filter is proposed. The common and the individual dynamics of the sub-objects belonging to the same extended target or target group are described by means of the combination between the overall maneuver model and the sub-object models. For the merging of updating components, an improved merging criterion and a new merging method are derived. A specific implementation of prediction partition with pseudo-likelihood method is presented. Two scenarios for non-maneuvering and maneuvering NETT and NGTT are simulated. The results demonstrate the effectiveness of the proposed algorithms.

SIS: An R Package for Sure Independence Screening in Ultrahigh-Dimensional Statistical Models

We revisit sure independence screening procedures for variable selection in generalized linear models and the Cox proportional hazards model. Through the publicly available R package SIS, we provide a unified environment to carry out variable selection using iterative sure independence screening (ISIS) and all of its variants. For the regularization steps in the ISIS recruiting process, available penalties include the LASSO, SCAD, and MCP while the implemented variants for the screening steps are sample splitting, data-driven thresholding, and combinations thereof. Performance of these feature selection techniques is investigated by means of real and simulated data sets, where we find considerable improvements in terms of model selection and computational time between our algorithms and traditional penalized pseudo-likelihood methods applied directly to the full set of covariates.

Marginal Pseudo-Likelihood Learning of Discrete Markov Network Structures

Markov networks are a popular tool for modeling multivariate distributions over a set of discrete variables. The core of the Markov network representation is an undirected graph which elegantly captures the dependence structure over the variables. Traditionally, the Bayesian approach of learning the graph structure from data has been done under the assumption of chordality since non-chordal graphs are difficult to evaluate for likelihood-based scores. Recently, there has been a surge of interest towards the use of regularized pseudo-likelihood methods as such approaches can avoid the assumption of chordality. Many of the currently available methods necessitate the use of a tuning parameter to adapt the level of regularization for a particular dataset. Here we introduce the marginal pseudo-likelihood which has a built-in regularization through marginalization over the graph-specific nuisance parameters. We prove consistency of the resulting graph estimator via comparison with the pseudo-Bayesian information criterion. To identify high-scoring graph structures in a high-dimensional setting we design a two-step algorithm that exploits the decomposable structure of the score. Using synthetic and existing benchmark networks, the marginal pseudo-likelihood method is shown to perform favorably against recent popular structure learning methods.

Statistical mechanics of the inverse Ising problem and the optimal objective function

The inverse Ising problem seeks to reconstruct the parameters of an Ising Hamiltonian on the basis of spin configurations sampled from the Boltzmann measure. Over the last decade, many applications of the inverse Ising problem have arisen, driven by the advent of large-scale data across different scientific disciplines. Recently, strategies to solve the inverse Ising problem based on convex optimisation have proven to be very successful. These approaches maximise particular objective functions with respect to the model parameters. Examples are the pseudolikelihood method and interaction screening. In this paper, we establish a link between approaches to the inverse Ising problem based on convex optimisation and the statistical physics of disordered systems. We characterise the performance of an arbitrary objective function and calculate the objective function which optimally reconstructs the model parameters. We evaluate the optimal objective function within a replica-symmetric ansatz and compare the results of the optimal objective function with other reconstruction methods. Apart from giving a theoretical underpinning to solving the inverse Ising problem by convex optimisation, the optimal objective function outperforms state-of-the-art methods, albeit by a small margin.

A Novel Approach for Markov Random Field With Intractable Normalizing Constant on Large Lattices

ABSTRACTThe pseudo likelihood method of Besag (1974) has remained a popular method for estimating Markov random field on a very large lattice, despite various documented deficiencies. This is partly because it remains the only computationally tractable method for large lattices. We introduce a novel method to estimate Markov random fields defined on a regular lattice. The method takes advantage of conditional independence structures and recursively decomposes a large lattice into smaller sublattices. An approximation is made at each decomposition. Doing so completely avoids the need to compute the troublesome normalizing constant. The computational complexity is O(N), where N is the number of pixels in the lattice, making it computationally attractive for very large lattices. We show through simulations, that the proposed method performs well, even when compared with methods using exact likelihoods. Supplementary material for this article is available online.

Fast, closed-form, and efficient estimators for hierarchical models with AR(1) covariance and unequal cluster sizes

ABSTRACTThis article is concerned with statistically and computationally efficient estimation in a hierarchical data setting with unequal cluster sizes and an AR(1) covariance structure. Maximum likelihood estimation for AR(1) requires numerical iteration when cluster sizes are unequal. A near optimal non-iterative procedure is proposed. Pseudo-likelihood and split-sample methods are used, resulting in computing weights to combine cluster size specific parameter estimates. Results show that the method is statistically nearly as efficient as maximum likelihood, but shows great savings in computation time.

A statistical physics approach to learning curves for the inverse Ising problem

Using methods of statistical physics, we analyse the error of learning couplings in large Ising models from independent data (the inverse Ising problem). We concentrate on learning based on local cost functions, such as the pseudo-likelihood method for which the couplings are inferred independently for each spin. Assuming that the data are generated from a true Ising model, we compute the reconstruction error of the couplings using a combination of the replica method with the cavity approach for densely connected systems. We show that an explicit estimator based on a quadratic cost function achieves minimal reconstruction error, but requires the length of the true coupling vector as prior knowledge. A simple mean field estimator of the couplings which does not need such knowledge is asymptotically optimal, i.e. when the number of observations is much larger than the number of spins. Comparison of the theory with numerical simulations shows excellent agreement for data generated from two models with random couplings in the high temperature region: a model with independent couplings (Sherrington–Kirkpatrick model), and a model where the matrix of couplings has a Wishart distribution.

Weighted pseudolikelihood for SNP set analysis with multiple secondary outcomes in case-control genetic association studies.

We propose a weighted pseudolikelihood method for analyzing the association of a SNP set, example, SNPs in a gene or a genetic pathway or network, with multiple secondary phenotypes in case-control genetic association studies. To boost analysis power, we assume that the SNP-specific effects are shared across all secondary phenotypes using a scaled mean model. We estimate regression parameters using Inverse Probability Weighted (IPW) estimating equations obtained from the weighted pseudolikelihood, which accounts for case-control sampling to prevent potential ascertainment bias. To test the effect of a SNP set, we propose a weighted variance component pseudo-score test. We also propose a penalized IPW pseudolikelihood method for selecting a subset of SNPs that are associated with the multiple secondary phenotypes. We show that the proposed variable selection procedure has the oracle properties and is robust to misspecification of the correlation structure among secondary phenotypes. We select the tuning parameter using a weighted Bayesian Information-like Criterion (wBIC). We evaluate the finite sample performance of the proposed methods via simulations, and illustrate the methods by the analysis of the multiple secondary smoking behavior outcomes in a lung cancer case-control genetic association study.

Neighborhood contributions to psychological distress among Latino New York City adults

ABSTRACTObjective: Neighborhood-level factors such as ethnic densities and social cohesion have been negatively associated with psychological distress among Latino Americans. Yet, existing evidence is based on either specific neighborhood factors or particular Latino subgroups. The objective of the study was to assess difference in psychological distress between each of four Latino subgroups (Puerto Ricans, Dominicans, Mexicans, other Latinos) and non-Latino white adults in New York City, and quantify total neighborhood-level influence on these differences.Design: We used the combined Community Health Survey data from 2009, 2010, and 2012 surveys. We estimated the odds ratios (ORs) for self-reported non-specific psychological distress (Kessler-6 questions scale ≥ 13) by race/ethnicity using logistic regression models. We further adjusted these estimates for both observed and unobserved neighborhood-level confounding using the conditional pseudolikelihood method for complex survey data.Results: Puerto Ricans were more likely to be psychologically distressed (OR = 1.58, 95% CI = 1.18, 2.12) compared with non-Latino whites, whereas the opposite was seen in other Latino subgroups. Accounting for full neighborhood-level confounding increased the disparity for Puerto Ricans (OR = 1.79, 95% CI = 1.26–2.54). For the other subgroups, lower odds of psychological distress were no longer observed or attenuated after accounting for neighborhood-level confounding. Overall neighborhood-level factors were associated with lower psychological distress at greater extent among all Latinos subgroups versus non-Latino whites in New York City.Conclusion: Despite substantial variations of psychological distress across Latino subgroups, the study shows evidence that neighborhood-level factors might play a protective role in all Latino subgroups.

Pairwise Likelihood Ratio Tests and Model Selection Criteria for Structural Equation Models with Ordinal Variables.

Correlated multivariate ordinal data can be analysed with structural equation models. Parameter estimation has been tackled in the literature using limited-information methods including three-stage least squares and pseudo-likelihood estimation methods such as pairwise maximum likelihood estimation. In this paper, two likelihood ratio test statistics and their asymptotic distributions are derived for testing overall goodness-of-fit and nested models, respectively, under the estimation framework of pairwise maximum likelihood estimation. Simulation results show a satisfactory performance of type I error and power for the proposed test statistics and also suggest that the performance of the proposed test statistics is similar to that of the test statistics derived under the three-stage diagonally weighted and unweighted least squares. Furthermore, the corresponding, under the pairwise framework, model selection criteria, AIC and BIC, show satisfactory results in selecting the right model in our simulation examples. The derivation of the likelihood ratio test statistics and model selection criteria under the pairwise framework together with pairwise estimation provide a flexible framework for fitting and testing structural equation models for ordinal as well as for other types of data. The test statistics derived and the model selection criteria are used on data on 'trust in the police' selected from the 2010 European Social Survey. The proposed test statistics and the model selection criteria have been implemented in the R package lavaan.

ACE: adaptive cluster expansion for maximum entropy graphical model inference.

Graphical models are often employed to interpret patterns of correlations observed in data through a network of interactions between the variables. Recently, Ising/Potts models, also known as Markov random fields, have been productively applied to diverse problems in biology, including the prediction of structural contacts from protein sequence data and the description of neural activity patterns. However, inference of such models is a challenging computational problem that cannot be solved exactly. Here, we describe the adaptive cluster expansion (ACE) method to quickly and accurately infer Ising or Potts models based on correlation data. ACE avoids overfitting by constructing a sparse network of interactions sufficient to reproduce the observed correlation data within the statistical error expected due to finite sampling. When convergence of the ACE algorithm is slow, we combine it with a Boltzmann Machine Learning algorithm (BML). We illustrate this method on a variety of biological and artificial datasets and compare it to state-of-the-art approximate methods such as Gaussian and pseudo-likelihood inference. We show that ACE accurately reproduces the true parameters of the underlying model when they are known, and yields accurate statistical descriptions of both biological and artificial data. Models inferred by ACE more accurately describe the statistics of the data, including both the constrained low-order correlations and unconstrained higher-order correlations, compared to those obtained by faster Gaussian and pseudo-likelihood methods. These alternative approaches can recover the structure of the interaction network but typically not the correct strength of interactions, resulting in less accurate generative models. The ACE source code, user manual and tutorials with the example data and filtered correlations described herein are freely available on GitHub at https://github.com/johnbarton/ACE CONTACTS: jpbarton@mit.edu, cocco@lps.ens.frSupplementary information: Supplementary data are available at Bioinformatics online.

Invariant tests based on M-estimators, estimating functions, and the generalized method of moments

ABSTRACTWe study the invariance properties of various test criteria which have been proposed for hypothesis testing in the context of incompletely specified models, such as models which are formulated in terms of estimating functions (Godambe, 1960) or moment conditions and are estimated by generalized method of moments (GMM) procedures (Hansen, 1982), and models estimated by pseudo-likelihood (Gouriéroux, Monfort, and Trognon, 1984b,c) and M-estimation methods. The invariance properties considered include invariance to (possibly nonlinear) hypothesis reformulations and reparameterizations. The test statistics examined include Wald-type, LR-type, LM-type, score-type, and C(α)−type criteria. Extending the approach used in Dagenais and Dufour (1991), we show first that all these test statistics except the Wald-type ones are invariant to equivalent hypothesis reformulations (under usual regularity conditions), but all five of them are not generally invariant to model reparameterizations, including measurement unit changes in nonlinear models. In other words, testing two equivalent hypotheses in the context of equivalent models may lead to completely different inferences. For example, this may occur after an apparently innocuous rescaling of some model variables. Then, in view of avoiding such undesirable properties, we study restrictions that can be imposed on the objective functions used for pseudo-likelihood (or M-estimation) as well as the structure of the test criteria used with estimating functions and generalized method of moments (GMM) procedures to obtain invariant tests. In particular, we show that using linear exponential pseudo-likelihood functions allows one to obtain invariant score-type and C(α)−type test criteria, while in the context of estimating function (or GMM) procedures it is possible to modify a LR-type statistic proposed by Newey and West (1987) to obtain a test statistic that is invariant to general reparameterizations. The invariance associated with linear exponential pseudo-likelihood functions is interpreted as a strong argument for using such pseudo-likelihood functions in empirical work.

Local Autoencoding for Parameter Estimation in a Hidden Potts-Markov Random Field.

A local-autoencoding (LAE) method is proposed for the parameter estimation in a Hidden Potts-Markov random field model. Due to sampling cost, Markov chain Monte Carlo methods are rarely used in real-time applications. Like other heuristic methods, LAE is based on a conditional independence assumption. It adapts, however, the parameters in a block-by-block style with a simple Hebbian learning rule. Experiments with given label fields show that the LAE is able to converge in far less time than required for a scan. It is also possible to derive an estimate for LAE based on a Cramer–Rao bound that is similar to the classical maximum pseudolikelihood method. As a general algorithm, LAE can be used to estimate the parameters in anisotropic label fields. Furthermore, LAE is not limited to the classical Potts model and can be applied to other types of Potts models by simple label field transformations and straightforward learning rule extensions. Experimental results on image segmentations demonstrate the efficiency and generality of the LAE algorithm.

A sensibility study of the autobinomial model estimation methods based on a feature similarity index

The estimation of parameters in the Autobinomial model is an important task for characterizing the content of an image and generating synthetic textures. This paper compares the performance of three estimation methods of the model: coding, maximum pseudo-likelihood and conditional least squares, under textures with different levels of additive contamination, using a feature similarity image index, via Monte Carlo studies. This novel framework quantifies the similarity between the original texture and its texture regenerated by each method. Differences in performance were tested with a Repeated Measures ANOVA model design. Simulation results show that the Conditional Least Squares method is associated with the highest value of the similarity image measure in contaminated textures, while Coding and Maximum Pseudo-Likelihood methods have a comparable behavior and there is no clear pattern whether to prefer one over the other. An application for landscape classification using real Landsat images with different spatial resolutions is described.

Optimal B-robust posterior distributions for operational risk

The aim of operational risk modeling is to provide a reasonably accurate, reasonably precise and reasonably robust estimation of capital requirements, including a level of sensitivity that is consistent with the changes of the risk profile. A way to obtain robust capital estimates is through optimal B-robust (OBR) methods. Previous research has shown that OBR methods might mitigate the bias in capital risk quantification when compared with classical maximum likelihood estimation. Motivated by requirements related to operational risk measurement, the aim of this work is to integrate prior information into a robust parameter estimation framework via OBR-estimating functions. Unfortunately, the evaluation of OBR-estimating functions for different parameter values is cumbersome, and this rules out the use of many pseudo-likelihood methods. To deal with this issue, we suggest resorting to approximate Bayesian computation (ABC) machinery, using the OBR-estimating function as the summary statistic. Unlike other methods, the proposed ABC-OBR algorithm requires the evaluation of the OBR-estimating function at a fixed parameter value but using different data samples, which is computationally trivial. The method is illustrated using a small simulation study and applications to two real operational risk data sets.

The infrared database of extragalactic observables from Spitzer – I. The redshift catalogue

This is the first of a series of papers on the Infrared Database of Extragalactic Observables from Spitzer (IDEOS). In this work we describe the identification of optical counterparts of the infrared sources detected in Spitzer Infrared Spectrograph (IRS) observations, and the acquisition and validation of redshifts. The IDEOS sample includes all the spectra from the Cornell Atlas of Spitzer/IRS Sources (CASSIS) of galaxies beyond the Local Group. Optical counterparts were identified from correlation of the extraction coordinates with the NASA Extragalactic Database (NED). To confirm the optical association and validate NED redshifts, we measure redshifts with unprecedented accuracy on the IRS spectra ({\sigma}(dz/(1+z))=0.0011) by using an improved version of the maximum combined pseudo-likelihood method (MCPL). We perform a multi-stage verification of redshifts that considers alternate NED redshifts, the MCPL redshift, and visual inspection of the IRS spectrum. The statistics is as follows: the IDEOS sample contains 3361 galaxies at redshift 0<z<6.42 (mean: 0.48, median: 0.14). We confirm the default NED redshift for 2429 sources and identify 124 with incorrect NED redshifts. We obtain IRS-based redshifts for 568 IDEOS sources without optical spectroscopic redshifts, including 228 with no previous redshift measurements. We provide the entire IDEOS redshift catalog in machine-readable formats. The catalog condenses our compilation and verification effort, and includes our final evaluation on the most likely redshift for each source, its origin, and reliability estimates.

Inference in log-alpha-power and log-skew-normal multivariate models

ABSTRACTRandom vectors with positive components are common in many applied fields, for example, in meteorology, when daily precipitation is measured through a region Marchenko and Genton (2010). Frequently, the log-normal multivariate distribution is used for modeling this type of data. This modeling approach is not appropriate for data with high asymmetry or kurtosis. Consequently, more flexible multivariate distributions than the log-normal multivariate are required. As an alternative to this distribution, we propose the log-alpha-power multivariate and log-skew-normal multivariate models. The first model is an extension for positive data of the fractional order statistics model Durrans (1992). The second one is an extension of the log-skew-normal model studied by Mateu-Figueras and Pawlowsky-Glahn (2007). We study parameter estimation for these models by means of pseudo-likelihood and maximum likelihood methods. We illustrate the proposal analyzing a real dataset.

Inference of the sparse kinetic Ising model using the decimation method.

In this paper we study the inference of the kinetic Ising model on sparse graphs by the decimation method. The decimation method, which was first proposed in Decelle and Ricci-Tersenghi [Phys. Rev. Lett. 112, 070603 (2014)] for the static inverse Ising problem, tries to recover the topology of the inferred system by setting the weakest couplings to zero iteratively. During the decimation process the likelihood function is maximized over the remaining couplings. Unlike the ℓ(1)-optimization-based methods, the decimation method does not use the Laplace distribution as a heuristic choice of prior to select a sparse solution. In our case, the whole process can be done auto-matically without fixing any parameters by hand. We show that in the dynamical inference problem, where the task is to reconstruct the couplings of an Ising model given the data, the decimation process can be applied naturally into a maximum-likelihood optimization algorithm, as opposed to the static case where pseudolikelihood method needs to be adopted. We also use extensive numerical studies to validate the accuracy of our methods in dynamical inference problems. Our results illustrate that, on various topologies and with different distribution of couplings, the decimation method outperforms the widely used ℓ(1)-optimization-based methods.

Pseudo-likelihood Estimation of Multivariate Normal Parameters in the Presence of Left-Censored Data

Environmental data often include left-censored values reported to be less than some limit of detection (LOD). While simple imputation of a specific value such as LOD/2 is common, maximum likelihood methods accounting for censoring provide alternate ways of analyzing such data. Concentration levels of contaminants in water, for example, are typically modeled with normal or lognormal distributions. Corresponding maximum likelihood estimates (MLEs) of means and variances in univariate analyses can be obtained from standard software packages; however, multivariate analyses may be more appropriate when multiple measurements come from the same entity. For example, measures of several dissolved trace metals may be derived from freshwater stream samples. In less-polluted areas, one or more of these measures fall below the LOD. An index of overall contamination may be formed as a linear function of these measures. The desire to estimate this index led to the need to estimate the parameters in the presence of nondetects, which led to our proposed method. We propose a pseudo-likelihood method utilizing pairs of variables that provides MLEs of mean and unstructured covariance parameters corresponding to a multivariate normal or lognormal distribution in the presence of left-censored data. In conducting hypothesis tests and estimating functions of MLEs with standard errors, we apply this method to trace metal concentration data collected from freshwater streams across Virginia.