Joint Maximum Likelihood Estimation Research Articles

Inference on regression model with misclassified binary response

Misclassification of binary responses, if ignored, may severely bias the maximum likelihood estimators (MLEs) of regression parameters. For such data, a binary regression model incorporating non-differential classification errors is extensively used by researchers in different application contexts. We strongly caution against indiscriminate use of this model considering the fact that it suffers from a serious estimation problem due to confounding of the unknown misclassification probabilities with the regression parameters, and thus, may lead to a highly biased estimate. To overcome this problem, we propose here the use of an internal validation sample in addition to the main sample. Assuming differential classification errors, we consider MLEs of the regression parameters based on the joint likelihood of the main sample and the internal validation sample. We then develop a rigorous asymptotic theory for the joint MLEs under standard assumptions. To facilitate its easy implementation for inference, we propose a bootstrap approximation to the asymptotic distribution and prove its consistency. The results of the simulation studies suggest that even an extremely small validation sample may lead to a vastly improved inference. Finally, the methodology is illustrated with a real-life survey data.

Forecasting VIX using two-component realized EGARCH model

In this paper, we propose the two-component realized EGARCH (REGARCH-2C) model, which accommodates the high-frequency information and the long memory volatility through the realized measure of volatility and the component volatility structure, to forecast VIX. We obtain the risk-neutral dynamics of the REGARCH-2C model and derive the corresponding model-implied VIX formula. The parameter estimates of the REGARCH-2C model are obtained via the joint maximum likelihood estimation using observations on the returns, realized measure and VIX. Our empirical results demonstrate that the proposed REGARCH-2C model provides more accurate VIX forecasts compared to a variety of competing models, including the GARCH, GJR-GARCH, nonlinear GARCH, Heston–Nandi GARCH, EGARCH, REGARCH and two two-component GARCH models. This result is found to be robust to alternative realized measure. Our empirical evidence highlights the importance of incorporating the realized measure as well as the component volatility structure for VIX forecasting.

Clock synchronization with partial timestamp information for wireless sensor networks

Most existing clock synchronization algorithms in wireless sensor networks require full timestamp information to estimate clock skew and offset so as to maintain the same time scale between nodes. However, in many scenarios, with the purpose of decreasing energy consumption, reducing communication overhead, or improving security, timestamps are not exchanged in synchronization process, resulting in insufficient access to all time information by nodes to be synchronized. Hence, how to jointly estimate clock skew and offset with limited observation is a significant challenge. In order to solve this issue, this paper proposes a partial timestamp synchronization (PTS) scheme, which can implement complete synchronization function only according to the local timestamps and the timestamp relationships obtained by the basic synchronization protocol. Then, based on PTS, a joint Maximum Likelihood Estimator is developed to estimate clock skew and offset when stochastic delays follow Gaussian distribution. In addition, a Best Linear Unbiased Estimator for PTS with exponential random delays is presented, which can also estimate clock skew and offset simultaneously. Finally, simulations are conducted to illustrate the effectiveness of the proposed algorithms.

Covariance dependent kernels, a Q-affine GARCH for multi-asset option pricing

This paper introduces a class of multivariate GARCH models that extends the existing literature by explicitly modeling correlation dependent pricing kernels. A large subclass admits closed-form recursive solutions for the moment generating function under the risk-neutral measure, which permits efficient pricing of multi-asset options. We perform a full calibration to three bivariate series of index returns and their corresponding volatility indexes in a joint maximum likelihood estimation. The results empirically confirm the presence of correlation dependance in addition to the well known variance dependance in the pricing kernel. The model improves both the overall likelihood and the VIX-implied likelihoods, with a better fitting of marginal distributions, e.g., 15% less error on one-asset option prices. The new degree of freedom is also shown to significantly impact the shape of marginal and joint pricing kernels, and leads to up to 53% differences for out-of-the-money two-asset correlation option prices.

Shortened version of TAS using IRT: The 10-item scale for adolescents

Alexithymia is a multidimensional construct characterized by difficulty in identifying and describing feelings. Considering that the assessment of alexithymia in adolescents is very difficult, the aim of the present study was to examine the TAS-20 psychometric properties and its efficiency among a sample of Italian adolescents. In order to achieve this goal, the Rasch model, partial credit (PCM) specification with joint maximum likelihood estimation, was used. The Rasch results were used to select a set of items with high validity and discriminating power for adolescents, without any loss in measurement invariance. The final brief 10-item version of the scale, TAS-10, resulted to be valid and reliable: its correlation with the original 20-item score was remarkably high and the network with convergent and divergent measures gave the expected relationships. Furthermore, the concordance of the classification obtained from the 20 and 10-item versions was adequate, but the latter had the noticeable quality of reducing the quota of possible alexithymic adolescents, distinguishing non-alexithymic or alexithymic persons from those who are not clearly classified. Finally, the TAS-10 is definitively more suitable for adolescents in clinical and research practice.

A Unified Bayesian Framework for Joint Estimation and Anomaly Detection in Environmental Sensor Networks

Advanced large-scale environmental monitoring systems relying on the emerging aerial/terrestrial technologies of wireless sensor networks (WSNs), unmanned aerial vehicles (UAVs), and mobile crowdsensing, impose strong requirements on the reliability of the collected data. Unfortunately, sensing units can suddenly suffer unexpected anomalies due to accidental faults or malicious causes. Outlier detection methods have been widely employed to identify and discard unreliable measurements from large data sets, but further improvements in the sensing processes can be obtained by adopting advanced signal processing algorithms that take full advantage of all the collected information without rejecting the measurements. In this paper, we propose a novel unified Bayesian framework that enable simultaneous estimation of a common parameter of interest and identification of multiple and possibly different types of anomalies that can affect sensors in environmental sensor networks. Specifically, we consider two rather general error models based on Gaussian mixtures able to capture different variations affecting the quality of the collected measurements. For each model, we illustrate the optimal joint maximum-likelihood and maximum a-posteriori (ML-MAP) estimation method, which represents the benchmark for the problem at hand, and propose novel reduced-complexity two-step algorithms able to achieve almost the same performance of the joint ML-MAP, but at a fraction of its computational cost. The derivations of all the algorithms are also extended to handle the more general case in which the probability of occurrence of anomalies is unknown and should be inferred from the data using an Empirical Bayes approach. Extensive performance analyses using both synthetic and real experimental data acquired in a network of environmental monitoring stations deployed in the Apulia region, south of Italy, demonstrate the effectiveness of the proposed framework.

AutoRasch: An R Package to Do Semi-Automated Rasch Analysis.

The R package autoRasch has been developed to perform a Rasch analysis in a (semi-)automated way. The automated part of the analysis is achieved by optimizing the so-called in-plus-out-of-questionnaire log-likelihood (IPOQ-LL) or IPOQ-LL-DIF when differential item functioning (DIF) is included. These criteria measure the quality of fit on a pre-collected survey, depending on which items are included in the final instrument. To compute these criteria, autoRasch fits the generalized partial credit model (GPCM) or the generalized partial credit model with differential item functioning (GPCM-DIF) using penalized joint maximum likelihood estimation (PJMLE). The package further allows the user to reevaluate the output of the automated method and use it as a basis for performing a manual Rasch analysis and provides standard statistics of Rasch analyses (e.g., outfit, infit, person separation reliability, and residual correlation) to support the model reevaluation.

A Geometry-Based Stochastic Model for Truck Communication Channels in Freeway Scenarios

Vehicle-to-vehicle (V2V) wireless communication systems are fundamental in many intelligent transportation applications, e.g., traffic load control, driverless vehicle, and collision avoidance. Hence, developing appropriate V2V communication systems and standardization require realistic V2V propagation channel models. However, most existing V2V channel modeling studies focus on car-to-car channels; only a few investigate truck-to-car (T2C) or truck-to-truck (T2T) channels. In this paper, a hybrid geometry-based stochastic model (GBSM) is proposed for T2X (T2C or T2T) channels in freeway environments. Next, we parameterize this GBSM from the extensive channel measurements. We extract the multipath components (MPCs) by using a joint maximum likelihood estimation (RiMAX) and then determine the cluster types based on their evolution patterns. We classify the determined clusters into line-of-sight, single-bounce reflections from static interaction objects (IOs), single-bounce reflections from mobile IOs, multiple-bounce reflections, and density multipath components (DMCs). Particularly, we model multiple-bounce reflections as double clusters following the COST 273/COST2100 method. This paper presents the complete parameterization of the channel model. We validate this model by comparing the delay spread and the angular spreads of arrival/departure obtained from the proposed model with the measurement data.

ML and MAP Device Activity Detections for Grant-Free Massive Access in Multi-Cell Networks

Device activity detection is one main challenge in grant-free massive access, which is recently proposed to support massive machine-type communications (mMTC). Existing solutions for device activity detection fail to consider inter-cell interference generated by massive IoT devices or important prior information on device activities and inter-cell interference. In this paper, given different numbers of observations and network parameters, we consider both non-cooperative device activity detection and cooperative device activity detection in a multi-cell network, consisting of many access points (APs) and IoT devices. Under each activity detection mechanism, we consider the joint maximum likelihood (ML) estimation and joint maximum a posterior probability (MAP) estimation of both device activities and interference powers, utilizing tools from probability, stochastic geometry, and optimization. Each estimation problem is a challenging non-convex problem, and a coordinate descent algorithm is proposed to obtain a stationary point. Each proposed joint ML estimation extends the existing one for a single-cell network by considering the estimation of interference powers, together with the estimation of device activities. Each proposed joint MAP estimation further enhances the corresponding joint ML estimation by exploiting prior distributions of device activities and interference powers. The proposed joint ML estimation and joint MAP estimation under cooperative detection outperform the respective ones under non-cooperative detection at the costs of increasing backhaul burden, knowledge of network parameters, and computational complexities. Numerical results show the substantial gains of the proposed designs over well-known existing designs and reveal the importance of explicit consideration of inter-cell interference, the value of prior information, and the advantage of AP cooperation in device activity detection.

Assessing Rasch measurement estimation methods across R packages with yes/no vocabulary test data

Instrument measurement conducted with Rasch analysis is a common process in language assessment research. A recent systematic review of 215 studies involving Rasch analysis in language testing and applied linguistics research reported that 23 different software packages had been utilized. However, none of the analyses were conducted with one of the numerous R-based Rasch analysis software packages, which generally employ one of the three estimation methods: conditional maximum likelihood estimation (CMLE), joint maximum likelihood estimation (JMLE), or marginal maximum likelihood estimation (MMLE). For this study, eRm, a CMLE-based R package, was utilized to conduct a dichotomous Rasch analysis of a Yes/No vocabulary test based on the academic word list. The resulting parameters and diagnostic statistics were compared with the equivalent results from four other R-based Rasch measurement software packages and Winsteps. Finally, all of the packages were utilized in the analysis of 1000 simulated datasets to investigate the extent to which results generated from the contrasting estimation methods converged or diverged. Overall, the differences between the results produced with the three estimation methods were negligible, and the discrepancies observed between datasets were attributable to the software choice as opposed to the estimation method.

Maximum Likelihood Algorithm for Time-Delay Based Multistatic Target Localization

In this letter we address the problem of multistatic target localization using time-delay measurements corrupted by noise with unknown non-uniform variances. More concretely, we consider the problem of joint maximum likelihood (ML) estimation of the target position and the noise variances, for which we propose a majorization-minimization based algorithm. The proposed approach is compared with state-of-the-art algorithms, and the simulation results show the excellent accuracy of our algorithm.

Joint ML/MAP Estimation of the Frequency and Phase of a Single Sinusoid With Wiener Carrier Phase Noise

We address here the issue of jointly estimating the angle parameters of a single sinusoid with Wiener carrier phase noise and observed in additive, white, Gaussian noise (AWGN). We develop the theoretical foundation for time-domain, phase-based, joint maximum likelihood (ML) estimation of the unknown carrier frequency and the initial carrier phase, with simultaneous maximum a posteriori probability (MAP) estimation of the time-varying carrier phase noise. The derivation is based on the amplitude and phase-form of the noisy received signal model together with the use of the best, linearized, additive observation phase noise model due to AWGN. Our newly derived estimators are closed-form expressions, consisting of both the phase and the magnitude of all the received signal samples. More importantly, they all have a low-complexity, sample-by-sample iterative processing structure, which can be implemented iteratively in real-time. As a basis for comparison, the Cramer-Rao lower bound (CRLB) for the ML estimators and the Bayesian CRLB (BCRLB) for the MAP estimator are derived in the presence of carrier phase noise, and the results simply depend on the signal-to-noise ratio (SNR), the observation length and the phase noise variance. It is theoretically shown that the estimates obtained are unbiased, and the mean-square error (MSE) of the estimators attain the CRLB/BCRLB at high SNR. The MSE performance as a function of the SNR, the observation length and the phase noise variance is verified using Monte Carlo simulation, which shows a remarkable improvement in estimation accuracy in large phase noise.

Joint Maximum Likelihood Estimation of Frequency Offset and Channel and Intrinsic Interference Cancellation Using Cyclic Pilot in Fbmc/Qam Systems

Joint Maximum Likelihood Estimation of Frequency Offset and Channel and Intrinsic Interference Cancellation Using Cyclic Pilot in Fbmc/Qam Systems

A Comprehensive Simulation Study of Estimation Methods for the Rasch Model

The Rasch model is one of the most prominent item response models. In this article, different item parameter estimation methods for the Rasch model are systematically compared through a comprehensive simulation study: Different alternatives of joint maximum likelihood (JML) estimation, different alternatives of marginal maximum likelihood (MML) estimation, conditional maximum likelihood (CML) estimation, and several limited information methods (LIM). The type of ability distribution (i.e., nonnormality), the number of items, sample size, and the distribution of item difficulties were systematically varied. Across different simulation conditions, MML methods with flexible distributional specifications can be at least as efficient as CML. Moreover, in many situations (i.e., for long tests), penalized JML and JML with ε adjustment resulted in very efficient estimates and might be considered alternatives to JML implementations currently used in statistical software. Moreover, minimum chi-square (MINCHI) estimation was the best-performing LIM method. These findings demonstrate that JML estimation and LIM can still prove helpful in applied research.

Seeking the Truth in a Decentralized Manner

In networks where massive sources make observations of same entities, we intend to seek the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">truth</i> – the most trustworthy value of each entity from conflicting information claimed by multiple sources. Various methods are proposed for accurately inferring both source reliability and truths, yet relying heavily on centralized settings that incur tremendous overhead to source side. In this paper, we offer a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">decentralized</i> design of truth discovery task that can fit favorably to the environments with limited resources. Considering that sources forming the connected network and making individual observations, we undertake the joint maximum likelihood estimation (MLE) of truth and source reliability. Our decentralization framework simply allows each source to maintain local information exchange at a time, and computes very basic functions of data observations. To this end, we facilitate the decentralization by simplifying the MLE problem into optimizing an objective function. Upon the proof of NP-hardness, two proposed decentralized algorithms (exact and approximation) are decentralized and randomized via a combination of algorithms from their centralized counterparts that ensure performance guarantee. The derived time complexity features explicit data/network dependent terms, which leads to further acceleration in truth finding. Remarkably, in two well connected networks like random geometric and preferential attachment graphs, the accelerated approximation method enjoys logarithmic time complexity while preserving comparable accuracy to the centralized counterparts. The effectiveness of the proposed decentralizations are further empirically confirmed.

A Joint MLE Approach to Large-Scale Structured Latent Attribute Analysis

Structured latent attribute models (SLAMs) are a family of discrete latent variable models widely used in education, psychology, and epidemiology to model multivariate categorical data. A SLAM assumes that multiple discrete latent attributes explain the dependence of observed variables in a highly structured fashion. Usually, the maximum marginal likelihood estimation approach is adopted for SLAMs, treating the latent attributes as random effects. The increasing scope of modern assessment data involves large numbers of observed variables and high-dimensional latent attributes. This poses challenges to classical estimation methods and requires new methodology and understanding of latent variable modeling. Motivated by this, we consider the joint maximum likelihood estimation (MLE) approach to SLAMs, treating latent attributes as fixed unknown parameters. We investigate estimability, consistency, and computation in the regime where sample size, number of variables, and number of latent attributes all can diverge. We establish the statistical consistency of the joint MLE and propose efficient algorithms that scale well to large-scale data for several popular SLAMs. Simulation studies demonstrate the superior empirical performance of the proposed methods. An application to real data from an international educational assessment gives interpretable findings of cognitive diagnosis.

A plotless density estimator with a Norton-Rice distribution for ordered distances

A Norton-Rice distribution (NRD) is a versatile, flexible distribution for k ordered distances from a random location to the k nearest objects. In a context of plotless density estimation (PDE) with n randomly chosen sample locations, and distances measured to the k = 6 nearest objects, the NRD provided a good fit to distance data from seven populations with a census of forest tree stem locations. More importantly, the three parameters of a NRD followed a simple trend with the order (1, …, 6) of observed distances. The trend is quantified and exploited in a proposed new PDE through a joint maximum likelihood estimation of the NRD parameters expressed as a functions of distance order. In simulated probability sampling from the seven populations, the proposed PDE had the lowest overall bias with a good performance potential when compared to three alternative PDEs. However, absolute bias increased by 0.8 percentage points when sample size decreased from 20 to 10. In terms of root mean squared error (RMSE), the new proposed estimator was at par with an estimator published in Ecology when this study was wrapping up, but otherwise superior to the remaining two investigated PDEs. Coverage of nominal 95% confidence intervals averaged 0.94 for the new proposed estimators and 0.90, 0.96, and 0.90 for the comparison PDEs. Despite tangible improvements in PDEs over the last decades, a globally least biased PDE remains elusive.

Downlink Single-Snapshot Localization and Mapping With a Single-Antenna Receiver

5G mmWave MIMO systems enable accurate estimation of the user position and mapping of the radio environment using a single snapshot when both the base station (BS) and user are equipped with large antenna arrays. However, massive arrays are initially expected only at the BS side, likely leaving users with one or very few antennas. In this paper, we propose a novel method for single-snapshot localization and mapping in the more challenging case of a user equipped with a single-antenna receiver. The joint maximum likelihood (ML) estimation problem is formulated and its solution formally derived. To avoid the burden of a full-dimensional search over the space of the unknown parameters, we present a novel practical approach that exploits the sparsity of mmWave channels to compute an approximate joint ML estimate. A thorough analysis, including the derivation of the Cramér-Rao lower bounds, reveals that accurate localization and mapping can be achieved also in a MISO setup even when the direct line-of-sight path between the BS and the user is severely attenuated.

A Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis.

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

A non-elliptical orthogonal GARCH model for portfolio selection under transaction costs

Covariance matrix forecasts for portfolio optimization have to balance sensitivity to new data points with stability in order to avoid excessive rebalancing. To achieve this, a new orthogonal GARCH model for a multivariate set of non-Gaussian asset returns is proposed. The conditional return distribution is multivariate generalized hyperbolic and the dispersion matrix dynamics are driven by the leading factors in a principal component decomposition. Each of these leading factors is endowed with a univariate GARCH structure, while the remaining eigenvalues are kept constant over time. Joint maximum likelihood estimation of all model parameters is performed via an expectation maximization algorithm, and is applicable in high dimensions. The new model generates realistic correlation forecasts even for large asset universes and captures rising pairwise correlations in periods of market distress better than numerous competing models. When applied to portfolio optimization, it generates strategies with lower turnover and maximum drawdown, and superior risk-adjusted returns net of transaction costs. Moreover, unlike its competitors, it performs well in the sudden market downturn triggered by the global COVID-19 pandemic.