Expectation-conditional Maximization Either Research Articles

Time series clustering based on latent volatility mixture modeling with applications in finance

Modeling financial time series data poses a significant challenge in the realm of time series analysis. The Autoregressive Conditional Heteroskedasticity (ARCH) model stands out as a potent tool for capturing time-varying volatility and heteroskedasticity in financial data. However, conventional ARCH models display sensitivity to departures from normality, leading to the development of extensions employing more flexible distributions. In this context, we propose a robust enhancement to the mixture of ARCH (MoARCH) model by integrating normal mean–variance mixture (NMVM) distributions to model component errors. The stochastic representation of the proposed model allows for a straightforward implementation of an Expectation Conditional Maximization Either (ECME) algorithm for obtaining Maximum Penalized Likelihood estimates (MPL). To thoroughly evaluate the model, we conduct four simulation studies to explore finite-sample properties, assess MPL estimators, scrutinize model robustness, and evaluate the accuracy of our proposal in fitting, clustering, and forecasting. Practical applications further highlight the effectiveness of our methodology, showcasing successful implementations across diverse real datasets.

Estimation for vector autoregressive model under multivariate skew-t-normal innovations

Current procedures for estimating the parameters of [Formula: see text]th order vector autoregressive (VAR [Formula: see text]) model are usually based on assuming that the ensuing error distribution is multivariate normal. But there exists large body of evidence that several data encountered in real life are skewed; thereby making estimators derived based on normality assumption not suitable in such scenarios. This prompts for the search of appropriate methods for skewed distributions. Therefore, this article proposes estimators for the mean and covariance matrices of the [Formula: see text] model under multivariate skew- [Formula: see text]-normal (MSTN) distribution. Also, estimators for the shape and skewness parameters are provided. The expectation conditional maximization (ECM) and its extension the expectation conditional maximization either (ECME) algorithms are the tools used to derive the estimators. The performance of the estimators were examined through extensive simulations, and results show that they compete favourably with other numerical methods especially when the underlying distribution is skewed. The usefulness of our estimators was illustrated using a real data set on some US economic indicators. The VAR [Formula: see text] model under MSTN distribution provides a good fit, better than [Formula: see text] model under the assumption of normality.

Stochastic Maximum Likelihood Direction Finding in the Presence of Nonuniform Noise Fields

The maximum likelihood (ML) technique plays an important role in direction-of-arrival (DOA) estimation. In this paper, we employ and design the expectation–conditional maximization either (ECME) algorithm, a generalization of the expectation–maximization algorithm, for solving the ML direction finding problem of stochastic sources, which may be correlated, in unknown nonuniform noise. Unlike alternating maximization, the ECME algorithm updates both the source and noise covariance matrix estimates by explicit formulas, and can guarantee that both estimates are positive semi-definite and definite, respectively. Thus, the ECME algorithm is computationally efficient and operationally stable. Simulation results confirm that the ECME algorithm can efficiently obtain the ML based DOA estimate of each stochastic source.

Multivariate measurement error models with normal mean‐variance mixture distributions

The class of normal mean‐variance mixture (NMVM) distributions is a rich family of asymmetric and heavy‐tailed distributions and has been widely considered in parametric modeling of the data for robust statistical inference. This paper proposes an extension of measurement error models by assuming the NMVM distributions for the unobserved covariates and error terms in the model, referred to as the NMVM‐MEM. An expectation conditional maximization either (ECME) algorithm is developed to compute the maximum likelihood (ML) estimates of model parameters. Furthermore, an information‐based approach is performed to derive the asymptotic covariance matrix of ML estimators. The analysis of a blood pressure dataset illustrates the superiority of NMVM‐MEM to accommodate asymmetry and outliers over the normal counterpart. Two simulation studies are undertaken to validate our proposed techniques.

Robust mixture regression modeling based on the normal mean-variance mixture distributions

Mixture regression models (MRMs) are widely used to capture the heterogeneity of relationships between the response variable and one or more predictors coming from several non-homogeneous groups. Since the conventional MRMs are quite sensitive to departures from normality caused by extra skewness and possible heavy tails, various extensions built on more flexible distributions have been put forward in the last decade. The class of normal mean-variance mixture (NMVM) distributions that arise from scaling both the mean and variance of a normal random variable with a common mixing distribution encompasses many prominent (symmetric or asymmetrical) distributions as special cases. A unified approach to robustifying MRMs is proposed by considering the class of NMVM distributions for component errors. An expectation conditional maximization either (ECME) algorithm, which incorporates membership indicators and the latent scaling variables as the missing data, is developed for carrying out maximum likelihood (ML) estimation of model parameters. Four simulation studies are conducted to examine the finite-sample property of ML estimators and the robustness of the proposed model against outliers for contaminated and noisy data. The usefulness and superiority of our methodology are demonstrated through applications to two real datasets.

Locally optimum image watermark detector based on statistical modeling of SWT-EFMs magnitudes

There are three indispensable, yet contrasting requirements for a watermarking scheme: perceptual transparency, watermark capacity, and robustness against attacks. Therefore, a watermarking scheme should provide a trade-off among these requirements from the information-theoretic perspective. Improving the ability of imperceptibility, watermark capacity, and robustness at the same time has been a challenge for all image watermarking algorithms. The statistical model-based transform domain multiplicative watermarking is an effective way to achieve the tradeoff among imperceptibility, robustness and data payload. In this paper, we propose a locally optimum statistical image watermark detector by incorporating stationary wavelet transform (SWT), exponent-Fourier moments (EFMs), and locally optimum (LO) decision rule. SWT has many characteristics, such as high redundancy, high reconstruction accuracy, low time complexity and multi-resolution representation, and EFMs can provide the excellent robustness. In order to obtain valuable samples, we first decompose the digital image by using SWT, and then perform EFMs calculation in the corresponding frequency domain. These robust frequency-domain EFMs magnitudes are treated as the basis for the following research work. In our scheme, the watermark is hidden into the target location of frequency-domain EFMs magnitudes by using multiplicative embedding method. These magnitudes at the target location are abbreviated as SWT-EFMs magnitudes for the convenience of subsequent description. We analyze the statistical characteristics of SWT-EFMs magnitudes in detail. In order to achieve accurate statistical modeling of SWT-EFMs magnitudes, the Beta Generalized Weibull mixtures-based hidden Markov tree (BGWM-HMT) model is constructed in SWT-EFMs magnitudes. The expectation-conditional maximization either (ECME) and the upward–downward algorithms are used to obtain model parameters. Our proposed statistical model can simultaneously capture the special distribution characteristics of SWT-EFMs magnitudes and the dependencies between scales. At the receiver, a statistical watermark detector based on the LO decision rule and BGWM-HMT model is developed in SWT-EFMs magnitudes. We derive closed-form expressions about the LO statistic and analyze the receiver operating characteristic (ROC) of the proposed detector. In addition, we measure the proposed scheme in terms of robustness, invisibility and time complexity. Experimental results on some test images and comparison with well-known existing methods demonstrate the efficacy and superiority of the proposed statistical image watermark detector.

A Comparison of Different Estimation Methods to Handle Missing Data in Explanatory Variables

Missing data is one of the problems that may occur in regression models. This problem is usually handled by deletion mechanism available in statistical software. This method reduces statistical inference values because deletion affects sample size. In this paper, Expectation Maximization algorithm (EM), Multicycle-Expectation-Conditional Maximization algorithm (MC-ECM), Expectation-Conditional Maximization Either (ECME), and Recurrent Neural Networks (RNN) are used to estimate multiple regression models when explanatory variables have some missing values. Experimental dataset were generated using Visual Basic programming language with missing values of explanatory variables according to a missing mechanism at random general pattern and some ratios of missing values (10%, 20%, and 30%) with error variance values of 0.5, 1. 5, and 2, which were included in sample sizes of 25, 50, 100, and 500 and evaluated using Mean Squared Error (MSE). Simulation results show that RNN outperforms the other methods, followed by EM at small sample sizes.

Multivariate-t linear mixed models with censored responses, intermittent missing values and heavy tails.

Multivariate longitudinal data arisen in medical studies often exhibit complex features such as censored responses, intermittent missing values, and atypical or outlying observations. The multivariate-t linear mixed model (MtLMM) has been recognized as a powerful tool for robust modeling of multivariate longitudinal data in the presence of potential outliers or fat-tailed noises. This paper presents a generalization of MtLMM, called the MtLMM-CM, to properly adjust for censorship due to detection limits of the assay and missingness embodied within multiple outcome variables recorded at irregular occasions. An expectation conditional maximization either (ECME) algorithm is developed to compute parameter estimates using the maximum likelihood (ML) approach. The asymptotic standard errors of the ML estimators of fixed effects are obtained by inverting the empirical information matrix according to Louis' method. The techniques for the estimation of random effects and imputation of missing responses are also investigated. The proposed methodology is illustrated on two real-world examples from HIV-AIDS studies and a simulation study under a variety of scenarios.

Linear scalar-on-surface random effects regression models

Many research fields increasingly involve analyzing data of a complex structure. Models investigating the dependence of a response on a predictor have moved beyond the ordinary scalar-on-vector regression. We propose a regression model for a scalar response and a surface (or a bivariate function) predictor. The predictor has a random component and the regression model falls in the framework of linear random effects models. We estimate the model parameters via maximizing the log-likelihood with the ECME (Expectation/Conditional Maximization Either) algorithm. We use the approach to analyze a data set where the response is the neuroticism score and the predictor is the resting-state brain function image. In the simulations we tried, the approach has better performance than two other approaches, a functional principal component regression approach and a smooth scalar-on-image regression approach.

Multivariate-$t$ nonlinear mixed models with application to censored multi-outcome AIDS studies.

In multivariate longitudinal HIV/AIDS studies, multi-outcome repeated measures on each patient over time may contain outliers, and the viral loads are often subject to a upper or lower limit of detection depending on the quantification assays. In this article, we consider an extension of the multivariate nonlinear mixed-effects model by adopting a joint multivariate-$t$ distribution for random effects and within-subject errors and taking the censoring information of multiple responses into account. The proposed model is called the multivariate-$t$ nonlinear mixed-effects model with censored responses (MtNLMMC), allowing for analyzing multi-outcome longitudinal data exhibiting nonlinear growth patterns with censorship and fat-tailed behavior. Utilizing the Taylor-series linearization method, a pseudo-data version of expectation conditional maximization either (ECME) algorithm is developed for iteratively carrying out maximum likelihood estimation. We illustrate our techniques with two data examples from HIV/AIDS studies. Experimental results signify that the MtNLMMC performs favorably compared to its Gaussian analogue and some existing approaches.

Extracting Low-Dimensional Latent Structure from Time Series in the Presence of Delays.

Noisy, high-dimensional time series observations can often be described by a set of low-dimensional latent variables. Commonly used methods to extract these latent variables typically assume instantaneous relationships between the latent and observed variables. In many physical systems, changes in the latent variables manifest as changes in the observed variables after time delays. Techniques that do not account for these delays can recover a larger number of latent variables than are present in the system, thereby making the latent representation more difficult to interpret. In this work, we introduce a novel probabilistic technique, time-delay gaussian-process factor analysis (TD-GPFA), that performs dimensionality reduction in the presence of a different time delay between each pair of latent and observed variables. We demonstrate how using a gaussian process to model the evolution of each latent variable allows us to tractably learn these delays over a continuous domain. Additionally, we show how TD-GPFA combines temporal smoothing and dimensionality reduction into a common probabilistic framework. We present an expectation/conditional maximization either (ECME) algorithm to learn the model parameters. Our simulations demonstrate that when time delays are present, TD-GPFA is able to correctly identify these delays and recover the latent space. We then applied TD-GPFA to the activity of tens of neurons recorded simultaneously in the macaque motor cortex during a reaching task. TD-GPFA is able to better describe the neural activity using a more parsimonious latent space than GPFA, a method that has been used to interpret motor cortex data but does not account for time delays. More broadly, TD-GPFA can help to unravel the mechanisms underlying high-dimensional time series data by taking into account physical delays in the system.

Compound Poisson Processes, Latent Shrinkage Priors and Bayesian Nonconvex Penalization

In this paper we discuss Bayesian nonconvex penalization for sparse learning problems. We explore a nonparametric formulation for latent shrinkage parameters using subordinators which are one-dimensional L\'{e}vy processes. We particularly study a family of continuous compound Poisson subordinators and a family of discrete compound Poisson subordinators. We exemplify four specific subordinators: Gamma, Poisson, negative binomial and squared Bessel subordinators. The Laplace exponents of the subordinators are Bernstein functions, so they can be used as sparsity-inducing nonconvex penalty functions. We exploit these subordinators in regression problems, yielding a hierarchical model with multiple regularization parameters. We devise ECME (Expectation/Conditional Maximization Either) algorithms to simultaneously estimate regression coefficients and regularization parameters. The empirical evaluation of simulated data shows that our approach is feasible and effective in high-dimensional data analysis.

Methods to assess an exercise intervention trial based on 3-level functional data.

Motivated by data recording the effects of an exercise intervention on subjects' physical activity over time, we develop a model to assess the effects of a treatment when the data are functional with 3 levels (subjects, weeks and days in our application) and possibly incomplete. We develop a model with 3-level mean structure effects, all stratified by treatment and subject random effects, including a general subject effect and nested effects for the 3 levels. The mean and random structures are specified as smooth curves measured at various time points. The association structure of the 3-level data is induced through the random curves, which are summarized using a few important principal components. We use penalized splines to model the mean curves and the principal component curves, and cast the proposed model into a mixed effects model framework for model fitting, prediction and inference. We develop an algorithm to fit the model iteratively with the Expectation/Conditional Maximization Either (ECME) version of the EM algorithm and eigenvalue decompositions. Selection of the number of principal components and handling incomplete data issues are incorporated into the algorithm. The performance of the Wald-type hypothesis test is also discussed. The method is applied to the physical activity data and evaluated empirically by a simulation study.

Simultaneous inferences based on empirical Bayes methods and false discovery rates ineQTL data analysis

BackgroundGenome-wide association studies (GWAS) have identified hundreds of genetic variants associated with complex human diseases, clinical conditions and traits. Genetic mapping of expression quantitative trait loci (eQTLs) is providing us with novel functional effects of thousands of single nucleotide polymorphisms (SNPs). In a classical quantitative trail loci (QTL) mapping problem multiple tests are done to assess whether one trait is associated with a number of loci. In contrast to QTL studies, thousands of traits are measured alongwith thousands of gene expressions in an eQTL study. For such a study, a huge number of tests have to be performed (). This extreme multiplicity gives rise to many computational and statistical problems. In this paper we have tried to address these issues using two closely related inferential approaches: an empirical Bayes method that bears the Bayesian flavor without having much a priori knowledge and the frequentist method of false discovery rates. A three-component t-mixture model has been used for the parametric empirical Bayes (PEB) method. Inferences have been obtained using Expectation/Conditional Maximization Either (ECME) algorithm. A simulation study has also been performed and has been compared with a nonparametric empirical Bayes (NPEB) alternative.ResultsThe results show that PEB has an edge over NPEB. The proposed methodology has been applied to human liver cohort (LHC) data. Our method enables to discover more significant SNPs with FDR<10% compared to the previous study done by Yang et al. (Genome Research, 2010).ConclusionsIn contrast to previously available methods based on p-values, the empirical Bayes method uses local false discovery rate (lfdr) as the threshold. This method controls false positive rate.

Robust radio interferometric calibration using the t-distribution

A major stage of radio interferometric data processing is calibration or the estimation of systematic errors in the data and the correction for such errors. A stochastic error (noise) model is assumed, and in most cases, this underlying model is assumed to be Gaussian. However, outliers in the data due to interference or due to errors in the sky model would have adverse effects on processing based on a Gaussian noise model. Most of the shortcomings of calibration such as the loss in flux or coherence, and the appearance of spurious sources, could be attributed to the deviations of the underlying noise model. In this paper, we propose to improve the robustness of calibration by using a noise model based on Student's t distribution. Student's t noise is a special case of Gaussian noise when the variance is unknown. Unlike Gaussian noise model based calibration, traditional least squares minimization would not directly extend to a case when we have a Student's t noise model. Therefore, we use a variant of the Expectation Maximization (EM) algorithm, called the Expectation-Conditional Maximization Either (ECME) algorithm when we have a Student's t noise model and use the Levenberg-Marquardt algorithm in the maximization step. We give simulation results to show the robustness of the proposed calibration method as opposed to traditional Gaussian noise model based calibration, especially in preserving the flux of weaker sources that are not included in the calibration model.

Sparse Signal Reconstruction via ECME Hard Thresholding

We propose a probabilistic model for sparse signal reconstruction and develop several novel algorithms for computing the maximum likelihood (ML) parameter estimates under this model. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown deterministic sparse signal component and a zero-mean white Gaussian component with an unknown variance. Our reconstruction schemes are based on an expectation-conditional maximization either (ECME) iteration that aims at maximizing the likelihood function with respect to the unknown parameters for a given signal sparsity level. Compared with the existing iterative hard thresholding (IHT) method, the ECME algorithm contains an additional multiplicative term and guarantees monotonic convergence for a wide range of sensing (regression) matrices. We propose a double overrelaxation (DORE) thresholding scheme for accelerating the ECME iteration. We prove that, under certain mild conditions, the ECME and DORE iterations converge to local maxima of the likelihood function. The ECME and DORE iterations can be implemented exactly in small-scale applications and for the important class of large-scale sensing operators with orthonormal rows used e.g., partial fast Fourier transform (FFT). If the signal sparsity level is unknown, we introduce an unconstrained sparsity selection (USS) criterion and a tuning-free automatic double overrelaxation (ADORE) thresholding method that employs USS to estimate the sparsity level. We compare the proposed and existing sparse signal reconstruction methods via one-dimensional simulation and two-dimensional image reconstruction experiments using simulated and real X-ray CT data.