Stat Softw Research Articles

Computational issues in parameter estimation for hidden Markov models with template model builder

A popular way to estimate the parameters of a hidden Markov model (HMM) is direct numerical maximization (DNM) of the (log-)likelihood function. The advantages of employing the TMB [Kristensen K, Nielsen A, Berg C, et al. TMB: automatic differentiation and Laplace approximation. J Stat Softw Articles. 2016;70(5):1–21] framework in R for this purpose were illustrated recently [Bacri T, Berentsen GD, Bulla J, et al. A gentle tutorial on accelerated parameter and confidence interval estimation for hidden Markov models using template model builder. Biom J. 2022 Oct;64(7):1260–1288]. In this paper, we present extensions of these results in two directions. First, we present a practical way to obtain uncertainty estimates in form of confidence intervals (CIs) for the so-called smoothing probabilities at moderate computational and programming effort via TMB. Our approach thus permits to avoid computer-intensive bootstrap methods. By means of several examples, we illustrate patterns present for the derived CIs. Secondly, we investigate the performance of popular optimizers available in R when estimating HMMs via DNM. Hereby, our focus lies on the potential benefits of employing TMB. Investigated criteria via a number of simulation studies are convergence speed, accuracy, and the impact of (poor) initial values. Our findings suggest that all optimizers considered benefit in terms of speed from using the gradient supplied by TMB. When supplying both gradient and Hessian from TMB, the number of iterations reduces, suggesting a more efficient convergence to the maximum of the log-likelihood. Last, we briefly point out potential advantages of a hybrid approach.

Association Between Immunosuppressive Therapy and Incident Risk of Interstitial Lung Disease in Systemic Sclerosis

Association Between Immunosuppressive Therapy and Incident Risk of Interstitial Lung Disease in Systemic Sclerosis

Is COVID-19 infection more severe in kidney transplant recipients?

There are no studies which have compared the risk of severe COVID‐19 and related mortality between transplant recipients and nontransplant patients. We enrolled two groups of patients hospitalized for COVID‐19, that is, kidney transplant recipients (KTR) from the French Registry of Solid Organ Transplant (n = 306) and a single‐center cohort of nontransplant patients (n = 795). An analysis was performed among subgroups matched for age and risk factors for severe COVID‐19 or mortality. Severe COVID‐19 was defined as admission (or transfer) to an intensive care unit, need for mechanical ventilation, or death. Transplant recipients were younger and had more comorbidities compared to nontransplant patients. They presented with higher creatinine levels and developed more episodes of acute kidney injury. After matching, the 30‐day cumulative incidence of severe COVID‐19 did not differ between KTR and nontransplant patients; however, 30‐day COVID‐19‐related mortality was significantly higher in KTR (17.9% vs 11.4%, respectively, p = .038). Age >60 years, cardiovascular disease, dyspnea, fever, lymphopenia, and C‐reactive protein (CRP) were associated with severe COVID‐19 in univariate analysis, whereas transplant status and serum creatinine levels were not. Age >60 years, hypertension, cardiovascular disease, diabetes, CRP >60 mg/L, lymphopenia, kidney transplant status (HR = 1.55), and creatinine level >115 µmol/L (HR = 2.32) were associated with COVID‐19‐related mortality in univariate analysis. In multivariable analysis, cardiovascular disease, dyspnea, and fever were associated with severe disease, whereas age >60 years, cardiovascular disease, dyspnea, fever, and creatinine level>115 µmol/L retained their independent associations with mortality. KTR had a higher COVID‐19‐related mortality compared to nontransplant hospitalized patients.

Asthma is associated with increased risk of intubation but not hospitalization or death in coronavirus disease 2019

Asthma is associated with increased risk of intubation but not hospitalization or death in coronavirus disease 2019

Implementing Convex Optimization in R: Two Econometric Examples

Economists specify high-dimensional models to address heterogeneity in empirical studies with complex big data. Estimation of these models calls for optimization techniques to handle a large number of parameters. Convex problems can be effectively executed in modern programming languages. We complement Koenker and Mizera (J Stat Softw 60(5):1–23, 2014)’s work on numerical implementation of convex optimization, with focus on high-dimensional econometric estimators. Combining R and the convex solver MOSEK achieves speed gain and accuracy, demonstrated by examples from Su et al. (Econometrica 84(6):2215–2264, 2016) and Shi (J Econom 195(1):104–119, 2016). Robust performance of convex optimization is witnessed across platforms. The convenience and reliability of convex optimization in R make it easy to turn new ideas into executable estimators.

Computing and graphing probability values of pearson distributions: a SAS/IML macro

BackgroundAny empirical data can be approximated to one of Pearson distributions using the first four moments of the data (Elderton WP, Johnson NL. Systems of Frequency Curves. 1969; Pearson K. Philos Trans R Soc Lond Ser A. 186:343–414 1895; Solomon H, Stephens MA. J Am Stat Assoc. 73(361):153–60 1978). Thus, Pearson distributions made statistical analysis possible for data with unknown distributions. There are both extant, old-fashioned in-print tables (Pearson ES, Hartley HO. Biometrika Tables for Statisticians, vol. II. 1972) and contemporary computer programs (Amos DE, Daniel SL. Tables of percentage points of standardized pearson distributions. 1971; Bouver H, Bargmann RE. Tables of the standardized percentage points of the pearson system of curves in terms of β1 and β2. 1974; Bowman KO, Shenton LR. Biometrika. 66(1):147–51 1979; Davis CS, Stephens MA. Appl Stat. 32(3):322–7 1983; Pan W. J Stat Softw. 31(Code Snippet 2):1–6 2009) available for obtaining percentage points of Pearson distributions corresponding to certain pre-specified percentages (or probability values; e.g., 1.0%, 2.5%, 5.0%, etc.), but they are little useful in statistical analysis because we have to rely on unwieldy second difference interpolation to calculate a probability value of a Pearson distribution corresponding to a given percentage point, such as an observed test statistic in hypothesis testing.ResultsThe present study develops a SAS/IML macro program to identify the appropriate type of Pearson distribution based on either input of dataset or the values of four moments and then compute and graph probability values of Pearson distributions for any given percentage points.ConclusionsThe SAS macro program returns accurate approximations to Pearson distributions and can efficiently facilitate researchers to conduct statistical analysis on data with unknown distributions.

Forecasting the unemployment rate using the degree of agreement in consumer unemployment expectations

This study aims to refine unemployment forecasts by incorporating the degree of consensus in consumers’ expectations. With this objective, we first model the unemployment rate in eight European countries using the step-wise algorithm proposed by Hyndman and Khandakar (J Stat Softw 27(3):1–22, 2008). The selected optimal autoregressive integrated moving average (ARIMA) models are then used to generate out-of-sample recursive forecasts of the unemployment rates, which are used as benchmark. Finally, we replicate the forecasting experiment including as predictors both an indicator of unemployment, based on the degree of agreement in consumer unemployment expectations, and a measure of disagreement based on the dispersion of expectations. In both cases, we obtain an improvement in forecast accuracy in most countries. These results reveal that the degree of agreement in consumers’ expectations contains useful information to predict unemployment rates, especially for the detection of turning points.

Nonparametric Regression Using Clusters

We present a fundamentally unique method of nonparametric regression using clusters and test it against classically established methods. We compare two nonlinear regression estimation packages called ‘NNS’, Viole (NNS: nonlinear nonparametric statistics, 2016), and ‘np’, Hayfield and Racine (J Stat Softw 27(5):1–32, 2008), with the help of a simulation using deterministic (DT) and stochastic (ST) regressor models. We find the respective coefficients of determination $$(R^2)$$ are close for DT models, while finding an advantage to NNS in ST and large sample cases. Regression coefficients are sometimes regarded as approximations to partial derivatives, especially in social sciences. Then, NNS alone has the ability to compute a range of partials evaluated at points within the sample and also out-of-sample. Thus NNS can provide a viable alternative to kernel based nonparametric regressions without using bandwidths for smoothing.

Measuring spatial inhomogeneity at different spatial scales using hybrids of Gibbs point process models

Infectious diseases give rise to complex spatial patterns exhibiting aggregation at different scales. Baddeley (J Stat Softw 55:1–43, 2013) proposed a technique for constructing new Gibbs models for spatial point patterns, combining existing models available in the literature. We use their proposal to model the spatial point pattern of varicella, a highly contagious airborne disease, in Valencia, Spain. We employed descriptive analysis to get a glimpse of the basic properties of the point pattern. Covariate information such as the density of population (children under 14 years old) living in the study region, the distance to the nearest school, and the composition of families (expressed as the average number of persons per family) is used to describe the intensity of the process. We used SatScan to identify main clusters of schools, and to feed the model with this further information. Our analysis shows the relation between varicella cases and school locations, and highlights aggregation in the data at different spatial scales.

Mondrian

AbstractMondrian is a general‐purpose statistical data‐visualization system [Theus M. Interactive data visualization using Mondrian. J Stat Softw 2003, 7:1–9]. It features outstanding visualization techniques for data of almost any kind, and has its particular strength compared to other tools when working with categorical data, geographical data, and large data sets. Data displays in Mondrian are interactive, i.e., plot parameters may be changed interactively resulting in an instantaneous update of the views. Furthermore, all plots in Mondrian are fully linked, i.e., any case selected or marked in a plot in Mondrian is highlighted in all other linked plots. Mondrian offers interactive queries, which show information of the data objects in any plot. Copyright © 2010 John Wiley & Sons, Inc.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Software for Computational Statistics > Software/Statistical Software Statistical and Graphical Methods of Data Analysis > Statistical Graphics and Visualization

Visualizing hypothesis tests in multivariate linear models: the heplots package for R

Hypothesis-error (or “HE”) plots, introduced by Friendly (J Stat Softw 17(6):1–42, 2006a; J Comput Graph Stat 16:421–444, 2006b), permit the visualization of hypothesis tests in multivariate linear models by representing hypothesis and error matrices of sums of squares and cross-products as ellipses. This paper describes the implementation of these methods in the heplots package for R, as well as their extension, for example from two to three dimensions and by scaling hypothesis ellipses and ellipsoids in a natural manner relative to error.