Penalized Maximum Likelihood Research Articles

Bayesian inference for nonstationary marginal extremes

We propose a simple piecewise model for a sample of peaks‐over‐threshold, nonstationary with respect to multidimensional covariates, and estimate it using a carefully designed and computationally efficient Bayesian inference. Model parameters are themselves parameterized as functions of covariates using penalized B‐spline representations. This allows detailed characterization of non‐stationarity extreme environments. The approach gives similar inferences to a comparable frequentist penalized maximum likelihood method, but is computationally considerably more efficient and allows a more complete characterization of uncertainty in a single modelling step. We use the model to quantify the joint directional and seasonal variation of storm peak significant wave height at a northern North Sea location and estimate predictive directional–seasonal return value distributions necessary for the design and reliability assessment of marine and coastal structures.

Model Selection and Minimax Estimation in Generalized Linear Models

We consider model selection in generalized linear models (GLM) for high-dimensional data and propose a wide class of model selection criteria based on penalized maximum likelihood with a complexity penalty on the model size. We derive a general nonasymptotic upper bound for the Kullback–Leibler risk of the resulting estimators and establish the corresponding minimax lower bounds for the sparse GLM. For the properly chosen (nonlinear) penalty, the resulting penalized maximum likelihood estimator is shown to be asymptotically minimax and adaptive to the unknown sparsity. We also discuss possible extensions of the proposed approach to model selection in the GLM under additional structural constraints and aggregation.

Sparse factor regression via penalized maximum likelihood estimation

In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.

Gradient shimming based on regularized estimation for B0-field and shim functions

Mapping B0-field and shim functions spatially is a crucial step in the gradient shimming. The conventional estimation method used in the phase difference imaging technique takes no account for noise and T2∗ effects, and is prone to create noisy and distorted field maps. This paper describes a new gradient shimming based on the regularized estimation for B0-field and shim functions. Based on a statistical model, the B0-field and shim function maps are estimated by a Penalized Maximum Likelihood method that minimizes two regularized least-squares cost functions, respectively. The first cost function of B0-field exploits the two facts that the noise in the phase difference measurements is Gaussian and the B0-field maps tend to be smooth. And the other one adds an additional fact that each shim function corresponds to a given spherical harmonic of the magnetic field. Significant improvements in the quality of field mapping and in the final shimming results are demonstrated through computer simulations as well as experiments, especially when the magnetic field homogeneity is poor.

On estimation of the diagonal elements of a sparse precision matrix

In this paper, we present several estimators of the diagonal elements of the inverse of the covariance matrix, called precision matrix, of a sample of independent and identically distributed random vectors. The main focus is on the case of high dimensional vectors having a sparse precision matrix. It is now well understood that when the underlying distribution is Gaussian, the columns of the precision matrix can be estimated independently form one another by solving linear regression problems under sparsity constraints. This approach leads to a computationally efficient strategy for estimating the precision matrix that starts by estimating the regression vectors, then estimates the diagonal entries of the precision matrix and, in a final step, combines these estimators for getting estimators of the off-diagonal entries. While the step of estimating the regression vector has been intensively studied over the past decade, the problem of deriving statistically accurate estimators of the diagonal entries has received much less attention. The goal of the present paper is to fill this gap by presenting four estimators—that seem the most natural ones—of the diagonal entries of the precision matrix and then performing a comprehensive empirical evaluation of these estimators. The estimators under consideration are the residual variance, the relaxed maximum likelihood, the symmetry-enforced maximum likelihood and the penalized maximum likelihood. We show, both theoretically and empirically, that when the aforementioned regression vectors are estimated without error, the symmetry-enforced maximum likelihood estimator has the smallest estimation error. However, in a more realistic setting when the regression vector is estimated by a sparsity-favoring computationally efficient method, the qualities of the estimators become relatively comparable with a slight advantage for the residual variance estimator.

A comparative study of methods for logistic regression with separated or nearly separated data

Objective To introduce a comparative study of methods for Logistic regression with separated or nearly separated data. Methods Human immunodeficiency virus (HIV) infection and its influential factors, 75 drug users were analyzed by maximum Likelihood estimate, exact Logistic regression and Firth's penalized maximum Likelihood estimate, then results of the three methods were compared. Results Both of penalized maximum Likelihood estimation and exact Logistic regression produced valid parameter estimates and confidence interval of the latter was wider. The results of penalized maximum Likelihood estimation and exact Logistic regression showed that race was significantly associated with HIV infection, and Yi people with HIV infection was higher than Han people. Conclusions The maximum Likelihood estimate for separated or nearly separated data is invalid. However, exact Logistic regression and Firth's penalized maximum Likelihood estimate can get valid estimates. Since exact Logistic regression have problems of complex calculations, over-conditioning and conditional distributions degenerated, Firth's penalized maximum Likelihood estimate is recommended to Logistic regression with separated or nearly separated data.

Penalized: AMATLABToolbox for Fitting Generalized Linear Models with Penalties

penalized is a flexible, extensible, and efficient MATLAB toolbox for penalized maximum likelihood. penalized allows you to fit a generalized linear model (gaussian, logistic, poisson, or multinomial) using any of ten provided penalties, or none. The toolbox can be extended by creating new maximum likelihood models or new penalties. The toolbox also includes routines for cross-validation and plotting.

Penalized maximum likelihood reconstruction for x-ray differential phase-contrast tomography.

The purpose of this work is to propose a cost function with regularization to iteratively reconstruct attenuation, phase, and scatter images simultaneously from differential phase contrast (DPC) acquisitions, without the need of phase retrieval, and examine its properties. Furthermore this reconstruction method is applied to an acquisition pattern that is suitable for a DPC tomographic system with continuously rotating gantry (sliding window acquisition), overcoming the severe smearing in noniterative reconstruction. We derive a penalized maximum likelihood reconstruction algorithm to directly reconstruct attenuation, phase, and scatter image from the measured detector values of a DPC acquisition. The proposed penalty comprises, for each of the three images, an independent smoothing prior. Image quality of the proposed reconstruction is compared to images generated with FBP and iterative reconstruction after phase retrieval. Furthermore, the influence between the priors is analyzed. Finally, the proposed reconstruction algorithm is applied to experimental sliding window data acquired at a synchrotron and results are compared to reconstructions based on phase retrieval. The results show that the proposed algorithm significantly increases image quality in comparison to reconstructions based on phase retrieval. No significant mutual influence between the proposed independent priors could be observed. Further it could be illustrated that the iterative reconstruction of a sliding window acquisition results in images with substantially reduced smearing artifacts. Although the proposed cost function is inherently nonconvex, it can be used to reconstruct images with less aliasing artifacts and less streak artifacts than reconstruction methods based on phase retrieval. Furthermore, the proposed method can be used to reconstruct images of sliding window acquisitions with negligible smearing artifacts.

Implications of PCR and ELISA results on the routes of bulk-tank contamination with Mycobacterium avium ssp. paratuberculosis

Mycobacterium avium ssp. paratuberculosis (MAP), the etiologic agent of Johne’s disease in dairy cattle, may enter the bulk tank via environmental contamination or direct excretion into milk. Traditionally, diagnostics to identify MAP in milk target either MAP antibodies (by ELISA) or the organism itself (by culture or PCR). High ELISA titers may be directly associated with excretion of MAP into milk but only indirectly linked to environmental contamination of the bulk tank. Patterns of bulk-milk ELISA and bulk-milk PCR results could therefore provide insight into the routes of contamination and level of infection or environmental burden. Coupled with questionnaire responses pertaining to management, the results of these diagnostic tests could reveal correlations with herd characteristics or on-farm practices that distinguish herds with high and low environmental bulk-tank MAP contamination. A questionnaire on hygiene, management, and Johne’s specific parameters was administered to 292 dairy farms in New York, Oregon, and Wisconsin. Bulk-tank samples were collected from each farm for evaluation by real-time PCR and ELISA. Before DNA extraction and testing of the unknown samples, bulk-milk template preparation was optimized with respect to parameters such as MAP fractionation patterns and lysis. Two regression models were developed to explore the relationships among bulk-tank PCR, ELISA, environmental predictors, and herd characteristics. First, ELISA optical density (OD) was designated as the outcome in a linear regression model. Second, the log odds of being PCR positive in the bulk tank were modeled using binary logistic regression with penalized maximum likelihood. The proportion of PCR-positive bulk tanks was highest for New York and for organic farms, providing a clue as to the geographical patterns of MAP-positive bulk-tank samples and relationship to production type. Bulk-milk PCR positivity was also higher for large relative to small herds. The models revealed that bulk-milk PCR result could predict ELISA OD, with PCR-positive results corresponding to high bulk-milk ELISA titers. Similarly, ELISA was a predictor of PCR result, although the association was stronger for organic farms. Despite agreement between high bulk-milk ELISA titers and positive PCR results, a large proportion of high ELISA farms had PCR-negative bulk tanks, suggesting that farms are able to maintain satisfactory hygiene and management despite a presence of MAP in these herds.

Theoretical Analysis of Penalized Maximum-Likelihood Patlak Parametric Image Reconstruction in Dynamic PET for Lesion Detection.

Detecting cancerous lesions is a major clinical application of emission tomography. In a previous work, we studied penalized maximum-likelihood (PML) image reconstruction for lesion detection in static PET. Here we extend our theoretical analysis of static PET reconstruction to dynamic PET. We study both the conventional indirect reconstruction and direct reconstruction for Patlak parametric image estimation. In indirect reconstruction, Patlak parametric images are generated by first reconstructing a sequence of dynamic PET images, and then performing Patlak analysis on the time activity curves (TACs) pixel-by-pixel. In direct reconstruction, Patlak parametric images are estimated directly from raw sinogram data by incorporating the Patlak model into the image reconstruction procedure. PML reconstruction is used in both the indirect and direct reconstruction methods. We use a channelized Hotelling observer (CHO) to assess lesion detectability in Patlak parametric images. Simplified expressions for evaluating the lesion detectability have been derived and applied to the selection of the regularization parameter value to maximize detection performance. The proposed method is validated using computer-based Monte Carlo simulations. Good agreements between the theoretical predictions and the Monte Carlo results are observed. Both theoretical predictions and Monte Carlo simulation results show the benefit of the indirect and direct methods under optimized regularization parameters in dynamic PET reconstruction for lesion detection, when compared with the conventional static PET reconstruction.

Unsupervised learning of regression mixture models with unknown number of components

ABSTRACTWe propose a new unsupervised learning algorithm to fit regression mixture models with unknown number of components. The developed approach consists in a penalized maximum likelihood estimation carried out by a robust expectation–maximization (EM)-like algorithm. We derive it for polynomial, spline, and B-spline regression mixtures. The proposed learning approach is unsupervised: (i) it simultaneously infers the model parameters and the optimal number of the regression mixture components from the data as the learning proceeds, rather than in a two-fold scheme as in standard model-based clustering using afterward model selection criteria, and (ii) it does not require accurate initialization unlike the standard EM for regression mixtures. The developed approach is applied to curve clustering problems. Numerical experiments on simulated and real data show that the proposed algorithm performs well and provides accurate clustering results, and confirm its benefit for practical applications.

Evictions and suicide: a follow-up study of almost 22 000 Swedish households in the wake of the global financial crisis

BackgroundMillions of families across the world are evicted every year. However, very little is known about the impact that eviction has on their lives. This lack of knowledge is also...

Efficient estimation of approximate factor models via penalized maximum likelihood

We study an approximate factor model in the presence of both cross sectional dependence and heteroskedasticity. For efficient estimations it is essential to estimate a large error covariance matrix. We estimate the common factors and factor loadings based on maximizing a Gaussian quasi-likelihood, through penalizing a large covariance sparse matrix. The weighted ℓ1 penalization is employed. While the principal components (PC) based methods estimate the covariance matrices and individual factors and loadings separately, they require consistent estimation of residual terms. In contrast, the penalized maximum likelihood method (PML) estimates the factor loading parameters and the error covariance matrix jointly. In the numerical studies, we compare PML with the regular PC method, the generalized PC method (Choi 2012) combined with the thresholded covariance matrix estimator (Fan et al. 2013), as well as several related methods, on their estimation and forecast performances. Our numerical studies show that the proposed method performs well in the presence of cross-sectional dependence and heteroskedasticity.

Multigrid reconstruction with block-iterative updates for breast tomosynthesis.

The authors wish to evaluate the possible advantages of using a multigrid approach to maximum-a-posteriori reconstruction in digital breast tomosynthesis together with block-iterative updates in the form of either plane-by-plane updates or ordered subsets. The authors previously developed a penalized maximum likelihood reconstruction algorithm with resolution model dedicated to breast tomosynthesis [K. Michielsen et al., "Patchwork reconstruction with resolution modeling for digital breast tomosynthesis," Med. Phys. 40, 031105 (10pp.) (2013)]. This algorithm was extended with ordered subsets and multigrid updates, and the effects on the convergence and on limited angle artifact appearance were evaluated on a mathematical phantom and patient data. To ensure a fair comparison, the analysis was performed at the same computational cost for all methods. To assess convergence and artifact creation in the phantom reconstructions, the authors looked at posterior likelihood, sum of squared residuals, contrast of identical calcifications at different positions, and the standard deviation between the contrasts of these calcifications. For the patient cases, the authors calculated posterior likelihood, measured the signal difference to noise ratio of subtle microcalcifications, and visually evaluated the reconstructions. The authors selected multigrid sequences scoring in the best 10% of the four evaluated parameters, except for the reconstructions with subsets where a low standard deviation of the contrast was incompatible with the three other parameters. In further evaluation of phantom reconstructions from noisy data and patient data, the authors found improved convergence and a reduction in artifacts for our chosen multigrid reconstructions compared to the single grid reconstructions with equivalent computational cost, although there was a diminishing return for an increasing number of subsets. Multigrid reconstruction improves upon reconstruction with a fixed grid when evaluated at a fixed computational cost. For multigrid reconstruction, using plane-by-plane updates or applying ordered subsets resulted in similar performance.

Mixed Influence of Electronic Health Record Implementation on Diabetes Order Patterns for Michigan Medicaid Adults.

These 2011-2013 analyses examined the authors' hypothesis that relative diabetes care order changes would be measured after electronic health record (EHR) implementation for 291 Medicaid adults who received all of their office-based care at one midwestern federally qualified health center (FQHC) over a 24-month period (n = 2727 encounters, 2489 claims). Beneficiary sociodemographic, clinical, and claims data were validated with clinic EHR and state Medicaid claims linked to providers' national identifier numbers. Overall pre-post order rate comparisons, and a series of controlled within group binary logistic models were conducted under penalized maximum likelihood estimation terms. After EHR implementation, both the overall order rates and odds ratios of per beneficiary hemoglobin A1C (HbA1C) orders increased significantly (ie, from mean of 0.65 [SD = 1.19] annual tests to 0.96 tests [SD = 1.24] [P < .001]). Although the overall post-EHR order rates of dilated eye exams and microalbumin urine tests appeared fairly stable, the odds of eye exam orders being placed at the claims level decreased significantly (OR = 0.774, P = .0030). These mixed results provide evidence of the varied diabetes care ordering patterns likely seen from increased office use of EHR technologies. The authors attempt to explain these post-EHR differences (or lack of) that generally resemble some of the authors' results from another funded project. Ideally, these findings provide Medicaid and health care officials with a more realistic indication of how EHRs may, or may not, influence diabetes care ordering patterns for vulnerable lower-income primary health care consumers.

Clinico-pathological and transcriptomic determinants of SLFN11 expression in invasive breast carcinoma

SLFN11 is a putative DNA/RNA helicase we discovered as causally associated with sensitivity to DNA damaging agents, such as platinum salts, topoisomerase I and II inhibitors, and other alkylators in the NCI-60 panel of cancer cell lines [1]. Later, SLFN11 was identified as an early interferon response gene, in association with HIV infection [2]. Here we assessed SLFN11 determinants in a gene expression meta-set of 5,061 breast cancer patients annotated with clinical data and multigene signatures obtained with the package genefu [3]. By correlation analysis, we found 537 transcripts above the 95th percentile of Pearson’s coefficients with SLFN11, identifying “immune response”, “lymphocyte activation”, and “T cell activation” as top Gene Ontology enriched processes [4]. Through multiple correspondence analysis, we discovered a subgroup of patients characterized by high SLFN11 levels, ER negativity, basal phenotype, elevated CD3D, STAT1 signature [5], and young age. Fitting a penalized maximum likelihood lasso regression model [6], we found a strong multivariable association of SLN11 with the stroma 1 and stroma 2 signatures [7,8], associated with basal cancer and response to chemotherapy in ER- tumors. Finally, using Cox proportional hazard regression, ER-, high proliferation, high SLFN11 patients undergoing chemotherapy treatment showed a significantly longer disease-free interval than other patient categories included in our model.

Score-based causal learning in additive noise models

Given data sampled from a number of variables, one is often interested in the underlying causal relationships in the form of a directed acyclic graph. In the general case, without interventions on some of the variables it is only possible to identify the graph up to its Markov equivalence class. However, in some situations one can find the true causal graph just from observational data, for example, in structural equation models with additive noise and nonlinear edge functions. Most current methods for achieving this rely on nonparametric independence tests. One of the problems there is that the null hypothesis is independence, which is what one would like to get evidence for. We take a different approach in our work by using a penalized likelihood as a score for model selection. This is practically feasible in many settings and has the advantage of yielding a natural ranking of the candidate models. When making smoothness assumptions on the probability density space, we prove consistency of the penalized maximum likelihood estimator. We also present empirical results for simulated scenarios and real two-dimensional data sets (cause–effect pairs) where we obtain similar results as other state-of-the-art methods.

Development and validation of a Clinical Index of Severe Febrile Neutropenia: A prospective multicenter study.

9617 Background: To develop and validate a prognostic score for serious neutropenic events based on characteristics available prior to starting chemotherapy Methods: FINITE is a prospective, multicenter study to assess prognostic factors and patterns of care in adult cancer patients with seemingly stable febrile neutropenia. For this analysis, we used only those clinical variables that are routinely available on Day 1 of chemotherapy in all solid tumor patients. The main outcome measure was occurrence of serious complications. A logistic regression with Penalized Maximum Likelihood Estimations (PMLE) was applied; to build an additive score, estimates were rounded as integers. To validate the model, the bootstrap bias-corrected c-index was calculated as a measure of prediction performance. A separate dataset with 216 patients was used for external validation. Results: We enrolled 1238 patients from 25 centers. The primary end point (serious complications) ensued in 161 patients (13%; 95% CI, 11.1%-14.9%) a...

Testing the geocentric axial dipole hypothesis using regional paleomagnetic intensity records from 0 to 300 ka

Absolute and relative geomagnetic paleointensity records reveal variations in geomagnetic dipole strength, either via averaging time series of virtual axial dipole moments, or through formal inversion strategies like the penalized maximum likelihood (PML) method used for the PADM2M (Paleomagnetic Axial Dipole Moment for 0–2 Ma) model. However, departures from the most basic geocentric axial dipole (GAD) structure are obvious on centennial to millennial time scales, and paleomagnetic records from igneous rocks suggest small deviations persist on million year time scales. Spatial variations in heat flow at the core–mantle boundary (inferred from large low shear velocity provinces, LLSVPs) are widely suspected to influence both the average geomagnetic field and its regional secular variation. Long term departures from a GAD configuration should be visible from regional differences in paleointensity reconstructions. We use a PML method to construct time-varying models of regional axial dipole moment (RADMs) from a combined set of absolute and relative paleointensity data, and compare results from the last 300 kyr. RADMs are created from sediment records selected from specific latitude and longitude bands. We also test whether grouping records lying above each of the 2 major LLSVPs (centered on Africa and the Pacific) produce RADMs that are distinct from those above regions lacking anomalous seismic structure. Systematic differences appear in the various regional results. In the most recent part of the record regional differences are broadly similar to the Holocene, CALS10k.1b, time-varying geomagnetic field model spanning 0–10 ka. However, lack of Southern hemisphere records prevents direct confirmation of the hemispheric asymmetry present in CALS10k.1b in both average virtual axial dipole moment and its variability. As expected, the 300 kyr RADMs exhibit greater overall temporal field variability than is seen over 0–10 ka. Average RADM is higher in the Pacific and in Equatorial regions than in the Atlantic and in mid–high latitude northern hemisphere regions. Higher average RADMs are associated with lower overall field variability and less pronounced excursional signatures. Notably, the lower variability in the Pacific sector seen here (defined by either longitude band or LLSVP location) suggests that the modern low paleosecular variation there extends over at least the past few hundred thousand years. RADMs identified with LLSVPs show systematic deviations from the non-LLSVP group of records, with distinct characteristics for the African and Pacific provinces. The African LLSVP generates more pronounced RADM minima associated with geomagnetic excursions, and in general paleointensity decreases associated with excursions occur first in the Atlantic longitude sector and over the African LLSVP.

Reduction of Metal Artifact in Single Photon-Counting Computed Tomography by Spectral-Driven Iterative Reconstruction Technique

PurposeThe exciting prospect of Spectral CT (SCT) using photon-counting detectors (PCD) will lead to new techniques in computed tomography (CT) that take advantage of the additional spectral information provided. We introduce a method to reduce metal artifact in X-ray tomography by incorporating knowledge obtained from SCT into a statistical iterative reconstruction scheme. We call our method Spectral-driven Iterative Reconstruction (SPIR).MethodThe proposed algorithm consists of two main components: material decomposition and penalized maximum likelihood iterative reconstruction. In this study, the spectral data acquisitions with an energy-resolving PCD were simulated using a Monte-Carlo simulator based on EGSnrc C++ class library. A jaw phantom with a dental implant made of gold was used as an object in this study. A total of three dental implant shapes were simulated separately to test the influence of prior knowledge on the overall performance of the algorithm. The generated projection data was first decomposed into three basis functions: photoelectric absorption, Compton scattering and attenuation of gold. A pseudo-monochromatic sinogram was calculated and used as input in the reconstruction, while the spatial information of the gold implant was used as a prior. The results from the algorithm were assessed and benchmarked with state-of-the-art reconstruction methods.ResultsDecomposition results illustrate that gold implant of any shape can be distinguished from other components of the phantom. Additionally, the result from the penalized maximum likelihood iterative reconstruction shows that artifacts are significantly reduced in SPIR reconstructed slices in comparison to other known techniques, while at the same time details around the implant are preserved. Quantitatively, the SPIR algorithm best reflects the true attenuation value in comparison to other algorithms.ConclusionIt is demonstrated that the combination of the additional information from Spectral CT and statistical reconstruction can significantly improve image quality, especially streaking artifacts caused by the presence of materials with high atomic numbers.