Nonlinear Link Function Research Articles

Incompatibility of Model Specifications in Generalized Linear Models

Summary The choice of regression models can vary depending on a study’s objectives and the characteristics of the dataset. Yet it is crucial to recognize that the properties of conditional expectation impose a natural consistency requirement on the formulation of these models. This paper endeavors to demonstrate that, even under the most favorable assumptions concerning the covariates’ structure, inconsistent specifications can arise in generalized linear models, particularly when nonlinear link functions are employed.

Efficient Algorithms for Non-gaussian Single Index Models with Generative Priors

In this work, we focus on high-dimensional single index models with non-Gaussian sensing vectors and generative priors. More specifically, our goal is to estimate the underlying signal from i.i.d. realizations of the semi-parameterized single index model, where the underlying signal is contained in (up to a constant scaling) the range of a Lipschitz continuous generative model with bounded low-dimensional inputs, the sensing vector follows a non-Gaussian distribution, the noise is a random variable that is independent of the sensing vector, and the unknown non-linear link function is differentiable. Using the first- and second-order Stein's identity, we introduce efficient algorithms to obtain estimated vectors that achieve the near-optimal statistical rate. Experimental results on image datasets are provided to support our theory.

Nonlinear weight learning model for incipient fault detection and degradation modelling and its interpretability for fault diagnosis

Optimization-based weight learning approaches that incorporate multiple sensor signals or vibration spectral components have demonstrated their effectiveness in monitoring mechanical systems. However, the current weight learning methods are limited by their reliance on linear fusion functions. The exploration of nonlinear link functions, particularly in the context of fusing spectral components in vibration spectra for health indices (HIs) construction, remains largely unexplored. Additionally, the development of formulas for interpreting HIs constructed using nonlinear fusion functions poses significant theoretical challenges. To address these limitations, this study proposes a novel nonlinear optimization-based weight learning model for constructing an HI that captures the nonlinear relationships among spectral components. This HI is designed to pinpoint fault characteristic frequencies while also highlighting other spectral lines associated with bearing degradation modelling. Furthermore, we derive formulas that enable the calculation of the contributions of frequency components to nonlinearly constructed HIs, thereby enhancing the interpretability of the model. By selecting an appropriate kernel function, subtle changes resulting from incipient faults and the underlying degradation process can be effectively captured. The effectiveness of the proposed model in detecting incipient faults, modelling degradation, and identifying fault types is validated through two run-to-failure case studies and a gearbox bearing experiment.

An efficient approach for identifying important biomarkers for biomedical diagnosis

In this paper, we explore the challenges associated with biomarker identification for diagnosis purpose in biomedical experiments, and propose a novel approach to handle the above challenging scenario via the generalization of the Dantzig selector. To improve the efficiency of the regularization method, we introduce a transformation from an inherent nonlinear programming due to its nonlinear link function into a linear programming framework under a reasonable assumption on the logistic probability range. We illustrate the use of our method on an experiment with binary response, showing superior performance on biomarker identification studies when compared to their conventional analysis. Our proposed method does not merely serve as a variable/biomarker selection tool, its ranking of variable importance provides valuable reference information for practitioners to reach informed decisions regarding the prioritization of factors for further investigations.

Variable selection in high dimensions for discrete-outcome individualized treatment rules: Reducing severity of depression symptoms.

Despite growing interest in estimating individualized treatment rules, little attention has been given the binary outcome setting. Estimation is challenging with nonlinear link functions, especially when variable selection is needed. We use a new computational approach to solve a recently proposed doubly robust regularized estimating equation to accomplish this difficult task in a case study of depression treatment. We demonstrate an application of this new approach in combination with a weighted and penalized estimating equation to this challenging binary outcome setting. We demonstrate the double robustness of the method and its effectiveness for variable selection. The work is motivated by and applied to an analysis of treatment for unipolar depression using a population of patients treated at Kaiser Permanente Washington.

An extension to the enhanced Gaussian noise model in optical bus considering various links spectrum

Physical layer impairment modeling has considerably influenced optical networking in optimizing resource allocation for fulfilling a required quality of transmission. Among these impairments, the modeling of optical fiber non-linearity has enjoyed widespread applications by vendors and network operators, giving rise to the two well established Gaussian noise (GN) and enhanced Gaussian noise (EGN) models. In this work, we introduce an extension to the EGN model for a point-to-point bus transmission, where a demand is not typically routed over a single link. More specifically, we assume that a lightpath is routed over a bus and multiple interferers occasionally add and drop along the links, resulting in a mixture of coherent and incoherent non-linear propagation. We propose a mathematical formulation in the optical bus scenario, taking into account the spectrum variation of the links along the bus. To more accurately address their mutual statistical correlation in non-linear interference estimation, we make use of extra lightpath information. Along with presenting the new formulation, we modify the non-linear link function to include the spectrum variation of a lightpath links. The exploited optical bus topology is capable of approximating a real network lightpaths to enable design optimization of optical networks. Our numerical results show approximately 0.4 dB and 0.6 dB improvements versus the GN and EGN models in long bus scenarios with moderate number of channels and lightpaths, and around 0.3 dB minimum improvement.

Estimating functional single index models with compact support

AbstractThe functional single index models are widely used to describe the nonlinear relationship between a scalar response and a functional predictor. The conventional functional single index model assumes that the coefficient function is nonzero in the entire time domain. In other words, the functional predictor always has a nonzero effect on the response all the time. We propose a new compact functional single index model, in which the coefficient function is only nonzero in a subregion. We also propose an efficient method that can simultaneously estimate the nonlinear link function, the coefficient function and also the nonzero region of the coefficient function. Hence, our method can identify the region in which the functional predictor is related to the response. Our method is illustrated by an application example in which the total number of daily bike rentals is predicted based on hourly temperature data. The finite sample performance of the proposed method is investigated by comparing it to the conventional functional single index model in a simulation study

Semiparametric multiple inflation count model with application to a smoking cessation study.

Count data are observed by practitioners across various fields. Often, a substantially large proportion of one or some values causes extra variation and may lead to a particular case of mixed structured data. In these cases, a standard count model may lead to poor inference of the parameters involved because of its inability to account for extra variation. Furthermore, we hypothesize a possible nonlinear relationship of a continuous covariate with the logarithm of the mean count and with the probability of belonging to an inflated category. We propose a semiparametric multiple inflation Poisson (MIP) model that considers the two nonlinear link functions. We develop a sieve maximum likelihood estimator (sMLE) for the regression parameters of interest. We establish the asymptotic behavior of the sMLE. Simulations are conducted to evaluate the performance of the proposed sieve MIP (sMIP). Then, we illustrate the methodology on data from a smoking cessation study. Finally, some remarks and opportunities for future research conclude the article.

Semi-supervised empirical risk minimization: Using unlabeled data to improve prediction

We present a general methodology for using unlabeled data to design semi supervised learning (SSL) variants of the Empirical Risk Minimization (ERM) learning process. Focusing on generalized linear regression, we analyze of the effectiveness of our SSL approach in improving prediction performance. The key ideas are carefully considering the null model as a competitor, and utilizing the unlabeled data to determine signal-noise combinations where SSL outperforms both supervised learning and the null model. We then use SSL in an adaptive manner based on estimation of the signal and noise. In the special case of linear regression with Gaussian covariates, we prove that the non-adaptive SSL version is in fact not capable of improving on both the supervised estimator and the null model simultaneously, beyond a negligible O(1∕n) term. On the other hand, the adaptive model presented in this work, can achieve a substantial improvement over both competitors simultaneously, under a variety of settings. This is shown empirically through extensive simulations, and extended to other scenarios, such as non-Gaussian covariates, misspecified linear regression, or generalized linear regression with non-linear link functions.

The investment development path and human development: Is there a nexus?

To investigate the nexus between the investment development path (IDP) and human development, we modelled the human development (human development index, HDI) using generalized estimation equations and accounted for the unit interval property using non-linear link functions. The gross domestic product per capita was used as the proxy for the IDP. Using panel data of 2,568 observations from 135 countries for 1990–2019, we found a strong and positive effect of IDP on HDI. Thus, there is a nexus between IDP and human development. The 4% elasticity of the IDP, confirms the broader scope of the HDI than the IDP in capturing development. Coupled with the 60% correlation between the IDP variable and HDI, aside from serving as a proxy for each other, these could also be instruments for each other provided they are uncorrelated with the error term in the model under consideration. Trade, infrastructure, and human capital were found to enhance human development. These offer policy options for enhancing human development.

Power calculation for analyses of cross-sectional stepped-wedge cluster randomized trials with binary outcomes via generalized estimating equations.

Power calculation for stepped-wedge cluster randomized trials (SW-CRTs) presents unique challenges, beyond those of standard parallel cluster randomized trials, due to the need to consider temporal within cluster correlations and background period effects. To date, power calculation methods specific to SW-CRTs have primarily been developed under a linear model. When the outcome is binary, the use of a linear model corresponds to assessing a prevalence difference; yet trial analysis often employs a nonlinear link function. We propose power calculation methods for cross-sectional SW-CRTs under a logistic model fitted by generalized estimating equations. Firstly, under an exchangeable correlation structure, we show the power based on a logistic model is lower than that from assuming a linear model in the absence of period effects. We then evaluate the impact of background prevalence changes over time on power. To allow the correlation among outcomes in the same cluster to change over time and with treatment status, we generalize the methods to more complex correlation structures. Our simulation studies demonstrate that the proposed power calculation methods perform well with the model-based variance under the true correlation structure and reveal that a working independence structure can result in substantial efficiency loss, while a working exchangeable structure performs well even when the underlying correlation structure deviates from exchangeable. An extension to our methods accounts for variable cluster sizes and reveals that unequal cluster sizes have a modest impact on power. We illustrate the approaches by application to a quality of care improvement trial for acute coronary syndrome.

Multimodal Data Fusion in High-Dimensional Heterogeneous Datasets Via Generative Models

The commonly used latent space embedding techniques, such as Principal Component Analysis, Factor Analysis, and manifold learning techniques, are typically used for learning effective representations of homogeneous data. However, they do not readily extend to heterogeneous data that are a combination of numerical and categorical variables, e.g., arising from linked GPS and text data. In this paper, we are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. The learned generative model provides latent unified representations that capture the factors common to the multiple dimensions of the data, and thus enable fusing multimodal data for various machine learning tasks. Following a Bayesian approach, we propose a general framework that combines disparate data types through the natural parameterization of the exponential family of distributions. To scale the model inference to millions of instances with thousands of features, we use the Laplace-Bernstein approximation for posterior computations involving nonlinear link functions. The proposed algorithm is presented in detail for the commonly encountered heterogeneous datasets with real-valued (Gaussian) and categorical (multinomial) features. Experiments on two high-dimensional and heterogeneous datasets (NYC Taxi and MovieLens-10M) demonstrate the scalability and competitive performance of the proposed algorithm on different machine learning tasks such as anomaly detection, data imputation, and recommender systems.

A marginal estimate for the overall treatment effect on a survival outcome within the joint modeling framework.

Joint models for longitudinal and survival data are increasingly used and enjoy a wide range of application areas. In this article, we focus on the application of joint models on clinical trial data with special interest in the treatment effect on the survival outcome. Within a joint model, the estimated treatment effect on the survival outcome is an aggregate comprising the indirect treatment effect through the longitudinal outcome and the direct treatment effect on the survival outcome. This overall treatment effect is, however, conditional on random effects, and therefore has a subject‐specific interpretation. The conditional interpretation arises from the shared random effects between the longitudinal and survival process in combination with the nonlinear link function of the survival model. The overall treatment effect is, therefore, not valid for population‐based inference, which is the goal for most clinical trials. We propose a method to obtain a marginal estimate of the overall treatment effect on the survival outcome in a joint model. Additionally, we extend our proposal to allow for different parameterizations for the association between the longitudinal and survival outcome. The proposed method is demonstrated on data of a clinical study on the effect of synbiotic on the gut microbiota of cesarean delivered infants, where we estimate the marginal overall treatment effect on the risk of eczema or atopic dermatitis using longitudinal information on fecal bifidobacteria.

A Weighted Covariate Balancing Method of Estimating Causal Effects in Case-Control Studies

Propensity score methods have dominated the estimation of treatment effects based on observational data and particularly in the health and medical sciences. We propose a weighting method based on rank-based Mahalanobis distance, namely the covariate balancing rank-based Mahalanobis distance method, to estimate causal effects for observational data. Using Monte Carlo simulations, under different data structures and type of outcome variables, the proposed method is shown to have better performance, in terms of bias reduction and treatment effect estimation. Specifically, under the generalized linear model framework, we simulated datasets based on the Lalonde-PSID study, for linear link function; while datasets were simulated based on the Lindner study, for non-linear link functions. We further apply the proposed method to data extracted from the Nigeria Demographic Health Survey (2013), to investigate the effect of educational exposure on ideal family size among married couples in Nigeria. The proposed method is a viable alternative method that can improve covariates balance, bias reduction, and efficient estimation of treatment effects.

Fast Algorithms for Demixing Sparse Signals From Nonlinear Observations

We study the problem of demixing a pair of sparse signals from noisy, nonlinear observations of their superposition. Mathematically, we consider a nonlinear signal observation model, $y_i = g(a_i^Tx) + e_i, \ i=1,\ldots,m$, where $x = \Phi w+\Psi z$ denotes the superposition signal, $\Phi$ and $\Psi$ are orthonormal bases in $\mathbb{R}^n$, and $w, z\in\mathbb{R}^n$ are sparse coefficient vectors of the constituent signals, and $e_i$ represents the noise. Moreover, $g$ represents a nonlinear link function, and $a_i\in\mathbb{R}^n$ is the $i$-th row of the measurement matrix, $A\in\mathbb{R}^{m\times n}$. Problems of this nature arise in several applications ranging from astronomy, computer vision, and machine learning. In this paper, we make some concrete algorithmic progress for the above demixing problem. Specifically, we consider two scenarios: (i) the case when the demixing procedure has no knowledge of the link function, and (ii) the case when the demixing algorithm has perfect knowledge of the link function. In both cases, we provide fast algorithms for recovery of the constituents $w$ and $z$ from the observations. Moreover, we support these algorithms with a rigorous theoretical analysis, and derive (nearly) tight upper bounds on the sample complexity of the proposed algorithms for achieving stable recovery of the component signals. We also provide a range of numerical simulations to illustrate the performance of the proposed algorithms on both real and synthetic signals and images.

Partially linear single-index proportional hazards model with current status data

A partially linear single-index proportional hazards model with current status data is introduced, where the cumulative hazard function is assumed to be nonparametric and a nonlinear link function is assumed to take a parametric spline function. Efficient estimation and effective algorithm are established. Polynomial spline smoothing is invoked for the estimation of the cumulative baseline hazard function with monotonicity constraint on the functional, while a simultaneous sieve maximum likelihood (SML) estimation is proposed to estimate regression parameters. The proposed SML estimator for the parameter vector is shown to be asymptotically normal and semiparametric efficient. The spline estimator of the functional of the cumulative hazard function is shown to achieve the optimal nonparametric rate of convergence. A simulation study is conducted to examine the finite sample performance of the proposed estimators and algorithm, and an analysis of renal function recovery data is presented.

Generalized linear mixed models in hearing science

Linear mixed-effects models (LMEs), by virtue of allowing for the quantification of both random effects (e.g., associated with sampling of subjects) and the effects of fixed predictor variables, provide a powerful statistical modeling framework that can accommodate a large number of common experimental designs used in hearing science. Examples of designs for which LMEs are suitable include repeated-measures designs, longitudinal studies, and multilevel designs. An extensive literature of robust methods exists for fitting such models for normally distributed responses. On the other hand, often, data in hearing science experiments are binary (e.g., detected or missed), proportions, or counts (number of correct responses) and are better modeled using non-normal distributions (e.g., binomial). Generalized linear mixed models (GLMMs) provide a framework to retain the advantages of LMEs in modeling random effects by modeling the non-normal response variables using LMEs through (nonlinear) link functions. In thi...

Identifying predictive markers for personalized treatment selection.

It is now well recognized that the effectiveness and potential risk of a treatment often vary by patient subgroups. Although trial-and-error and one-size-fits-all approaches to treatment selection remain a common practice, much recent focus has been placed on individualized treatment selection based on patient information (LaThangue and Kerr, 2011; Ong etal., 2012). Genetic and molecular markers are becoming increasingly available to guide treatment selection for various diseases including HIV and breast cancer (Mallal etal., 2008; Zujewski and Kamin, 2008). In recent years, many statistical procedures for developing individualized treatment rules (ITRs) have been proposed. However, less focus has been given to efficient selection of predictive biomarkers for treatment selection. The standard Wald test for interactions between treatment and the set of markers of interest may not work well when the marker effects are nonlinear. Furthermore, interaction-based test is scale dependent and may fail to capture markers useful for predicting individualized treatment differences. In this article, we propose to overcome these difficulties by developing a kernel machine (KM) score test that can efficiently identify markers predictive of treatment difference. Simulation studies show that our proposed KM-based score test is more powerful than the Wald test when there is nonlinear effect among the predictors and when the outcome is binary with nonlinear link functions. Furthermore, when there is high-correlation among predictors and when the number of predictors is not small, our method also over-performs Wald test. The proposed method is illustrated with two randomized clinical trials.

Generalized varying index coefficient models

In this paper, we propose a new semiparametric model called generalized varying index coefficient models (GVICMs). The GVICM is a generalization of the varying index coefficient model (VICM) proposed by Ma and Song (2014), by allowing for non-Gaussian data and nonlinear link functions. The GVICM serves as a good tool for modeling and assessing nonlinear interaction effects between grouped covariates and the response variable. Our main goal is to estimate the unknown parameters and nonparametric functions. Firstly, we develop a profile spline quasi-likelihood estimation procedure to estimate the regression parameters and nonparametric coefficients in which the nonparametric functions are approximated by B-spline basis functions. Under some mild conditions, we establish asymptotic normalities of parameter estimations as well as the convergence rates of nonparametric estimators. Secondly, we develop a two-step spline backfitted local quasi-likelihood estimation for achieving asymptotic distribution of nonparametric function. Moreover, the oracle property of the nonparametric estimator is also established by utilizing the two-step estimation approach. Simulation study and a set of real data are carried out to investigate the performance of the proposed method.

Trimmed likelihood estimators for lifetime experiments and their influence functions

We study the behaviour of trimmed likelihood estimators (TLEs) for lifetime models with exponential or lognormal distributions possessing a linear or nonlinear link function. In particular, we investigate the difference between two possible definitions for the TLE, one called original trimmed likelihood estimator (OTLE) and one called modified trimmed likelihood estimator (MTLE) which is the finite sample version of a form for location and linear regression used by Bednarski and Clarke [Trimmed likelihood estimation of location and scale of the normal distribution. Aust J Statist. 1993;35:141–153, Asymptotics for an adaptive trimmed likelihood location estimator. Statistics. 2002;36:1–8] and Bednarski et al. [Adaptive trimmed likelihood estimation in regression. Discuss Math Probab Stat. 2010;30:203–219]. The OTLE is always an MTLE but the MTLE may not be unique even in cases where the OLTE is unique. We compare especially the functional forms of both types of estimators, characterize the difference with the implicit function theorem and indicate situations where they coincide and where they do not coincide. Since the functional form of the MTLE has a simpler form, we use it then for deriving the influence function, again with the help of the implicit function theorem. The derivation of the influence function for the functional form of the OTLE is similar but more complicated.