Multivariate Nonparametric Regression Research Articles

Discussion of: “Nonparametric regression using deep neural networks with ReLU activation function”

I would like to congratulate Johannes Schmidt–Hieber on a very interesting paper in which he considers regression functions belonging to the class of so-called compositional functions and analyzes the ability of estimators based on the multivariate nonparametric regression model of deep neural networks to achieve minimax rates of convergence. In my discussion, I will first regard such a type of result from the general viewpoint of the theoretical foundations of deep neural networks. This will be followed by a discussion from the viewpoint of expressivity, optimization and generalization. Finally, I will consider some specific aspects of the main result.

Nonparametric regression using deep neural networks with ReLU activation function

Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log n$-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constraints such as (generalized) additive models. While there is a lot of flexibility in the network architecture, the tuning parameter is the sparsity of the network. Specifically, we consider large networks with number of potential network parameters exceeding the sample size. The analysis gives some insights into why multilayer feedforward neural networks perform well in practice. Interestingly, for ReLU activation function the depth (number of layers) of the neural network architectures plays an important role, and our theory suggests that for nonparametric regression, scaling the network depth with the sample size is natural. It is also shown that under the composition assumption wavelet estimators can only achieve suboptimal rates.

Estimation of Matrix Variance-Covariance on Nonparametric Regression Spline Truncated for Longitudinal Data

Regression analysis is a statistical analysis used to determine the pattern of relationships between predictor variables and response variables. There are two models estimation approaches in regression analysis, namely parametric regression and nonparametric regression. The parametric regression approach is used if the shape of the regression curve is known. In cases with unknown relationship patterns, the development is done using nonparametric regression. Nonparametric regression is a model estimation method which is based on an approach that is not bound by certain assumptions of the regression curve shape. One of the most commonly used nonparametric regression curve estimates is the spline approach. Spline is one of the nonparametric approaches that is often used and has a very good ability to handle data characters that are smooth as well as data whose behavior changes at certain sub-intervals. Theoretically, the spline approach is not only able to estimate nonparametric regression curves for cross section data, but can also be developed for longitudinal data. Longitudinal data is a combination of cross section data and time series data where the longitudinal data between subjects are mutually independent, but between observations are dependent. This study aims to find the estimation curves of spline truncated multivariable nonparametric regression for longitudinal data using the Weighted Least Square (WLS) method and to find the estimation of variance-covariance matrix using the Maximum Likelihood Estimate (MLE) method.

Assessing the effects of the nursing education reform on the educational environment in Tajikistan: a repeated cross-sectional analysis

BackgroundA well-functioning education system for family nurses is a priority of the primary health care reform in Tajikistan. In 2015/2016, a baseline study was carried out to measure the educational environment at two nursing colleges, in Kulob and Dushanbe. Building on the study’s recommendations, the educational reform has addressed several key issues to improve the educational environment among nursing students with a focus on strengthening competency-based learning and clinical skills. A follow-up study was carried out in late 2018 to comparatively analyse progress in the educational environment against the baseline and assess potential impacts of tailored interventions.MethodA repeated cross-sectional survey involving 1239 students was applied to measure changes in the educational environment between 2015/2016 (baseline) and 2018 (endline) using the Dundee Ready Education Environment Measure (DREEM). We compared mean scores over time using Welch’s two sample t-test and the Wilcoxon-Mann-Whitney test. Single items have additionally been analysed using critical threshold (flags) for mean scores, and the percentage of answers falling above or below predefined values. A multivariate non-parametric regression was applied to control for confounding factors. Internal consistency was examined using Cronbach’s α.ResultsCronbach’s α for overall scores ranged between 0.87 and 0.89. Between 2015/2016 and 2018 the perceived educational environment improved with an increase of the mean total DREEM score from 131.8 to 146.9 in Dushanbe and from 134.9 to 151.2 in Kulob. Mean comparisons and multivariate regression revealed a significant increase of all sub-scores between 2015/2016 and 2018 with students’ social self-perception exhibiting the smallest progress. Despite the general improvements observed, analysis at the level of single items revealed persistent weaknesses including a lack of competency-based learning and stress.ConclusionsThe education environment has improved in several important ways between 2015/2016 and 2018 which points to a likely positive contribution of the nursing education reform. This progress notwithstanding, there is still notable room for further improvement. Targeted efforts aimed at a better organization of practical trainings, improved didactical competences of teachers, and support structures for lonely and stressed students still seem to be lacking for the achievement of a good nursing education system in Tajikistan.

Hydrodynamic Conditions Effects on Soft-Bottom Subtidal Nearshore Benthic Community Structure and Distribution

This study assesses the impacts of wave action and freshwater outflow on soft-bottom benthic macrofauna spatial distribution and temporal stability along the highly exposed French Basque coast. Sediment characteristics and macrofauna abundance have been seasonally investigated during two years for nine stations located at the same (6 m) depth and spread over three subtidal sites showing distinct exposure levels. Wave climate has been determined through an operational numerical model. A total of 121 taxa were recorded, gathered in three main faunal assemblages, as revealed by classification and ordination methods. Non-parametric multivariate multiple regression (distance-based linear model) showed that the variations in macrofaunal distribution can be explained by hydrodynamic conditions. Wave exposure strongly linked to estuarine inputs were the most relevant abiotic factors influencing distributional patterns and functional structure as described by biological traits analysis. Despite the influence of these abiotic variables affecting sedimentary dynamics, seasonal stability was observed in macrobenthic assemblages composition suggesting an ability to recover from natural disturbances such as (e.g.) winter storms. In this way, these results provide baseline knowledge for future ecosystem and resource management in shallow subtidal areas strongly exposed to swell and freshwater outflow where soft-bottom macrozoobenthic communities are less frequently studied.

Mental Illness and BRCA1/2 Genetic Testing Intention Among Multiethnic Women Undergoing Screening Mammography.

To examine associations between patient-reported mental illness diagnosis and symptoms and BRCA1/2 genetic testing intention among women undergoing screening mammography. 100 multiethnic women of lower socioeconomic status who were undergoing mammography screening and met family history criteria for BRCA1/2 genetic testing. Descriptive and bivariate nonparametric statistics and multivariate logistic regression were used to examine associations between mental illness and genetic testing intention. Variables were anxiety, depression, patient-reported mental illness diagnosis and symptoms, and testing intention. Prevalence rates of mental illness symptoms were 36% for clinically significant depression and 36% for anxiety. Although 76% of participants intended to undergo genetic testing, only 5% had completed testing. History of mental illness and elevated levels of anxiety and depressive symptoms were positively correlated with testing intention in the bivariate analysis. In multivariate analysis, only younger age and less education were associated with testing intention. Future studies should address psychosocial needs and other competing barriers at the patient, provider, and healthcare system levels to increase access to BRCA1/2 genetic testing among multiethnic women.

Asymptotics of estimators for nonparametric multivariate regression models with long memory

In this paper, a nonparametric multivariate regression model with long memory covariates and long memory errors is considered. We approximate the nonparametric multivariate regression function by the weighted additive one-dimensional functions. The local linear smoothing and least squares method are proposed for the one-dimensional regression estimation and the weight parameters estimation, respectively. The asymptotic behaviors of the proposed estimators are investigated.

Bacterial salivary microbiome associates with asthma among african american children and young adults.

Several studies have shown that the airways of asthma patients contain higher diversity of bacteria and are enriched in pathogenic species. However, sampling the airways in children is challenging. Here we aimed to identify differences in the salivary bacterial composition between African Americans children with and without asthma. Saliva samples from 57 asthma cases and 57 healthy controls were analyzed by means of 16S ribosomal RNA amplicon profiling. Measurements of bacterial diversity and genus relative abundance were compared between cases and controls using the nonparametric Wilcoxon test and multivariate regression models. A total of five phyla and a mean of 56 genera were identified. Among them, 15 genera had a relative abundance greater than 1%, being Prevotella, Haemophilus, Streptococcus, and Veillonella the most abundant genera. Differences between cases and controls were found in terms of diversity, as well as in relative abundance for Streptococcus genus (13.0% in cases vs 18.3% in controls; P = .003) and Veillonella genus (11.1% in cases vs 8.0% in controls; P = .002). These differences remained significant after correction for multiple comparisons and when potential confounders were taken into account in logistic regression models. In conclusion, we identified changes in the salivary microbiota associated with asthma among African Americans.

Identifying Entrenched Users of Social Networking Sites: Empirical Study on Facebook

Recently, the rapid growth of users has considered ending their Facebook use or actually done so due to privacy issues, concerns about overexposing personal information online, or interpersonal interventions or distress. This study conducted research on users of Facebook, the most representative social networking site in the world, to determine the effect of Big Five personality traits, self-monitoring, and Facebook usage behaviors on continuance usage intention. A total of 498 valid samples were collected from a questionnaire survey and a multivariate nonparametric regression technique -multivariate adaptive regression splines-was employed to screen critical variables and predict users' continuance usage intention in the future. The results revealed that when the main motivation for Facebook use is not documenting life or expressing emotions, continuance usage intention notably decreases. In addition, the degree of agreeableness has a moderating effect on the relationship between Facebook usage behaviors and continuance usage intention.

An optimal test for the additive model with discrete or categorical predictors

In multivariate nonparametric regression, the additive models are very useful when a suitable parametric model is difficult to find. The backfitting algorithm is a powerful tool to estimate the additive components. However, due to complexity of the estimators, the asymptotic p value of the associated test is difficult to calculate without a Monte Carlo simulation. Moreover, the conventional tests assume that the predictor variables are strictly continuous. In this paper, a new test is introduced for the additive components with discrete or categorical predictors, where the model may contain continuous covariates. This method is also applied to the semiparametric regression to test the goodness of fit of the model. These tests are asymptotically optimal in terms of the rate of convergence, as they can detect a specific class of contiguous alternatives at a rate of $$n^{-1/2}$$ . An extensive simulation study and a real data example are presented to support the theoretical results.

Empirical performance of nonparametric regression over LRM and IGRM addressing influential observations

AbstractNonparametric regression is commonly used for summarizing the relationship between variables without requiring the assumptions of model. Generalized linear model and linear regression model are usually used to examine the relationship of variables, but both are badly affected by influential observations. Due to this, detection and removal of outliers attain a lot of attention of researchers to obtain reliable estimates. We focus on such robust technique whose performance is acceptable in the presence of outliers. The present article empirically compared the performance of linear regression model and generalized linear model with multivariate nonparametric kernel regression. Here, multivariate nonparametric kernel regression is used with Gaussian kernel and six different bandwidths on Aerial biomass data. The performance of nonparametric regression with Bayesian bandwidth was found to be better as compared with other methods.

Simultaneous Hypothesis Testing of Multivariable Nonparametric Spline Regression in the GWR Model

In this research, studied multivariable nonparametric geographically weighted regression use truncated spline approach. The model is an expansion of nonparametric truncated spline regression that takes into account geographical or spatial factors. The purpose of this study was to find statistics test and distribution for the simultaneous hypothesis test. This study obtains the statistic test used the maximum likelihood ratio test (MLRT) method. Results of the research obtained statistics test based on the ratio between the maximum of the likelihood function under the set of H_0  and the maximum of the set likelihood function below the population with each have a spatial factor. Distribution of statistical tests has been proven to have a distribution of F. The modeling application used the percentage of the death of Dengue Hemorrhagic Fever (DHF) in 38 districts/cities in East Java Province. The modeling resulted in the determination coefficient of 80.7% and SSE value that is 0.0043.

Cash transfers, clientelism, and political enfranchisement: Evidence from Brazil

This paper uses Brazil's Bolsa Família to show that redistributive policies that are shielded from the influence of political intermediaries can reduce incumbency advantage for mayors, increase both electoral competition and candidate quality, reduce support for clientelistic parties, and lead incumbents to increase redistributive spending. The paper exploits a nonparametric multivariate regression discontinuity design and employs a novel identification strategy for the variation in program coverage. The theory proposes that cash transfers, by reducing the vulnerability of poor voters, make clientelism a less attractive strategy to incumbent mayors. Consequently, incumbents reallocate effort away from the practice into public good distribution.

Simulations Study Combined Estimator Fourier Series and Spline Truncated in Multivariable Nonparametric Regression

The problems one of that appeared in everyday life is how to explain the shape of the relation pattern between a response with some predictor. Regression analysis is a statistical method used to estimate the relation pattern between response (y) with predictor (x). Development of nonparametric regression of Fourier series involving multiple predictor, has been more developed for predictors of the same pattern. We need to develop different estimator for different predictors of multivariable nonparametric regression. Theoretical research will be focused on estimator form, and applied to simulation data. The estimation of the combined regression function of the Fourier and spline truncated sequence is obtained through the optimization of penalized least square (PLS). Based on the simulation result obtained that the larger the sample size and the smaller the size of the variance, it will result in better estimation value of parameter and knot.

Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

Sparse model identification and learning for ultra-high-dimensional additive partially linear models

The additive partially linear model (APLM) combines the flexibility of nonparametric regression with the parsimony of regression models, and has been widely used as a popular tool in multivariate nonparametric regression to alleviate the “curse of dimensionality”. A natural question raised in practice is the choice of structure in the nonparametric part, i.e., whether the continuous covariates enter into the model in linear or nonparametric form. In this paper, we present a comprehensive framework for simultaneous sparse model identification and learning for ultra-high-dimensional APLMs where both the linear and nonparametric components are possibly larger than the sample size. We propose a fast and efficient two-stage procedure. In the first stage, we decompose the nonparametric functions into a linear part and a nonlinear part. The nonlinear functions are approximated by constant spline bases, and a triple penalization procedure is proposed to select nonzero components using adaptive group LASSO. In the second stage, we refit data with selected covariates using higher order polynomial splines, and apply spline-backfitted local-linear smoothing to obtain asymptotic normality for the estimators. The procedure is shown to be consistent for model structure identification. It can identify zero, linear, and nonlinear components correctly and efficiently. Inference can be made on both linear coefficients and nonparametric functions. We conduct simulation studies to evaluate the performance of the method and apply the proposed method to a dataset on the Shoot Apical Meristem (SAM) of maize genotypes for illustration.

Multiple risks analysis for aplastic anemia in Zhejiang, China: A case-control study.

To understand the risks associated with aplastic anemia (AA) in 4 cities of Zhejiang Province, China, with special focus on the joint contributions of multiple risks.Based on an Electronic Data Capture (EDC), a case control study was carried out. Data regarding socio-demographic, diseases history, living habits, and exposures to toxic substances, etc., were collected through survey questionnaires. t Test, chi-square test, or non-parametric rank sum test, and univariate and multivariate Logistic regression analysis were conducted to analyze data.The univariate logistic regression analysis results indicated that among all study participants (n = 1802), AA was associated with over 30 risks, in terms of their individual behaviors, daily and environmental exposures, diseases history, and family history. Multivariate logistic regression analysis further confirmed that the independent risks related to AA included presence of chemical factory within 3 km of living residence (odds ratio [OR] = 8.73, 95% CI: 1.42–53.74, P = .019), living in a newly decorated house/apartment (OR = 25.37, 95% CI: 4.44–144.81, P < .001), vegetarian diet (OR = 131.60, 95% CI: 3.45–5020.16, P = .009), preference of sugar (OR = 89.38, 95% CI: 7.22–1106.44, P < .001), preference of oily food (OR = 55.68, 95% CI: 5.12–605.26, P = .001), drinking lake water or pond water (OR = 58.05, 95% CI: 3.21–1049.81, P < .001), habit of staying up late (OR = 11.87, 95% CI: 3.43–41.02, P < .001), infection history (OR = 10.08, 95% CI: 2.75–36.93, P < .001). Result of receiver operating characteristic curve (ROC) analysis on the joint contribution of multiple risks indicated that AA was 13.835 times likely to occur when exposed to ≥1 risks than those exposed to 0 risks (95% CI: 9.995–19.149).Our study results demonstrated a comprehensive epidemiological pattern, in which the joint contributions of individual inherited health status, environment exposure, and individual behaviors lead to the occurrence of AA.

PEMODELAN INDEKS HARGA SAHAM GABUNGAN MENGGUNAKAN REGRESI SPLINE MULTIVARIABEL

The Composite Stock Price Index (CSPI) is a composite index of all types of shares listed on the stock exchange and their movements indicate the conditions occurring in the stock market. CSPI movement is an important indicator for investors to determine whether they will sell, hold, or buy a stock. One of the factors that influence the movement of CSPI is Inflation (X1), Exchange Rate (X2) and SBI rate (X3). This study aims to obtain the best CSPI model using a multivariable nonparametric spline regression approach. The approach is done by nonparametric regression because the regression curve obtained does not show a certain relationship pattern. Spline is very dependent on the order and location of the knot point. The best spline model is the model that has the minimum MSE (Mean Square Error) value. In this study, the best spline regression model is when X1 is 4 order, X2 is 2 order and X3 is 2 order. The number of knots on X1 is 1 knot at 8.22, X2 is 2 knots at 13066.82 and 13781.75 While X3 is 2 knots at 6.6 and 6.67 with value of MSE equal to 6686.85.Keywords: Composite Stock Price Index, Multivariable Spline Regression, MSE

Specific Cognitive-Psychopathological Phenotypes in Patients With Early Stages of Subcortical Vascular Neurocognitive Disorders: A Hospital-Based Case-Control Study.

Diagnosis of subcortical vascular neurocognitive disorders' (ScVNCDs) is currently based on neuropsychological and neuroimaging approaches; nevertheless, clinical features, apart from cognitive impairments (CI), may provide additional information about ScVNCD phenotypes. We aimed to determine whether CI and neuropsychiatric symptoms (NPS) form such clinical phenotypes in the mild and early stage of major ScVNCD. Our sample included 88 cognitively normal elderly individuals, 100 patients with mild ScVNCD, and 60 patients with early major ScVNCD. All participants had neuropsychological, neuropsychiatric, neurological, and functional evaluations. The prevalence of NPS was based on the neuropsychiatric inventory. The statistical analyses included parametric and nonparametric tests and multivariate regression. The severity of executive dysfunction increased through stages of ScVNCD progression ( P < .0001). The NPS with significant predictive value for mild ScVNCD membership was depression (odds ratio [OR] = 7.4), whereas for early major ScVNCD were depression (OR = 5.5) and apathy (OR = 7.6). Those distinguishing NPS and impairments of executive tests' performance significantly correlated ( P < .05) in patients with mild/major ScVNCD. Significant correlation between pathognomonic cognitive and NPS in compared groups suggest that dysexecutive-depressive syndrome can be the main phenotype in mild ScVNCD, while dysexecutive-depressive-apathetic syndrome in the early stage of major ScVNCD. Obtained cognitive-psychopathological phenotypes may allow a better comprehension of the ScVNCD pathophysiology and improve the diagnostic and therapeutic approach.

Market channel selections by US nursery plant producers: a multivariate nonparametric fractional regression analysis

ABSTRACTAvailability of market channel alternatives has helped the growth of ornamental plant sales in the United States. To identify the factors affecting the choice and allocation of outputs to different market channels by nursery producers, we first use a mixture of experts model to select clusters of homogenous subpopulations of US nursery producers based on a 2009 National Nursery Survey. The impact of growers’ business characteristics on shares of sales to these channels was estimated using multivariate parametric and nonparametric fractional regression models. Specification tests indicated a nonparametric model was superior to a parametric model in some clusters. Important explanatory variables affecting the sales volume to different channels were sales of plant groups, kinds of contract sales, promotional expenses, and farm size. Results indicated the existence of clear market segmentation of nursery producers in the United States.