Kernel Regression Technique Research Articles

Super-Resolution Imaging Enhancement through a 2D Scanning Galvanometer: Algorithm Formulation and Application in Aerial Optoelectronic Systems

As the fields of aviation and aerospace optics continue to evolve, there is an increasing demand for enhanced detection capabilities in equipment. Nonetheless, in applications where both optical and mechanical constraints are stringent, the continuous expansion of optical aperture and focal length is impractical. Given the existing technological landscape, employing super-resolution algorithms to enhance the imaging capability of optical systems is both practical and highly relevant. This study capitalizes on using a 2D scanning galvanometer in optical systems to acquire micro-displacement information. Initially, an imaging model for optical systems equipped with a 2D scanning galvanometer was established, and the displacement vectors for both forward and sweep image motions were defined. On this foundation, we incorporated micro-displacement information that can induce high-frequency aliasing. Subsequently, the motion paths of the galvanometer were planned and modeled. To align image sequences with micro-displacement correlations, the Lucas–Kanade (L-K) optical flow method was employed with multi-layer pyramid iteration. Then, super-resolution reconstruction was performed using kernel regression techniques. Ultimately, we tested the algorithm on an aeronautical optoelectronic pod to evaluate its impact on optical resolution and imaging quality. Compared with the original images, the 16-frame image demonstrated a 39% improvement in optical resolution under laboratory conditions. Moreover, the algorithm exhibited satisfactory performance under both nighttime and daytime conditions, as well as during aerial tests.

Prediction of Significant Wave Heights with Engineered Features from GNSS Reflectometry

Estimating reflector heights at stationary GNSS sites with interferometric reflectometry (IR) is a well-established technique in ocean remote sensing. Additionally, IR offers the possibility to estimate the significant wave height (SWH) with a linear model using the damping coefficient from an inverse modelling applied to GNSS signal-to-noise ratio (SNR) observations. Such a linear model serves as a benchmark in the present study, where an alternative approach for the estimation of both SWH and reflector height is presented that is based on kernel regression and clustering techniques. In this alternative approach, the reflector height is estimated by analyzing local extrema occurring in the interference pattern that is present in GNSS SNR observations. Various predictors are derived from clustering statistics and the estimated reflector heights. These predictors are used for the SWH determination with supervised machine learning. Linear models, bagged regression trees, and artificial neural networks are applied and respective results are compared for various predictor sets. In a second step, damping coefficients obtained from the inverse modelling mentioned above are additionally taken into account as predictors. The usability of the alternative approach is demonstrated. Compared to the benchmark, a significant improvement in terms of accuracy is found for an artificial neural network with predictors from both the alternative and the inverse modelling approach.

Model‐based clustering of semiparametric temporal exponential‐family random graph models

Model‐based clustering of time‐evolving networks has emerged as one of the important research topics in statistical network analysis. It is a fundamental research question to model time‐varying network parameters. However, due to difficulties in modelling functional network parameters, there is little progress in the current literature to model time‐varying network parameters effectively. In this work, we model network parameters as univariate nonparametric functions instead of constants. We effectively estimate those functional network parameters in temporal exponential‐family random graph models using a kernel regression technique and a local likelihood approach. Furthermore, we propose a semiparametric finite mixture of temporal exponential‐family random graph models by adopting finite mixture models, which simultaneously allows both modelling and detecting groups in time‐evolving networks. Also, we use a conditional likelihood to construct an effective model selection criterion and network cross‐validation to choose an optimal bandwidth. The power of our method is demonstrated in simulation studies and real‐world applications to dynamic international trade networks and dynamic arm trade networks.

A robust strategy for sensor fault detection in nuclear power plants based on principal component analysis

The principal component analysis (PCA) method has been widely used in sensor fault detection. However, outliers of training data may affect the projection directions of both principal component (PC) and residual space, thereby reducing the fault detection rate (FDR). The high sensitivity of PCA to random noise in the test data can also lead to an increase in the false alarm rate (FAR). To improve the performance of the PCA, this paper proposes a robust PCA approach for sensor fault detection in nuclear power plants (NPPs). A statistical method based on Euclidean distance is used to clean outliers in the training data pre-processing phase. Subsequently in the fault detection phase, the moving average (MA) filtering method is adopted to process Q-statistic to reduce false alarms caused by random noise in the test data. Simulation and plant signals are used to verify the effectiveness of the proposed method. Finally, comparisons with the conventional PCA, auto-associative kernel regression (AAKR) and multivariate state estimation technique (MSET) highlight the advantages of the proposed method.

Sparse kernel regression technique for self-cleansing channel design

The application of a robust learning technique is inevitable in the development of a self-cleansing sediment transport model. This study addresses this problem and advocates the use of sparse kernel regression (SKR) technique to design a self-cleaning model. The SKR approach is a regression technique operating in the kernel space which also benefits from the desirable properties of a sparse solution. In order to develop a model applicable to a wide range of channel characteristics, five different experimental data sets from 14 different channels are utilized in this study. In this context, the efficacy of the SKR model is compared against the support vector regression (SVR) approach along with several other methods from the literature. According to the statistical analysis results, the SKR method is found to outperform the SVR and other regression equations. In particular, while empirical regression models fail to generate accurate results for other channel cross-section shapes and sizes, the SKR model provides promising results due to the inclusion of a channel parameter at the core of its structure and also by operating on an extensive range of experimental data. The superior efficacy of the SKR approach is also linked to its formulation in the kernel space while also benefiting from a sparse representation method to select the most useful training samples for model construction. As such, it also circumvents the requirement to evaluate irrelevant or noisy observations during the test phase of the model, and thus improving on the test phase running time.

Machine Learning for Solar Array Monitoring, Optimization, and Control

The efficiency of solar energy farms requires detailed analytics and information on each panel regarding voltage, current, temperature, and irradiance. Monitoring utility-scale solar arrays was shown to minimize the cost of maintenance and help optimize the performance of the photo-voltaic arrays under various conditions. We describe a project that includes development of machine learning and signal processing algorithms along with a solar array testbed for the purpose of PV monitoring and control. The 18kW PV array testbed consists of 104 panels fitted with smart monitoring devices. Each of these devices embeds sensors, wireless transceivers, and relays that enable continuous monitoring, fault detection, and real-time connection topology changes. The facility enables networked data exchanges via the use of wireless data sharing with servers, fusion and control centers, and mobile devices. We develop machine learning and neural network algorithms for fault classification. In addition, we use weather camera data for cloud movement prediction using kernel regression techniques which serves as the input that guides topology reconfiguration. Camera and satellite sensing of skyline features as well as parameter sensing at each panel provides information for fault detection and power output optimization using topology reconfiguration achieved using programmable actuators (relays) in the SMDs. More specifically, a custom neural network algorithm guides the selection among four standardized topologies. Accuracy in fault detection is demonstrate at the level of 90+% and topology optimization provides increase in power by as much as 16% under shading.

Time varying mean extraction for stationary and nonstationary winds

This paper discusses different strategies for the extraction of the time-varying mean from wind speed time histories. Due to the advantage of allowing analytical evaluations, the attention is focused on kernel regression techniques, considering different weighting functions, namely a constant, a Gaussian and a cardinal sine weighting function. The problem is firstly treated analytically, and the frequency-domain properties of the filter associated to different kinds of weighting functions in the definition of the slowly varying mean through kernel regression are analysed. Then, different weighting functions are adopted for the analysis of digitally-simulated stationary wind speed time histories and for the time histories of thunderstorm outflows recorded by a tri-axial anemometer. The consequences of the adoption of different weighting functions on the harmonic content and statistical properties of turbulence are studied. The same features are found also for thunderstorm outflow records.

Estimation of a Finite Population Mean under Random Nonresponse Using Kernel Weights

Nonresponse is a potential source of errors in sample surveys. It introduces bias and large variance in the estimation of finite population parameters. Regression models have been recognized as one of the techniques of reducing bias and variance due to random nonresponse using auxiliary data. In this study, it is assumed that random nonresponse occurs in the survey variable in the second stage of cluster sampling, assuming full auxiliary information is available throughout. Auxiliary information is used at the estimation stage via a regression model to address the problem of random nonresponse. In particular, auxiliary information is used via an improved Nadaraya–Watson kernel regression technique to compensate for random nonresponse. The asymptotic bias and mean squared error of the estimator proposed are derived. Besides, a simulation study conducted indicates that the proposed estimator has smaller values of the bias and smaller mean squared error values compared to existing estimators of a finite population mean. The proposed estimator is also shown to have tighter confidence interval lengths at 95% coverage rate. The results obtained in this study are useful for instance in choosing efficient estimators of a finite population mean in demographic sample surveys.

Myelin water fraction estimation using small-tip fast recovery MRI.

To demonstrate the feasibility of an optimized set of small-tip fast recovery (STFR) MRI scans for rapidly estimating myelin water fraction (MWF) in the brain. We optimized a set of STFR scans to minimize the Cramér-Rao Lower Bound of MWF estimates. We evaluated the RMSE of MWF estimates from the optimized scans in simulation. We compared STFR-based MWF estimates (both modeling exchange and notmodeling exchange) to multi-echo spin echo (MESE)-based estimates. We used the optimized scans to acquire in vivo data from which a MWF map was estimated. We computed the STFR-based MWF estimates using PERK, a recently developed kernel regression technique, and the MESE-based MWF estimates using both regularized non-negative least squares (NNLS) and PERK. In simulation, the optimized STFR scans led to estimates of MWF with low RMSE across a range of tissue parameters and across white matter and gray matter. The STFR-based MWF estimates that modeled exchange compared well to MESE-based MWF estimates in simulation. When the optimized scans were tested in vivo, the MWF map that was estimated using a 3-compartment model with exchange was closer to the MESE-based MWF map. The optimized STFR scans appear to be well suited for estimating MWF in simulation and in vivo when we model exchange in training. In this case, the STFR-based MWF estimates are close to the MESE-based estimates.

Grafting metal complexes onto amorphous supports: from elementary steps to catalyst site populationsviakernel regression

We use transition state theory, kernel regression, and population balance modeling techniques to model the grafting of metal complexes onto amorphous catalyst supports.

Site-averaged kinetics for catalysts on amorphous supports: an importance learning algorithm

We combine importance sampling and kernel regression techniques to efficiently predict site-averaged kinetics for isolated catalyst sites on amorphous supports.

Evaluating Climate Model Simulations of the Radiative Forcing and Radiative Response at Earth’s Surface

AbstractWe analyze the radiative forcing and radiative response at Earth’s surface, where perturbations in the radiation budget regulate the atmospheric hydrological cycle. By applying a radiative kernel-regression technique to CMIP5 climate model simulations where CO2 is instantaneously quadrupled, we evaluate the intermodel spread in surface instantaneous radiative forcing, radiative adjustments to this forcing, and radiative responses to surface warming. The cloud radiative adjustment to CO2 forcing and the temperature-mediated cloud radiative response exhibit significant intermodel spread. In contrast to its counterpart at the top of the atmosphere, the temperature-mediated cloud radiative response at the surface is found to be positive in some models and negative in others. Also, the compensation between the temperature-mediated lapse rate and water vapor radiative responses found in top-of-atmosphere calculations is not present for surface radiative flux changes. Instantaneous radiative forcing at the surface is rarely reported for model simulations; as a result, intermodel differences have not previously been evaluated in global climate models. We demonstrate that the instantaneous radiative forcing is the largest contributor to intermodel spread in effective radiative forcing at the surface. We also find evidence of differences in radiative parameterizations in current models and argue that this is a significant, but largely overlooked, source of bias in climate change simulations.

Estimation and testing of nonparametric hidden Markov model with application in stock market

In this paper, a fully nonparametric Hidden Markov Model is proposed, where both the state dependent emission model and transition matrix are all nonparametric and depend on a covariate. By reducing modeling bias, the new model extends the applicability of both the semiparametric HMM with nonparametric emission model and nonhomogeneous HMM. A modified EM algorithm, which combines the kernel regression technique, is proposed for estimation. Model selection procedures are provided and a conditional bootstrap method is employed to assess the standard errors of the estimates. A generalized likelihood ratio test procedure is further proposed and the Wilk’s type of phenomenon is verified to hold for the proposed model. Simulation studies are conducted to demonstrate proposed procedure, and the proposed model is applied in analyzing the impact of economic on stock market.

'Choosing Factors' by Fama and French (2018): A Comment

This paper features a statistical analysis of the monthly three factor Fama/French return series. We apply rolling OLS regressions to explore the relationship between the 3 factors, using monthly data from July 1926 to June 2018, that are available on Ken French's website. The results suggest there are significant and time varying relationships between the factors. We then switch to a sub-sample from July 1990 to July 2018, also taken from Ken French's website. The three series and their inter-relationships are analysed using two stage least squares and the Hausman test to check for issues related to endogeneity, the Sargan over-identification test and the Cragg-Donald weak instrument test. The relaationship between factors is also examined using OLS incorporating Ramsey's RESET tests of functional form misspecification, plus Naradaya-Watson kernel regression techniques. The empirical results suggest that the factors, when combined in OLS regression analysis, as suggested by Fama and French (2018), are likely to suffer from endogeneity. OLS regression analysis and the the application of Ramsey's RESET tests suggest a non-linear relationship exists between the three series, in which squared and cubed terms are significant. This non-linearity is also confirmed by the Kernel regression analysis. We use two instruments to estimate the market betas, and then use the factor estimates in a second set of panel data tests using a small sample of monthly returns for US firms that are drawn from the on-line data source “tingo”. These issues are analysed using methods suggested by Petersen (2009) to permit clustering in the panels by date and firm. The empirical results suggest that using an instrument to capture endogeneity reduces the standard error of market beta in subsequent cross-sectional tests, but that clustering effects, as suggested by Petersen (2009), will also impact on the estimated standard errors. The empirical results suggest that using these factors in linear regression analysis, such as suggested by Fama and French (2018), as a method of screening factor relevance, is problematic in that the estimated standard errors are highly sensitive to the correct model specification.

Statistical inference under imputation for proportional hazard model with missing covariates

ABSTRACTMissing covariate data are common in biomedical studies. In this article, by using the non parametric kernel regression technique, a new imputation approach is developed for the Cox-proportional hazard regression model with missing covariates. This method achieves the same efficiency as the fully augmented weighted estimators (Qi et al. 2005. Journal of the American Statistical Association, 100:1250) and has a simpler form. The asymptotic properties of the proposed estimator are derived and analyzed. The comparisons between the proposed imputation method and several other existing methods are conducted via a number of simulation studies and a mouse leukemia data.

Fast lithography aerial image calculation method based on machine learning.

Aerial image calculation for thick masks is an indispensable but time-consuming step in most lithography simulations. This paper develops a fast thick-mask aerial image calculation method based on machine learning for partially coherent lithography systems. First, some sparse sampling points are chosen from the source plane to represent the partially coherent illumination. Then, the training libraries of thick-mask diffraction near-fields are built up for all sampling points based on a set of representative mask features. For an arbitrary thick mask, we calculate its aerial image using the nonparametric kernel regression technique and the pre-calculated training libraries. Subsequently, a post-processing method is applied to compensate for the estimation error and improve the computational accuracy. In addition, this paper also studies the impacts of several key factors on the accuracy and efficiency of the proposed method. Finally, the proposed method is verified by the simulations at 45 nm and 14 nm technology nodes.

A Maximum a Posteriori Estimation Framework for Robust High Dynamic Range Video Synthesis.

High dynamic range (HDR) image synthesis from multiple low dynamic range exposures continues to be actively researched. The extension to HDR video synthesis is a topic of significant current interest due to potential cost benefits. For HDR video, a stiff practical challenge presents itself in the form of accurate correspondence estimation of objects between video frames. In particular, loss of data resulting from poor exposures and varying intensity makes conventional optical flow methods highly inaccurate. We avoid exact correspondence estimation by proposing a statistical approach via maximum a posterior estimation, and under appropriate statistical assumptions and choice of priors and models, we reduce it to an optimization problem of solving for the foreground and background of the target frame. We obtain the background through rank minimization and estimate the foreground via a novel multiscale adaptive kernel regression technique, which implicitly captures local structure and temporal motion by solving an unconstrained optimization problem. Extensive experimental results on both real and synthetic data sets demonstrate that our algorithm is more capable of delivering high-quality HDR videos than current state-of-the-art methods, under both subjective and objective assessments. Furthermore, a thorough complexity analysis reveals that our algorithm achieves better complexity-performance tradeoff than conventional methods.

Facial Skin Smoothening by Iterative Steering Kernel Regression Technique

Facial Skin Smoothening by Iterative Steering Kernel Regression Technique

A fast and manufacture-friendly optical proximity correction based on machine learning

Pixel-based optical proximity correction (PBOPC) is currently a key resolution enhancement technique to push the resolution limit of optical lithography. However, the increasing scale, density and complexity of modern integrated circuits pose new challenges to both of the OPC computational intensity and mask manufacturability. This paper aims at developing a practical OPC algorithm based on a machine learning technique to effectively reduce the PBOPC runtime and mask complexity. We first divide the target layout into small regions around corners and edge fragments. Using a nonparametric kernel regression technique, these small regions are then filled in by the weighted linear combination of a subset of training OPC examples selected from the pre-calculated libraries. To keep balance between the image fidelity and mask complexity, we use an edge-based OPC (EBOPC) library to synthesize the OPC patterns in non-critical areas, while use another PBOPC library for hotspots. In addition, a post-processing method is developed to refine the regressed OPC pattern so as to guarantee the final image fidelity and mask manufacturability. Experimental results show that, compared to a currently professional PBOPC software, the proposed algorithm can achieve approximately two-fold speedup and more manufacture-friendly OPC patterns.

Estimating a nonparametric triangular model with binary endogenous regressors

We consider identification and estimation in a nonparametric triangular system with a binary endogenous regressor and nonseparable errors. For identification, we take a control function approach utilizing the Dynkin system idea. We articulate various trade‐offs, including continuity, monotonicity and differentiability. For estimation, we use the idea of local instruments under smoothness assumptions, but we do not assume additive separability in latent variables. Our estimator uses nonparametric kernel regression techniques and its statistical properties are derived using the functional delta method. We establish that it is n 2 / 7 ‐consistent and has a limiting normal distribution. We apply the method to estimate the returns on a college education. Unlike existing work, we find that returns on a college education are consistently positive. Moreover, the returns curves we estimate are inconsistent with the shape restrictions imposed in those papers.