Kernel Regression Function Research Articles

Face Hallucination via Coarse-to-Fine Recursive Kernel Regression Structure

In recent years, patch-based face hallucination algorithms have attracted considerable interest due to their effectiveness. These approaches produce a high-resolution (HR) face image according to the corresponding low-resolution (LR) input by learning a reconstruction model from the given training image set. The critical problem in these algorithms is establishing the underlying relationship between LR and HR patch pairs. Most previous methods aim to denote each input LR patch by the linear combination of the training set in the LR space while utilizing the combination weights to reconstruct the target HR patch. However, this assumes that the same combination weights should be shared between various resolution spaces, which is truly difficult to satisfy because of the one-to-many mapping relation between LR and HR patches. In this paper, we directly train a series of adaptive kernel regression mappings for predicting the lost high-frequency information from the LR patch, which avoids dealing with the above difficult problem. During the training process, we first establish a local optimization function on each LR/HR training pair according to the geometric structure of neighboring patches. The objective of local optimization can be presented in two aspects: 1) ensure the reconstruction consistency between each LR patch and the corresponding HR patch and 2) preserve the intrinsic geometry between each HR training patch and its original neighbors after the reconstruction process. The local optimizations are finally incorporated as the global optimization for calculating the optimal kernel regression function. To better approximate the target HR patch, we further propose a recursive structure to compensate for the residual reconstruction error of high-frequency details by a series of regression mappings. The proposed method is rather fast yet very effective in producing HR face images. Experimental results show that the proposed approach achieves superior performance with reasonable computational time compared with the state-of-the-art methods.

Caging a novel object using multi-task learning method

Caging grasps provide a way to manipulate an object without full immobilization and enable dealing with the pose uncertainties of the object. Most previous works have constructed caging sets by using the geometric models of the object. This work aims to present a learning-based method for caging a novel object only with its image. A caging set is first defined using the constrained region, and a mapping from the image feature to the caging set is then constructed with kernel regression function. Avoiding the collection of large number of samples, a multi-task learning method is developed to build the regression function, where several different caging tasks are trained with a joint model. In order to transfer the caging experience to a new caging task rapidly, shape similarity for caging knowledge transfer is utilized. Thus, given only the shape context for a novel object, the learner is able to accurately predict the caging set through zero-shot learning. The proposed method can be applied to the caging of a target object in a complex real-world environment, for which the user only needs to know the shape feature of the object, without the need for the geometric model. Several experiments prove the validity of our method.

Complex Video Scene Analysis Using Kernelized-Collaborative Behavior Pattern Learning Based on Hierarchical Representative Object Behaviors

This paper considers an unsupervised learning algorithm that can automatically discover key behavior patterns to characterize a complex video scene. For behavior features (bFs) extracted at multiple spatial-temporal scales, an optimization problem is formulated to cluster bFs in their scales while establishing a collaborative nonlinear relationship in the form of a kernel regression function among clustered bFs across different spatial scales. The relationship allows features extracted in one scale to be considered as contextual information in the analysis of another scale. This optimization problem is solved using linear programming to reduce computational complexity. The proposed algorithm is evaluated on four crowded traffic scenes and two sports video data sets. Experimental results show that the proposed algorithm achieves a better performance compared with the current state-of-the-art algorithms in terms of video segmentation accuracy.

Super resolution of aerial image by means of polyphase components reconstruction

Multi-frame super resolution reconstruction is a technology for obtaining a high resolution image from a set of blurred and aliased low resolution images. The most popular and widely used super resolution methods are motion based. However, the estimation of motion information (registration) is very difficult, computationally expensive and inaccurate, especially for aerial image. The sub-pixel registration error restricts the performance of the subsequent super resolution. Instead of trying to parameterize the motion estimation model, this paper proposes an image super resolution framework based on the polyphase components reconstruction algorithm and an improved steering kernel regression algorithm. Given an image observation model, a reversible 2D polyphase decomposition, which breaks down a high resolution image into polyphase components, is obtained. Though the assumption of diversity sampling, this paper adopts a fundamentally different approach, in which the low-resolution frames is used as the basis and the reference frame as the reference sub-polyphase component of the high resolution image for recovering the polyphase components of the high resolution image. The polyphase components, which fuse the low resolution frames with the complementary details, can be obtained by computing their expansion coefficients in terms of this basis using the available sub-polyphase components and then inversely transforming them into a high resolution image. This paper accomplishes this by formulating the problem as the maximum likelihood estimation, which guarantees a close-to-perfect solution. Furthermore, this paper proposes an improved steering kernel regression algorithm, to help restore the fusion image with mild blur and random noise. This paper adaptively refines the steering kernel regression function according to the local region context and structures. Thus, this new algorithm not only effectively combines denoising and deblurring together, but also preserves the edge information. Our framework develops an efficient and stable algorithm to tackle the huge size and ill-posedness of the super resolution problem, and improves the computational efficiency via avoiding registration and iterative computation. Several experimental results on synthetic data illustrate that our method outperforms the state-of-the-art methods in quantitative and qualitative comparisons. The proposed super resolution algorithm can indeed reconstruct high-frequency information which is otherwise unavailable in the single LR image. It can effectively suppress blur and noise, and produce visually pleasing resolution enhancement in aerial images.

An improved kernel regression method based on Taylor expansion

Many regression functions obtained by nonparametric regression method often appear inconsonance between smoothness and fitness. This phenomenon is extremely outstanding near the vertex regions. How to improve the fitness and smoothness simultaneously becomes an important problem in the nonparametric regression field. In this paper, an improved kernel regression is proposed by introducing second derivative estimation into kernel regression function based on Taylor expansion theorem. Experimental results on regression problems show that this new method is feasible and enables us to get regression function that is both smooth and well-fitting. The application of the method to grey image enhancement indicates that this approach is fruitful to the enhancement of weak information in the images.

INTEGRATED CROSS-VALIDATION FOR THE RANDOM DESIGN NONPARAMETRIC REGRESSION

For the random design nonparametric regression, cross-validation is a popular bandwidth selector. It is constructed by using the criterion of ``weighted" integrated square error. In practice, however, the weighting scheme by the design density in the criterion causes that its associated cross-validation function puts more emphasis in regions with more data, gives little attention to regions with few data, but has no consideration for regions without data. In such a case, the value of the cross-validated bandwidth depends on the distribution of the design points, but is independent of the location of the interval on which the regression function value is estimated. Hence, if there are sparse regions in the realization of the design, then the resulting cross-validated bandwidth is usually not large enough in magnitude such that its corresponding kernel regression function estimate has rough appearance in these sparse regions. To avoid this drawback to cross-validation, we suggest using the criterion of ''unweighted'' integrated square error to construct the bandwidth selector. Under the criterion, a bandwidth selector called integrated cross-validation is proposed, and the resulting bandwidth is shown to be asymptotically optimal. Empirical studies demonstrate that the kernel regression function estimate obtained by using our proposed bandwidth is better than that employing the ordinary cross-validated bandwidth, in both senses of having smoother appearance and yielding smaller sample unweighted integrated square error.

INTEGRATED CROSS-VALIDATION FOR THE RANDOM DESIGN NONPARAMETRIC REGRESSION

For the random design nonparametric regression, cross-validation is a popular bandwidth selector. It is constructed by using the criterion of ``weighted integrated square error. In practice, however, the weighting scheme by the design density in the criterion causes that its associated cross-validation function puts more emphasis in regions with more data, gives little attention to regions with few data, but has no consideration for regions without data. In such a case, the value of the cross-validated bandwidth depends on the distribution of the design points, but is independent of the location of the interval on which the regression function value is estimated. Hence, if there are sparse regions in the realization of the design, then the resulting cross-validated bandwidth is usually not large enough in magnitude such that its corresponding kernel regression function estimate has rough appearance in these sparse regions. To avoid this drawback to cross-validation, we suggest using the criterion of ''unweighted'' integrated square error to construct the bandwidth selector. Under the criterion, a bandwidth selector called integrated cross-validation is proposed, and the resulting bandwidth is shown to be asymptotically optimal. Empirical studies demonstrate that the kernel regression function estimate obtained by using our proposed bandwidth is better than that employing the ordinary cross-validated bandwidth, in both senses of having smoother appearance and yielding smaller sample unweighted integrated square error.

Uniform convergence rate of kernel estimation with mixed categorical and continuous data

We consider nonparametric estimation of regression functions with mixed categorical and continuous data, and we smooth both the discrete and continuous variables so that the approach does not suffer the sample splitting problem associated with the conventional frequency-based nonparametric estimation method. We derive the uniform convergence rate of the kernel regression function estimation with weakly dependent data.

On study of kernel regression function polygons

In the case of the random design nonparametric regression, the regression function estimate is produced practically by joining every two consecutive kernel estimates of regression function values by a straight line segment. Hence, it is of polygon type, and is called the kernel regression function polygon (KRFP) in this paper. The KRFP is analyzed by its asymptotic integrated mean square error (AIMSE). This AIMSE precisely quantifies both effects of the kernel function and of the distance between the points on which kernel estimates of regression function values are calculated on the KRFP. By studying the AIMSE, we have the following findings. First of all, if the distance is of smaller order in magnitude than the bandwidth used by the kernel regression function estimator, then Epanechnikov kernel is still the optimal kernel function for the KRFP. Secondly, if the distance is of the same order in magnitude as the bandwidth, then Epanechnikov kernel is no longer optimal for the KRFP. In this case, using the AIMSE of the KRFP, we obtain the optimal kernel for the KRFP over the class of two-degreepolynomials by numerical calculation. As the distance increases, the computation time of the KRFP decreases. However, the resulting performance of the KRFP deteriorates, since the minimum AIMSE of the KRFP over both the bandwidth and the kernelfunction increases. Finally, if the distance is of larger order in magnitude than the bandwidth, then the uniform kernel is the optimal kernel function for the KRFP.

Gaussian approximation of local empirical processes indexed by functions

An extended notion of a local empirical process indexed by functions is introduced, which includes kernel density and regression function estimators and the conditional empirical process as special cases. Under suitable regularity conditions a central limit theorem and a strong approximation by a sequence of Gaussian processes are established for such processes. A compact law of the iterated logarithm (LIL) is then inferred from the corresponding LIL for the approximating sequence of Gaussian processes. A number of statistical applications of our results are indicated.

A strong law of large numbers for triangular mixingale arrays

In this paper a strong law of large numbers for triangular mixingale arrays is proven. The mixingale condition is one of asymptotically weak dependence. A strong law of large numbers for triangular mixingale arrays has not been established previously in the literature. The result is applied to kernel regression function estimation.

Approximations to the mean integrated squared error with applications to optimal bandwidth selection for nonparametric regression function estimators

Discrete versions of the mean integrated squared error (MISE) provide stochastic measures of accuracy to compare different estimators of regression fuctions. These measures of accuracy have been used in Monte Carlo trials and have been employed for the optimal bandwidth selection for kernel regression function estimators, as shown in Härdle and Marron (1983), Optimal Bandwidth Selection in Nonparametric Regression Function Estimation. Inst. of Statistics Mimeo Series No. 1530, Univ. of North Carolina, Chapel Hill). In the present paper it is shown that these stochastic measures of accuracy converge to a weighted version of the MISE of kernel regression function estimators, extending a result of Hall (1982, Biometrika 69, 383–390) and Marron (1983, J. Multivariate Anal. 18, No. 2) to regression function estimation.

Probability Inequalities for Sums of Independent Random Variables

Probability Inequalities for Sums of Independent Random Variables

Remarks on Non-Parametric Estimates for Density Functions and Regression Curves

Remarks on Non-Parametric Estimates for Density Functions and Regression Curves