Graph frequency-domain regression

The ability to model irregular data and the interactions between them have extended the traditional signal processing tools to the graph domain. Under these circumstances, the emergence of graph signal processing has offered a brand new framework for dealing with complex data. In particular, the graph Fourier transform (GFT) lets us analyze the spectral components of a graph signal in the graph frequency domain. Based on the GFT, graph filters provide useful tools to modify or extract spectral parts in terms of different objectives, e.g., using a low-pass graph filter to construct graph signals without noise. This thesis mainly focuses on designing and implementing graph filters. Similar to traditional signal processing, we investigate two types of graph filters: finite impulse response (FIR) and infinite impulse response (IIR) graph filters. Moreover, this thesis takes both undirected and directed graphs into account for the design methods and implementations.

We consider the problem of recovering random graph signals from nonlinear measurements. For this setting, closed-form Bayesian estimators are usually intractable and even numerical evaluation may be difficult to compute for large networks. In this paper, we propose a graph signal processing (GSP) framework for random graph signal recovery that utilizes information on the structure behind the data. First, we develop the GSP-linear minimum mean-squared-error (GSP-LMMSE) estimator, which minimizes the mean-squared-error (MSE) among estimators that are represented as an output of a graph filter. The GSP-LMMSE estimator is based on <i>diagonal</i> covariance matrices in the graph frequency domain, and thus, has reduced complexity compared with the LMMSE estimator. This property is especially important when using the sample-mean estimators that are based on a training dataset. We then state conditions under which the low-complexity GSP-LMMSE estimator coincides with the optimal LMMSE estimator. Next, we develop an approximate parametrization of the GSP-LMMSE estimator by graph filters. We present three implementations of the parametric GSP-LMMSE estimator for typical graph filters. These parametric graph filters are more robust to outliers and to network topology changes. In our simulations, we evaluate the performance of the proposed GSP-LMMSE estimators for the problem of state estimation in power systems, which can be interpreted as a graph signal recovery task. We show that the proposed sample-GSP estimators outperform the sample-LMMSE estimator for a limited training dataset and that the parametric GSP-LMMSE estimators are more robust to topology changes in the form of adding/removing vertices/edges.

We consider a framework for generalized sampling of graph signals that extends sampling results in shift-invariant (SI) spaces to the graph setting. We assume that the input signal lies in a periodic graph spectrum subspace, which generalizes the standard SI assumption to graph signals. Sampling is performed in the graph frequency domain by an arbitrary graph filter. We show that under a mild condition on the sampling filter, perfect recovery is possible using a correction filter that can be represented as a spectral graph filter whose response depends on the prior subspace spectrum and on the sampling filter. This filter parallels the correction filter in SI sampling in standard signal processing. Since the input space and the sampling filter are almost arbitrary, our framework allows perfect recovery of many classes of input signals from a variety of different sampling patterns using a simple correction filter. For example, our method enables perfect recovery of non-bandlimited graph signals from their bandlimited measurements.

Graph filtering, which is founded on the theory of graph signal processing, is proved as a useful tool for image denoising. Most graph filtering methods focus on learning an ideal lowpass filter to remove noise, where clean images are restored from noisy ones by retaining the image components in low graph frequency bands. However, this lowpass filter has limited ability to separate the low-frequency noise from clean images such that it makes the denoising procedure less effective. To address this issue, we propose an adaptive weighted graph filtering (AWGF) method to replace the design of traditional ideal lowpass filter. In detail, we reassess the existing low-rank denoising method with adaptive regularizer learning (ARLLR) from the view of graph filtering. A shrinkage approach subsequently is presented on the graph frequency domain, where the components of noisy image are adaptively decreased in each band by calculating their component significances. As a result, it makes the proposed graph filtering more explainable and suitable for denoising. Meanwhile, we demonstrate a graph filter under the constraint of subspace representation is employed in the ARLLR method. Therefore, ARLLR can be treated as a special form of graph filtering. It not only enriches the theory of graph filtering, but also builds a bridge from the low-rank methods to the graph filtering methods. In the experiments, we perform the AWGF method with a graph filter generated by the classical graph Laplacian matrix. The results show our method can achieve a comparable denoising performance with several state-of-the-art denoising methods.

This paper studies the stability and control of networked systems from the perspective of the new field of signal processing over graphs. Specifically, we reformulate the linear quadratic optimal controller as a graph filter by demonstrating that it becomes separable when applying the graph Fourier transform. Accordingly, the graph-frequency components of the input signal are processed independently by a set of parallel controllers which are given in closed form by solving their Riccati equations. Additionally, we prove that the joint graph-and-temporal-frequency transfer function of the controlled system satisfies the condition for stability, that is, that the complement of its region of convergence fits inside a cylinder of unit radius. The results are universal in the sense that they hold true for any graph-frequency and do not depend on the specific eigenvalues of the network shift operator. Numerical tests in an directed circulant graph show that the unstable poles corresponding to graph-frequencies larger than one are shrunk towards the origin.

One of the main challenges of graph filters is the stability of their design. While classical graph filters allow for a stable design using optimal polynomial approximation theory, generalized graph filters tend to suffer from the ill-conditioning of the involved system matrix. This issue, accentuated for increasing graph filter orders, naturally leads to very large (small) filter coefficients or error saturation, casting a shadow on the benefits of these richer graph filter structures. In addition to this, data-driven design/learning of graph filters with large filter orders, even in the case of classical graph filters, suffers from the eigenvalue spread of the input data covariance matrix and mode coupling, leading to convergence-related issues as the ones observed when identifying time-domain filters with large orders. To alleviate these conditioning and convergence problems, and to reduce the overall design complexity, in this work, we propose a cascaded implementation of generalized graph filters and an efficient algorithm for designing the graph filter coefficients in both model- and data-driven settings. Further, we establish the connections of this implementation with so-called graph convolutional neural networks and demonstrate the performance of the proposed structure in different network applications. By the proposed approach, further error reduction and better design stability are achieved.

In graph signal processing, signals are processed by explicitly taking into account their underlying structure, which is generally characterized by a graph. In this field, graph filters play a major role to process such signals in the so-called graph frequency domain. In this paper, we focus on the design of autoregressive moving average (ARMA) graph filters and basically present two design approaches. The first approach is inspired by Prony's method, which considers a modified error between the modeled and the desired frequency response. The second approach is based on an iterative method, which finds the filter coefficients by iteratively minimizing the true error (instead of the modified error) between the modeled and the desired frequency response. The performance of the proposed design algorithms is evaluated and compared with finite impulse response (FIR) graph filters. The obtained results show that ARMA filters outperform FIR filters in terms of approximation accuracy even for the same computational cost.

Recently, the stability of graph filters has been studied as one of the key theoretical properties driving the highly successful graph convolutional neural networks (GCNs). The stability of a graph filter characterizes the effect of topology perturbation on the output of a graph filter, a fundamental building block for GCNs. Many existing results have focused on the regime of small perturbation with a small number of edge rewires. However, the number of edge rewires can be large in many applications. To study the latter case, this work departs from the previous analysis and proves a bound on the stability of graph filter relying on the filter’s frequency response. Assuming the graph filter is low pass, we show that the stability of the filter depends on perturbation to the community structure. As an application, we show that for stochastic block model graphs, the graph filter distance converges to a small constant when the number of nodes approaches infinity. Numerical simulations validate our findings.

Dynamic systems of graph signals are encountered in various applications, including social networks, power grids, and transportation. While such systems can often be described as state space (SS) models, tracking graph signals via conventional tools based on the Kalman filter (KF) and its variants is typically challenging. This is due to the nonlinearity, high dimensionality, irregularity of the domain, and complex modeling associated with real-world dynamic systems of graph signals. In this work, we study the tracking of graph signals using a hybrid model-based/data-driven approach. We develop the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GSP-KalmanNet</i>, which tracks the hidden graphical states from the graphical measurements by jointly leveraging graph signal processing (GSP) tools and deep learning (DL) techniques. The derivations of the GSP-KalmanNet are based on extending the KF to exploit the inherent graph structure via designing a graph frequency domain filtering and replacing the Kalman gain (KG) with a graph filter that minimizes the prediction error. Thus, it considerably simplifies the computational complexity entailed in processing high-dimensional signals and increases the robustness to small topology changes. Then, we use data to learn the KG, namely, the graph filter, following the recently proposed KalmanNet framework, which copes with partial and approximated modeling, without forcing a specific model over the noise statistics. Restricting the KG to a graph filter in the proposed GSP-KalmanNet reduces learned parameters, thereby enhancing stability. Our empirical results demonstrate that the GSP-KalmanNet achieves enhanced accuracy and run time performance, and improved robustness to model misspecifications compared with both model-based and data-driven benchmarks.

In this paper, we consider the problem of recovering random graph signals with complex values. For general Bayesian estimation of complex-valued vectors, it is known that the widely-linear minimum mean-squared-error (WLMMSE) estimator can achieve a lower mean-squared-error (MSE) than that of the linear minimum MSE (LMMSE) estimator. Inspired by the WLMMSE estimator, in this paper we develop the graph signal processing (GSP)-WLMMSE estimator, which minimizes the MSE among estimators that are represented as a two-channel output of a graph filter, i.e. widely-linear GSP estimators. We discuss the properties of the proposed GSP-WLMMSE estimator. In particular, we show that the MSE of the GSP-WLMMSE estimator is always equal to or lower than the MSE of the GSP-LMMSE estimator. The GSP-WLMMSE estimator is based on diagonal covariance matrices in the graph frequency domain, and thus has reduced complexity compared with the WLMMSE estimator. This property is especially important when using the sample-mean versions of these estimators that are based on a training dataset. We then state conditions under which the low-complexity GSP-WLMMSE estimator coincides with the WLMMSE estimator. In the simulations, we investigate two synthetic estimation problems (with linear and nonlinear models) and the problem of state estimation in power systems. For these problems, it is shown that the GSP-WLMMSE estimator outperforms the GSP-LMMSE estimator and achieves similar performance to that of the WLMMSE estimator.

This paper proposes a class of $M$-channel spectral graph filter banks with a symmetric structure, that is, the transform has sampling operations and spectral graph filters on both the analysis and synthesis sides. The filter banks achieve maximum decimation, perfect recovery, and orthogonality. Conventional spectral graph transforms with decimation have significant limitations with regard to the number of channels, the structures of the underlying graph, and their filter design. The proposed transform uses sampling in the graph frequency domain. This enables us to use any variation operators and apply the transforms to arbitrary graphs even when the filter banks have symmetric structures. We clarify the perfect reconstruction conditions and show design examples. An experiment on graph signal denoising conducted to examine the performance of the proposed filter bank is described.

In this article, we consider the parameter estimation problem for a class of subdiffusion processes which are characterized by the time-changed Ornstein–Uhlenbeck processes. Least squares method is used to obtain the estimator for the drift coefficient. First, we get the strong consistency, asymptotical normality and asymptotical mixed normality for the estimator on the condition that we can observe the process continuously. After that, weak consistent and asymptotic properties are derived basing on discrete observations when the time-change process is an inverse -stable () subordinator.

Recently deterministic dynamic models have become very popular in biomedical research and other scientific areas; examples include modeling human immunodeficiency virus (HIV) dynamics, pharmacokinetic/pharmacodynamic analysis, tumor cell kinetics, and genetic network modeling. In this article we propose estimation methods for the time-varying coefficients in deterministic dynamic systems that are usually described by a set of differential equations. Three two-stage local polynomial estimators are proposed, and their asymptotic normality is established. An alternative approach, a discretization method that is widely used in stochastic diffusion models, is also investigated. We show that the discretization method that uses the simple Euler discretization approach for the deterministic dynamic model does not achieve the optimal convergence rate compared with the proposed two-stage estimators. We use Monte Carlo simulations to study the finite-sample performance, and use a real data application to HIV dynamics to illustrate the proposed methods.

Motivated by recent work studying massive imaging data in the neuroimaging literature, we propose multivariate varying coefficient models (MVCM) for modeling the relation between multiple functional responses and a set of covariates. We develop several statistical inference procedures for MVCM and systematically study their theoretical properties. We first establish the weak convergence of the local linear estimate of coefficient functions, as well as its asymptotic bias and variance, and then we derive asymptotic bias and mean integrated squared error of smoothed individual functions and their uniform convergence rate. We establish the uniform convergence rate of the estimated covariance function of the individual functions and its associated eigenvalue and eigenfunctions. We propose a global test for linear hypotheses of varying coefficient functions, and derive its asymptotic distribution under the null hypothesis. We also propose a simultaneous confidence band for each individual effect curve. We conduct Monte Carlo simulation to examine the finite-sample performance of the proposed procedures. We apply MVCM to investigate the development of white matter diffusivities along the genu tract of the corpus callosum in a clinical study of neurodevelopment.

Motivated by recent work studying massive imaging data in the neuroimaging literature, we propose multivariate varying coefficient models (MVCM) for modeling the relation between multiple functional responses and a set of covariates. We develop several statistical inference procedures for MVCM and systematically study their theoretical properties. We first establish the weak convergence of the local linear estimate of coefficient functions, as well as its asymptotic bias and variance, and then we derive asymptotic bias and mean integrated squared error of smoothed individual functions and their uniform convergence rate. We establish the uniform convergence rate of the estimated covariance function of the individual functions and its associated eigenvalue and eigenfunctions. We propose a global test for linear hypotheses of varying coefficient functions, and derive its asymptotic distribution under the null hypothesis. We also propose a simultaneous confidence band for each individual effect curve. We conduct Monte Carlo simulation to examine the finite-sample performance of the proposed procedures. We apply MVCM to investigate the development of white matter diffusivities along the genu tract of the corpus callosum in a clinical study of neurodevelopment.