Large-scale Estimation Problems Research Articles

Block Coordinate Regularization by Denoising

We consider the problem of reconstructing an image from its noisy measurements using a prior specified only with an image denoiser. Recent work on plug-and-play priors (PnP) and regularization by denoising (RED) has shown the state-of-the-art performance of image reconstruction algorithms under such priors in a range of imaging problems. In this work, we develop a new block coordinate RED algorithm that decomposes a large-scale estimation problem into a sequence of updates over a small subset of the unknown variables. We theoretically analyze the convergence of the algorithm and discuss its relationship to the traditional proximal optimization. Our analysis complements and extends recent theoretical results for RED-based estimation methods. We numerically validate our method using several denoising priors, including those based on deep neural nets.

Large-scale estimation in Rasch models: asymptotic results, approximations of the variance–covariance matrix of item parameters, and divide-and-conquer estimation

Rasch-type item response models are often estimated via a conditional maximum-likelihood approach. This article elaborates on the asymptotics of conditional maximum-likelihood estimates for an increasing number of items, important for modern data settings where a large number of items need to be scaled. Using approximations of the variance–covariance matrix based on Edgeworth expansions, the problem is studied theoretically as well as computationally. In a subsequent step, these results are used to split the large-scale estimation problem into smaller sub-problems involving blocks of items. These item blocks are estimated separately from each other and, finally, merged back into the full parameter vector (divide-and-conquer). By means of simulation studies, and in conjunction with the asymptotic results, it was found that block sizes in the range of 30–40 items approximate the full-scale estimators with a negligible loss in precision. It is also shown how varying block sizes affect the running time needed to fit the model.

Localized Distributed Kalman Filters for Large-Scale Systems

This paper presents a scalable method to design large-scale Kalman-like filters for a class of linear systems. In particular, we consider systems for which both the propagation of dynamics through the plant and the exchange of information between estimators/sensors is subject to delays. Under suitable assumptions on these delays, our proposed Kalman-like filter has the following desirable properties: (1) each local estimator only needs to collect the information within a localized region to estimate its local state, and (2) each local estimator can be designed by solving a local optimization problem using local plant model information. The decomposition of the global problem into local subproblems thus allows for the method to scale to arbitrarily large heterogeneous systems - this is clearly an extremely favorable property for large-scale estimation problems. The effectiveness of our algorithm is demonstrated on a randomized heterogeneous example with 51200 states, in which the traditional Kalman filter cannot be computed within a reasonable amount of time.

Distributed Multiple Shooting for Optimal Control of Large Interconnected Systems

Large interconnected systems consist of a multitude of subsystems with their own dynamics, but coupled with each other via input-output connections. Each subsystem is typically modelled by ordinary differential equations or differential-algebraic equations. Simulation and optimal control of such systems pose a challenge both with respect to CPU time and memory requirements. We address optimal control of such systems by applying “distributed multiple shooting”, a generalization of the direct multiple shooting method, which uses the decomposable structure of the system in order to obtain a highly parallel algorithm. The interconnections are allowed to be infeasible during the iterations but are driven to feasibility by a Newtontype optimization algorithm. We evaluate the performance of the distributed multiple shooting method on a large scale estimation problem.

Embedded Trees: Estimation of Gaussian Processes on Graphs with Cycles

Graphical models provide a powerful general framework for encoding the structure of large-scale estimation problems. However, the graphs describing typical real-world phenomena contain many cycles, making direct estimation procedures prohibitively costly. In this paper, we develop an iterative inference algorithm for general Gaussian graphical models. It operates by exactly solving a series of modified estimation problems on spanning trees embedded within the original cyclic graph. When these subproblems are suitably chosen, the algorithm converges to the correct conditional means. Moreover, and in contrast to many other iterative methods, the tree-based procedures we propose can also be used to calculate exact error variances. Although the conditional mean iteration is effective for quite densely connected graphical models, the error variance computation is most efficient for sparser graphs. In this context, we present a modeling example suggesting that very sparsely connected graphs with cycles may provide significant advantages relative to their tree-structured counterparts, thanks both to the expressive power of these models and to the efficient inference algorithms developed herein. The convergence properties of the proposed tree-based iterations are characterized both analytically and experimentally. In addition, by using the basic tree-based iteration to precondition the conjugate gradient method, we develop an alternative, accelerated iteration that is finitely convergent. Simulation results are presented that demonstrate this algorithm's effectiveness on several inference problems, including a prototype distributed sensing application.

A development of a decomposed estimation algorithm for large scale systems

This paper introduces a developed estimation algorithm using a partitioning method. The main advantage of the proposed approach is that it can handle large scale estimation problems in which a large number of parameters have to be identified based on sampling measurements. Therefore, the obstacles resulting from the dimensionality of the problem and the numerical instability when applying classical estimation procedures can be completely avoided. The application of the developed method on the problem of studying the impact of socio-economic level on the communication structure in rural Egypt showed its excellent characteristics and behaviour in large scale estimation problems.