Near-optimal Sampling Research Articles

Evolving Improved Sampling Protocols for Dose–Response Modelling Using Genetic Algorithms with a Profile-Likelihood Metric

Practical limitations of quality and quantity of data can limit the precision of parameter identification in mathematical models. Model-based experimental design approaches have been developed to minimise parameter uncertainty, but the majority of these approaches have relied on first-order approximations of model sensitivity at a local point in parameter space. Practical identifiability approaches such as profile-likelihood have shown potential for quantifying parameter uncertainty beyond linear approximations. This research presents a genetic algorithm approach to optimise sample timing across various parameterisations of a demonstrative PK-PD model with the goal of aiding experimental design. The optimisation relies on a chosen metric of parameter uncertainty that is based on the profile-likelihood method. Additionally, the approach considers cases where multiple parameter scenarios may require simultaneous optimisation. The genetic algorithm approach was able to locate near-optimal sampling protocols for a wide range of sample number (n = 3–20), and it reduced the parameter variance metric by 33–37% on average. The profile-likelihood metric also correlated well with an existing Monte Carlo-based metric (with a worst-case r > 0.89), while reducing computational cost by an order of magnitude. The combination of the new profile-likelihood metric and the genetic algorithm demonstrate the feasibility of considering the nonlinear nature of models in optimal experimental design at a reasonable computational cost. The outputs of such a process could allow for experimenters to either improve parameter certainty given a fixed number of samples, or reduce sample quantity while retaining the same level of parameter certainty.

Predictive and Near-Optimal Sampling for View Materialization in Video Databases

Scalable video query optimization has re-emerged as an attractive research topic in recent years. The OTIF system, a video database with cutting-edge efficiency, has introduced a new paradigm of utilizing view materialization to facilitate online query processing. Specifically, it stores the results of multi-object tracking queries to answer common video queries with sub-second latency. However, the cost associated with view materialization in OTIF is prohibitively high for supporting large-scale video streams. In this paper, we study efficient MOT-based view materialization in video databases. We first conduct a theoretical analysis and establish two types of optimality measures that serve as lower bounds for video frame sampling. In order to minimize the number of processed video frames, we propose a novel predictive sampling framework, namely LEAP, exhibits near-optimal sampling performance. Its efficacy relies on a data-driven motion manager that enables accurate trajectory prediction, a compact object detection model via knowledge distillation, and a robust cross-frame associator to connect moving objects in two frames with a large time gap. Extensive experiments are conducted in 7 real datasets, with 7 baselines and a comprehensive query set, including selection, aggregation and top-k queries. The results show that with comparable query accuracy to OTIF, our LEAP can reduce the number of processed video frames by up to 9× and achieve 5× speedup in query processing time. Moreover, LEAP demonstrates impressive throughput when handling large-scale video streams, as it leverages a single NVIDIA RTX 3090ti GPU to support real-time MOT-based view materialization from 160 video streams simultaneously.

Robust low-rank diffraction separation and imaging by CUR matrix decomposition

Diffractions from underground discontinuities, which appear as common wavefields in seismic records, contain rich geologic information regarding small-scale structures. As a result of their weak amplitude characteristics, a key preliminary task in imaging subsurface inhomogeneities using seismic diffractions is to simultaneously eliminate strong reflections and separate weak diffractions. Traditional low-rank diffraction separation methods predict linear reflections and separate diffractions by applying a low-rank approximation, such as truncated singular value decomposition (TSVD). However, these methods require the accurate estimation of the rank, which influences the separation and imaging quality of diffractions. A robust low-rank diffraction separation method is developed using CUR matrix decomposition rather than the TSVD calculation to avoid the rank estimate. CUR matrix decomposition expresses a data matrix as a product of the matrices [Formula: see text], [Formula: see text], and [Formula: see text] by randomly selecting a small number of actual columns and rows from the matrix to achieve a low-rank approximation. A near-optimal sampling algorithm is used to randomly select columns and rows from the Hankel matrix and calculate the CUR decomposition. Oversampling of columns and rows effectively eliminates the requirement for an accurate rank. Moreover, synthetic and field applications demonstrate the good performance of our CUR-based diffraction separation method in attenuating reflections and highlighting diffractions.

Optimal Sampling and Christoffel Functions on General Domains

We consider the problem of reconstructing an unknown function \(u\in L^2(D,\mu )\) from its evaluations at given sampling points \(x^1,\dots ,x^m\in D\), where \(D\subset {\mathbb {R}}^d\) is a general domain and \(\mu \) a probability measure. The approximation is picked from a linear space \(V_n\) of interest where \(n=\dim (V_n)\). Recent results (Cohen and Migliorati in SMAI J Comput Math 3:181–203, 2017, Doostan and Hampton in Comput Methods Appl Mech Eng 290:73–97, 2015, Jakeman et al. in Math Comput 86:1913–1947, 2017) have revealed that certain weighted least-squares methods achieve near best (or instance optimal) approximation with a sampling budget m that is proportional to n, up to a logarithmic factor \(\ln (2n/\varepsilon )\), where \(\varepsilon >0\) is a probability of failure. The sampling points should be picked at random according to a well-chosen probability measure \(\sigma \) whose density is given by the inverse Christoffel function that depends both on \(V_n\) and \(\mu \). While this approach is greatly facilitated when D and \(\mu \) have tensor product structure, it becomes problematic for domains D with arbitrary geometry since the optimal measure depends on an orthonormal basis of \(V_n\) in \(L^2(D,\mu )\) which is not explicitly given, even for simple polynomial spaces. Therefore, sampling according to this measure is not practically feasible. One computational solution recently proposed in Adcock and Huybrechs (Approximating smooth, multivariate functions on irregular domains, forum of mathematics, sigma, Cambridge University Press, Cambridge, 2020) relies on using the restrictions of an orthonormal basis of \(V_n\) defined on a simpler bounding domain and sampling according to the original probability measure \(\mu \), in turn giving up on the optimal sampling budget \(m\sim n\). In this paper, we discuss practical sampling strategies, which amounts to using a perturbed measure \(\widetilde{\sigma }\) that can be computed in an offline stage, not involving the measurement of u, as recently proposed in Adcock and Cardenas (SIAM J Math Data Sci 2:607–630, 2020) and Migliorati (IMA J Numer Anal, 2020. https://doi.org/10.1093/imanum/draa023). We show that near best approximation is attained by the resulting weighted least-squares method at near-optimal sampling budget and we discuss multilevel approaches that preserve optimality of the cumulated sampling budget when the spaces \(V_n\) are iteratively enriched. These strategies rely on the knowledge of a-priori upper bounds B(n) on the inverse Christoffel function for the space \(V_n\) and the domain D. We establish bounds of the form \(\mathcal O(n^r)\) for spaces \(V_n\) of multivariate algebraic polynomials of given total degree, and for general domains D. The exact growth rate r depends on the regularity of the domain, in particular \(r=2\) for domains with Lipschitz boundaries and \(r=\frac{d+1}{d}\) for smooth domains.

Node Generation for RBF-FD Methods by QR Factorization

Polyharmonic spline (PHS) radial basis functions (RBFs) have been used in conjunction with polynomials to create RBF finite-difference (RBF-FD) methods. In 2D, these methods are usually implemented with Cartesian nodes, hexagonal nodes, or most commonly, quasi-uniformly distributed nodes generated through fast algorithms. We explore novel strategies for computing the placement of sampling points for RBF-FD methods in both 1D and 2D while investigating the benefits of using these points. The optimality of sampling points is determined by a novel piecewise-defined Lebesgue constant. Points are then sampled by modifying a simple, robust, column-pivoting QR algorithm previously implemented to find sets of near-optimal sampling points for polynomial approximation. Using the newly computed sampling points for these methods preserves accuracy while reducing computational costs by mitigating stencil size restrictions for RBF-FD methods. The novel algorithm can also be used to select boundary points to be used in conjunction with fast algorithms that provide quasi-uniformly distributed nodes.

Optimal sampling design for spatial capture-recapture.

Spatial capture-recapture (SCR) has emerged as the industry standard for estimating population density by leveraging information from spatial locations of repeat encounters of individuals. The precision of density estimates depends fundamentally on the number and spatial configuration of traps. Despite this knowledge, existing sampling design recommendations are heuristic and their performance remains untested for most practical applications. To address this issue, we propose a genetic algorithm that minimizes any sensible, criteria-based objective function to produce near-optimal sampling designs. To motivate the idea of optimality, we compare the performance of designs optimized using three model-based criteria related to the probability of capture. We use simulation to show that these designs outperform those based on existing recommendations in terms of bias, precision, and accuracy in the estimation of population size. Our approach, available as a function in the R package oSCR, allows conservation practitioners and researchers to generate customized and improved sampling designs for wildlife monitoring.

Homeostatic Plasticity Shapes the Retinal Response to Photoreceptor Degeneration.

Homeostatic plasticity stabilizes input and activity levels during neural development, but whether it can restore connectivity and preserve circuit function during neurodegeneration is unknown. Photoreceptor degeneration is the most common cause of blindness in the industrialized world. Visual deficits are dominated by cone loss, which progresses slowly, leaving a window during which rewiring of second-order neurons (i.e., bipolar cells) could preserve function. Here we establish a transgenic model to induce cone degeneration with precise control and analyze bipolar cell responses and their effects on vision through anatomical reconstructions, invivo electrophysiology, and behavioral assays. In young retinas, we find that three bipolar cell types precisely restore input synapse numbers when 50% of cones degenerate but one does not. Of the three bipolar cell types that rewire, two contact new cones within stable dendritic territories, whereas one expands its dendrite arbors to reach new partners. In mature retinas, only one of four bipolar cell types rewires homeostatically. This steep decline in homeostatic plasticity is accompanied by reduced light responses of bipolar cells and deficits in visual behaviors. By contrast, light responses and behavioral performance are preserved when cones degenerate in young mice. Our results reveal unexpected cell type specificity and a steep maturational decline of homeostatic plasticity. The effect of homeostatic plasticity on functional outcomes identify it as a promising therapeutic target for retinal and other neurodegenerative diseases.

Deep Reinforced Learning Tree for Spatiotemporal Monitoring With Mobile Robotic Wireless Sensor Networks

This paper concerns the deployment problem of wireless sensor networks (WSNs) with mobile robotic sensor nodes for spatiotemporal monitoring. The proposed approach, deep reinforced learning tree (DRLT), utilizes deep reinforcement learning (DRL) to improve the efficiency of searching the most informative sampling locations. The parameterized sampling locations in an infinite horizon space are modeled according to their spatiotemporal correlations and subject to various constraints, including field estimation error and information gain. And the model-based information gain can be calculated efficiently over an infinite horizon. In this manner, the effectiveness of the sampling locations is learned through DRLT during the exploration by the robotic sensors. Then DRLT can instruct the robotic sensors to avoid unnecessary sampling locations in future iterations. Also, it is proved in this paper that the proposed algorithm is capable of searching for the near-optimal sampling locations effectively and guaranteeing a minimum field estimation error. Simulation based on national oceanic and atmospheric administration (NOAA) datasets is presented, which demonstrates the significant enhancements made by the proposed algorithm. Compared with the traditional approaches, such as the information theory-based greedy approach or random sampling, the proposed method shows a superior performance with regard to both estimation error and planning efficiency.

Cross entropy-based importance sampling using Gaussian densities revisited

The computation of the probability of a rare (failure) event is a common task in structural reliability analysis. In most applications, the numerical model defining the rare event is nonlinear and the resulting failure domain often multimodal. One strategy for estimating the probability of failure in this context is the importance sampling method. The efficiency of importance sampling depends on the choice of the importance sampling density. A near-optimal sampling density can be found through application of the cross entropy method. The cross entropy method is an adaptive sampling approach that determines the sampling density through minimizing the Kullback-Leibler divergence between the theoretically optimal importance sampling density and a chosen parametric family of distributions. In this paper, we investigate the suitability of the multivariate normal distribution and the Gaussian mixture model as importance sampling densities within the cross entropy method. Moreover, we compare the performance of the cross entropy method to sequential importance sampling, another recently proposed adaptive sampling approach, which uses the Gaussian mixture distribution as a proposal distribution within a Markov Chain Monte Carlo algorithm. For the parameter updating of the Gaussian mixture within the cross entropy method, we propose a modified version of the expectation-maximization algorithm that works with weighted samples. To estimate the number of distributions in the mixture, the density-based spatial clustering of applications with noise (DBSCAN) algorithm is adapted to the use of weighted samples. We compare the performance of the different methods in several examples, including component reliability problems, system reliability problems and reliability in varying dimensions. The results show that the cross entropy method using a single Gaussian outperforms the cross entropy method using Gaussian mixture and that both distribution types are not suitable for high dimensional reliability problems.

Sparse Sampling for Inverse Problems With Tensors

We consider the problem of designing sparse sampling strategies for multidomain signals, which can be represented using tensors that admit a known multilinear decomposition. We leverage the multidomain structure of tensor signals and propose to acquire samples using a Kronecker-structured sensing function, thereby circumventing the curse of dimensionality. For designing such sensing functions, we develop low-complexity greedy algorithms based on submodular optimization methods to compute near-optimal sampling sets. We present several numerical examples, ranging from multiantenna communications to graph signal processing, to validate the developed theory.

A near-optimal sampling strategy for sparse recovery of polynomial chaos expansions

Compressive sampling has become a widely used approach to construct polynomial chaos surrogates when the number of available simulation samples is limited. Originally, these expensive simulation samples would be obtained at random locations in the parameter space. It was later shown that the choice of sample locations could significantly impact the accuracy of resulting surrogates. This motivated new sampling strategies or design-of-experiment approaches, such as coherence-optimal sampling, which aim at improving the coherence property. In this paper, we propose a sampling strategy that can identify near-optimal sample locations that lead to improvement in local-coherence property and also enhancement of cross-correlation properties of measurement matrices. We provide theoretical motivations for the proposed sampling strategy along with several numerical examples that show that our near-optimal sampling strategy produces substantially more accurate results, compared to other sampling strategies.

Uncertainty Quantification of Stochastic Simulation for Black-box Computer Experiments

Stochastic simulations applied to black-box computer experiments are becoming more widely used to evaluate the reliability of systems. Yet, the reliability evaluation or computer experiments involving many replications of simulations can take significant computational resources as simulators become more realistic. To speed up, importance sampling coupled with near-optimal sampling allocation for these experiments is recently proposed to efficiently estimate the probability associated with the stochastic system output. In this study, we establish the central limit theorem for the probability estimator from such procedure and construct an asymptotically valid confidence interval to quantify estimation uncertainty. We apply the proposed approach to a numerical example and present a case study for evaluating the structural reliability of a wind turbine.

Compressive Sampling Using Annihilating Filter-Based Low-Rank Interpolation

While the recent theory of compressed sensing provides an opportunity to overcome the Nyquist limit in recovering sparse signals, a solution approach usually takes the form of an inverse problem of an unknown signal, which is crucially dependent on specific signal representation. In this paper, we propose a drastically different two-step Fourier compressive sampling framework in a continuous domain that can be implemented via measurement domain interpolation, after which signal reconstruction can be done using classical analytic reconstruction methods. The main idea originates from the fundamental duality between the sparsity in the primary space and the low-rankness of a structured matrix in the spectral domain, showing that a low-rank interpolator in the spectral domain can enjoy all of the benefits of sparse recovery with performance guarantees. Most notably, the proposed low-rank interpolation approach can be regarded as a generalization of recent spectral compressed sensing to recover large classes of finite rate of innovations (FRI) signals at a near-optimal sampling rate. Moreover, for the case of cardinal representation, we can show that the proposed low-rank interpolation scheme will benefit from inherent regularization and an optimal incoherence parameter. Using a powerful dual certificate and the golfing scheme, we show that the new framework still achieves a near-optimal sampling rate for a general class of FRI signal recovery, while the sampling rate can be further reduced for a class of cardinal splines. Numerical results using various types of FRI signals confirm that the proposed low-rank interpolation approach offers significantly better phase transitions than conventional compressive sampling approaches.

Cross-entropy-based Adaptive Importance Sampling for Probabilistic Seismic Risk Assessment of Lifeline Networks Considering Spatial Correlation

There exist pressing research needs for developing sophisticated methodologies for probabilistic seismic risk assessment (PSRA) of complex lifeline networks, which are critical backbones of today's urban communities. In particular, ground-motion intensities at different locations of lifeline networks show significant variability and spatial-correlation, which should be properly considered in evaluating structural damage of lifeline networks and their corresponding network-level performance. In addition, the probabilities of the post-disaster states of the network often shows large variability, so simulation-based approaches may require a huge number of computational simulations if the set of the possible network states includes rare events. To improve the efficiency of simulation-based approach while maintaining its broad applicability, adaptive importance sampling methods have been recently introduced. In this study, a new PSRA approach is developed for lifeline networks by extending a cross-entropy-based adaptive importance sampling (CE-AIS) to estimate the probabilities of multiple possible states of the network utility concurrently. To test and demonstrate the proposed approach, a hypothetical traffic network under seismic hazard from surrounding active faults is investigated. The results of the numerical example show that the proposed approach, termed as CE-based concurrent AIS (CE-CAIS), identifies a near-optimal sampling density through only a few rounds of pre-sampling and drastically improves efficiency of simulation-based PSRA. It is also noted that CE-CAIS does not rely on a subjective intuition to select importance sampling density or require additional reliability analysis to identify important regions. The proposed method is expected to improve PSRA of lifeline networks in terms of efficiency and applicability, and provide new insights into the risk assessment and management of such urban infrastructures.

Cross-entropy-based adaptive importance sampling using von Mises-Fisher mixture for high dimensional reliability analysis

In order to address challenges in performing importance sampling in a high dimensional space of random variables, the paper develops a cross-entropy-based adaptive importance sampling technique that employs a von Mises-Fisher mixture as the sampling density model. By small-size pre-samplings, the proposed approach first finds a near-optimal sampling density by minimizing the Kullback–Leibler cross entropy between a von Mises-Fisher mixture model and the absolute best importance sampling density. To facilitate the minimization process, updating rules for parameters of the von Mises-Fisher mixture model are derived. Various practical issues associated with the updating rules are discussed and heuristic rules to improve the performance of the importance sampling are introduced. At the stage of final sampling, two slightly different sampling strategies are proposed to provide analysis options. Three numerical examples are investigated to test and demonstrate the proposed importance sampling method. The numerical examples show that the proposed approach, applicable to both component and system reliability problems, has superior performance for high dimensional reliability analysis problems with low failure probabilities.

Adaptive Sampling of RF Fingerprints for Fine-Grained Indoor Localization

Indoor localization is a supporting technology for a broadening range of pervasive wireless applications. One promis- ing approach is to locate users with radio frequency fingerprints. However, its wide adoption in real-world systems is challenged by the time- and manpower-consuming site survey process, which builds a fingerprint database a priori for localization. To address this problem, we visualize the 3-D RF fingerprint data as a function of locations (x-y) and indices of access points (fingerprint), as a tensor and use tensor algebraic methods for an adaptive tubal-sampling of this fingerprint space. In particular using a recently proposed tensor algebraic framework in [1] we capture the complexity of the fingerprint space as a low-dimensional tensor-column space. In this formulation the proposed scheme exploits adaptivity to identify reference points which are highly informative for learning this low-dimensional space. Further, under certain incoherency conditions we prove that the proposed scheme achieves bounded recovery error and near-optimal sampling complexity. In contrast to several existing work that rely on random sampling, this paper shows that adaptivity in sampling can lead to significant improvements in localization accuracy. The approach is validated on both data generated by the ray-tracing indoor model which accounts for the floor plan and the impact of walls and the real world data. Simulation results show that, while maintaining the same localization accuracy of existing approaches, the amount of samples can be cut down by 71% for the high SNR case and 55% for the low SNR case.

Continuous adaptive radial sampling of k-space from real-time physiologic feedback in MRI

Results Across several combinations of shots (number of previous periods) and segments (radial projects per period), ARKS resulted in higher sampling uniformity than random radial sampling as well as golden-angle sampling. An example of ARKS sampling for a subject with a regular arrhythmia is shown in Fig 1. The rhythm in this subject featured a normal beat, a second beat interrupted by a pre-ventricular contraction, followed by a third long beat with a longer RR-interval. Each of the three beats were correctly differentiated and segmented for ARKS, despite distortion of the ECG waveform by the electrohemodynamic effect of the MRI scanner. Simulated ARKS images correctly depicted the tri-beat arrhythmia morphology in real-time (Fig. 1F). Conclusions Adaptive real-time k-space sampling is a near-optimal sampling scheme that shares the benefits of real-time display, improves the signal-to-noise and artifact levels of a multi-shot radial k-space trajectory and, by adapting to a physiologic signal, mitigates deleterious motion artifacts. We anticipate that this approach to k-space sampling would have benefits for clinical examinations of the heart.

A Tale of Two Retinal Domains: Near-Optimal Sampling of Achromatic Contrasts in Natural Scenes through Asymmetric Photoreceptor Distribution

For efficient coding, sensory systems need to adapt to the distribution of signals to which they are exposed. In vision, natural scenes above and below the horizon differ in the distribution of chromatic and achromatic features. Consequently, many species differentially sample light in the sky and on the ground using an asymmetric retinal arrangement of short- (S, "blue") and medium- (M, "green") wavelength-sensitive photoreceptor types. Here, we show that in mice this photoreceptor arrangement provides for near-optimal sampling of natural achromatic contrasts. Two-photon population imaging of light-driven calcium signals in the synaptic terminals of cone-photoreceptors expressing a calcium biosensor revealed that S, but not M cones, preferred dark over bright stimuli, in agreement with the predominance of dark contrasts in the sky but not on the ground. Therefore, the different cone types do not only form the basis of "color vision," but in addition represent distinct (achromatic) contrast-selective channels.

Cross-entropy-based adaptive importance sampling using Gaussian mixture

Structural reliability analysis frequently requires the use of sampling-based methods, particularly for the situation where the failure domain in the random variable space is complex. One of the most efficient and widely utilized methods to use in such a situation is importance sampling. Recently, an adaptive importance sampling method was proposed to find a near-optimal importance sampling density by minimizing Kullback–Leibler cross entropy, i.e. a measure of the difference between the absolute best sampling density and the one being used for the importance sampling. In this paper, the adaptive importance sampling approach is further developed by incorporating a nonparametric multimodal probability density function model called the Gaussian mixture as the importance sampling density. This model is used to fit the complex shape of the absolute best sampling density functions including those with multiple important regions. An efficient procedure is developed to update the Gaussian mixture model toward a near-optimal density using a small size of pre-samples. The proposed method needs only a few steps to achieve a near-optimal sampling density, and shows significant improvement in efficiency and accuracy for a variety of component and system reliability problems. The method requires far less samples than both crude Monte Carlo simulation and the cross-entropy-based adaptive importance sampling method employing a unimodal density function; thus achieving relatively small values of the coefficient of variation efficiently. The computational efficiency and accuracy of the proposed method are not hampered by the probability level, dimension of random variable space, and curvatures of limit-state function. Moreover, the distribution model parameters of the Gaussian densities in the obtained near-optimal density help identify important areas in the random variable space and their relative importance.

Rats track odour trails accurately using a multi-layered strategy with near-optimal sampling

Tracking odour trails is a crucial behaviour for many animals, often leading to food, mates or away from danger. It is an excellent example of active sampling, where the animal itself controls how to sense the environment. Here we show that rats can track odour trails accurately with near-optimal sampling. We trained rats to follow odour trails drawn on paper spooled through a treadmill. By recording local field potentials (LFPs) from the olfactory bulb, and sniffing rates, we find that sniffing but not LFPs differ between tracking and non-tracking conditions. Rats can track odours within ~1 cm, and this accuracy is degraded when one nostril is closed. Moreover, they show path prediction on encountering a fork, wide 'casting' sweeps on encountering a gap and detection of reappearance of the trail in 1-2 sniffs. We suggest that rats use a multi-layered strategy, and achieve efficient sampling and high accuracy in this complex task.