Noisy Estimates Research Articles

Are price limits cooling off agricultural futures markets?

AbstractIn the past few years, the lean hog and live cattle futures markets have experienced significantly heightened volatility and frequent limit moves. In this paper, we study whether price limits help resolve uncertainty and facilitate liquidity provision in the two markets and how they affect trading in the limit‐free options market. In contrast to previous literature that is based on daily data, we use intraday futures and options data from 2014 to 2019 that allows for a better characterization of market behavior around limit moves. Consistent with microstructure theories, we find that price limits neither reduce volatility nor improve liquidity. Instead, they add to the high uncertainty that precedes the limit move, leading to significantly higher volatility and lower liquidity when trading resumes. Further, contrary to the notion that trading migrates to the limit‐free options market, options volume drops and liquidity drains during limit moves. The options‐implied futures price is only a biased, inefficient, and highly noisy estimate of the equilibrium futures price on locked‐limit days. Overall, the findings suggest that rather than allowing the markets to cool off, price limits adversely affect market quality in presence of large price movements.

Spatial Bayesian GLM on the cortical surface produces reliable task activations in individuals and groups

The general linear model (GLM) is a widely popular and convenient tool for estimating the functional brain response and identifying areas of significant activation during a task or stimulus. However, the classical GLM is based on a massive univariate approach that does not explicitly leverage the similarity of activation patterns among neighboring brain locations. As a result, it tends to produce noisy estimates and be underpowered to detect significant activations, particularly in individual subjects and small groups. A recently proposed alternative, a cortical surface-based spatial Bayesian GLM, leverages spatial dependencies among neighboring cortical vertices to produce more accurate estimates and areas of functional activation. The spatial Bayesian GLM can be applied to individual and group-level analysis. In this study, we assess the reliability and power of individual and group-average measures of task activation produced via the surface-based spatial Bayesian GLM. We analyze motor task data from 45 subjects in the Human Connectome Project (HCP) and HCP Retest datasets. We also extend the model to multi-run analysis and employ subject-specific cortical surfaces rather than surfaces inflated to a sphere for more accurate distance-based modeling. Results show that the surface-based spatial Bayesian GLM produces highly reliable activations in individual subjects and is powerful enough to detect trait-like functional topologies. Additionally, spatial Bayesian modeling enhances reliability of group-level analysis even in moderately sized samples (n=45). Notably, the power of the spatial Bayesian GLM to detect activations above a scientifically meaningful effect size is nearly invariant to sample size, exhibiting high power even in small samples (n=10). The spatial Bayesian GLM is computationally efficient in individuals and groups and is convenient to implement with the open-source BayesfMRI R package.

Effectiveness Assessment of the Search-Based Statistical Structural Testing

Search-based statistical structural testing (SBSST) is a promising technique that uses automated search to construct input distributions for statistical structural testing. It has been proved that a simple search algorithm, for example, the hill-climber is able to optimize an input distribution. However, due to the noisy fitness estimation of the minimum triggering probability among all cover elements (Tri-Low-Bound), the existing approach does not show a satisfactory efficiency. Constructing input distributions to satisfy the Tri-Low-Bound criterion requires an extensive computation time. Tri-Low-Bound is considered a strong criterion, and it is demonstrated to sustain a high fault-detecting ability. This article tries to answer the following question: if we use a relaxed constraint that significantly reduces the time consumption on search, can the optimized input distribution still be effective in fault-detecting ability? In this article, we propose a type of criterion called fairness-enhanced-sum-of-triggering-probability (p-L1-Max). The criterion utilizes the sum of triggering probabilities as the fitness value and leverages a parameter p to adjust the uniformness of test data generation. We conducted extensive experiments to compare the computation time and the fault-detecting ability between the two criteria. The result shows that the 1.0-L1-Max criterion has the highest efficiency, and it is more practical to use than the Tri-Low-Bound criterion. To measure a criterion’s fault-detecting ability, we introduce a definition of expected faults found in the effective test set size region. To measure the effective test set size region, we present a theoretical analysis of the expected faults found with respect to various test set sizes and use the uniform distribution as a baseline to derive the effective test set size region’s definition.

System Equivalent Flux Density of a Polarimetric Tripole Radio Interferometer

Electromagnetic and information properties of tripole antennas have been studied since the 1980s. In radio astronomy, tripole antennas find an application in space telescopes. More recently, a radio interferometer with satellite-borne tripole antennas is being considered for a lunar orbiting radio telescope to observe very long wavelengths. System equivalent flux density (SEFD) is an important figure of merit of a radio telescope. This paper aims to derive a general expression for SEFD of a polarimetric tripole interferometer. The derivation makes only two basic and reasonable assumptions. First, the noise under consideration is zero mean and when expressed in complex phasor domain, has independent and identically distributed (iid) real and imaginary components. Correlated and non-identically distributed noise sources are allowed as long as the real and imaginary components remain iid. Second, the system noise is uncorrelated between the elements separated by a baseline distance. The SEFD expression is derived from first principles, that is the standard deviation of the noisy flux estimate in a target direction due to system noise. The resulting SEFD expression is expressed as a simple matrix operation that involves a mixture of the system temperatures of each antenna and the Jones matrix elements. It is not limited to tripoles, but rather, fully extensible to multipole antennas; it is not limited to mutually orthogonal antennas. To illustrate the usefulness of the expression and how the formula is applied, we discuss an example calculation based on a tripole interferometer on lunar orbit for ultra-long wavelengths observation. We compared the SEFD results based on a formula assuming short dipoles and the general expression. As expected, the SEFDs converge at the ultra-long wavelengths where the dipoles are well-approximated as short dipoles. The general SEFD expression can be applied to any multipole antenna systems with arbitrary shapes.

A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning

We study the consensus decentralized optimization problem where the objective function is the average of <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> agents private non-convex cost functions; moreover, the agents can only communicate to their neighbors on a given network topology. The stochastic learning setting is considered in this paper where each agent can only access a noisy estimate of its gradient. Many decentralized methods can solve such problem including EXTRA, Exact-Diffusion/D<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula>, and gradient-tracking. Unlike the famed <small>Dsgd</small> algorithm, these methods have been shown to be robust to the heterogeneity across the local cost functions. However, the established convergence rates for these methods indicate that their sensitivity to the network topology is worse than <small>Dsgd</small>. Such theoretical results imply that these methods can perform much worse than <small>Dsgd</small> over sparse networks, which, however, contradicts empirical experiments where <small>Dsgd</small> is observed to be more sensitive to the network topology. In this work, we study a general <u>s</u>tochastic <u>u</u>nified <u>d</u>ecentralized <u>a</u>lgorithm (<b>SUDA</b>) that includes the above methods as special cases. We establish the convergence of <b>SUDA</b>&#x00A0;under both non-convex and the Polyak-&#x0141;ojasiewicz condition settings. Our results provide improved network topology dependent bounds for these methods (such as Exact-Diffusion/D<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> and gradient-tracking) compared with existing literature. Moreover, our results show that these methods are often less sensitive to the network topology compared to <small>Dsgd</small>, which agrees with numerical experiments.

Mode-Suppression: A Simple, Stable and Scalable Chunk-Sharing Algorithm for P2P Networks

The ability of a P2P network to scale its throughput up in proportion to the arrival rate of peers has recently been shown to be crucially dependent on the chunk sharing policy employed. Some policies can result in low frequencies of a particular chunk, known as the missing chunk syndrome, which can dramatically reduce throughput and lead to instability of the system. For instance, commonly used policies that nominally "boost" the sharing of infrequent chunks such as the well known rarest-first algorithm have been shown to be unstable. Recent efforts have largely focused on the careful design of boosting policies to mitigate this issue. We take a complementary viewpoint, and instead consider a policy that simply prevents the sharing of the most frequent chunk(s). Following terminology from statistics wherein the most frequent value in a data set is called the mode, we refer to this policy as mode-suppression. We also consider a more general version that suppresses the mode only if the mode frequency is larger than the lowest frequency by a fixed threshold. We prove the stability of mode-suppression using Lyapunov techniques, and use a Kingman bound argument to show that the total download time does not increase with peer arrival rate. We then design versions of mode-suppression that sample a small number of peers at each time, and construct noisy mode estimates by aggregating these samples over time. We show numerically that the variants of mode-suppression yield near-optimal download times, and outperform all other recently proposed chunk sharing algorithms.

Path graphs

To render higher quality images from the samples generated by path tracing with a low sample count, we propose a novel path reuse approach that processes a fixed collection of paths to iteratively refine and improve radiance estimates throughout the scene. Our method operates on a path graph consisting of the union of the traced paths with additional neighbor edges inserted among clustered nearby vertices. Our approach refines the initial noisy radiance estimates via an aggregation operator, treating vertices within clusters as independent sampling techniques that can be combined using MIS. In a novel step, we also introduce a propagation operator to forward the refined estimates along the paths to successive bounces. We apply the aggregation and propagation operations to the graph iteratively, progressively refining the radiance values, converging to fixed-point radiance estimates with lower variance than the original ones. We also introduce a decorrelation (final gather) step, which uses information already in the graph and is cheap to compute, allowing us to combine the method with standard denoisers. Our approach is lightweight, in the sense that it can be easily plugged into any standard path tracer and neural final image denoiser. Furthermore, it is independent of scene complexity, as the graph size only depends on image resolution and average path depth. We demonstrate that our technique leads to realistic rendering results starting from as low as 1 path per pixel, even in complex indoor scenes dominated by multi-bounce indirect illumination.

Evaluating approval-based multiwinner voting in terms of robustness to noise

Approval-based multiwinner voting rules have recently received much attention in the Computational Social Choice literature. Such rules aggregate approval ballots and determine a winning committee of alternatives. To assess effectiveness, we propose to employ new noise models that are specifically tailored for approval votes and committees. These models take as input a ground truth committee and return random approval votes to be thought of as noisy estimates of the ground truth. A minimum robustness requirement for an approval-based multiwinner voting rule is to return the ground truth when applied to profiles with sufficiently many noisy votes. Our results indicate that approval-based multiwinner voting can indeed be robust to reasonable noise. We further refine this finding by presenting a hierarchy of rules in terms of how robust to noise they are.

On fair selection in the presence of implicit and differential variance

Discrimination in selection problems such as hiring or college admission is often explained by implicit bias from the decision maker against disadvantaged demographic groups. In this paper, we consider a model where the decision maker receives a noisy estimate of each candidate's quality, whose variance depends on the candidate's group—we argue that such differential variance is a key feature of many selection problems. We analyze two notable settings: in the first, the noise variances are unknown to the decision maker who simply picks the candidates with the highest estimated quality independently of their group; in the second, the variances are known and the decision maker picks candidates having the highest expected quality given the noisy estimate. We show that both baseline decision makers yield discrimination, although in opposite directions: the first leads to underrepresentation of the low-variance group while the second leads to underrepresentation of the high-variance group. We study the effect on the selection utility of imposing a fairness mechanism that we term the γ-rule (it is an extension of the classical four-fifths rule and it also includes demographic parity). In the first setting (with unknown variances), we prove that under mild conditions, imposing the γ-rule increases the selection utility—here there is no trade-off between fairness and utility. In the second setting (with known variances), imposing the γ-rule decreases the utility but we prove a bound on the utility loss due to the fairness mechanism.

Fast and Accurate Multidimensional Free Energy Integration.

Enhanced sampling and free energy calculation algorithms of the thermodynamic integration family (such as the adaptive biasing force (ABF) method) are not based on the direct computation of a free energy surface but rather of its gradient. Integrating the free energy surface is nontrivial in dimensions higher than one. Here, the author introduces a flexible, portable implementation of a Poisson equation formalism to integrate free energy surfaces from estimated gradients in dimensions 2 and 3 using any combination of periodic and nonperiodic (Neumann) boundary conditions. The algorithm is implemented in portable C++ and provided as a standalone tool that can be used to integrate multidimensional gradient fields estimated on a grid using any algorithm, such as umbrella integration as a post-treatment of umbrella sampling simulations. It is also included in the implementation of ABF (and its extended-system variant eABF) in the Collective Variables Module, enabling the seamless computation of multidimensional free energy surfaces within ABF and eABF simulations. A Python-based analysis toolchain is provided to easily plot and analyze multidimensional ABF simulation results, including metrics to assess their convergence. The Poisson integration algorithm can also be used to perform Helmholtz decomposition of noisy gradient estimates on the fly, resulting in an efficient implementation of the projected ABF (pABF) method proposed by Leliévre and co-workers. In numerical tests, pABF is found to lead to faster convergence with respect to ABF in simple cases of low intrinsic dimension but seems detrimental to convergence in a more realistic case involving degenerate coordinates and hidden barriers due to slower exploration. This suggests that variance reduction schemes do not always yield convergence improvements when applied to enhanced sampling methods.

Achieving proportional fairness in WiFi networks via bandit convex optimization

In this paper, we revisit proportional fair channel allocation in IEEE 802.11 networks. Traditional approaches are either based on the explicit solution of the optimization problem or use iterative solvers to converge to the optimum. Instead, we propose an algorithm able to learn the optimal slot transmission probability only by monitoring the throughput of the network. We have evaluated this algorithm both (i) using the true value of the function to optimize and (ii) considering estimation errors. We provide a comprehensive performance evaluation that includes assessing the sensitivity of the algorithm to different learning and network parameters as well as its reaction to network dynamics. We also evaluate the effect of noisy estimates on the convergence rate and propose a method to alleviate them. We believe our approach is a practical solution to improve the performance of wireless networks as it does not require knowing the network parameters in advance. Yet, we conclude that the setting of the parameters of the algorithm is crucial to guarantee fast convergence.

Adaptive batching for Gaussian process surrogates with application in noisy level set estimation

AbstractWe develop adaptive replicated designs for Gaussian process metamodels of stochastic experiments. Adaptive batching is a natural extension of sequential design heuristics with the benefit of replication growing as response features are learned, inputs concentrate, and the metamodeling overhead rises. Motivated by the problem of learning the level set of the mean simulator response, we develop five novel schemes: Multi‐Level Batching (MLB), Ratchet Batching (RB), Adaptive Batched Stepwise Uncertainty Reduction (ABSUR), Adaptive Design with Stepwise Allocation (ADSA), and Deterministic Design with Stepwise Allocation (DDSA). Our algorithms simultaneously (MLB, RB, and ABSUR) or sequentially (ADSA and DDSA) determine the sequential design inputs and the respective number of replicates. Illustrations using synthetic examples and an application in quantitative finance (Bermudan option pricing via Regression Monte Carlo) show that adaptive batching brings significant computational speed‐ups with minimal loss of modeling fidelity.

Spatiotemporal imputation of MODIS land surface temperature using machine learning techniques (Case study: New Mexico's Lower Rio Grande Valley)

Land surface temperature (LST) is one of the main factors in the physical processes of land surface energy and water balance at the global scale. The importance of LST is being progressively increased in a variety of water resource fields. However, the time series of LST observations regularly have missing values because of various reasons such as cloudiness. Missing value imputation is a technique to use a realistic value to estimate or replace the missing value. The objective of this study was to investigate and compare the potential of machine learning (ML) models including K-Nearest Neighbors (KNN), Support Vector Regression (SVR), Boosted Regression Trees (BRT), and Extreme Learning Machine (ELM) for spatiotemporal imputation of LST satellite images in New Mexico's Lower Rio Grande Valley (LRGV) where LST is a critical variable for water resource studies. The cross-validation approach was applied to obtain the optimized parameters for each model separately. The model comparison results showed that the SVR model is the most accurate model compared to other models from missing ratio 0.1 to 0.8 (0.24 < mean RMSE (°C) < 0.33). The BRT and ELM models showed the same performance level in general with almost the same mean CV-RMSE of 0.5°C. However, the ELM imputed maps showed noisy estimations particularly when the missing ratios were increased. The BRT model showed a better performance compared to ELM with a lower missing ratio from 0.1 to 0.5. The KNN model found to be the least reliable ML model for spatiotemporal imputation of LST among the all models with mean RMSE of 0.982°C. Also, blurring effects were more obvious in the KNN imputed maps than other models particularly in larger missing ratios. Overall, the findings confirmed the superiority of the SVR and BRT over the ELM and KNN for spatiotemporal imputations of LST.

Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics

With increasing adoption of face recognition systems, it is important to ensure adequate performance of these technologies across demographic groups, such as race, age, and gender. Recently, phenotypes such as skin tone, have been proposed as superior alternatives to traditional race categories when exploring performance differentials. However, there is little consensus regarding how to appropriately measure skin tone in evaluations of biometric performance or in AI more broadly. Biometric researchers have estimated skin tone, most notably focusing on face area lightness measures (FALMs) using automated color analysis or Fitzpatrick Skin Types (FST). These estimates have generally been based on the same images used to assess biometric performance, which are often collected using unknown and varied devices, at unknown and varied times, and under unknown and varied environmental conditions. In this study, we explore the relationship between FALMs estimated from images and ground-truth skin readings collected using a colormeter device specifically designed to measure human skin. FALMs estimated from different images of the same individual varied significantly relative to ground-truth FALMs. This variation was only reduced by greater control of acquisition (camera, background, and environmental conditions). Next, we compare ground-truth FALMs to FST categories obtained using the standard, in-person, medical survey. We found that there was relatively little change in ground-truth FALMs across different FST category and that FST correlated more with self-reported race than with ground-truth FALMs. These findings show FST is poorly predictive of skin tone and should not be used as such in evaluations of computer vision applications. Finally, using modeling, we show that when face recognition performance is driven by FALMs and independent of race, noisy FALM estimates can lead to erroneous selection of race as a key correlate of biometric performance. These results demonstrate that measures of skin type for biometric performance evaluations must come from objective, characterized, and controlled sources. Further, despite this being a currently practiced approach, estimating FST categories and FALMs from uncontrolled imagery does not provide an appropriate measure of skin tone.

Graph Matching between Bipartite and Unipartite Networks: to Collapse, or not to Collapse, that is the Question.

Graph matching consists of aligning the vertices of two unlabeled graphs in order to maximize the shared structure across networks; when the graphs are unipartite, this is commonly formulated as minimizing their edge disagreements. In this paper we address the common setting in which one of the graphs to match is a bipartite network and one is unipartite. Commonly, the bipartite networks are collapsed or projected into a unipartite graph, and graph matching proceeds as in the classical setting. This potentially leads to noisy edge estimates and loss of information. We formulate the graph matching problem between a bipartite and a unipartite graph using an undirected graphical model, and introduce methods to find the alignment with this model without collapsing. We theoretically demonstrate that our methodology is consistent, and provide non-asymptotic conditions that ensure exact recovery of the matching solution. In simulations and real data examples, we show how our methods can result in a more accurate matching than the naive approach of transforming the bipartite networks into unipartite, and we demonstrate the performance gains achieved by our method in simulated and real data networks, including a co-authorship-citation network pair, and brain structural and functional data.

High-dimensional structure learning of sparse vector autoregressive models using fractional marginal pseudo-likelihood

Learning vector autoregressive models from multivariate time series is conventionally approached through least squares or maximum likelihood estimation. These methods typically assume a fully connected model which provides no direct insight to the model structure and may lead to highly noisy estimates of the parameters. Because of these limitations, there has been an increasing interest towards methods that produce sparse estimates through penalized regression. However, such methods are computationally intensive and may become prohibitively time-consuming when the number of variables in the model increases. In this paper we adopt an approximate Bayesian approach to the learning problem by combining fractional marginal likelihood and pseudo-likelihood. We propose a novel method, PLVAR, that is both faster and produces more accurate estimates than the state-of-the-art methods based on penalized regression. We prove the consistency of the PLVAR estimator and demonstrate the attractive performance of the method on both simulated and real-world data.

Consensus-Based Stochastic Control for Model-Free Cell Balancing

Balancing the state of charge (SOC) of the cells improves both the capacity and the health of the battery. State of the art cell balancing algorithms assume highly accurate SOC estimation which can only be achieved using model-based control solutions. Unfortunately, electrochemical models of batteries are complicated and are still not reliable enough. We propose a simple consensus-based cell balancing algorithm that does not assume any knowledge of the electrochemical model of the battery. Instead, our algorithm only uses SOC estimates from online measurements to make sequential balancing decisions. Using martingale techniques, we prove that our algorithm equalizes the SOC of all cells with probability 1 even with very noisy SOC estimates under a general non-i.i.d. noise model. We then analyze the convergence time of our algorithm as a function of the problem parameters. We also prove performance guarantees for the more challenging case where balancing is done while the battery is charging or discharging. Finally, we present simulations that support our theoretical claims and show the reliability of our algorithm. Our model-free cell balancing is easy to implement and therefore is a major step towards making active cell balancing practical.

Quantum Metrology for Non-Markovian Processes.

Quantum metrology is a rapidly developing branch of quantum technologies. While various theories have been established on quantum metrology for Markovian processes, i.e., quantum channel estimation, quantum metrology for non-Markovian processes is much less explored. In this Letter, we establish a general framework of non-Markovian quantum metrology. For any parametrized non-Markovian process on a finite-dimensional system, we derive a formula for the maximal amount of quantum Fisher information that can be extracted from it by an optimally controlled probe state. In addition, we design an algorithm that evaluates this quantum Fisher information via semidefinite programming. We apply our framework to noisy frequency estimation, where we find that the optimal performance of quantum metrology is better in the non-Markovian scenario than in the Markovian scenario and explore the possibility of efficient sensing via simple variational circuits.

A Particle Method for Solving Fredholm Equations of the First Kind

Fredholm integral equations of the first kind are the prototypical example of ill-posed linear inverse problems. They model, among other things, reconstruction of distorted noisy observations and indirect density estimation and also appear in instrumental variable regression. However, their numerical solution remains a challenging problem. Many techniques currently available require a preliminary discretization of the domain of the solution and make strong assumptions about its regularity. For example, the popular expectation maximization smoothing (EMS) scheme requires the assumption of piecewise constant solutions which is inappropriate for most applications. We propose here a novel particle method that circumvents these two issues. This algorithm can be thought of as a Monte Carlo approximation of the EMS scheme which not only performs an adaptive stochastic discretization of the domain but also results in smooth approximate solutions. We analyze the theoretical properties of the EMS iteration and of the corresponding particle algorithm. Compared to standard EMS, we show experimentally that our novel particle method provides state-of-the-art performance for realistic systems, including motion deblurring and reconstruction of cross-section images of the brain from positron emission tomography.

TransPose

Motion capture is facing some new possibilities brought by the inertial sensing technologies which do not suffer from occlusion or wide-range recordings as vision-based solutions do. However, as the recorded signals are sparse and quite noisy, online performance and global translation estimation turn out to be two key difficulties. In this paper, we present TransPose, a DNN-based approach to perform full motion capture (with both global translations and body poses) from only 6 Inertial Measurement Units (IMUs) at over 90 fps. For body pose estimation, we propose a multi-stage network that estimates leaf-to-full joint positions as intermediate results. This design makes the pose estimation much easier, and thus achieves both better accuracy and lower computation cost. For global translation estimation, we propose a supporting-foot-based method and an RNN-based method to robustly solve for the global translations with a confidence-based fusion technique. Quantitative and qualitative comparisons show that our method outperforms the state-of-the-art learning- and optimization-based methods with a large margin in both accuracy and efficiency. As a purely inertial sensor-based approach, our method is not limited by environmental settings (e.g., fixed cameras), making the capture free from common difficulties such as wide-range motion space and strong occlusion.