Variance Of Estimator Research Articles

Improved windowed phase difference method for frequency estimation of distorted power signals

Frequency is one of the most significant indicators of power system, which needs accurate estimation to provide a reliable basis for monitoring, control, and protection. Owning to the efficiency, DFT is widely used in the frequency measurement of power systems. However, the inherent defects, that is, the spectral leakage and picket fence effect, decrease the accuracy of DFT. One of the variants, named the windowed phase difference method (WPD), addresses the problem using the phase variation of two windowed sequences in the frequency domain and achieves accurate frequency estimation. Suffering from the ignored negative component and contamination of noise, there are still undesired errors in WPD-based frequency estimation. To reduce the influence of the undesired errors, this paper first analyzes the systematic error of WPD introduced by spectrum leakage and proposes an improved WPD (IWPD) through systematic error compensation to achieve more accurate frequency estimations. Then, the windowing effect on IWPD-based frequency estimation of a noisy signal is studied by deducing the theoretical expression of frequency estimation variance. Finally, the proposed IWPD is validated by extensive computer simulations and practical experiments.

Efficient Estimation of Generative Models Using Tukey Depth

Generative models have recently received a lot of attention. However, a challenge with such models is that it is usually not possible to compute the likelihood function, which makes parameter estimation or training of the models challenging. The most commonly used alternative strategy is called likelihood-free estimation, based on finding values of the model parameters such that a set of selected statistics have similar values in the dataset and in samples generated from the model. However, a challenge is how to select statistics that are efficient in estimating unknown parameters. The most commonly used statistics are the mean vector, variances, and correlations between variables, but they may be less relevant in estimating the unknown parameters. We suggest utilizing Tukey depth contours (TDCs) as statistics in likelihood-free estimation. TDCs are highly flexible and can capture almost any property of multivariate data, in addition, they seem to be as of yet unexplored for likelihood-free estimation. We demonstrate that TDC statistics are able to estimate the unknown parameters more efficiently than mean, variance, and correlation in likelihood-free estimation. We further apply the TDC statistics to estimate the properties of requests to a computer system, demonstrating their real-life applicability. The suggested method is able to efficiently find the unknown parameters of the request distribution and quantify the estimation uncertainty.

Unraveling the mechanism of social media application addiction among college students: the moderating role of self-regulation

PurposeThis study seeks to explore the impact of “fear of missing out” (FOMO) and “psychological enhancement” (PE) on addiction to social media applications, subsequently influencing users' life satisfaction and continuous usage intention.Design/methodology/approachThis research involved the administration of two sets of questionnaires during distinct periods: December 15 to December 30, 2022 and August 26 to September 2, 2023. The participants were college students from three universities in China, and the data collection utilized the “Questionnaire Star” platform. Only responses deemed valid and consistent were included in the subsequent statistical analysis. A total of 1,108 valid samples were used for the final analysis. Analyses including reliability, validity, path analysis, structural equation modeling, mediation effects and moderation effects were conducted using SPSS and AMOS software.FindingsThe study revealed that both FOMO and PE exerted positive influences on users' addiction to social media applications. Furthermore, this addiction was found to have a negative effect on users' life satisfaction while simultaneously contributing positively to their intention to continue using these platforms. The mediating effect of social media application addiction and the moderating impact of self-regulation were also substantiated through the analysis.Research limitations/implicationsFirstly, it is important to note that the research population of this study is limited to college students, which may limit its generalizability and representativeness. Although college students are a group known for their familiarity with and frequent use of smartphones and social media apps, the findings may not fully capture the behaviors of social media app users in other age groups. To enhance the understanding of social media app addiction across different age groups, future studies should consider expanding the research population and conducting multi-group difference analyses. Secondly, while focusing on specific users within a particular region can minimize unexplained variance in model estimation, it may also restrict the broader applicability of the study results. Therefore, future studies should consider testing the research model with diverse groups from different regions and cultural backgrounds. This approach will provide valuable insights into how social media app addiction may vary across various contexts, thereby enriching our understanding of this phenomenon.Practical implicationsOur findings reveal that in the “attention economy” environment shaped by addiction, social media app managers should leverage technology to swiftly and accurately target audiences, attract them to their platforms and cultivate long-term relationships. Encouraging users to develop new beneficial habits through app-specific functions and precise services will foster continuous usage and unlock revenue and marketing opportunities for app companies.Social implicationsDespite the extensive scholarly discourse on social media application addiction, there is a lack of a well-defined framework delineating how addictive user behaviors can be leveraged in the marketing strategies of social media application platforms. The present study seeks to address these gaps, contributing to a better understanding of the formation mechanisms and knowledge systems related to social media application addiction. By investigating the causes and consequences of such addiction, this research offers valuable insights and recommendations for the innovative development of these apps, given their widespread popularity. Concurrently, the study establishes a theoretical basis for the concept that users can mitigate the negative effects of social media addiction by exercising their own self-regulation.Originality/valueAs the functionalities and features of social media apps converge, their individual uniqueness starts to diminish, intensifying the competition among social media companies. This escalating rivalry places higher demands on these companies. This study aims to aid social media app companies in comprehending and analyzing the diverse psychological needs of users. By enriching their platform features and services, leading users towards addiction and gaining an edge in the “Attention Economy” competition. Understanding and catering to users' needs will be instrumental in thriving within this dynamic and evolving attention economy landscape.

Evolution of intraspecific floral variation in a generalist-specialist pollination system.

Intraspecific processes impact macroevolutionary patterns through individual variation, selection, and ecological specialisation. According to the niche variation hypothesis, the broader ecological niche of gen- eralist species results in an increased morphological variation among individuals, either because they are constituted of diversified specialised individuals each exploiting a fraction of the species' niche, or because they are constituted of true generalist individuals that experience relaxed selection. To test this hypoth- esis, we surveyed the individual floral morphology of species of Antillean Gesneriaceae, a group that has transitioned between specialisation for hummingbird pollination and generalisation multiple times throughout its evolutionary history. We characterised the profiles of corollas using geometric morpho- metrics and compared the intraspecific shape variance of specialists and generalists in a phylogenetic context. We used three approaches that differently accounted for the high dimensionality of morphologi- cal traits, the ancestral reconstruction of pollination syndromes over time, and the error associated with the estimation of the intraspecific variance. Our findings provide partial support for the niche variation hypothesis. If considering the whole shape in the analysis corroborated this idea, decomposing the shape into principal components indicated that not all aspects of the corolla exhibit the same pattern of vari- ation. Specifically, pollination generalists tend to display greater intraspecific variation than specialists in terms of tubularity, but not of curvature. Accounting for the error in the variance estimation also reduced the support for the hypothesis, suggesting that larger sample sizes may be required to reach stronger conclusions. This study emphasises the reciprocal influence between plants and their pollinators on floral morphology at different biodiversity scales, and suggests that ecological strategies of species can affect patterns of morphological variation at macroevolutionary scales.

Estimation of heterogeneous population variance using memory-type estimators based on EWMA statistic in the presence of measurement error for time-scaled surveys

In this present article, we have suggested memory-type ratio, exponential ratio, product and exponential product estimators based on exponentially weighted moving average statistic for the estimation of heterogeneous population variance using stratified sampling design in presence of measurement error for time-scaled surveys. Mathematical expressions of approximate mean square error are derived using Taylor and exponential expansions for the proposed memory-type estimators. We have also discussed the situations in which the memory-type estimators would perform efficiently than the conventional estimators. The results of extensive simulation study revealed that the proposed memory-type estimators based exponentially weighted moving average statistic would perform better than the conventional estimators in the presence measurement error for time-scaled surveys under certain condition.

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

The stochastic linear quadratic (LQ) problem of discrete-time linear Markov jumping systems with multiplicative noise is investigated in this paper. Two reinforcement learning algorithms, one model-based and one model-free, are designed. The algorithms can be readily adapted to handle nonlinear cases by employing an appropriate function approximator. In the linear case, the network structure is designed to enable the problem to be solved without succumbing to modeling errors. For the model-free algorithm, an approach based on moving averages is proposed to reduce estimation variance. A proof of convergence is also provided. Finally, the effectiveness of the proposed algorithms is demonstrated through a numerical example.

Improved estimation of population variance in stratified successive sampling using calibrated weights under non-response

This paper introduces a new method to estimate the population variance of a study variable in stratified successive sampling over two occasions, while accounting for random non-response. The method uses a logarithmic type estimator and leverages information from a highly positively correlated auxiliary variable. The paper also presents calibrated weights for the new estimator and examines its properties through numerical and simulation studies. The results indicate that the suggested estimator is more effective than the standard estimator for estimating the population variance.

Controlling underestimation bias in reinforcement learning via minmax operation

Obtaining the accurate value estimation and reducing the estimation bias are the key issues in reinforcement learning. However, current methods that address the overestimation problem tend to introduce underestimation, which face a challenge of precise decision-making in many fields. To address this issue, we conduct a theoretical analysis of the underestimation bias and propose the minmax operation, which allow for flexible control of the estimation bias. Specifically, we select the maximum value of each action from multiple parallel state-action networks to create a new state-action value sequence. Then, a minimum value is selected to obtain more accurate value estimations. Moreover, based on the minmax operation, we propose two novel algorithms by combining Deep Q-Network (DQN) and Double DQN (DDQN), named minmax-DQN and minmax-DDQN. Meanwhile, we conduct theoretical analyses of the estimation bias and variance caused by our proposed minmax operation, which show that this operation significantly improves both underestimation and overestimation biases and leads to the unbiased estimation. Furthermore, the variance is also reduced, which is helpful to improve the network training stability. Finally, we conduct numerous comparative experiments in various environments, which empirically demonstrate the superiority of our method.

Automated importance sampling via optimal control for stochastic reaction networks: A Markovian projection–based approach

We propose a novel alternative approach to our previous work (Ben Hammouda et al., 2023) to improve the efficiency of Monte Carlo (MC) estimators for rare event probabilities for stochastic reaction networks (SRNs). In the same spirit of Ben Hammouda et al. (2023), an efficient path-dependent measure change is derived based on a connection between determining optimal importance sampling (IS) parameters within a class of probability measures and a stochastic optimal control formulation, corresponding to solving a variance minimization problem. In this work, we propose a novel approach to address the encountered curse of dimensionality by mapping the problem to a significantly lower-dimensional space via a Markovian projection (MP) idea. The output of this model reduction technique is a low-dimensional SRN (potentially even one dimensional) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained by solving a related optimization problem via a discrete L2 regression. By solving the resulting projected Hamilton–Jacobi–Bellman (HJB) equations for the reduced-dimensional SRN, we obtain projected IS parameters, which are then mapped back to the original full-dimensional SRN system, resulting in an efficient IS-MC estimator for rare events probabilities of the full-dimensional SRN. Our analysis and numerical experiments reveal that the proposed MP-HJB-IS approach substantially reduces the MC estimator variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators.

Is Fisher inference inferior to Neyman inference for policy analysis?

The increasing computational power has led to an increasing interest in Fisher’s test in social science. As the Fisher and Neyman inference are based on different principles there is also an increasing interest in understanding the differential features of the two procedures. For example, Young (2018) found that the Fisher test has better size properties than the Neyman test in the situation with influential observations. Ding (2017), on the other hand, showed that the asymptotic variance of the mean-difference estimator (MDE) under Fisher inference is larger than that under Neyman inference, and that the asymptotic Fisher test is less powerful than the t-test even for the simplest case of homogeneous effect. Since MDE plays an important role for policy evaluation, these latter results are a concern for using Fisher’s test as argued in Young (2018). With the aim of providing an understanding of the usefulness of the exact Fisher test for inference to the sample and to the population, this paper clarifies the results in Ding (2017). Using a novel Monte Carlo simulation following the same data generating processes as in Ding (2017), we demonstrate that the Fisher test has no worse power properties than the t-test even with heterogeneous effects.

Identification of genomic regions and pathways associated with traits related to rumen acidosis in feedlot Nellore cattle.

There may be an increased risk of metabolic disorders, such as rumen acidosis, in cattle fed high-concentrate diets, particularly those from Bos taurus indicus genotypes, which have shown to be more sensitive to ruminal acidification. Therefore, this study aimed to estimate (co)variance components and identify genomic regions and pathways associated with ruminal acidosis in feedlot Nellore cattle fed high-concentrate diets. It was utilized a dataset containing a total of 642 Nellore bulls that were genotyped from seven feedlot nutrition studies. The GGP Indicus 35k panel was used with the single step genome-wide association study methodology in which the effects of the markers were obtained from the genomic values estimated by the GBLUP model. A bivariate model to estimate genetic correlations between the economically important traits and indicator traits for acidosis was used. The traits evaluated in this study that were nutritionally related to rumen acidosis included average daily gain (ADG), final body weight, time spent eating (TSE), time spent ruminating, rumenitis score (RUM), rumen absorptive surface area (ASA), rumen keratinized layer thickness (KER) and hot carcass weight (HCW). The identified candidate genes were mainly involved in the negative or non-regulation of the apoptotic process, salivary secretion, and transmembrane transport. The genetic correlation between HCW and ASA was low positive (0.27 ± 0.23), and between ADG and ASA was high moderate (0.58 ± 0.59). A positive genetic correlation between RUM and all performance traits was observed, and TSE correlated negatively with HCW (-0.33 ± 0.21), ASA (-0.75 ± 0.48), and KER (-0.40 ± 0.27). The genetic association between economically important traits and indicator traits for acidosis suggested that Nellore cattle may be more sensitive to acidosis in feedlot systems.

Scalable phylogenetic Gaussian process models improve the detectability of environmental signals on local extinctions for many Red List species

Abstract Conservation biologists have a daunting task of understanding the causes of species decline associated with anthropogenic factors and predicting the extinction risk of a growing number of endangered species. By reducing variances of estimates with information on closely related species, phylogenetic information among species can bridge gaps in information on species with small range sizes when modelling large numbers of endangered species. However, modelling many species with the Gaussian process (GP), which underlies the evolutionary process of phylogenetic random effects, remains a challenge owing to the computational burden in estimating the large variance–covariance matrix. Here, we applied a phylogenetic generalised mixed model with random slopes and random intercepts to 1010 endangered vascular plant taxa in Japan following phylogenetic GPs implemented by nearest neighbour GP (NNGP) approximation. NNGP enables flexibility in changing the proximity on the phylogenetic tree of species from which information is borrowed to reduce the variances of estimates with a realistic computational burden. We evaluated the effectiveness of phylogenetic models by comparing the predictive performance and descriptive power of phylogenetic and non‐phylogenetic models and identified the anthropogenic factors contributing to the decline of each of the studied endangered species. We found that the model with phylogenetic information had better prediction performance than the model without phylogenetic information. The results showed that across all explanatory variables, the phylogenetic model could detect interspecific differences in response to environmental factors in a number of species more clearly. Combined with the phylogenetic signal results, we could also detect a phylogenetic bias in the species that could benefit from the positive effects of protected areas but reduce the local extinctions of 95% of all studied taxa. In conclusion, our model, considering phylogenetic information with NNGP, allows the elucidation of factors causing the decline of many endangered species. In future analyses, the estimation of extinction probability linked to environmental change might be applied to future climate–land use scenarios, advancing the comprehensive assessment of biodiversity degradation and threats to species at multiple scales.

Functional analysis of variance (ANOVA) for carbon flux estimates from remote sensing data

Abstract. The constellation of Earth-observing satellites has now produced atmospheric greenhouse gas concentration estimates covering a period of several years. Their global coverage is providing additional information on the global carbon cycle. These products can be combined with complex inversion systems to infer the magnitude of carbon sources and sinks around the globe. Multiple factors, including the atmospheric transport model and satellite product aggregation method, can impact such flux estimates. Analysis of variance (ANOVA) is a well-established statistical framework for estimating common signals while partitioning variability across factors in the analysis of experiments. Functional ANOVA extends this approach with a statistical model that incorporates spatiotemporal correlation for each ANOVA component. The approach is illustrated on inversion experiments with different satellite retrieval aggregation methods and identifies consistent significant patterns in flux increments that span large spatial scales. Functional ANOVA identifies these patterns while accounting for the uncertainty at small spatial scales that is attributed to differences in the aggregation method. Functional ANOVA is also applied to a recent flux model intercomparison project (MIP), and the relative magnitudes of inversion system effects and data source (satellite versus in situ) are similar but exhibit slightly different importance for fluxes over different continents. In all examples, the unexplained residual variability is locally sizable in magnitude but with limited spatial and temporal correlation. These common behaviors across flux inversion experiments demonstrate the diagnostic capability for functional ANOVA to simultaneously distinguish the spatiotemporal coherence of carbon cycle processes and algorithmic factors.

Genetic correlation estimates between calving ease in primiparous cows and economically important traits in Nellore cattle.

This study aimed to estimate (co)variance components and genetic parameters for calving ease (CE) and their genetic correlations with growth, reproductive, carcass, and feed efficiency traits in Nellore cattle. Phenotypes for CE are scored in two categories: normal calving and assisted calving. The traits considered were probability of precocious calving, age at first calving, stayability, adjusted scrotal circumference at 365 days of age, accumulated cow productivity, age at puberty of males, gestation length, birth weight, adjusted weights at 210 and 450 days of age, adult cow weight, frame score, hip height, rib eye area, subcutaneous backfat thickness, rump fat thickness, intramuscular fat percentage, residual feed intake and dry matter intake. The estimation of genetic parameters was performed using a two-trait threshold-linear animal model, except for CE, stayability, and probability of precocious calving, which were evaluated through a two-trait threshold animal model. The direct (0.27) and maternal (0.19) heritability estimates for CE in heifers primiparous Nellore indicated that selecting for this trait is feasible. The selection to improve the female sexual precocity should consider CE during the selection and mating decisions to reduce calving problems. Genetic correlation estimates between CE and BW suggest that selecting low birth weight to reduce calving problems is not an appropriate strategy to improve calving ease in heifers Nellore. Therefore, adopting a multi-trait selection model with CE and BW in the Nellore breed would reduce calving difficulties, particularly in sexually precocious heifers, without impairing the growth, reproductive, feed efficiency conversion, and carcass indicator traits.

Effect of Increasing Sample Size on Multi-Collinearity in Multilevel Non-Linear Model

A multilevel logistic regression model demonstrating high correlations among predictor variables is susceptible to multi-collinearity. Multi-collinearity significantly impacts the robustness and interpretability of multilevel non-linear models. In multilevel non-linear models, the effects of multi-collinearity can be amplified, leading to distorted parameter estimates and inflated standard errors. This phenomenon contributes to an escalation in the variances of parameter estimates, thereby resulting in inaccurate inferences regarding the relationships between the response and explanatory factors. The primary objective of this study is to investigate the impact of multi-collinearity on multilevel non-linear models. The research aims to assess whether the quantity of independent variables influences the Multilevel Variance Inflation Factor and to explore the effect of altering the correlation degree at one level on multi-collinearity within a multilevel non-linear model. Additionally, the research seeks to determine how multi-collinearity affects the standard errors of parameters in a multilevel non-linear model. In a 2-level logistic regression, a binary variable was the dependent variable, while pre-established standard variables functioned as regressors. The Monte Carlo analysis incorporated three distinct correlation strengths (0.2, 0.5, and 0.9) and sample sizes (500, 100, and 30). The Multilevel Variance Inflation Factor was employed for multi-collinearity diagnosis. The outcomes revealed that, within the logistic multilevel regression model, an increase in sample size correlated with a reduction in multi-collinearity. Notably, the influence of multi-collinearity on standard errors in a multilevel non-linear model was more pronounced. It was observed that increasing the sample size remains an effective strategy to mitigate multi-collinearity errors in a multilevel non-linear model. This approach is particularly crucial due to the reliance on maximum likelihood estimation in logistic regression, as opposed to ordinary least squares (OLS) regression, which contrasts with the methodology of OLS regression.

Rescaling bootstrap variance estimation technique under dual frame surveys with unknown domain sizes

Dual frame (DF) surveys are a special case of multiple frame (MF) surveys considering two frames covering the entire population. Dual frame surveys are applicable in those situations, where, one frame may cover the entire population but is very expensive to sample; so an alternate frame may be available that does not cover the entire population but is easily available. Unbiased variance estimation in dual frame surveys can be difficult and complicated than corresponding estimators under single frame surveys. Again, the variance of dual frame estimator involves population variances of the individual domains which are generally unknown. Due to this reason, obtaining an unbiased estimate of the variance of the dual frame estimator is quite complex in the case of dual frame surveys. In this article, we propose a Post-stratified Rescaling Bootstrap with Unknown Domain size (PstRBUD) method for variance estimation of the dual frame estimator of population total. The proposed rescaled bootstrap method was compared to that of standard bootstrap methods in simulation analysis. The proposed PstRBUD method provides an unbiased estimation of the variance of the dual frame estimator of population total, according to simulation results.

Impact of individual level uncertainty of lung cancer polygenic risk score (PRS) on risk stratification.

Although polygenic risk score (PRS) has emerged as a promising tool for predicting cancer risk from genome-wide association studies (GWAS), the individual-level accuracy of lung cancer PRS and the extent to which its impact on subsequent clinical applications remains largely unexplored. Lung cancer PRSs and confidence/credible interval (CI) were constructed using two statistical approaches for each individual: (1) the weighted sum of 16 GWAS-derived significant SNP loci and the CI through the bootstrapping method (PRS-16-CV) and (2) LDpred2 and the CI through posteriors sampling (PRS-Bayes), among 17,166 lung cancer cases and 12,894 controls with European ancestry from the International Lung Cancer Consortium. Individuals were classified into different genetic risk subgroups based on the relationship between their own PRS mean/PRS CI and the population level threshold. Considerable variances in PRS point estimates at the individual level were observed for both methods, with an average standard deviation (s.d.) of 0.12 for PRS-16-CV and a much larger s.d. of 0.88 for PRS-Bayes. Using PRS-16-CV, only 25.0% of individuals with PRS point estimates in the lowest decile of PRS and 16.8% in the highest decile have their entire 95% CI fully contained in the lowest and highest decile, respectively, while PRS-Bayes was unable to find any eligible individuals. Only 19% of the individuals were concordantly identified as having high genetic risk (> 90th percentile) using the two PRS estimators. An increased relative risk of lung cancer comparing the highest PRS percentile to the lowest was observed when taking the CI into account (OR = 2.73, 95% CI: 2.12-3.50, P-value = 4.13 × 10-15) compared to using PRS-16-CV mean (OR = 2.23, 95% CI: 1.99-2.49, P-value = 5.70 × 10-46). Improved risk prediction performance with higher AUC was consistently observed in individuals identified by PRS-16-CV CI, and the best performance was achieved by incorporating age, gender, and detailed smoking pack-years (AUC: 0.73, 95% CI = 0.72-0.74). Lung cancer PRS estimates using different methods have modest correlations at the individual level, highlighting the importance of considering individual-level uncertainty when evaluating the practical utility of PRS.

The estimation of additive genetic variance of body size in a wild passerine is sensitive to the method used to estimate relatedness among the individuals.

Assessing additive genetic variance is a crucial step in predicting the evolutionary response of a target trait. However, the estimated genetic variance may be sensitive to the methodology used, e.g., the way relatedness is assessed among the individuals, especially in wild populations where social pedigrees can be inaccurate. To investigate this possibility, we investigated the additive genetic variance in tarsus length, a major proxy of skeletal body size in birds. The model species was the collared flycatcher (Ficedula albicollis), a socially monogamous but genetically polygamous migratory passerine. We used two relatedness matrices to estimate the genetic variance: (1) based solely on social links and (2) a genetic similarity matrix based on a large array of single-nucleotide polymorphisms (SNPs). Depending on the relatedness matrix considered, we found moderate to high additive genetic variance and heritability estimates for tarsus length. In particular, the heritability estimates were higher when obtained with the genetic similarity matrix instead of the social pedigree. Our results confirm the potential for this crucial trait to respond to selection and highlight methodological concerns when calculating additive genetic variance and heritability in phenotypic traits. We conclude that using a social pedigree instead of a genetic similarity matrix to estimate relatedness among individuals in a genetically polygamous wild population may significantly deflate the estimates of additive genetic variation.

Cooperative Multiagent Reinforcement Learning With Partial Observations

In this paper, we propose a distributed zeroth-order policy optimization method for Multi-Agent Reinforcement Learning (MARL). Existing MARL algorithms often assume that every agent can observe the states and actions of all the other agents in the network. This can be impractical in large-scale problems, where sharing the state and action information with multi-hop neighbors may incur significant communication overhead. The advantage of the proposed zeroth-order policy optimization method is that it allows the agents to compute the local policy gradients needed to update their local policy functions using local estimates of the global accumulated rewards that depend on partial state and action information only and can be obtained using consensus. Specifically, to calculate the local policy gradients, we develop a new distributed zeroth-order policy gradient estimator that relies on one-point residual-feedback which, compared to existing zeroth-order estimators that also rely on one-point feedback, significantly reduces the variance of the policy gradient estimates improving, in this way, the learning performance. We show that the proposed distributed zeroth-order policy optimization method with constant stepsize converges to the neighborhood of a policy that is a stationary point of the global objective function. The size of this neighborhood depends on the agents' learning rates, the exploration parameters, and the number of consensus steps used to calculate the local estimates of the global accumulated rewards. Moreover, we provide numerical experiments that demonstrate that our new zeroth-order policy gradient estimator is more sample-efficient compared to other existing one-point estimators.

Inferring community transmission of SARS-CoV-2 in the United Kingdom using the ONS COVID-19 Infection Survey

Key epidemiological parameters, including the effective reproduction number, R(t), and the instantaneous growth rate, r(t), generated from an ensemble of models, have been informing public health policy throughout the COVID-19 pandemic in the four nations of the United Kingdom of Great Britain and Northern Ireland (UK). However, estimation of these quantities became challenging with the scaling down of surveillance systems as part of the transition from the “emergency” to “endemic” phase of the pandemic.The Office for National Statistics (ONS) COVID-19 Infection Survey (CIS) provided an opportunity to continue estimating these parameters in the absence of other data streams. We used a penalised spline model fitted to the publicly-available ONS CIS test positivity estimates to produce a smoothed estimate of the prevalence of SARS-CoV-2 positivity over time. The resulting fitted curve was used to estimate the “ONS-based” R(t) and r(t) across the four nations of the UK. Estimates produced under this model are compared to government-published estimates with particular consideration given to the contribution that this single data stream can offer in the estimation of these parameters.Depending on the nation and parameter, we found that up to 77% of the variance in the government-published estimates can be explained by the ONS-based estimates, demonstrating the value of this singular data stream to track the epidemic in each of the four nations. We additionally find that the ONS-based estimates uncover epidemic trends earlier than the corresponding government-published estimates.Our work shows that the ONS CIS can be used to generate key COVID-19 epidemiological parameters across the four UK nations, further underlining the enormous value of such population-level studies of infection. This is not intended as an alternative to ensemble modelling, rather it is intended as a potential solution to the aforementioned challenge faced by public health officials in the UK in early 2022.