Estimation Of Correlation Function Research Articles

Recovery of Discrete-Time Signal Based on the Moving Average Model and Estimation of the Samples Correlation in Forward and Reverse Forecasting

The article discusses the development of mathematical support for the recovery of the values of discrete-time sequence samples obtained as a result of uniform sampling of a continuous signal. The recovery problem of discrete-time sequence samples is solved for a signal that can be considered stationary or stationary at least in a broad sense (quasi-stationary). The development of mathematical support for the recovery of the values of signal samples was carried out on the basis of constructing a moving average model and estimating the correlation of signal samples over time with forward and reverse forecasting. Estimates of the signal correlation function necessary to recover sample sections with lost values are calculated from samples with known values. Correlation function estimates can be calculated regardless of the location of the recovery area when the condition of stationarity of the signal is met. The obtained estimates of the correlation function samples can be used for both forward and reverse forecasting. Moreover, even if it is necessary to recover several problem sections, it is enough to calculate only once the sample of correlation function estimates necessary for their restoration. The resulting mathematical solution to the problem became the basis for the development of algorithmic support. Test tests and functional checks of the algorithmic support were carried out on the basis of simulation using a signal model representing an additive sum of harmonic components with random initial phases. The simulation results showed that the calculation of estimates of the lost sample values is carried out with a fairly low error, both in forward and reverse forecasting, as well as when they are used together. In practice, the choice of a sequence recovery algorithm based on forward or reverse forecasting will be determined based on the actual conditions of its processing. In particular, if previous samples with known values are not enough to carry out forward forecasting, then the reverse forecasting procedure is implemented and vice versa. The developed algorithmic support can be implemented in the form of metrologically significant software for digital signal processing systems.

Galaxy clustering measurements out to redshift z ˜ 8 from Hubble Legacy Fields

ABSTRACT We present a novel approach for measuring the two-point correlation function of galaxies in narrow pencil beam surveys with varying depths. Our methodology is utilized to expand high-redshift galaxy clustering investigations up to z ∼ 8 by analysing a comprehensive sample consisting of Ng = 160 Lyman break galaxy candidates obtained through optical and near-infrared photometric data within the CANDELS GOODS data sets from the Hubble Space Telescope Legacy Fields. For bright sources with MUV < −19.8, we determine a galaxy bias of b = 9.33 ± 4.90 at $\overline{z} = 7.7$ and a correlation length of r0 = 10.74 ± 7.06 $h^{-1}\, \mathrm{Mpc}$. We obtain similar results for the XDF, with a galaxy bias measurement of b = 8.26 ± 3.41 at the same redshift for a slightly fainter sample with a median luminosity of MUV = −18.4. By comparing with dark-matter halo bias and employing abundance matching, we deduce a characteristic halo mass of Mh ∼ 1011.5 M⊙ and a duty cycle close to unity. To validate our approach for variable-depth data sets, we replicate the analysis in a region with near-uniform depth using a standard two-point correlation function estimator, yielding consistent outcomes. Our study not only provides a valuable tool for future utilization in JWST data sets but also suggests that the clustering of early galaxies continues to increase with redshift beyond z ≳ 8, potentially contributing to the existence of protocluster structures observed in early JWST imaging and spectroscopic surveys at z ≳ 8.

Algorithm for Estimating the Spectral Correlation Function Using the 2D Fast Fourier Transform

An algorithm for estimating the spectral correlation function of cyclostationary random processes on the basis of their finite-time realization is proposed. The main algorithm steps are described and the main operations performed at each of them are formalized. The presented estimates of the spectral correlation function in the presence of a correlated stationary noise overlapping the analyzed cyclostationary random process in the frequency domain confirm the efficiency of the algorithm with a low signal-to-noise ratio. The number of required computational operations and the amount of computer memory required for storing intermediate results are estimated.

Bootstrap bandwidth selection for the pair correlation function of inhomogeneous spatial point processes

This work focuses on kernel estimation of the pair correlation function (PCF) for inhomogeneous spatial point processes. We propose a bootstrap bandwidth selector based on minimizing the mean integrated squared error (MISE). The variance term is estimated by nonparametric bootstrap, and the bias by a plug-in approach using a pilot estimator of the PCF. Kernel estimators of the PCF also require a pilot estimator of the first-order intensity. We test the performance of the bandwidth selector and the role of the pilot intensity estimator in a simulation study. The bootstrap bandwidth selector is competitive with cross-validation procedures, but the contribution of the bandwidth parameter to the goodness-of-fit of the kernel PCF estimator is minor in comparison with that of the pilot intensity function. The data-based kernel intensity estimator leads to biased kernel PCF estimators, while both kernel and parametric covariate-based intensities provide accurate estimators of the PCF.

Bayesian estimation of correlation functions

We apply Bayesian statistics to the estimation of correlation functions. We give the probability distributions of auto- and cross-correlations as functions of the data. Our procedure uses the measured data optimally and informs about the certainty level of the estimation. Our results apply to general stationary processes and their essence is a nonparametric estimation of spectra. It allows one to better understand the statistical noise fluctuations, assess the correlations between two variables, and postulate parametric models of spectra that can be further tested. We also propose a method to numerically generate correlated noise with a given spectrum.

ConKer: An algorithm for evaluating correlations of arbitrary order

Context. High order correlations in the cosmic matter density have become increasingly valuable in cosmological analyses. However, computing these correlation functions is computationally expensive. Aims. We aim to circumvent these challenges by developing a new algorithm called ConKer for estimating correlation functions. Methods. This algorithm performs convolutions of matter distributions with spherical kernels using FFT. Since matter distributions and kernels are defined on a grid, it results in some loss of accuracy in the distance and angle definitions. We study the algorithm setting at which these limitations become critical and suggest ways to minimize them. Results. ConKer is applied to the CMASS sample of the SDSS DR12 galaxy survey and corresponding mock catalogs, and is used to compute the correlation functions up to correlation order n = 5. We compare the n = 2 and n = 3 cases to traditional algorithms to verify the accuracy of the new algorithm. We perform a timing study of the algorithm and find that three of the four distinct processes within the algorithm are nearly independent of the catalog size N, while one subdominant component scales as O(N). The dominant portion of the calculation has complexity of O(Nc4/3 log Nc), where Nc is the of cells in a three-dimensional grid corresponding to the matter density. Conclusions. We find ConKer to be a fast and accurate method of probing high order correlations in the cosmic matter density, then discuss its application to upcoming surveys of large-scale structure.

Improving the accuracy of estimators for the two-point correlation function

Aims. We show how to increase the accuracy of estimates of the two-point correlation function without sacrificing efficiency. Methods. We quantify the error of the pair-counts and of the Landy & Szalay estimator by comparing them with exact reference values. The standard method, using random point sets, is compared to geometrically motivated estimators and estimators using quasi-Monte Carlo integration. Results. In the standard method, the error scales proportionally to 1/√Nr, with Nr being the number of random points. In our improved methods, the error scales almost proportionally to 1/Nq, where Nq is the number of points from a low-discrepancy sequence. We study the run times of the new estimator in comparison to those of the standard estimator, keeping the same level of accuracy. For the considered case, we always see a speedup ranging from 50% up to a factor of several thousand. We also discuss how to apply these improved estimators to incompletely sampled galaxy catalogues.

Accurate predictions from small boxes: variance suppression via the Zel'dovich approximation

Simulations have become an indispensable tool for accurate modelling of observables measured in galaxy surveys, but can be expensive if very large dynamic range in scale is required. We describe how to combine Lagrangian perturbation theory models with N-body simulations to reduce the effects of finite computational volume in the prediction of ensemble average properties in the simulations within the context of control variates. In particular we use the fact that Zel'dovich displacements, computed during initial condition generation for any simulation, correlate strongly with the final density field. Since all the correlators of biased tracers can be computed with arbitrary precision for these displacements, pairing the Zel'dovich `simulation' with the N-body realization allows hundredfold reductions in sample variance for power spectrum or correlation function estimation. Zel'dovich control variates can accurately extend matter or tracer field emulators to larger scales than previously possible, as well as improving measurements of statistics in simulations which are inherently limited to small volumes, such as hydrodynamical simulations of galaxy formation and reionization.

The Spatial Distribution of Gamma-Ray Bursts with Measured Redshifts from 24 Years of Observation

Gamma-ray bursts (GRBs) are the most luminous objects known: they outshine their host galaxies, making them ideal candidates for probing large-scale structure. Our aim is to determine the Spatial Two-Point Correlation Function of the GRBs with spectroscopic redshifts. We used all observations till 4 June 2021, and we also split the data according to the origin of the redshift (afterglow or host galaxy). We analyzed the possible correlation between the GRB redshift and sky position. There is only one region, called the Faraway GRB Patch, where nine distant GRBs shows deviation from the randomness with an ≈1% significance, showing that the sky and the radial component of the GRB distribution could be factorized and both can be determined independently. The GRBs’ cumulative monthly distribution was used to show that it is impossible to synthesize the Sky Exposure Function, even from the perfect observational logs. We estimated the Sky Exposure Function using the Gaussian kernel, and with the radial distribution, it was used for the estimation of the Spatial Two-Point Correlation Function. Monte Carlo simulations were used to determine the Poissonian errors. Our result shows that the current Spatial Two-Point Correlation Function estimations are consistent with zero for the full and the afterglow/host galaxy datasets.

Applications of Operational Modal Analysis in Gearbox and Induction Motor, Based on Random Decrement Technique and Enhanced Ibrahim Time Method

There have been steadily growing requirements from the academia and industry, demanding non-invasive methods and reliable measurement systems of research devoted to operational mode analysis (OMA). Due to the simplicity of performing only structures surface vibration measurements, OMA is frequently applied in machine fault diagnosis (MFD) and structure health monitoring (SHM). OMA can handle big structures, such as bridges, buildings, machines, etc. However, there is still an open question: how to properly handle the harmonic effects of rotating components and the difficulty of closely estimating space modes are still a nightmare to deal with. Therefore, the main objective of this paper is to identify the structure of natural frequencies by the regeneration of frequency response functions (FRFs) for complex structures based on OMA. The novelty of our approach is to use the random decrement technique (RDT), correlation function estimation (CFE), and enhanced Ibrahim time method (EITM) to overcome OMA’s difficulties and limitations. To reduce further rotational harmonics effects, gear mesh and side band frequencies, digital signal processing techniques based on notching filters, and liftering analysis techniques were also used. All the experiments were performed at the laboratory test rig and conducted by using three accelerometers, one impedance hammer, one force sensor, and one data acquisition board. To reduce data’s variabilities, each test was measured three times for 5 min. The data sampling frequency for all the experiments was 25.6 kHz. To validate the proposed methodology, extensive OMA tests were performed for the generation of FRFs. The measured objects were a steel bar, induction motor, and gearbox. Five structural natural frequencies for the induction motor and eight structural natural frequencies for the gearbox were generated, respectively.

Semiparametric Goodness-of-Fit Test for Clustered Point Processes with a Shape-Constrained Pair Correlation Function

Specification of a parametric model for the intensity function is a fundamental task in statistics for spatial point processes. It is, therefore, crucial to be able to assess the appropriateness of a suggested model for a given point pattern dataset. For this purpose, we develop a new class of semiparametric goodness-of-fit tests for the specified parametric first-order intensity, without assuming a full data generating mechanism that is needed for the existing popular Monte Carlo tests. The proposed tests crucially rely on accurate nonparametric estimation of the second-order properties of a point process. To address this we propose a new nonparametric pair correlation function (PCF) estimator for clustered spatial point processes under some mild shape constraints, which is shown to achieve uniform consistency. The proposed test statistics are computationally efficient owing to closed-form asymptotic distributions and achieve the nominal size even for testing composite hypotheses. In practice, the proposed estimation and testing procedures provide effective tools to improve parametric intensity function modeling, which is demonstrated through extensive simulation studies as well as a real data analysis of street crime activity in Washington DC. Supplementary materials for this article are available online.

Estimation of integral length scales across the neutral atmospheric boundary layer depth: A Large Eddy Simulation study

Integral length scales of atmospheric boundary layer (ABL) flows, as well as mean velocities and turbulence intensities, play an important role in determining the response of structures subjected to wind loads. For tall building design, however, sufficient relevant data at high elevations are currently not available from experimental/field measurement data or in building standards. This study uses Large Eddy Simulation (LES) with a constant horizontal pressure gradient and streamwise periodic boundary conditions to develop horizontally homogeneous neutral ABL flow. Based on two-point velocity correlation functions (ρux, ρuy, ρuz, ρvy, ρwz), the corresponding integral length scales (Lux,Luy,Luz,Lvy,Lwz) are presented at heights across the ABL depth for better characterization of the energetic turbulent eddies. In particular, this study addresses numerical artifacts inherent in the CFD simulations, which are due to insufficient longitudinal domain lengths and streamwise periodic boundary conditions, frequently used in computational studies, that affect the estimation of longitudinal correlation function in the streamwise direction, ρux, and the corresponding integral length scale, Lux. An analytical expression for ρux is proposed and calibrated using simulations with various longitudinal domain lengths, to obtain domain-independent Lux estimates across the ABL depth. The results clearly show that the turbulent eddy structures are highly elongated (streamwise) in the near-ground region, and become shorter in the outer region of the ABL. Estimates of the integral length scales are provided at heights much greater than the upper limit of standards like ASCE 7–16. This study provides an insight into the three-dimensional coherent structures of flows throughout the ABL depth and presents data on modeling approach flows in wind tunnel tests, in numerical simulations, and in associated standard provisions.

On Stochastic Differential Equations Generating Non-gaussian Continuous Markov Process

Continuous Markov processes widely used as a tool for modeling random phenomena in numerous applications, can be defined as solutions of generally nonlinear stochastic differential equations (SDEs) with certain drift and diffusion coefficients which together governs the process’ probability density and correlation functions. Usually it is assumed that the diffusion coefficient does not depend on the process' current value. For presentation of non-Gaussian real processes this assumption becomes undesirable, leads generally to complexity of the correlation function estimation. We consider its analysis for the process with particular pairs of the drift and diffusion coefficients providing the given stationary probability distribution of the considered process

Технологии вычисления коэффициента корреляции между полезным сигналом и помехой по оценке их релейной взаимной корреляционной функции

Показано, что часто при формировании зашумленных сигналов нарушается условие отсутствия корреляции между полезным сигналом и помехой. Поэтому при корреляционном анализе этих сигналов возникают определенные погрешности, становящиеся причиной неадекватности полученных результатов. Кроме того, существующие технологии корреляционного анализа не позволяют использовать помехи как носители ценной информации. Поэтому для полного использования колоссального информационного потенциала зашумленных сигналов необходимо создать новые технологии, которые как при выполнении известных классических условий, так и при их невыполнении исключили бы потерю ценной информации. Разработаны алгоритмы определения оценки коэффициента корреляции между полезным сигналом и помехой, которые невозможно измерить непосредственно или выделить из зашумленного сигнала. С этой целью использована нормированная взаимная корреляционная функция между полезным сигналом и помехой. Разработан алгоритм вычисления оценок нормированной взаимной корреляционной функции между полезным сигналом и помехой по оценкам релейной корреляционной функции на зашумление сигнала. Показано, что значение этой оценки, вычисленное при нулевом временном сдвиге, представляет собой оценку коэффициента корре-ляции между полезным сигналом и помехой. Проведена технология вычислительных экспериментов, сравнительный анализ, подтверждена достоверность предложенных алгоритмов и технологий. Показано, что при нормальном техническом состоянии объекта оценки релейной корреляционной взаимной функции и коэффициента корреляции между полезным сигналом и помехой будут близки нулю. При зарождении различных дефектов, предшествующих неполадкам на объекте, эти оценки в зависимости от степени повреждения будут изменяться. Поэтому в системах мониторинга и контроля именно оценки взаимной корре-ляционной функции и коэффициента корреляции между полезным сигналом и помехой целесообразно использовать как информативные признаки для сигнализации и контроля начала изменений технического состояния объектов и динамики развития их неисправностей. Благодаря использованию этих новых эффективных информативных признаков можно повысить степень достоверности и надежности функционирования современных информационных систем.

Improved two-point correlation function estimates using glass-like distributions as a reference sample

ABSTRACT All estimators of the two-point correlation function are based on a random catalogue, a set of points with no intrinsic clustering following the selection function of a survey. High-accuracy estimates require the use of large random catalogues, which imply a high computational cost. We propose to replace the standard random catalogues by glass-like point distributions or glass catalogues whose power spectrum P(k) ∝ k4 exhibits significantly less power on scales larger than the mean interparticle separation than a Poisson distribution with the same number of points. We show that these distributions can be obtained by iteratively applying the technique of Zeldovich reconstruction commonly used in studies of baryon acoustic oscillations (BAO). We provide a modified version of the widely used Landy–Szalay estimator of the correlation function adapted to the use of glass catalogues and compare its performance with the results obtained using random samples. Our results show that glass-like samples do not add any bias with respect to the results obtained using Poisson distributions. On scales larger than the mean interparticle separation of the glass catalogues, the modified estimator leads to a significant reduction of the variance of the Legendre multipoles ξℓ(s) with respect to the standard Landy–Szalay results with the same number of points. The size of the glass catalogue required to achieve a given accuracy in the correlation function is significantly smaller than when using random samples. Their use could help us to drastically reduce the computational cost of configuration-space clustering analysis of future surveys while maintaining high-accuracy requirements.

Identification of signals as one of the stages of synthesis of the system of automatic stabilization of the height of the fluidized bed in the grain dryer

Drying is one of the most responsible and energy-intensive post-harvest grain pro-cessing operations. The constant rise in heat prices has led to the search for new designs of grain dryers, or improvements to existing ones. Studies of well-known scientists such as Romankov, Rezchikov, Rashkovskaya have proven the potential effectiveness of using a fluid-ized bed for drying grain.The degree to which this drying efficiency approaches the potential is determined by how much the change in the actual humidity of the material during drying approaches the theoretical drying curve at the same resource consumption and the same initial state of the material to be dried.The study of the designs of fluidized bed grain dryers showed that they show irregular fluctuations in the height of the fluidized bed, and, accordingly, the efficiency of energy use for drying. The reason for this phenomenon is related, on the one hand, to random fluctua-tions in the amount of grain in the fluidized bed, and on the other - to the shortcomings of the system of automatic stabilization of its height.Taking into account the provisions of the theory of optimal control indicates that max-imizing the accuracy of stabilization of a given value of the height of the fluidized bed should be achieved by developing an optimal multidimensional stabilization system of the specified height. The basis for solving the first of these is the study of the design of a fluidized bed grain dryer.The decomposition of the grain dryer allows to present it in the form of a set of multidimensional objects with pure delay. The size of this set is determined by the number of cascades, and the dynamic characteristics of its elements depend on the distance of the cascade to the gate.The degree of connections between input-output signals in the form of correlation and cross-correlation functions was studied, and the physical processes occurring in the grain dryer were analyzed. As a result of applying the Blackman-Tukey algorithm to the implemen-tations of the components of the control signal vectors and output signals, estimates of the corresponding correlation functions and spectral densities for all elements of the set of con-trol objects were obtained. The discrete Fourier transform of smoothed estimates of correla-tion and cross-correlation functions allowed us to calculate estimates of the spectral and mutual spectral densities of the respective signals. Approximation of spectral and mutual spectral density estimates by the method of generalized logarithmic frequency characteristics allowed to determine the mathematical description of these signals in the form of spectral density expressions, which will be the basis for the next stage of object identification.

Two-point Statistics without Bins: A Continuous-function Generalization of the Correlation Function Estimator for Large-scale Structure

Abstract The two-point correlation function (2pcf) is the key statistic in structure formation; it measures the clustering of galaxies or other density field tracers. Estimators of the 2pcf, including the standard Landy–Szalay (LS) estimator, evaluate the 2pcf in hard-edged separation bins, which is scientifically inappropriate and results in a poor trade-off between bias and variance. We present a new 2pcf estimator, the Continuous-Function Estimator, which generalizes LS to a continuous representation and obviates binning in separation or any other pair property. Our estimator, inspired by the mathematics of least-squares fitting, replaces binned pair counts with projections onto basis functions; it outputs the best linear combination of basis functions to describe the 2pcf. The choice of basis can take into account the expected form of the 2pcf, as well as its dependence on pair properties other than separation. We show that the Continuous-Function Estimator with a cubic-spline basis better represents the shape of the 2pcf compared to LS. We also estimate directly the baryon acoustic scale, using a small number of physically motivated basis functions. Critically, this leads to a reduction in the number of mock catalogs required for covariance estimation, which is currently the limiting step in many 2pcf analyses. We discuss further applications of the Continuous-Function Estimator, including determination of the dependence of clustering on galaxy properties and searches for potential inhomogeneities or anisotropies in large-scale structure.

Dependent radius marks of Laguerre tessellations: a case study

SummaryWe study a particular marked three‐dimensional point process sample that represents a Laguerre tessellation. It comes from a polycrystalline sample of aluminium alloy material. The ‘points’ are the cell generators while the ‘marks’ are radius marks that control the size and shape of the tessellation cells. Our statistical mark correlation analyses show that the marks of the sample are in clear and plausible spatial correlation: the marks of generators close together tend to be small and similar and the form of the correlation functions does not justify geostatistical marking. We show that a simplified modelling of tessellations by Laguerre tessellations with independent radius marks may lead to wrong results. When we started from the aluminium alloy data and generated random marks by random permutation we obtained tessellations with characteristics quite different from the original ones. We observed similar behaviour for simulated Laguerre tessellations. This fact, which seems to be natural for the given data type, makes fitting of models to empirical Laguerre tessellations quite difficult: the generator points and radius marks have to be modelled simultaneously. This may imply that the reconstruction methods are more efficient than point‐process modelling if only samples of similar Laguerre tessellations are needed. We also found that literature recipes for bandwidth choice for estimating correlation functions should be used with care.

Radar imaging with EISCAT 3D

Abstract. A new incoherent scatter radar called EISCAT 3D is being constructed in northern Scandinavia. It will have the capability to produce volumetric images of ionospheric plasma parameters using aperture synthesis radar imaging. This study uses the current design of EISCAT 3D to explore the theoretical radar imaging performance when imaging electron density in the E region and compares numerical techniques that could be used in practice. Of all imaging algorithms surveyed, the singular value decomposition with regularization gave the best results and was also found to be the most computationally efficient. The estimated imaging performance indicates that the radar will be capable of detecting features down to approximately 90×90 m at a height of 100 km, which corresponds to a ≈0.05∘ angular resolution. The temporal resolution is dependent on the signal-to-noise ratio and range resolution. The signal-to-noise ratio calculations indicate that high-resolution imaging of auroral precipitation is feasible. For example, with a range resolution of 1500 m, a time resolution of 10 s, and an electron density of 2×1011m-3, the correlation function estimates for radar scatter from the E region can be measured with an uncertainty of 5 %. At a time resolution of 10 s and an image resolution of 90×90 m, the relative estimation error standard deviation of the image intensity is 10 %. Dividing the transmitting array into multiple independent transmitters to obtain a multiple-input–multiple-output (MIMO) interferometer system is also studied, and this technique is found to increase imaging performance through improved visibility coverage. Although this reduces the signal-to-noise ratio, MIMO has successfully been applied to image strong radar echoes as meteors and polar mesospheric summer echoes. Use of the MIMO technique for incoherent scatter radars (ISRs) should be investigated further.

Fast analytical calculation of the random pair counts for realistic survey geometry

Galaxy clustering is a standard cosmological probe that is commonly analysed through two-point statistics. In observations, the estimation of the two-point correlation function crucially relies on counting pairs in a random catalogue. The latter contains a large number of randomly distributed points, which accounts for the survey window function. Random pair counts can also be advantageously used for modelling the window function in the observed power spectrum. Since pair counting scales as 𝒪(N2), where N is the number of points, the computational time to measure random pair counts can be very expensive for large surveys. In this work, we present an alternative approach for estimating those counts that does not rely on the use of a random catalogue. We derived an analytical expression for the anisotropic random-random pair counts that accounts for the galaxy radial distance distribution, survey geometry, and possible galaxy weights. We show that a prerequisite is the estimation of the two-point correlation function of the angular selection function, which can be obtained efficiently using pixelated angular maps. Considering the cases of the VIPERS and SDSS-BOSS redshift surveys, we find that the analytical calculation is in excellent agreement with the pair counts obtained from random catalogues. The main advantage of this approach is that the primary calculation only takes a few minutes on a single CPU and it does not depend on the number of random points. Furthermore, it allows for an accuracy on the monopole equivalent to what we would otherwise obtain when using a random catalogue with about 1500 times more points than in the data at hand. We also describe and test an approximate expression for data-random pair counts that is less accurate than for random-random counts, but still provides subpercent accuracy on the monopole. The presented formalism should be very useful in accounting for the window function in next-generation surveys, which will necessitate accurate two-point window function estimates over huge observed cosmological volumes.