Least-squares Estimation Research Articles

Maximum-likelihood estimation of the Po-MDDRCINAR(p) model with analysis of a COVID-19 data

Integer-valued data are frequently encountered in time series studies. A pth-order mixed dependence-driven random coefficient integer-valued autoregressive time series model (Po-MDDRCINAR(p)) in view of binomial and negative binomial operators, where the innovation sequence follows a Poisson distribution, is investigated to provide meaningful theoretical explanations. Strict stationary and ergodicity of the model are demonstrated. Furthermore, the conditional least-squares and conditional maximum-likelihood methods are adopted to estimate the parameters, where the asymptotic characterization of the estimators is derived. Finite-sample properties of the conditional maximum-likelihood estimator are examined in relation to the widely used conditional least-squares estimator. The conclusion is that, if the Poisson assumption of the innovation sequence can be justified, conditional maximum-likelihood method performs better in terms of MADE and MSE. Finally, the practical performance of the model is illustrated by a set of COVID-19 data of suspected cases in China with a comparison with relevant models that exist so far in the literature.

Statistical and computational efficiency for smooth tensor estimation with unknown permutations

We consider the problem of structured tensor denoising in the presence of unknown permutations. Such data problems arise commonly in recommendation systems, neuroimaging, community detection, and multiway comparison applications. Here, we develop a general family of smooth tensor models up to arbitrary index permutations; the model incorporates the popular tensor block models and Lipschitz hypergraphon models as special cases. We show that a constrained least-squares estimator in the block-wise polynomial family achieves the minimax error bound. A phase transition phenomenon is revealed with respect to the smoothness threshold needed for optimal recovery. In particular, we find that a polynomial of degree up to ( m − 2 ) ( m + 1 ) / 2 is sufficient for accurate recovery of order-m tensors, whereas higher degrees exhibit no further benefits. This phenomenon reveals the intrinsic distinction for smooth tensor estimation problems with and without unknown permutations. Furthermore, we provide an efficient polynomial-time Borda count algorithm that provably achieves the optimal rate under monotonicity assumptions. The efficacy of our procedure is demonstrated through both simulations and Chicago crime data analysis.

Kalman-based estimation of loading conditions from ultrasonic guided wave measurements

Abstract Ultrasonic guided wave-based structural health monitoring (SHM) of structures can be perturbed by environmental and operations conditions (EOCs) that alter wave propagation. In this work, we present an estimation procedure to reconstruct an EOC-free baseline of the structure from the only available Ultrasonic guided wave measurements. This procedure could typically be used as a prior step to increase the robustness of a more general ultrasonic imaging algorithm or SHM process dedicated to flaw detection. Our approach is model-based, i.e. we use a precise modeling of the wave propagation altered by structure loading conditions. This model is coupled with the acquired data through a data assimilation procedure to estimate the deformation caused by the unknown loading conditions. From a methodological point of view, our approach is original since we have proposed an iterated reduced-order unscented Kalman strategy, which we justify as an alternative to a Levenberg–Marquardt strategy for minimizing the non quadratic least-squares estimation criteria. Therefore, from a data assimilation perspective, we provide a quasi-sequential strategy that can valuably replace more classical variational approaches. Indeed, our resulting algorithm proves to be computationally very effective, allowing us to successfully apply our strategy to realistic 3D industrial SHM configurations.

Iterative decomposition of water and fat with echo asymmetry and least-squares estimation (IDEAL-IQ) for evaluation of early bone mass changes in ageing osteoporosis patients

This study explored the application value of iterative decomposition of water and fat with echo asymmetry and least-squares estimation (IDEAL-IQ) technology in the early diagnosis of ageing osteoporosis (OP). 172 participants were enrolled and underwent magnetic resonance imaging (MRI) examinations on a 3.0T scanner. 100 cases were included in the normal group (50 males and 50 females; mean age: 45 years; age range: 20–84 years). 33 cases were included in the osteopenia group (17 males and 16 females; mean age: 55 years; age range: 43–83 years). 39 cases were included in the OP group (19 males and 20 females; mean age: 58 years; age range: 48–82 years). Conventional T1WI and T2WI were first obtained, followed by 3D-IDEAL-IQ-acquisition. Fat fraction (FF) and apparent transverse relaxation rate (R2*) results were automatically calculated from IDEAL-IQ-images on the console. Based on T1W and T2W-images, 300 ROIs for each participant were manually delineated in L1-L5 vertebral bodies of five middle slices. In each age group of all normal subjects, each parameter was significantly correlated with gender. In male participants from the normal, osteopenia, and OP groups, statistical analysis revealed F values of 11319.292 and 180.130 for comparisons involving FF and R2* values, respectively (all p<0.0001). The sensitivity and specificity of FF values were 0.906 and 0.950, 0.994 and 0.997, 0.865 and 0.820, respectively. For R2*, they were 0.665 and 0.616, 0.563 and 0.519, 0.571 and 0.368, respectively. In female participants from the normal, osteopenia, and OP-groups, statistical analysis revealed F values of 12461.658 and 548.274 for comparisons involving FF and R2* values, respectively (all p<0.0001). The sensitivity and specificity of FF values were 0.985 and 0.991, 0.996 and 0.996, 0.581 and 0.678, respectively. For R2*, they were 0.698 and 0.730, 0.603 and 0.665, 0.622 and 0.525, respectively. Significant differences were indicated in the quantitative values among the three groups. FF value had good performance, while R2* value had poor performance in discriminating osteopenia and OP-groups. Overall, the IDEAL-IQ technique offers specific reference indices that enable noninvasive and quantitative assessment of lumbar vertebrae bone metabolism, thereby providing diagnostic information for OP.

A new extension of Burr-Hatke exponential distribution with engineering and biomedical applications

In this study, the Topp-Leone family of distribution approach was used to modify the Burr Hatke Exponential distribution to provide adequate fits for some engineering and health data which previous existing distributions in the family of Burr Hatke Exponential have failed to do. The new distribution improves the robustness of Burr Hatke Exponential distribution by making it capable of modeling emerging new world complex data with varying features, possesses greater capacity and flexibility to model lifetime data, has better goodness of fit. Some mathematical properties of the derived distribution such as quantile function, moments, order statistics, entropies, etc were obtained and discussed. Some non-Bayesian estimation approaches like maximum likelihood estimation (ML), maximum Product spacing estimation (MPSE), Least squares estimation (LS), weighted least squares estimation (WLS), Cramer-von-Mises estimation (CVM), Anderson-Darling estimation (AD), and right-tailed Anderson Darling estimation (RTAD), as well as Bayesian method under independent gamma priors were adopted to estimate the parameters of the model and the various methods proved efficient. From the simulation results, the bias and root mean squared error for the parameters are relatively small and the become smaller as the sample size becomes larger. This shows convenience and improved estimation accuracy. We further constructed a regression model using the proposed distribution. Extensive simulation studies were used to determine the efficiency of the method in both the estimation of its parameters and that of the regression model. The TL-BHE regression was fitted on censored CD4 count data of HIV/AIDs patients. The competitiveness, applicability, and usefulness of the model were demonstrated using datasets and the results validate the theoretical findings. The results indicate that the TL-BHE distribution achieved better results than the baseline distribution and other variants of the classical distributions.

Corrected generalized cross-validation for finite ensembles of penalized estimators

Abstract Generalized cross-validation (GCV) is a widely used method for estimating the squared out-of-sample prediction risk that employs scalar degrees of freedom adjustment (in a multiplicative sense) to the squared training error. In this paper, we examine the consistency of GCV for estimating the prediction risk of arbitrary ensembles of penalized least-squares estimators. We show that GCV is inconsistent for any finite ensemble of size greater than one. Towards repairing this shortcoming, we identify a correction that involves an additional scalar correction (in an additive sense) based on degrees of freedom adjusted training errors from each ensemble component. The proposed estimator (termed CGCV) maintains the computational advantages of GCV and requires neither sample splitting, model refitting, or out-of-bag risk estimation. The estimator stems from a finer inspection of the ensemble risk decomposition and two intermediate risk estimators for the components in this decomposition. We provide a non-asymptotic analysis of the CGCV and the two intermediate risk estimators for ensembles of convex penalized estimators under Gaussian features and a linear response model. Furthermore, in the special case of ridge regression, we extend the analysis to general feature and response distributions using random matrix theory, which establishes model-free uniform consistency of CGCV.

Fine-tuning weibull distribution parameters in Morocco’s Tarfaya and Tangier wind farms using two-stage swarm optimization

This study assesses the efficacy of particle swarm optimization (PSO) in estimating scale ( c) and shape ( k) parameters of the Weibull distribution model for wind energy forecasting at two key wind farm sites in Morocco— Tarfaya in the south and Tangier in the north, utilizing real wind data from 2022. Employing a novel square frequency error objective function to enhance parameter accuracy, the study adopts a two-stage training approach involving recursive least-square estimation and PSO fine-tuning. Validation with artificial data underscores PSO’s effectiveness under diverse wind conditions. Parameter sensitivity analysis identifies four optimal PSO configurations, with the PSO-4 model exhibiting superior performance. Comparative analysis against traditional and heuristic optimization methods consistently demonstrates PSO-4’s lowest root-mean-square error (RMSE) and mean absolute error (MAE), high coefficients of determination ( R2), and shortest computation time. The research highlights PSO-4 model as a precise and efficient tool for Weibull distribution parameter estimation in wind energy forecasting, showcasing robust convergence across both wind farm sites.

LS-based multipath channel measurement method using software defined radio platform

Multipath channel measurement is the foundation of multipath channel modeling and analysis. Recently, the software defined radio (SDR) has provided new ideas for the design of multipath channel measurement systems due to universality and reconfigurability. Therefore, this paper introduces a multipath channel measurement method applicable to various SDR platforms. At first, to enhance the resolution of multipath measurements, a signal processing approach combining the least squares (LS) estimation algorithm within the SDR framework is proposed. Furthermore, to mitigate LS estimation errors stemming from high correlation among baseband signals, a scrambled measurement signal is introduced and its effectiveness is demonstrated through eigenvalue decomposition. Next, the systematic errors of In-phase and Quadrature(IQ) imbalance and bandwidth limitation are further analyzed and addressed within the SDR framework. Finally, the proposed measurement method is implemented based on a practical universal software radio peripheral (USRP) device and validated under simulated and real channels. The proposed measurement method mainly focuses on the baseband signal processing within the software component of SDR, making it applicable to most general USRP devices. The experiment results demonstrate that multipath channel measurement results with high accuracy can be obtained.

Strong laws for weighted sums of widely orthant dependent random variables and applications

Abstract In this study, the strong law of large numbers and the convergence rate for weighted sums of non-identically distributed widely orthant dependent random variables are established. As applications, the strong consistency for weighted estimator in nonparametric regression model and the rate of strong consistency for least-squares estimator in multiple linear regression model are obtained. Some numerical simulations are also provided to verify the validity of the theoretical results.

Forecasting Lattice and Point Spatial Data: Comparison of Unilateral and Multilateral SAR Models

Spatial auto-regressive (SAR) models are widely used in geosciences for data analysis; their main feature is the presence of weight (W) matrices, which define the neighboring relationships between the spatial units. The statistical properties of parameter and forecast estimates strongly depend on the structure of such matrices. The least squares (LS) method is the most flexible and can estimate systems of large dimensions; however, it is biased in the presence of multilateral (sparse) matrices. Instead, the unilateral specification of SAR models provides triangular weight matrices that allow consistent LS estimates and sequential prediction functions. These two properties are strictly related and depend on the linear and recursive nature of the system. In this paper, we show the better performance in out-of-sample forecasting of unilateral SAR (estimated with LS), compared to multilateral SAR (estimated with maximum likelihood, ML). This conclusion is supported by numerical simulations and applications to real geological data, both on regular lattices and irregularly distributed points.

Unscented recursive three-step filter based unbiased minimum-variance estimation for a class of nonlinear systems

For the direct feedthrough system where unknown input affects both process equation and measurement equation, an unscented recursive three-step filter based unbiased minimum-variance (URTSF-UMV) estimation algorithm is proposed. The algorithm is based on the unbiased minimum-variance (UMV) method and uses statistical linearisation technology to approximate the nonlinear system and the nonlinear measurement function to linear regression form. The estimation of unknown input is obtained from innovation through weighted least-squares (WLS) estimation. The approximate linear regression form allows us to process the nonlinear system with unknown input direct feedthrough by using the UMV state estimation framework, and derive the filter by minimising the trace of the state error covariance of the measurement update under the unbiased condition. The URTSF-UMV requires that the unknown input distribution matrix is full column rank. We use the full rank decomposition method to deal with the nonlinear system containing the unknown input rank-deficient distribution matrix. The effectiveness of the URTSF-UMV algorithm is demonstrated through an illustrative simulation example.

STATISTICAL INFERENCE ON SINE-EXPONENTIAL DISTRIBUTION PARAMETER

The Sine-Exponential (Sine-E) distribution is a probability distribution that combines the periodic behavior of the sine function with the decay characteristic of the exponential function. This study addresses the problem of identifying the most accurate and reliable estimation method for the parameter of the Sine-E distribution. The objective is to evaluate various parameter estimation techniques, including Maximum Likelihood Estimation (MLE), Least Squares Estimation (LSE), Weighted Least Squares Estimation (WLSE), Maximum Product of Spacing Estimation (MPSE), Cramer-von-Mises Estimation (CVME), and Anderson-Darling Estimation (ADE), using Mean Square Error (MSE) as the criterion for determining the technique with the minimum error. The study’s findings reveal that as sample size increases, the parameter estimates for all techniques converge to the true parameter value, with decreases in bias, MSE, and mean relative estimates. Among the techniques evaluated, the MPSE method consistently provides estimates closest to the true parameter value and exhibits the least bias and lowest MSE across small, moderate, and large sample sizes, making it the best estimator for the Sine-E distribution.

Regional tidal modeling on the European coast using tide gauges and satellite altimetry

The study investigates sea-level measurements along the coastal area of Europe for the 60-year (1961–2020) time span. Linear and quadratic modeling of tide gauge measurements showed an almost positive rate of trend of sea-level rise (0.09 to 3.6 mm/yr) and low acceleration (−0.05 to 0.40 mm/yr2). A least-squares harmonic estimation tidal modeling was carried out to estimate frequency (cycles per day) for a certain period. The smaller and higher tidal frequencies of these stations indicate their stability in terms of their surface variation. We used the 1993–2020 satellite altimetry data from the nearest grid points of the tide gauge station. The correlation coefficient between observed and satellite altimetry (lowest 0.53 and highest 0.93) varies at each station. This happens because of many factors that can affect the large difference in the sea-level trend between the satellite-derived and tide gauge results. Finally, to implement a global reference system for physical heights, the offshore topographic slope direction and slope range with contour spacing from the sea to the associated coastline were analyzed using bathymetry data. The abrupt change in slope from the coastline toward the sea can be seen toward the east, west, and southeast on the European coast. This is also an important factor that affects the variation of sea level.

Global assessment of linear trend and seasonal variations of GNSS-IR sea level retrievals with nearby tide gauges

Global Navigation Satellite System Interferometric Reflectometry (GNSS-IR) sea level retrievals and tide gauges at 40 globally distributed stations spanning from about 4.5 to 18 years are compared on a site-by-site basis, in terms of noise background, rate and seasonal variations by using the weighted least squares estimation (LSE) along with the Maximum likelihood estimation (MLE). The result shows that monthly GNSS-IR data agree well with tide gauges for most stations except the site Tuktoyaktuk, Canada. The mean correlation is 0.95 and the mean root mean square difference is 2.9 cm, respectively. The discrepancies of rate and seasonal amplitude estimates are within ± 1 cm for most stations. We confirm both the two sea level data exhibit temporal correlation, which has a great effect on the rate uncertainty estimates. Akaike Information Criteria (AIC) and Bayesian Information Criteria (BIC) favor Matérn and first-order autoregressive (AR1) the preferred stochastic model for the daily and monthly mean sea level time series, respectively. Owing to the data span dependence for the rate uncertainty estimates, to get an accuracy of sub-mm/yr in linear rate using the weighted LSE, at least 30 years of data (depends on data quality) is required. We recommend using long time series and a proper stochastic model for the rate estimation of sea level.

Blind power spectrum reconstruction for multi-sinusoid signals with generative multicoset sampling

As the target frequency increases, the Nyquist rate is hard to reach due to hardware limitations. Alternatives to high-rate sampling have attracted considerable research interest. Compressed covariance sensing based on multicoset sampling (MCS) is a prevalent sub-Nyquist sampling strategy for blind power spectrum reconstruction. However, it requires the sampling pattern of MCS to be a minimal sparse ruler. In this paper, a generative multicoset sampling (GMCS) is proposed to minimize the number of cosets. In the sampling stage, a minimal MCS, delay coprime scheme is proposed to ensure the recoverability of the power spectrum. In the recovery stage, a new function, i.e., multifold correlation is constructed for blind power spectrum reconstruction. We prove that the power spectrum of a multi-sinusoid signal (MSS) can be estimated by its multifold correlation and the sample density of multifold correlation can be increased by correlation operations. Owing to multifold correlation, GMCS can generate virtual cosets satisfying the sparse ruler criterion through only two physical cosets and then efficiently recover the power spectrum by least-squares estimation. Moreover, we demonstrate the feasibility of GMCS for linear frequency modulation (LFM) signals. Finally, the effectiveness of GMCS are verified by numerical simulations and blade tip timing experiments (a non-contact vibration measurement). By reducing the average sampling rate and hardware complexity, GMCS opens new avenues for blind sampling and power spectrum reconstruction.

Least Squares Estimation of Multifactor Uncertain Differential Equations with Applications to the Stock Market

Multifactor uncertain differential equations are powerful tools for studying dynamic systems under multi-source noise. A key challenge in this study is how to accurately estimate unknown parameters based on the framework of uncertainty theory in multi-source noise environments. To address this core problem, this paper innovatively proposes a least-squares estimation method. The essence of this method lies in constructing statistical invariants with a symmetric uncertainty distribution based on observational data and determining specific parameters by minimizing the distance between the population distribution and the empirical distribution of the statistical invariant. Additionally, two numerical examples are provided to help readers better understand the practical operation and effectiveness of this method. In addition, we also provide a case study of JD.com’s stock prices to illustrate the advantages of the method proposed in this paper, which not only provides a new idea and method for addressing the problem of dynamic system parameter estimation but also provides a new perspective and tool for research and application in related fields.

A Rigorous 2D-3D Registration Method for a High-Speed Bi-Planar Videoradiography Imaging System.

High-speed biplanar videoradiography can derive the dynamic bony translations and rotations required for joint cartilage contact mechanics to provide insights into the mechanical processes and mechanisms of joint degeneration or pathology. A key challenge is the accurate registration of 3D bone models (from MRI or CT scans) with 2D X-ray image pairs. Marker-based or model-based 2D-3D registration can be performed. The former has higher registration accuracy owing to corresponding marker pairs. The latter avoids bead implantation and uses radiograph intensity or features. A rigorous new method based on projection strategy and least-squares estimation that can be used for both methods is proposed and validated by a 3D-printed bone with implanted beads. The results show that it can achieve greater marker-based registration accuracy than the state-of-the-art RSA method. Model-based registration achieved a 3D reconstruction accuracy of 0.79 mm. Systematic offsets between detected edges in the radiographs and their actual position were observed and modeled to improve the reconstruction accuracy to 0.56 mm (tibia) and 0.64 mm (femur). This method is demonstrated on in vivo data, achieving a registration precision of 0.68 mm (tibia) and 0.60 mm (femur). The proposed method allows the determination of accurate 3D kinematic parameters that can be used to calculate joint cartilage contact mechanics.

Vertebral bone quality score was associated with paraspinal muscles fat infiltration, but not modic classification in patients with chronic low back pain: a prospective cross-sectional study

BackgroundThe lumbar vertebra and paraspinal muscles play an important role in maintaining the stability of the lumbar spine. Therefore, the aim of this study was to investigate the relationship between paraspinal muscles fat infiltration and vertebral body related changes [vertebral bone quality (VBQ) score and Modic changes (MCs)] in patients with chronic low back pain (CLBP).MethodsPatients with CLBP were prospectively collected in four hospitals and all patients underwent 3.0T magnetic resonance scanning. Basic clinical information was collected, including age, sex, course of disease (COD), and body mass index (BMI). MCs were divided into 3 types based on their signal intensity on T1 and T2-weighted imaging. VBQ was obtained by midsagittal T1-weighted imaging (T1WI) and calculated using the formula: SIL1−4/SICSF. The Proton density fat fraction (PDFF) values and cross-sectional area (CSA) of paraspinal muscles were measured on the fat fraction map from the iterative decomposition of water and fat with the echo asymmetry and least-squares estimation quantitation (IDEAL-IQ) sequences and in/out phase images at the central level of the L4/5 and L5/S1 discs.ResultsThis study included 476 patients with CLBP, including 189 males and 287 females. 69% had no Modic changes and 31% had Modic changes. There was no difference in CSA and PDFF for multifidus(MF) and erector spinae (ES) at both levels between Modic type I and type II, all P values>0.05. Spearman correlation analysis showed that VBQ was weakly negatively correlated with paraspinal muscles CSA (all r values < 0.3 and all p values < 0.05), moderately positive correlation with PDFF of MF at L4/5 level (r values = 0.304, p values<0.001) and weakly positively correlated with PDFF of other muscles (all r values<0.3 and all p values<0.001). Multivariate linear regression analysis showed that age (β = 0.141, p < 0.001), gender (β = 4.285, p < 0.001) and VBQ (β = 1.310, p = 0.001) were related to the total PDFF of muscles. For MCs, binary logistic regression showed that the odds ratio values of age, BMI and COD were 1.092, 1.082 and 1.004, respectively (all p values ＜ 0.05).ConclusionsPDFF of paraspinal muscles was not associated with Modic classification. In addition to age and gender, PDFF of paraspinal muscles is also affected by VBQ. Age and BMI are considered risk factors for the MCs in CLBP patients.

Confidence intervals in monotone regression

AbstractWe construct bootstrap confidence intervals for a monotone regression function. It has been shown that the ordinary nonparametric bootstrap, based on the nonparametric least squares estimator (LSE) , is inconsistent in this situation. We show that an ‐consistent bootstrap can be based on the smoothed , to be called the SLSE (Smoothed Least Squares Estimator). The asymptotic pointwise distribution of the SLSE is derived. The confidence intervals, based on the smoothed bootstrap, are compared to intervals based on the (not necessarily monotone) Nadaraya Watson estimator and the effect of Studentization is investigated. We also give a method for automatic bandwidth choice, correcting work in Sen and Xu (2015). Analogous methods for constructing confidence intervals in the current status model are discussed, improving on work in Groeneboom and Hendrickx (2018).

An Image-Based Prior Knowledge-Free Approach for a Multi-Material Decomposition in Photon-Counting Computed Tomography.

Photon-counting CT systems generally allow for acquiring multiple spectral datasets and thus for decomposing CT images into multiple materials. We introduce a prior knowledge-free deterministic material decomposition approach for quantifying three material concentrations on a commercial photon-counting CT system based on a single CT scan. We acquired two phantom measurement series: one to calibrate and one to test the algorithm. For evaluation, we used an anthropomorphic abdominal phantom with inserts of either aqueous iodine solution, aqueous tungsten solution, or water. Material CT numbers were predicted based on a polynomial in the following parameters: Water-equivalent object diameter, object center-to-isocenter distance, voxel-to-isocenter distance, voxel-to-object center distance, and X-ray tube current. The material decomposition was performed as a generalized least-squares estimation. The algorithm provided material maps of iodine, tungsten, and water with average estimation errors of 4% in the contrast agent maps and 1% in the water map with respect to the material concentrations in the inserts. The contrast-to-noise ratio in the iodine and tungsten map was 36% and 16% compared to the noise-minimal threshold image. We were able to decompose four spectral images into iodine, tungsten, and water.