Least-squares Estimation Research Articles

Improved deformation modelling of structures by least-squares variance component estimation based on multi-sensor data integration

In this contribution, to improve the deformation modelling based on data integration, the LS-VCE algorithm is proposed by obtaining a stochastic model of input multi-sensor data. So, one can achieve the accurate variance-covariance matrix of multi-sensor observations to participate in iterative least-squares. A practical application was made for the settlement observations from geotechnical settlement-meters and geodetic levelling (respectively known as internal and external sensors) to model the surface settlement variation of the Karkhe earth-dam. The determined variance component shows less contribution of the geotechnical settlements in the deformation modelling. An achievement of this paper is that the LS-VCE method improves the integration of the geotechnical with geodetic data by estimating an optimal stochastic model resulting in deformation model optimization. Validation results of estimated surface settlements on the check-points show an RMSE of about 3 cm and a relative-error of about 14%, which indicates the success of the modelling.

Response time correction of slow-response sensor data by deconvolution of the growth-law equation

Abstract. Accurate high-resolution measurements are essential to improve our understanding of environmental processes. Several chemical sensors relying on membrane separation extraction techniques have slow response times due to a dependence on equilibrium partitioning across the membrane separating the measured medium (i.e., a measuring chamber) and the medium of interest (i.e., a solvent). We present a new technique for deconvolving slow-sensor-response signals using statistical inverse theory; applying a weighted linear least-squares estimator with the growth law as a measurement model. The solution is regularized using model sparsity, assuming changes in the measured quantity occur with a certain time step, which can be selected based on domain-specific knowledge or L-curve analysis. The advantage of this method is that it (1) models error propagation, providing an explicit uncertainty estimate of the response-time-corrected signal; (2) enables evaluation of the solution self consistency; and (3) only requires instrument accuracy, response time, and data as input parameters. Functionality of the technique is demonstrated using simulated, laboratory, and field measurements. In the field experiment, the coefficient of determination (R2) of a slow-response methane sensor in comparison with an alternative fast-response sensor significantly improved from 0.18 to 0.91 after signal deconvolution. This shows how the proposed method can open up a considerably wider set of applications for sensors and methods suffering from slow response times due to a reliance on the efficacy of diffusion processes.

Reliability and maintainability optimization of load haul dump machines using genetic algorithm and particle swarm optimization

PurposeReliability and maintainability estimation of any system depends on the identification of the best-fitted probability distribution of failure and repair rates. The parameters of the best-fitted probability distribution are also contributing significantly to reliability estimation. In this work, a case study of load haul dump (LHD) machines is illustrated that consider the optimization of failure and repair rate parameters using two well established metaheuristic approaches, namely, genetic algorithm (GA) and particle swarm optimization (PSO). This paper aims to analyze the aforementioned points.Design/methodology/approachThe data on time between failures (TBF) and time to repairs (TTR) are collected for a LHD machine. The descriptive statistical analysis of TBF & TTR data is performed, trend and serial correlation tested and using Anderson–Darling (AD) value best-fitted distributions are identified for repair and failure times of various subsystems. The traditional methods of estimation like maximum likelihood estimation, method of moments, least-square estimation method help only in finding the local solution. Here, for finding the global solution two well-known metaheuristic approaches are applied.FindingsThe reliability of the LHD machine after 60 days on the real data set is 28.55%, using GA on 250 generations is 17.64%, and using PSO on 100 generations and 100 iterations is 30.25%. The PSO technique gives the global best value of reliability.Practical implicationsThe present work will be very convenient for reliability engineers, researchers and maintenance managers to understand the failure and repair pattern of LHD machines. The same methodology can be applied in other process industries also.Originality/valueIn this case study, initially likelihood function of the best-fitted distribution is optimized by GA and PSO. Reliability and maintainability of LHD machines evaluated by the traditional approach, GA and PSO are compared. These results will be very helpful for maintenance engineers to plan new maintenance strategies for better functioning of LHD machines.

The Optimization and Empirical Study of Stock Excess Return

In order to explore the influencing factors of stock excess return and provide a particular reference for predicting the development of stocks in the country, I comprehensively refer to relevant economic theories and academic research and use the 2018 and 2019 CSI 300 index data provided by the wind to establish an econometric model. Moreover, carry out inspection and evaluation. I choose the logarithm (R) of the stock excess return as the explained variable, and the risk coefficient, market capitalization, book-to-market value ratio, turnover rate, etc., as the explanatory variables for regression analysis. According to the support of economic theory and the observation of the scatter plot trend, I removed the two explanatory variables of price-earnings ratio and net profit. In order to reduce the influence of outliers on model parameter estimation, I use the SPSS case diagnosis function to remove outliers. I performed a least-squares estimation of the initial model and performed t-tests and F-tests for the parameters and the overall equation for the remaining data. The variance inflation factor shows that the model has a multicollinearity problem, which I remedy by removing the year*ln_ME term. The model did not pass White's heteroskedasticity test. I used Eviews to perform White's heteroscedasticity correction on the model to correct the model standard error and reduce the degree of heteroskedasticity. After remediation, I used the RESET test on the model again, and although the F value increased slightly, it was still in the receptive field, so the model remediation did not make the model setting severe problems. The Jacques Bella test is shown that the model residuals do not completely obey the normal distribution, but due to the relatively large sample size, the model residuals can be approximated by a normal distribution according to the central limit theorem. In addition, I also obtain numerical prediction results through point prediction, and the scatter plot shows a high degree of interval overlap. Finally, I summarize the form presented by the final model and return to the practical significance, which proposes a reference standard for evaluating excess return from the market environment and company size, and further affirm the economic significance of excess return as a diversified evaluation of stock quality.

Digital Transformation for Green Growth: Evidence from Micro Firms

In this paper we explore whether pursuing a digitalisation strategy, adopting advanced digital technologies, and building internal digital skills positively moderate the relationship between firms’ eco-innovation efforts and growth performance. Our econometric analysis of a sample of 5,015 micro firms from 39 countries accounts for selection bias and endogeneity via the estimation of an instrumental variable two-stage least squares (IV-2SLS) approach. The results provide evidence that eco-innovations boost firm growth when coupled with a well-defined digitalisation strategy. We also find that eco-innovations improve firm growth only for firms that adopt advanced digital technologies (such as big data and Artificial Intelligence applications) rather than basic ones. Finally, our results reveal that the positive relationship between eco-innovation and firm growth is enhanced by improvements in internal digital skills and capabilities. Our findings contribute to the literature and policy debates by underlying a new strategic alternative for micro firms that differs from traditional sole focus on eco-innovation driven growth. Instead, this paper advocates the viability and performance benefits of a twin digital and ecological transformation and showcases the need to align eco-innovation investments with digital transformation efforts for an economically successful net-zero transition.

Data-Driven Stabilization of Nonlinear Polynomial Systems With Noisy Data

In a recent article, we have shown how to learn controllers for unknown linear systems using finite-length noisy data by solving linear matrix inequalities. In this article, we extend this approach to deal with unknown nonlinear polynomial systems by formulating stability certificates in the form of data-dependent sum of squares programs, whose solution directly provides a stabilizing controller and a Lyapunov function. We then derive variations of this result that lead to more advantageous controller designs. The results also reveal connections to the problem of designing a controller starting from a least-square estimate of the polynomial system.

Adaptive real-time grid operation via Online Feedback Optimization with sensitivity estimation

In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input–output sensitivity estimation for real-time grid operation, e.g., at subsecond time scales. The OFO controller uses grid measurements as feedback to update the value of the controllable elements in the grid, and track the solution of a time-varying AC Optimal Power Flow (AC-OPF). Instead of relying on a full grid model, e.g., grid admittance matrix, OFO only requires the steady-state sensitivity relating a change in the controllable inputs, e.g., power injections set-points, to a change in the measured outputs, e.g., voltage magnitudes. Since an inaccurate sensitivity may lead to a model-mismatch and jeopardize the performance, we propose a recursive least-squares estimation that enables OFO to learn the sensitivity from measurements during real-time operation, turning OFO into a model-free approach. We analytically certify the convergence of the proposed OFO with sensitivity estimation, and validate its performance on a simulation using the IEEE 123-bus test feeder, and comparing it against a state-of-the-art OFO with constant sensitivity.

Voltage Sag Compensation Through Dynamic Voltage Restorer Under Non-Sinusoidal Electrical Grid Condition

This work presents a method of voltage insertion into the grid with harmonic distortions through a dynamic voltage restorer (DVR) for protection of sensitive loads to short-term voltage sags. The aim is to recover the nominal voltage level at the load by restoring only the fundamental frequency component without compensating the harmonic contents. Corrections carried out in this manner are justified as the DVR is a device dedicated to restore the amplitude of the sensitive load to its rated value, based on rapid variations from the power grid. The correction of distortions is not suitable for this equipment as harmonics are permanent problems and originate from nonlinearities of electrical loads. Therefore, the batteries that supply the DVR do not support the correction of permanent problems, only for rapid variations. Thus, it allows an extension in the lifespan of the batteries (needed in this type of compensator). Collaterally, restoring only the fundamental decreases the total harmonic distortion of the load, thus allowing this method to make a double contribution to improving the system’s power quality. For this purpose, a recursive least-square estimation is developed, suitable for voltages with harmonic content, here called HRLS. The construction of the reference voltage to be produced by the DC–AC converter also considers the synchronism with the fundamental component of the grid, which allows an optimization of the voltage level used on the DC-link. Correction scenarios only inserting the fundamental, as well as with harmonic content, are tested in simulation to compare performance in the use of the compensator power. The same scenarios are also tested on an experimental platform developed in the laboratory to confirm the proposal.

Adaptive estimation of autoregression models under long-tailed symmetric distribution

Non-normal innovations in autoregression models frequently occur in practice. In this situation, least squares (LS) estimators are known to be inefficient and non-robust, and maximum likelihood (ML) estimators need to be solved numerically, which becomes a daunting task. In the literature, the modified maximum likelihood (MML) estimation technique has been proposed to obtain the estimators of model parameters. While an explicit solution can be found via this method, the requirement of knowing the shape parameter becomes a drawback, especially in machine learning. In this study, we use the adaptive modified maximum likelihood (AMML) methodology, which combines the MML with Huber’s M-estimation so that this assumption is relaxed. The performance of the method in terms of efficiency and robustness is analyzed via simulation and compared to LS, MML and ML estimates that are obtained numerically via the Expectation Conditional Maximization (ECM) algorithm. Test statistics are proposed for the crucial parameters of the model. The results show that the AMML estimators are preferable in most of the settings according to the mean squared error (MSE) criterion and the test statistics based on AMML method are more robust than the others. Furthermore, both real life and synthetic data examples are given.

Identification of the phase connectivity in distribution systems through constrained least squares and confidence-based sequential assignment

This paper addresses the customer-phase identification problem in three-phase distribution grids including three-phase customers characterized by aggregated energy measurements. The proposed technique first solves a relaxed problem, in which the binary nature of the variables is ignored, which leads to a constrained, least-squares estimation, using as inputs the active and reactive energy readings provided by the smart meters, along with the energy delivered by each phase at the head of the feeder. With the estimated values of the decision variables, and their corresponding variances, a confidence-based selection technique is then applied for the sequential assignment of the customer with the highest joint probability of being connected to one of the three phases but not to the other two. The performance of the proposed procedure is assessed with five different scenarios in terms of accuracy for increasing number of loads and measurement errors. The robustness of the algorithm is additionally tested in the presence of model errors, and its performance is compared to that of existing methods.

Quantifying Viscous Damping and Stiffness in Parkinsonism Using Data-Driven Model Estimation and Admittance Control.

Rigidity of upper and lower limbs in Parkinson's disease (PD) is typically assessed via a clinical rating scale that is subject to human perception biases. Methodologies to quantify changes in rigidity associated with the angular position (stiffness) or velocity (viscous damping) are needed to enhance our understanding of PD pathophysiology and objectively assess therapies. In this proof of concept study, we developed a robotic system and a model-based approach to estimate viscous damping and stiffness of the elbow. Our methodology enables the subject to freely rotate the elbow using an admittance controller while torque perturbations tailored to identify the arm dynamics are delivered. The viscosity and stiffness are calculated based on the experimental data using least-squares estimation. We validated our technique using computer simulations and experiments with a nonhuman animal model of PD in the presence and absence of deep brain stimulation therapy. Our data show that stiffness and viscosity measurements can better differentiate rigidity changes than scores previously used for research, including the work and impulse scores, and the modified unified Parkinson's disease rating scale. Our estimation method is suitable for quantifying the effect of therapies on viscous damping and stiffness and studying the pathophysiological mechanisms underlying rigidity in PD.

Autoregressive Reconstruction of Total Water Storage within GRACE and GRACE Follow-On Gap Period

For 15 years, the Gravity Recovery and Climate Experiment (GRACE) mission have monitored total water storage (TWS) changes. The GRACE mission ended in October 2017, and 11 months later, the GRACE Follow-On (GRACE-FO) mission was launched in May 2018. Bridging the gap between both missions is essential to obtain continuous mass changes. To fill the gap, we propose a new approach based on a remove–restore technique combined with an autoregressive (AR) prediction. We first make use of the Global Land Data Assimilation System (GLDAS) hydrological model to remove climatology from GRACE/GRACE-FO data. Since the GLDAS mis-models real TWS changes for many regions around the world, we further use least-squares estimation (LSE) to remove remaining residual trends and annual and semi-annual oscillations. The missing 11 months of TWS values are then predicted forward and backward with an AR model. For the forward approach, we use the GRACE TWS values before the gap; for the backward approach, we use the GRACE-FO TWS values after the gap. The efficiency of forward–backward AR prediction is examined for the artificial gap of 11 months that we create in the GRACE TWS changes for the July 2008 to May 2009 period. We obtain average differences between predicted and observed GRACE values of at maximum 5 cm for 80% of areas, with the extreme values observed for the Amazon, Alaska, and South and Northern Asia. We demonstrate that forward–backward AR prediction is better than the standalone GLDAS hydrological model for more than 75% of continental areas. For the natural gap (July 2017–May 2018), the misclosures in backward–forward prediction estimated between forward- and backward-predicted values are equal to 10 cm. This represents an amount of 10–20% of the total TWS signal for 60% of areas. The regional analysis shows that the presented method is able to capture the occurrence of droughts or floods, but does not reflect their magnitudes. Results indicate that the presented remove–restore technique combined with AR prediction can be utilized to reliably predict TWS changes for regional analysis, but the removed climatology must be properly matched to the selected region.

Some properties and applications of half Cauchy extended exponential distribution

In this article, we have introduced a new probability distribution having three parameters using half Cauchy family of distribution named half Cauchy extended exponential distribution. The statistical properties and characteristics of the proposed distribution like the hazard rate function (HRF), cumulative hazard function, and the probability density function (PDF), and the cumulative distribution function (CDF), quantile function and the skewness, kurtosis are provided. The parameters of the proposed distribution are estimated using the Cramer-Von-Mises (CVM) least-square estimation (LSE), and maximum likelihood estimators (MLE) methods. A real data set is analyzed to test the goodness-of-fit of the proposed distribution. It is found that the half Cauchy extended exponential distribution performed well as compared to some competing distributions.

Asymptotic convergence of a distributed weighted least squares algorithm for networked systems with vector node variables

This paper studies the convergence properties of a recently proposed distributed algorithm for weighted least-squares (WLS) estimation in networked systems. This algorithm is suitable for large-scale networks with a vector parameter (variable) in each node of the network. By establishing the connection between this algorithm and the celebrated Gaussian Belief Propagation (BP) algorithm for statistical learning with scalar variables, asymptotic convergence of the algorithm is established under the assumption of generalised block diagonal dominance. This result generalises the known asymptotic convergence result of the Gaussian BP algorithm for networks with scalar variables. By extending the notion of diagonal dominance to block matrices, we are able to generalise the so-called walk-sum approach for convergence analysis of the Gaussian BP algorithm to this distributed WLS algorithm and show a similar asymptotic convergence property for networks with vector parameters. The significance of our work is that it gives theoretical guarantee for the distributed WLS algorithm for a new class of large-scale networked systems with vector parameters.

Synthetically trained convolutional neural networks for improved tensor estimation from free-breathing cardiac DTI.

Cardiac diffusion tensor imaging (cDTI) provides invaluable information about the state of myocardial microstructure. For further clinical dissemination, free-breathing acquisitions are desired, which however require image registration prior to tensor estimation. Due to the varying contrast and the intrinsically low signal-to-noise ratio (SNR), registration is very challenging and thus can introduce additional errors in the tensor estimation. In the work at hand it is hypothesized, that by incorporating spatial information and physiologically plausible priors into the fitting algorithm, the robustness of diffusion tensor estimation can be improved. To this end, we present a parameterized pipeline to generate synthetic data, that captures the statistics including spatial correlations of diffusion tensors and motion of the heart. The synthetic data is used to train a residual convolutional neural network (CNN) to estimate diffusion tensors from unregistered in-vivo cDTI data. Using in-silico data, the synthetically trained CNN is demonstrated to yield increased tensor estimation accuracy and precision when compared to conventional registration followed by least squares fitting. The network outputs fewer outliers especially at the myocardial borders. In-vivo feasibility using data from five healthy subjects demonstrates the utility of the synthetically trained network. The in-vivo results predicted by the synthetically trained CNN are found to be consistent with the registered least-squares estimates while showing fewer outliers and reduced noise. Even in low SNR regimes, the network results in robust tensor estimation, enabling scan time reduction by reduced-average acquisition in-vivo. Finally, to investigate the network's capability of discriminating between healthy and lesioned tissue, the in-vivo data was artificially augmented showing preserved classification of tissue states based on diffusion metrics.

Precision Motion Control of an Independent Metering Hydraulic System With Nonlinear Flow Modeling and Compensation

The independent metering system decouples the meter-in and meter-out flows of the chamber, giving extra control flexibility and can reducing energy consumption while keeping motion control accuracy. The control performance of the independent metering system is significantly determined by the valve characteristics.Valve nonlinearities, such as deadband, nonlinear flow gain, and leakage, may restrict the control accuracy. To address the issue and improve the system control precision, the nonlinearities of the valve need to be further modeled. In this article, a nonlinear valve flow model is proposed to accurately depict the relationships between the flow rate, pressure drop, and input voltage. The valve deadband is compensated by the static inverse function, while a certain polynomial projection function is used to mimic the nonlinear flow curve outside the deadband. To deal with uncertainties and nonlinearities in the independent metering system, an adaptive robust control approach is synthesized. The least-squares estimation law is adopted to online adapt the valve parameter and other system parameters. Comparative experiments were carried out to illustrate the effectiveness of the proposed approach. The results show that the valve flow nonlinearities are further captured by the proposed model, and the independent metering system performed a higher level control precision.

Performance Evaluation of Spectral Efficiency for Uplink and Downlink Multi-Cell Massive MIMO Systems

Massive multiple-input and multiple-output (MIMO) systems have become the most persuasive technology for 5G as it increased the energy efficiency gigantically as compared to other wireless communication systems. Being the most vibrant research technology in the communication sector, this research work is based on the optimal model development of energy-efficient massive MIMO systems. The proposed model is a realistic model that augmented the spectral efficiency (SE) of massive MIMO systems where a multi-cell model scenario is considered. Channel estimation is carried out at the base stations (BSs) based on uplink (UL) transmission while the minimum mean-squared error (MMSE), Element-wise MMSE, and Least-square (LS) estimators are used for the estimation. We analyze the achievable SE of the UL based on the MMSE channel estimator with different receive combining schemes. Moreover, the downlink (DL) transmission model is also modelled with different precoding schemes by taking the same vectors used in combining schemes. The simulation results show a significant improvement in spectral efficiency by developing UL and DL transmission models and also realized that the average sum of SE per cell can be improved by optimized MMSE channel estimation, installing multiple BS antennas, and serving multiple UEs per cell. The findings of this work specify that the massive MIMO system can be developed by optimizing the channel estimation for the augmentation of SE in UL and DL transmissions. Conclusively, it can be summarized that some complex computations of MMSE channel estimators can enhance the average sum of SE per cell as per the results verified in this model.

벌점화 회귀분석을 이용한 English Premier League의 승점 예측모형 개발 및 경기력 요인 탐색

Advancements in Information Communication Technology during the Fourth Industrial Revolution have led to rapid progress in soccer-related industries and research. Various countries have professional soccer leagues devoted to securing victory. These activities include studies that identify the performance factors that significantly influence the outcome of matches. Researchers often use linear regression analysis based on least-squares estimation for this process, but this method has several issues. As an alternative to this, penalized regression analysis that minimizes the penalty-pointed residual sum of squares has been developed as an alternative. However, there are few studies using the penalty point regression analysis to predict points in soccer. Therefore, in this study, theoretical contents were introduced to activate the penalty point regression analysis, and further examples of application were presented by empirically analyzing 2020-2021 EPL(EPL: The Football Association Premier League Limited) data. It was found to be a statistical predictive model. According to the results, the LASSO model and elastic net models were more suitable statistical prediction models from the perspective of the root mean square error (RMSE). Additionally, among the 19 variables used in the analysis, defensive factors such as tackles, clean sheets, and saves were identified as critical performance factors that impacted the score. Based on the results of this study, in order to enable more in-depth research in the future, the researchers hope that the EPL website will provide activity data, such as the total distance traveled, number of sprints, instantaneous speed, and distance between attackers and defenders.

Input layer regularization for magnetic resonance relaxometry biexponential parameter estimation.

Many methods have been developed for estimating the parameters of biexponential decay signals, which arise throughout magnetic resonance relaxometry (MRR) and the physical sciences. This is an intrinsically ill-posed problem so that estimates can depend strongly on noise and underlying parameter values. Regularization has proven to be a remarkably efficient procedure for providing more reliable solutions to ill-posed problems, while, more recently, neural networks have been used for parameter estimation. We re-address the problem of parameter estimation in biexponential models by introducing a novel form of neural network regularization which we call input layer regularization (ILR). Here, inputs to the neural network are composed of a biexponential decay signal augmented by signals constructed from parameters obtained from a regularized nonlinear least-squares estimate of the two decay time constants. We find that ILR results in a reduction in the error of time constant estimates on the order of 15%-50% or more, depending on the metric used and signal-to-noise level, with greater improvement seen for the time constant of the more rapidly decaying component. ILR is compatible with existing regularization techniques and should be applicable to a wide range of parameter estimation problems.

Correlation dispersion as a measure to better estimate uncertainty in remotely sensed glacier displacements

Abstract. In recent years a vast amount of glacier surface velocity data from satellite imagery has emerged based on correlation between repeat images. Thereby, much emphasis has been put on the fast processing of large data volumes and products with complete spatial coverage. The metadata of such measurements are often highly simplified when the measurement precision is lumped into a single number for the whole dataset, although the error budget of image matching is in reality neither isotropic nor constant over the whole velocity field. The spread of the correlation peak of individual image offset measurements is dependent on the image structure and the non-uniform flow of the ice and is used here to extract a proxy for measurement uncertainty. A quantification of estimation error or dispersion for each individual velocity measurement can be important for the inversion of, for instance, rheology, ice thickness and/or bedrock friction. Errors in the velocity data can propagate into derived results in a complex and exaggerating way, making the outcomes very sensitive to velocity noise and outliers. Here, we present a computationally fast method to estimate the matching precision of individual displacement measurements from repeat imaging data, focusing on satellite data. The approach is based upon Gaussian fitting directly on the correlation peak and is formulated as a linear least-squares estimation, making its implementation into current pipelines straightforward. The methodology is demonstrated for Sermeq Kujalleq (Jakobshavn Isbræ), Greenland, a glacier with regions of strong shear flow and with clearly oriented crevasses, and Malaspina Glacier, Alaska. Directionality within an image seems to be the dominant factor influencing the correlation dispersion. In our cases these are crevasses and moraine bands, while a relation to differential flow, such as shear, is less pronounced on the correlation spread.