Bad Data Analysis Research Articles

Reversing the trend of increasing surface analysis errors

Complex and unfamiliar techniques paired with funding and publishing incentives create an environment of bad data analysis that will take large-scale cooperation to fix.

Bad data analysis and detection using PMU with UPQC integration to grid during fault conditions

<span lang="EN-US">Analysis of bad data in an IEEE 14 bus system with <a name="_Hlk121211016"></a>phasor measuring units (PMU) devices is carried out in this paper. The normal operating condition data achieved from the PMU in the bus system is compared to the data achieved during faults in the bus system. A principal component analysis (PCA) technique is proposed in this paper for distinguishing the difference between data transmission during normal and fault conditions. The PCA approach detects the dynamical magnitudes of the measurements taken and also determines the noise caused by the disturbances. The grid system is updated with unified power quality conditioner (UPQC) improvising the parameters of the system to mitigate the fault. The PMU devices use PCA technique for a comparative analysis of the measured components analyzing the performance of the system under different operating conditions. The simulation of these modules is carried out in Simulink environment of MATLAB software with PCA done concerning time.</span>

Examining the role of virtue ethics and big data in enhancing viable, sustainable, and digital supply chain performance

In the real world of practice, data-driven supply chains have gained huge popularity in recent years. This has led operations and supply management researchers to the focus on the role of advanced information and technology, including big data. Literature highlights that the use of big data can enhance business performance. Nonetheless, big data is analyzed by humans and a lack of virtue ethics could lead to disastrous consequences (erroneous decision-making can stem from bad data analysis resulting in not only huge business losses but also deterioration of relationships with suppliers and customers in the supply chain). To address the calls of previous researchers, this study utilizes the Ethical Theory of Organizing framework and Stakeholder theory to develop the theoretical model and further examine the relationships. The samples are drawn from the manufacturing industry. Hypothesis testing is executed through covariance-based structural equation modeling and finally, the conclusions are drawn. The findings of this work provide a more nuanced understanding of virtue ethics and big data implications, thereby answering the important questions of “why” and “how” data-driven green and lean practices increase the stakeholders' trust and enhance viable, sustainable, and digital supply chain performance.

A robust dynamic line rating monitoring system through state estimation and bad data analysis

This paper introduces an approach to increase robustness and safety of a weather and tension-based dynamic line rating system through the use of state estimation and bad data analysis algorithms. Bad data are identified through a procedure based on the evaluation of normalized Lagrange multipliers and collinearity tests. Numerical results highlight the effectiveness of the state estimation and gross error verification algorithms as well as the impact of bad data on dynamic line rating assessments.

ANALYSIS OF BAD DATA IN POWER SYSTEM STATE ESTIMATION UNDER NON-GAUSSIAN MEASUREMENT NOISE

In practice, aging power system measurement equipment may deviate the noise statistics of errors associated with measurements provided by a supervisory control and data acquisition system from a Gaussian distribution. Furthermore, recent research conducted by the Pacific Northwest National Laboratory in the United States reveals that errors associated with phasor measurement units do not obey a Gaussian probability distribution. Under these conditions, the estimated state of an electric power system may deviate from the actual operation state because of the generally adopted assumption that measurements’ errors follow a Gaussian probability distribution. Hence, the question of what effect non-Gaussian data have on the outcome of state estimators and of bad data analyses used in power system control centers must be investigated. This paper addresses this question by providing a comprehensive analysis of the quality of estimated states using both the weighted least squares method and the largest normalized residual test, which are the ones currently used in todays' control centers, when the error distribution of measurements is not truly Gaussian. In this context, this paper also aims to provide a thorough discussion of the suitability of applying a recently proposed bad data analysis, which is based on the largest normalized estimated residual test (LNERT), for detecting, identifying and correcting or eliminating multiple measurements with gross errors under Gaussian and non-Gaussian noises. The effectiveness of the proposed LNERT-based bad data analysis is extensively compared with respect to other proposals for bad data identification by means of state estimation studies performed in the IEEE 14-bus and 118-bus test systems as well as on the very large 7659-bus representation of the Mexican interconnected power system.

Non-Technical Loss Identification by Using Data Analytics and Customer Smart Meters

The smart meters installed at the customer premises are one of the main apparatuses promoting the modernization of the distribution systems. These devices collect a huge amount of data, demanding the development of analytic techniques to transform these data into useful information. In addition, by proposing new applications to data supplied by smart meters, more value is added to this equipment, allowing higher return on the associated investments. Utilities can have a business case by increasing their operational efficiency if smart meters are used, for instance, to identify non-technical losses, which represent an important cause of revenue losses. This paper presents a new data analytic technique for detection and location of non-technical losses caused by illegal connections of loads to distribution systems in the presence of smart meters. The data analytic technique relies on bad data analysis, similar to the ones used in state estimation methods, developed specifically for this application. A real 34-bus low voltage system is used to illustrate the main concepts of the proposed algorithm. Systematic tests are also conducted on a real 1682-bus distribution system to evaluate the method performance considering electricity theft caused by medium and low voltage customers.

A new unified approach for the state estimation and bad data analysis of electric power transmission systems with multi-terminal VSC-based HVDC networks

The aim of this paper is the proposal of a static state estimation approach suitable for electric power transmission systems containing fully controlled multi-terminal high voltage direct current networks. An integrated electric circuit model of the voltage source converter station, including its on-load tap-changer (OLTC) transformer, is used to determine the state variables associated with the AC–DC interface. These controllable variables are added to the state vector, together with the nodal voltages associated with the AC and DC networks, as well as the duty cycle of the DC–DC converters, for a unified SE solution in a single frame of reference. The SE problem is formulated based on Hachtel’s augmented matrix method in order to directly consider the operational constraints associated with the AC–DC network. In addition, a new technique for performing multiple bad data analysis is proposed in which the original level of measurement redundancy is maintained unaltered during the estimation process. The applicability of the proposed SE approach and bad data analysis is demonstrated by estimating the operating state of two integrated AC–DC multi-terminal transmission networks. The numerical results show that the proposal retains good convergence properties in terms of the number of iterations required to achieve a reliable state estimation as well as robustness to deal with bad data.

Parallel Domain Decomposition Based Distributed State Estimation for Large-scale Power Systems

Growing system sizes and complexity, along with the large amount of data provided by phasor measurement units (PMUs), are the drivers to accurate state estimation algorithms for online monitoring and operation of power systems. In this paper, a distributed weighted-least-square state estimation method using an additive Schwarz domain decomposition technique is proposed to reduce the computational execution time. The proposed approach divides a data set into several subsets to be processed in parallel using a multiprocessor architecture considering data exchange among distributed areas. The slow coherency method and balanced partitioning are utilized to reduce the communication overhead and increase accuracy. Moreover, bad data analysis is also investigated in a distributed manner. The performance of the proposed distributed state estimator, along with the speed-up for several test systems, was compared with the traditional centralized state estimator. The simulation results show a speed-up of 6.5 for a 4992-bus system.

A Distributed Multiarea State Estimation

This paper presents a new distributed state estimation method for multiarea power systems. Each area performs its own state estimation, using local measurements, and exchanges border information (estimated boundary states and measurements) at a coordination state estimator, which computes the system-wide state. Furthermore, observability and bad data analysis are accomplished in a distributed manner. The proposed method is illustrated with the IEEE 14-bus system. Test results with the IEEE 118-bus system are given.

Bad data analysis in power system measurement estimation using complex artificial neural network based on the extended complex Kalman filter

AbstractThis paper proposes a method for bad data analysis in power system measurement estimation using complex artificial neural network (CANN) based on the extended complex Kalman filter (ECKF). The proposed algorithm is better in noise immunity since the link weighting in the CANN can be automatically adjusted with trained data through the ECKF. Moreover, the CANN is quite suitable for complex training data such as complex power in a power system since its input and output performs a nonlinear mapping. Four systems including a 6‐bus system, the IEEE 30‐bus system, IEEE 118‐bus system, and a practical system are used as examples to verify the feasibility of the ECKF‐CANN approach. Results show the proposed algorithm has increased sensitivity in identifying gross measurement errors with respect to the standard ANN. Copyright © 2009 John Wiley & Sons, Ltd.

Bad data pre-filter for state estimation

Systematic approach for pre-filter for state estimation based on wavelet analysis to detect and eliminate bad data, is developed in this work. Unbiased (random/Gaussian) bad data such as, transient meter failures, transient meter malfunction, and measurements captured during system transients, are inherently in the form of large abrupt change of short duration in a measurement-sequence. These should be detected in pre-filtering stage because their presence poses an extra burden on post-SE bad data analysis. The test results of the proposed pre-filter on two test systems establish that there is a significant reduction in the number of iterations required for bad data detection and elimination.

Generalized state estimation

Power system state estimation derives a real-time network model by extracting information from a redundant data set consisting of telemetered, predicted and static data items. This paper describes a generalized, fully developed, estimation approach that fundamentally improves the information extraction process. Its main contribution is the successful inclusion of topology and parameters in the estimation and bad data analysis processes. This is valuable both in the initial commissioning of a state estimator, and in its routine real-time and study mode application. The approach involves a variety of novel concepts and methods. It is usable in weighted least squares (WLS) and other estimation approaches.

Observability and bad data analysis using augmented blocked matrices (power system analysis computing)

Observability and bad data analysis in the context of the recently developed blocked sparse approach are discussed for power system analysis. The factorization-based observability analysis is extended to this method. Important statistical derivations involving the blocked augmented matrices are presented. The computational aspects of performing bad data analysis as well as guidelines for computing residual variances and sensitivity matrices are discussed. An efficient implementation of a bad data analysis algorithm based on normalized residuals and nonlinear residual computations is described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A linear recursive bad data identification method with real-time application to power system state estimation

A fast and efficient algorithm, the recursive measurement error estimation identification (RMEEI) method, for bad data (BD) analysis is further developed. By using a set of linear recursive formulas, state variables, residuals, and their variances are updated after the removal of a measurement from a suspected data set to the remaining data set, or in the reverse direction. Neither a re-estimation nor a residual sensitivity matrix are needed in the identification process, which increases the computational speed greatly. Digital tests have been done to compare the RMEEI method with other conventional BD identification methods in terms of identification performance and computational speed. The real-time operational experience of the RMEEI method in the northeast China power system is outlined. >

A recursive measurement error estimation identification method for bad data analysis in power system state estimation

A recursive measurement error estimation identification algorithm is proposed for identifying multiple interacting bad data in power system static state estimation. A set of linearized formulae are developed and used to recursively calculate normalized residuals and normalized measurement error estimates upon which the bad data identification method is based. Sparse vector and partial factor modification techniques are used in the recursive identification calculations. Neither the submatrix of the residual sensitivity matrix, W/sub ss/, nor state reestimation is needed in the whole identification process. Digital tests on various power systems, including a 171 bus real system, are done to show the validity and efficiency of the proposed bad data identification method. >

Implementation of the hypothesis testing identification in power system state estimation

The authors consider the online implementation of a general, reliable and efficient bad-data analysis procedure for power system state estimation. It is based on hypothesis testing identification, which was previously proposed and subsequently improved by the authors. The procedure involves a sequential measurement error estimator along with adequate sparsity programming techniques. Both make the procedure easy to implement on any state estimator. A criterion for multiple noninteracting bad-data identification is also proposed, which is applicable to any bad-data analysis method. Simulations are reported on systems of up to 700 buses. A thorough comparison with classical methods is also included. >

Power system observability and related functions Derivation of appropriate strategies and algorithms

Observability analysis deals with the adequacy of the available measurement configuration for state estimation purposes. This paper considers methods to explore observability and to face unobservable situations. More explicity, first, the problem of unobservable networks is examined; strategies currently used are discussed and a more general method is proposed, to build a complete and reliable database. Second, topological concepts derived from observability are used to enhance the reliability of the bad data analysis under stringent identification conditions. Third, a concrete algorithm for observability analysis is proposed.

Hypothesis Testing Identification: A New Method for Bad Data Analysis in Power System State Estimation

Hypothesis Testing Identification: A New Method for Bad Data Analysis in Power System State Estimation

A Two-Level State Estimator in Electric Power Systems with Bad Data Analysis

We present a hierarchical state estimator appropriate for the monitoring of large-scale electric power systems. Two interesting in practice organization schemes are considered and discussed. The problem of bad data analysis is also investigated; it is shown that the hierarchical approach contributes making this analysis easy, fast and efficient.

An Efficient Algorithm for Power System State Estimation

This paper reports a new and efficient algorithm for state estimation in power systems. The algorithm makes use of ‘Data Equations’ and Householder orthogonal transformations and is well suited for efficient batch processing with one or more measurements in a batch. Another attractive feature of the present algorithm is its efficiency in dealing with bad data. It evaluates automatically the unbiased estimates of the standard deviations of the measurement errors. This feature is exploited in proposing a Detection, Estimation and Correction (DEC) scheme to deal with bad data. The excellent numerical properties and the efficient bad data analysis capabilities, along with batch processing ability of the present algorithm are demonstrated by simulation results on an IEEE 30-bus test system.