Data Pre-processing System Research Articles

Short-term electricity load forecasting based on a novel data preprocessing system and data reconstruction strategy

Accurate forecasting of the electricity load plays a crucial role in the decision-making and operation of the smart grid. The characteristics of load series such as non-stationarity, non-linearity, and multiple-seasonality make such prediction a challenging task. In this study, a novel combined prediction model based on a feed-forward neural network (FNN) is established to address these challenges. A novel multi-step data preprocessing (DP) system is developed, which comprehensively eliminates the undesirable characteristics in electricity load data by identifying outliers, correcting outliers, and smoothing the data. Random forest is used to select high-impact parameters for electricity load to eliminate irrelevant variables. Moreover, a strategy combining seasonal-trend decomposition based on loess (STL) and Hodrick-Prescott (HP) filtering has been developed. STL is utilized to decompose the load series for exploring the inherent connection between electricity load trends and seasonal variations, while HP is employed to reconstruct the residual series, enabling a more in-depth exploration of the latent trend characteristics. The accuracy of the proposed framework was evaluated using publicly available datasets. The proposed DP-HSTL-FNN model achieved the lowest root mean square error (4.11 KW) and the coefficient of determination (R2) was closest to 1, demonstrating the highest predictive accuracy and reliable performance among all compared models. The results validate the significant advantage of the novel DP system in improving the quality of complex datasets, while confirming the potential of the data reconstruction strategy in enhancing prediction accuracy. Therefore, this novel model can be effectively applied in short-term electricity load forecasting for smart grids.

A novel hybrid wind speed prediction framework based on multi-strategy improved optimizer and new data pre-processing system with feedback mechanism

As a kind of renewable energy, wind energy has great potential for development and has been paid attention to by governments all over the world. However, due to the high uncertainty of wind speed, how to accurately predict wind speed and make use of wind energy has been recognized as a difficult problem. In order to solve this problem, a new hybrid wind speed prediction framework is proposed, which is composed of two subsystems, data preprocessing system and high-accuracy prediction system. In the system 1, the feedback mechanism is creatively added to the singular spectrum analysis (SSA) to find out the optimal decomposition-recombination strategy through the accuracy feedback. In the system 2, the unconstrained weighting mechanism is realized through the combination of combined neural network and multi-objective optimization algorithm to maximize the prediction accuracy on the premise of ensuring the stability of prediction. Besides, an improved meta-heuristic optimization algorithm based on cross-perturbation strategy (CP-JAYA) and its multi-objective form (MO-CPJAYA) are applied on two systems respectively to further improve the prediction ability of the framework. In 5 groups of experiments, the accuracy, advancement, generalization and sensitivity of the model are tested and compared with 13 other models. The proposed prediction framework has the best performance in all four sets of data. In 3 groups of discussions, we verify the advanced nature of CP-JAYA and MO-CPJAYA respectively through 13 single-objective test functions (CEC) and 4 multi-objective test functions (ZDT), and the speed advantage of the framework by recording the CPU running time.

An integrated system for the management of environmental data to support veterinary epidemiology.

Environmental and climatic fluctuations can greatly influence the dynamics of infectious diseases of veterinary concern, or interfere with the implementation of relevant control measures. Including environmental and climatic aspects in epidemiological studies could provide policy makers with new insights to assign resources for measures to prevent or limit the spread of animal diseases, particularly those with zoonotic potential. The ever-increasing number of technologies and tools permits acquiring environmental data from various sources, including ground-based sensors and Satellite Earth Observation (SEO). However, the high heterogeneity of these datasets often requires at least some basic GIS (Geographic Information Systems) and/or coding skills to use them in further analysis. Therefore, the high availability of data does not always correspond to widespread use for research purposes. The development of an integrated data pre-processing system makes it possible to obtain information that could be easily and directly used in subsequent epidemiological analyses, supporting both research activities and the management of disease outbreaks. Indeed, such an approach allows for the reduction of the time spent on searching, downloading, processing and validating environmental data, thereby optimizing available resources and reducing any possible errors directly related to data collection. Although multitudes of free services that allow obtaining SEO data exist nowadays (either raw or pre-processed through a specific coding language), the availability and quality of information can be sub-optimal when dealing with very small scale and local data. In fact, some information sets (e.g., air temperature, rainfall), usually derived from ground-based sensors (e.g., agro-meteo station), are managed, processed and redistributed by agencies operating on a local scale which are often not directly accessible by the most common free SEO services (e.g., Google Earth Engine). The EVE (Environmental data for Veterinary Epidemiology) system has been developed to acquire, pre-process and archive a set of environmental information at various scales, in order to facilitate and speed up access by epidemiologists, researchers and decision-makers, also accounting for the integration of SEO information with locally sensed data.

Application of Hybrid Support Vector Machine model for Streamflow Prediction in Barak valley, India

Forecasting streamflow (Qflow) is vital in flood and water management, determining potential of river water flow, agricultural practices, hydropower generation, and environmental flow study. This research aims to explore capability of hybrid support vector machines (SVM) with Whale Optimisation Algorithm (WOA) model for forecasting streamflow at Badarpur Ghat gauging station of Barak river basin and evaluate its enactment with the conventional SVM model. Root mean squared error (RMSE), mean absolute error (MAE), and Nash-Sutcliffe efficiency (NSE) statistical measures are considered as evaluating standards. Assessment of outcomes indicates that the optimization algorithm could enhance the accurateness of standalone SVM model in monthly streamflow forecasting. Compared to conventional artificial intelligence methods without a data pre-processing system, the comparatively good performance of applied hybrid model gives an effective alternate to achieve better precision in streamflow forecasting. Results confirm that enhanced SVM model can better process a multifaceted hydrogeological data set, have higher prediction accuracy, and possess better generalisation capability.

A hybrid air quality early-warning framework: An hourly forecasting model with online sequential extreme learning machines and empirical mode decomposition algorithms

Modelling air quality with a practical tool that produces real-time forecasts to mitigate risk to public health continues to face significant challenges considering the chaotic, non-linear and high dimensional nature of air quality predictor variables. The novelty of this research is to propose a hybrid early-warning artificial intelligence (AI) framework that can emulate hourly air quality variables (i.e., Particulate Matter 2.5, PM2.5; Particulate Matter 10, PM10 and lower atmospheric visibility, VIS), the atmospheric variables associated with increased respiratory induced mortality and recurrent health-care cost. Firstly, hourly air quality data series (January-2015 to December-2017) are demarcated into their respective intrinsic mode functions (IMFs) and a residual sub-series that reveal patterns and resolve data complexity characteristics, followed by partial autocorrelation function applied to each IMF and residual sub-series to unveil historical changes in air quality. To design the prescribed hybrid model, the data is partitioned into training (70%), validation (15%) and testing (15%) sub-sets. The online sequential-extreme learning machine (OS-ELM) algorithm integrated with improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) is designed as a data pre-processing system to robustly extract predictive patterns and fine-tune the model generalization to a near-optimal global solution, which represents modelled air quality at hourly forecast horizons. The resulting early warning AI-based framework denoted as ICEEMDAN-OS-ELM model, is individually constructed by forecasting each IMF and residual sub-series, with hourly PM2.5, PM10, and VIS obtained by the aggregated sum of forecasted IMFs and residual sub-series. The results are benchmarked with many competing predictive approaches; e.g., hybrid ICEEMDAN-multiple-linear regression (MLR), ICEEMDAN-M5 model tree and standalone versions: OS-ELM, MLR, M5 model tree. Statistical metrics including the root-mean-square error (RMSE), mean absolute error (MAE), Willmott's Index (WI), Legates & McCabe's Index (ELM) and Nash–Sutcliffe coefficients (ENS) are used to evaluate the model's accuracy. Both visual and statistical results show that the proposed ICEEMDAN-OS-ELM model registers superior results, outperforming alternative comparison approaches. For instance, for PM2.5,ELM values ranged from 0.65–0.82 vs. 0.59–0.77 for ICEEMDAN-M5 tree, 0.59–0.74 for ICEEMDAN-MLR, 0.28–0.54 for OS-ELM, 0.27–0.54 for M5 tree and 0.25–0.53 for the MLR model. For remaining air quality variables (i.e., PM10 & VIS), the objective model (ICEEMDAN-OS-ELM) outperformed the comparative models. In particular, ICEEMDAN-OS-ELM registered relatively low RMSE/MAE, ranging from approximately 0.7–1.03 μg/m3(MAE), 1.01–1.47 μg/m3(RMSE) for PM2.5 whereas for PM10, these metrics registered a value of 1.29–3.84 μg/m3(MAE), 3.01–7.04 μg/m3(RMSE) and for Visibility, they were 0.01–3.72 μg/m3 (MAE (Mm−1)), 0.04–5.98 μg/m3 (RMSE (Mm−1)). Visual analysis of forecasted and observed air quality through a Taylor diagram illustrates the objective model's preciseness, confirming the versatility of early warning AI-model in generating air quality forecasts. The excellent performance ascertains the hybrid model's potential utility for air quality monitoring and subsequent public health risk mitigation.

Application of Self-Organizing Neural Networks to Electrical Fault Classification in Induction Motors

Electrical winding faults, namely stator short-circuits and rotor bar damage, in total constitute around 50% of all faults of induction motors (IMs) applied in variable speed drives (VSD). In particular, the short circuits of stator windings are recognized as one of the most difficult failures to detect because their detection makes sense only at the initial stage of the damage. Well-known symptoms of stator and rotor winding failures can be visible in the stator current spectra; however, the detection and classification of motor windings faults usually require the knowledge of human experts. Nowadays, artificial intelligence methods are also used in fault recognition. This paper presents the results of experimental research on the application of the stator current symptoms of the converter-fed induction motor drive to electrical fault detection and classification using Kohonen neural networks. The experimental tests of a diagnostic setup based on a virtual measurement and data pre-processing system, designed in LabView, are described. It has been shown that the developed neural detectors and classifiers based on self-organizing Kohonen maps, trained with the instantaneous symmetrical components of the stator current spectra (ISCA), enable automatic distinguishing between the stator and rotor winding faults for supplying various voltage frequencies and load torque values.

A Non-stationary Analysis Using Ensemble Empirical Mode Decomposition to Detect Anomalies in Building Energy Consumption

Commercial buildings’ consumption is driven by multiple factors that include occupancy, system and equipment efficiency, thermal heat transfer, equipment plug loads, maintenance and operational procedures, and outdoor and indoor temperatures. A modern building energy system can be viewed as a complex dynamical system that is interconnected and influenced by external and internal factors. Modern large scale sensor measures some physical signals to monitor real-time system behaviors. Such data has the potentials to detect anomalies, identify consumption patterns, and analyze peak loads. The paper proposes a novel method to detect hidden anomalies in commercial building energy consumption system. The framework is based on Hilbert-Huang transform and instantaneous frequency analysis. The objectives are to develop an automated data pre-processing system that can detect anomalies and provide solutions with real-time consumption database using Ensemble Empirical Mode Decomposition(EEMD) method. The finding of this paper will also include the comparisons of Empirical mode decomposition and Ensemble empirical mode decomposition of three important type of institutional buildings.

천리안위성 기하보정 시스템의 궤도상 시험

천리안 위성의 궤도상 시험이 위성발사 직후인 2010년 7월부터 2011년 1월까지 약 7개월에 걸쳐 성공적으로 수행되었으며, 과학탑재체인 기상 및 해양탑재체 자료는 2011년 4월부터 공식적으로 배포되고 있다. 이 논문에서는 천리안위성의 궤도상 시험 중 핵심적인 요소인 탑재체 자료의 보정 과정 중 최종 단계인 기하보정의 궤도상 테스트 내용에 중점을 두고 있다. 천리안위성 기하보정 알고리즘의 구조와 내용을 간략하게 소개하였으며, 위성발사 후 행해진 기하보정시스템의 기능 및 검증 테스트의 단계별 과정들과 그 결과를 분석 정리하였다. 실시간 위성자료 전처리 시스템을 통해 확보한 자료들을 이용한 천리안 기하보정알고리즘의 최종 성능테스트는 요구 조건을 만족하는 우수한 정확도를 보여 주었다. COMS In-Orbit Tests(IOT), performed from July, 2010 to Jan, 2011, were successfully completed and the scientific data from MI and GOCI has been distributed officially from April, 2011. This paper focuses on the geometric calibration system tests conducted during the IOT. The geometric calibration process, which is one of the primary objectives of the IOT is the final step of COMS data pre-processing. The basic principles of the geometric calibration (or image navigation and registration, INR) algorithm for COMS are described and the functional and performance tests of COMS INR system were summarized according to the COMS IOT phases. Final performance testes were carried out using data sets acquired from the real-time COMS data pre-processing system. Geometric calibration accuracy of the COMS data showed excellent quality and met requirement specifications.