Accurate Wind Power Forecasting Research Articles

Short-to-Medium-Term Wind Power Forecasting through Enhanced Transformer and Improved EMD Integration

Accurate wind power forecasting (WPF) is critical in optimizing grid operations and efficiently managing wind energy resources. Challenges arise from the inherent volatility and non-stationarity of wind data, particularly in short-to-medium-term WPF, which extends to longer forecast horizons. To address these challenges, this study introduces a novel model that integrates Improved Empirical Mode Decomposition (IEMD) with an enhanced Transformer called TransIEMD. TransIEMD begins by decomposing the wind speed into Intrinsic Mode Functions (IMFs) using IEMD, transforming the scalar wind speed into a vector form that enriches the input data to reveal hidden temporal dynamics. Each IMF is then processed with channel attention, embedding, and positional encoding to prepare inputs for an enhanced Transformer. The Direct Embedding Module (DEM) provides an alternative viewpoint on the input data. The distinctive perspectives of IEMD and DEM offer interaction through cross-attention within the encoder, significantly enhancing the ability to capture dynamic wind patterns. By combining cross-attention and self-attention within the encoder–decoder structure, TransIEMD demonstrates enhanced proficiency in detecting and leveraging long-range dependencies and dynamic wind patterns, improving the forecasting precision. Extensive evaluations on a publicly available dataset from the National Renewable Energy Laboratory (NREL) demonstrate that TransIEMD significantly improves the forecasting accuracy across multiple horizons of 4, 8, 16, and 24 h. Specifically, at the 24 h forecast horizon, TransIEMD achieves reductions in the normalized mean absolute error and root mean square error of 4.24% and 4.37%, respectively, compared to the traditional Transformer. These results confirm the efficacy of integrating IEMD with attention mechanisms to enhance the accuracy of WPF.

A novel hybrid model based on Empirical Mode Decomposition and Echo State Network for wind power forecasting

Accurate wind power forecasting is essential for (i) the management of wind energy, (ii) increasing the integration of generated power into the electrical grid, and (iii) enhancing maintenance efficiency. This paper proposes a novel hybrid model for short-term (1-hour ahead) wind power forecasting. The model integrates the echo-state network (ESN) architecture with empirical mode decomposition (EMD) to improve forecast accuracy. Unlike existing approaches that use separate models for each decomposed signal after EMD method, proposed model uses a single ESN structure to predict wind power by incorporating all decomposed signals and their past values. The main advantage of this architecture is that it eliminates the need to train multiple models, thereby streamlining the forecasting process. To evaluate the performance of the proposed model, one year of data from the West of Duddon Sands, Barrow, and Horns Power offshore wind farms is utilized. Firstly, the classical ESN model and the EMD-ESN hybrid model are compared for the three datasets. Then, a comprehensive evaluation is performed by comparing the results of the proposed model with commonly used standalone and hybrid forecasting models such as Multi-Layer Perceptron (MLP), Adaptive-Neuro Fuzzy Inference System (ANFIS), Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), EMD-LSTM, EMD-BiLSTM, Variational Mode Decomposition based ESN (VMD-ESN), and Wavelet Decomposition based ESN (WD-ESN). The results show that the EMD-ESN hybrid model outperforms implemented models in predicting wind power in all three datasets. Furthermore, the proposed EMD-ESN model for an onshore wind turbine power data in Germany is compared with the SOTA models, such as Transformer, Informer, Autoformer, and Graph Patch Informer (GPI), in the literature. This study highlights the superior predictive capabilities of proposed model, making it a valuable tool for enhancing the accuracy of wind power forecasts, thereby contributing to the reliable integration of wind energy into the power grid.

Wind power forecasting technologies: A review

This study addresses the critical role of wind power forecasting in ensuring stable and reliable power system operations. Wind Power Forecasting is critical for the efficient operation of plant, time scheduling, and it’s balancing of power generation with grid integration systems. Due to its dependency on dynamic climatic conditions and associated factors, accurate wind power forecasting is challenging. The research delves into various aspects, including input data, input selection techniques, data pre-processing, and forecasting methods, with the aim of motivating researchers to design highly efficient online/offline models on weather-based data. The overarching goal is to enhance the reliability and stability of power systems while optimizing energy resource utilization. The analysis reveals that hybrid models offer more accurate results, highlighting their significance in the current era. This study investigates different Wind Power Forecasting (WPF) models from existing literature, focusing on input variables, time horizons, climatic conditions, pre-processing techniques, and sample sizes that affect model accuracy. It covers statistical models like ARMA and ARIMA, along with AI techniques including Deep Learning (DL), Machine Learning (ML), and neural networks, to estimate wind power.

Novel wind power ensemble forecasting system based on mixed-frequency modeling and interpretable base model selection strategy

Accurate wind power forecasting helps to maximize the utilization of wind energy resources, enhance wind power generation efficiency, and optimize grid operation. This study proposes an innovative mixed-frequency modeling and interpretable base model selection-based ensemble wind power forecasting system. Specifically, the data preprocessing module preprocesses wind speed and wind power data at different frequencies. The mixed-frequency modeling module then constructs 12 mixed-frequency and machine learning models to predict wind power, with a comprehensive evaluation metric to determine their optimal lags. Subsequently, the base model selection module effectively combines the elastic net and Shapley additive explanation methods to identify individual models that contribute significantly to the prediction target as base models. Finally, the ensemble module integrates the optimization algorithms with a machine learning model to ensemble the selected base models. The key findings are as follows: (1) mixed-frequency wind speed and wind power data effectively improve forecasting performance, and (2) the proposed base model selection strategy greatly enhances the accuracy and interpretability of the modeling process. This model could robustly predict two datasets from Inner Mongolian wind farms, with average absolute percentage errors of 2.4505% and 4.8270%, respectively, establishing this as a useful technique for wind power prediction.

Spatial correlation learning based on graph neural network for medium-term wind power forecasting

With the increasing penetration of wind power in power grid, accurate and reliable wind power forecasting is of great significance for the economic operation and safe dispatching of electrical power system. In practice, there exists complex spatial correlation between wind power variables, which brings great challenges to the accurate forecasting of wind power. However, traditional deep learning-based methods mostly focus on temporal feature while ignoring the spatial correlation between wind power variables, leading to low forecasting accuracy. To explore the spatial correlation among wind power variables and extract temporal features simultaneously, we propose a double attention-based spatial–temporal neural network (DA-STNet). First, graph attention network is employed to explore the spatial correlation between wind power variables based on a graph constructed by maximal information coefficient, which can consider the combined influences of multivariate on the wind power output. Then, by incorporating causal reasoning and data-driven element-wise attention measures, a novel temporal attention layer is proposed to extract the temporal feature of wind power sequences. Comprehensive experiments were conducted on one self-collected and one public dataset with three different multi-steps ahead forecasting tasks, and the experimental results demonstrated that the performance of proposed DA-STNet is superior to the existing methods on both real-world datasets. In the 24 h ahead experiment on the NWWPF dataset, the MSE of our model can reach as low as 0.136 and MAE can be decreased to 0.275.

Hybrid LSTM-Based Fractional-Order Neural Network for Jeju Island’s Wind Farm Power Forecasting

Efficient integration of wind energy requires accurate wind power forecasting. This prediction is critical in optimising grid operation, energy trading, and effectively harnessing renewable resources. However, the wind’s complex and variable nature poses considerable challenges to achieving accurate forecasts. In this context, the accuracy of wind parameter forecasts, including wind speed and direction, is essential to enhancing the precision of wind power predictions. The presence of missing data in these parameters further complicates the forecasting process. These missing values could result from sensor malfunctions, communication issues, or other technical constraints. Addressing this issue is essential to ensuring the reliability of wind power predictions and the stability of the power grid. This paper proposes a long short-term memory (LSTM) model to forecast missing wind speed and direction data to tackle these issues. A fractional-order neural network (FONN) with a fractional arctan activation function is also developed to enhance generated wind power prediction. The predictive efficacy of the FONN model is demonstrated through two comprehensive case studies. In the first case, wind direction and forecast wind speed data are used, while in the second case, wind speed and forecast wind direction data are used for predicting power. The proposed hybrid neural network model improves wind power forecasting accuracy and addresses data gaps. The model’s performance is measured using mean errors and R2 values.

Wind power forecasting: A transfer learning approach incorporating temporal convolution and adversarial training

Accurate forecasting of wind power is crucial for efficiently scheduling power grids that incorporate wind energy. However, distribution instability of time-varying data can undermine the generalization capability of forecasting models in future periods. This paper introduces a transfer learning approach incorporating adversarial training and temporal convolution to address wind power forecasting from the distribution-centric perspective. The proposed approach can be divided into two joint modules: the temporal domain split and domain-adversarial temporal convolutional network. The former segregates training data into temporal domains characterized by the most significant distribution differences, while the latter aims is designed to learn shared knowledge from the segmented temporal domains. The shared knowledge is independent of distribution shifts, which can be generalized to future testing well. Specifically, the domain-adversarial temporal convolutional network is composed of a base model incorporating temporal convolutional network and a domain classifier. Both components are jointly optimized by minimizing power prediction loss and maximizing temporal domain classification loss. The data from actual wind turbines are employed to validate the proposed approach. Experimental results show that the superior performance of the proposed approach compared to benchmark models, achieving a remarkable 24.56% improvement in accuracy by introducing adversarial training.

An ultra-short-term wind power prediction method based on spatial-temporal attention graph convolutional model

Accurate and robust wind power forecasting (WPF) is great significance to ensure the safe and stable operation of the power system and promote the transformation of low-carbon energy. However, the high randomness and intermittency of wind power bring great challenges when designing reliable forecasting models. In this paper, a novel spatial-temporal attention graph convolutional network model is proposed. Firstly, the spatial attention mechanism is used to aggregate and extract the spatial correlations of the raw wind power data. Secondly, the temporal attention mechanism is applied to capture the temporal correlations. Then, the extracted spatial-temporal correlations were put into the temporal convolution network and the spatial convolution network to further obtain the temporal and spatial dependencies. Finally, the wind power forecasting results is output through the full connection layer. The proposed method is verified by using wind power data from real wind farm in China. The experimental results reveal that the proposed depth spatiotemporal prediction model has more significant advantages than other advanced models in terms of prediction accuracy and stability.

Unleashing the economic potential of wind power for ancillary services

Abstract Ancillary services play a significant role in ensuring stable operation of power systems in the growing penetration of renewable energy. Some of the important functions for ancillary support include voltage support, frequency support, and system restoration. This work highlights the possibility of relevant ancillary support from wind power and investigates how such services will impact from an economic point of view. A crucial aspect of ancillary services from wind power relates to tailored and dynamic information about the conditional power production from the turbines and wind parks which could be obtained through accurate wind power forecasting. The study of the impact of forecast accuracy and correlation in errors on the possibility for wind power to provide ancillary services is analyzed. Furthermore, the limitations and challenges associated with the technical capabilities of wind turbines to support such services are reviewed and presented. An initial qualitative assessment of the value of providing such services is carried out and the capabilities combined with electricity market data from Nord Pool taking an example of the western region of Denmark is discussed. The result of this work shows that enhancing ancillary services from wind power combined with more accurate wind forecasting has a positive financial impact on wind power market.

Research on a Deep Ensemble Learning Model for the Ultra-Short-Term Probabilistic Prediction of Wind Power

An accurate method for predicting wind power is crucial in effectively mitigating wind energy fluctuations and ensuring a stable power supply. Nevertheless, the inadequacy of the stability of wind energy severely hampers the consistent functioning of the power grid and the reliable provision of electricity. To enhance the accuracy of wind power forecasting, this paper proposes an ensemble model named the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and convolutional bidirectional long short-term memory (CNN-BiLSTM), which incorporates a data preprocessing technique, feature selection method, deep ensemble model, and adaptive control. Initially, CEEMDAN is utilized to decompose wind speed and power sequences and hence obtain decomposed subsequences for further analysis. Subsequently, the CNN is used to extract features from each subsequence, whereas each subsequence is processed by BiLSTM to obtain an ultra-short-term deterministic prediction model. Additionally, the adaptive kernel density estimation (AKDE) method is employed to estimate the probabilistic distribution of prediction error, enabling ultra-short-term probabilistic wind power prediction. Finally, based on real datasets, the reliability of the model in probabilistic prediction is verified through the evaluation metrics of multi-step prediction intervals (PIs).

Short-term wind power forecasting based on dual attention mechanism and gated recurrent unit neural network

Accurate wind power forecasting is essential for both optimal grid scheduling and the massive absorption of wind power into the grid. However, the continuous changes in the contribution of various meteorological features to the forecasting of wind power output under different time or weather conditions, and the overlapping of wind power sequence cycles, make forecasting challenging. To address these problems, a short-term wind power forecasting model is established that integrates a gated recurrent unit (GRU) network with a dual attention mechanism (DAM). To compute the contributions of different features in real time, historical wind power data and meteorological information are first extracted using a feature attention mechanism (FAM). The feature sequences collected by the FAM are then used by the GRU network for preliminary forecasting. Subsequently, one-dimensional convolution employing several distinct convolution kernels is used to filter the GRU outputs. In addition, a multi-head time attention mechanism (MHTAM) is proposed and a Gaussian bias is introduced to assign different weights to different time steps of each modality. The final forecast results are produced by combining the outputs of the MHTAM. The results of the simulation experiment show that for 5-h, 10-h, and 20-h short-term wind power forecasting, the established DAM-GRU model performs better than comparative models on the basis of Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-squared (R2), Square sum error (SSE), Mean absolute percentile error (MAPE), and Relative root mean square error (RRMSE) index.

A multi-step wind power group forecasting seq2seq architecture with spatial–temporal feature fusion and numerical weather prediction correction

Accurate total wind power forecasting is essential to wind farms. Conventional methods predict each wind turbine separately, which fails to exploit the spatial correlation between turbines and causes error accumulation through summation. Additionally, numerical weather prediction (NWP) with limited spatial–temporal resolutions is difficult to assist in power forecasting effectively, resulting in decreased performance as the forecasting steps increase. To address these drawbacks, a novel “N encoder-1 decoder” multi-channel fusion group forecasting sequence-to-sequence (seq2seq) architecture with wind resource quality assisted spatial attention (WRQASA) and NWP correction (NWPC) is proposed in this paper. The encoders in the architecture extract the temporal features of each turbine, followed by WRQASA adaptively adjusting their impact on the decoder. NWPC constructs micro-scale meteorological data based on its large-scale counterpart. The decoder takes the output from WRQASA and NWPC, and predicts the total power of the turbine group. The model is evaluated on a real-world wind farm. Ablation studies revealed the effectiveness of the novel architecture, WRQASA and NWPC. In 12, 24, 36 and 72-h multi-step forecasting scenarios, the new method outperformed conventional methods, such as seq2seq, light gradient boosting machine and support vector regression, achieving a more effective prediction of total power.

Wind Power Forecasting in the presence of data scarcity: A very short-term conditional probabilistic modeling framework

The uncertainty of wind power (WP) poses a significant challenge to power systems with a high percentage of WP. Accurate WP forecasting is an important approach to mitigate this issue. With the increasing demand for electricity, some wind farms (WFs) have been built or expanded. Such WFs lack abundant historical data for establishing forecasting models, i.e., data scarcity. This paper proposes a novel method for very short-term probabilistic forecasting based on transfer learning (TL) and conditional screening for WP. Utilizing numerical weather prediction as predictive features to address deterministic forecast data scarcity. Considering the data orientation, combining bidirectional long short-term memory with attention mechanisms to enhance prediction accuracy. Based on TL and conditional screening, it achieves data completion and singular value filtering to enhance the comprehensive performance of probabilistic forecasting errors. Using real world data, the proposed WP forecasting model reduces MAPE by at least 0.87% and RMSE by at least 0.6483 compared to the benchmark at different forecast steps. The comprehensive performance of probabilistic forecasting improves by at least 1.9134 at different forecast steps and confidence levels. The results indicate that the proposed forecasting model for data scarcity WFs is feasible, providing new insights for data scarcity WF forecasting.

Research on multi-digital twin and its application in wind power forecasting

Digital twins are digital models that operate within the digital space to perform specific functions. It can positively impact the physical counterpart in terms of improved efficiency, reduced costs, increased safety, and reliability. However, most existing research has limited digital twins to a single digital twin model, and the inherent characteristics and practical applications of single digital twin models make it challenging to fully utilize their advantages in complex and dynamic environments. Therefore, this paper proposes the concept of a multi-digital twin (MDT), as well as its synergistic operation mechanism, and designs two specific implementation methods suitable for processing time series-related tasks: the single metric dynamic preference method and the multi-metrics dynamic fusion method both based on a time window. Then, the results of two experiments show that the average relative improvement of two methods in the MAE, RMSE, and R2 metrics were 7.83 %, 5.01 %, and 1.24 % in 2015, and 4.98 %, 2 %, and 0.29 % in 2017. This means that the proposed methods can improve the accuracy of wind power forecasting. The effectiveness of the MDT synergy operation mechanism is verified, providing new research ideas for digital twins to adapt to complex system changes.

Multiple decomposition‐aided long short‐term memory network for enhanced short‐term wind power forecasting

AbstractWith the increasing penetration of grid‐scale wind energy systems, accurate wind power forecasting is critical to optimizing their integration into the power system, ensuring operational reliability, and enabling efficient system asset utilization. Addressing this challenge, this study proposes a novel forecasting model that combines the long‐short‐term memory (LSTM) neural network with two signal decomposition techniques. The EMD technique effectively extracts stable, stationary, and regular patterns from the original wind power signal, while the VMD technique tackles the most challenging high‐frequency component. A deep learning‐based forecasting model, i.e. the LSTM neural network, is used to take advantage of its ability to learn from longer sequences of data and its robustness to noise and outliers. The developed model is evaluated against LSTM models employing various decomposition methods using real wind power data from three distinct offshore wind farms. It is shown that the two‐stage decomposition significantly enhances forecasting accuracy, with the proposed model achieving values up to 9.5% higher than those obtained using standard LSTM models.

Temporal collaborative attention for wind power forecasting

Wind power serves as a clean and sustainable form of energy. However, its generation is fraught with variability and uncertainty, owing to the stochastic and dynamic characteristics of wind. Accurate forecasting of wind power is indispensable for the efficient planning, operation, and grid integration of wind energy systems. In this paper, we introduce a novel forecasting method termed Temporal Collaborative Attention (TCOAT). This data-driven approach is designed to capture both temporal and spatial dependencies in wind power generation data, as well as discern long-term and short-term patterns. Utilizing attention mechanisms, TCOAT dynamically adjusts the weights of each input variable and time step based on their contextual relevance for forecasting. Furthermore, the method employs collaborative attention units to assimilate directional and global information from the input data. It also explicitly models the interactions and correlations among different variables or time steps through the use of self-attention and cross-attention mechanisms. To integrate long-term and short-term information effectively, TCOAT incorporates a temporal fusion layer that employs concatenation and mapping operations, along with hierarchical feature extraction and aggregation. We validate the efficacy of TCOAT through extensive experiments on a real-world wind power generation dataset from Greece and compare its performance against twenty-two state-of-the-art methods. Experimental results demonstrate that TCOAT outperforms existing methods in terms of both accuracy and robustness in wind power forecasting. Moreover, we conduct a generality study on an additional real-world dataset from a different climate condition and wind power characteristics. The results show that TCOAT can achieve comparable or better performance than the state-of-the-art methods, confirming the generalization ability of TCOAT.

A Hybrid Short‐Term Wind Power Forecasting Model Considering Significant Data Loss

Accurate wind power forecasting (WPF) is pivotal for the power system dominated by high penetration of renewable energy. Most forecasting techniques require sufficient data samples as a premise for achieving accurate prediction. Due to equipment faults during data collection, complete data is not always available, resulting that the forecasting accuracy is greatly diminished. To address this issue, this paper proposes a novel two‐stage hybrid forecasting approach including data restoration stage and forecasting stage. For the data restoration stage, the bidirectional long short‐term memory (Bi‐LSTM) is integrated into generative adversarial network (GAN) to recover the missing data with consideration of the complex time dynamics and correlations among heterogeneous data. To improve the prediction accuracy, the complete generated wind power sequence is decomposed into multiple time sequences with low volatility based on the enhanced variational mode decomposition (VMD). For the forecasting stage, a hybrid forecasting algorithm that combines convolutional neural network (CNN) and bidirectional gated recurrent unit (BiGRU) with an improved attention mechanism is proposed, strengthening the forecasting performance by assigning optimal weights to key features. The proposed hybrid forecasting method outperforms traditional methods based on real wind farm data with different shares of data loss from Guangxi province in China. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Advanced hyperparameter optimization of deep learning models for wind power prediction

The uncertainty of wind power as the main obstacle of its integration into the power grid can be addressed by an accurate and efficient wind power forecast. Among the various wind power forecasting methods, machine learning (ML) algorithms, are recognized as a powerful wind power forecasting tool, however, their performance is highly dependent on the proper tuning of their hyperparameters. Common hyperparameter tuning methods such as grid search or random search are time-consuming, computationally expensive, and unreliable for complex models such as deep learning neural networks. Therefore, there is an urgent need for automatic methods to discover optimal hyperparameters for higher accuracy and efficiency of prediction models. In this study, a novel investigation is contributed to the field of wind power forecasting by a comprehensive comparison of three advanced techniques – Scikit-opt, Optuna, and Hyperopt – for hyperparameter optimization of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) models, a facet that, to our knowledge, has not been systematically explored in existing literature. The impact of these optimization techniques on the accuracy and efficiency of the CNN and LSTM models are assessed by comparing the root mean square error (RMSE) of the predictions and the required time to tune the models. The results show that the Optuna algorithm, using a Tree-structured Parzen Estimator (TPE) search method and Expected Improvement (EI) acquisition function, has the best efficiency for both CNN and LSTM models. In terms of accuracy, it is demonstrated that while for the CNN model all the optimization methods achieve similar performances, the LSTM model optimized by the Hyperopt algorithm, based on the annealing search method, results in the highest accuracy. In addition, for the first time in this research, the impact of the random initialization features on the performance of the forecasting models with neural networks is investigated. The proposed structures for deep learning models were examined to determine the most robust structure with the minimal sensitivity to the randomness. What we have discovered from the comparison of advanced hyperparameter optimization methods can be used by researchers to tune the time series-based forecasting models.

Day-ahead wind power forecasting based on feature extraction integrating vertical layer wind characteristics in complex terrain

Accurate wind power forecasts help establish efficient power supply plans and stabilize power systems. For long-term forecasts, the outputs of numerical weather prediction (NWP) models are pipelined as inputs for the statistical post-processing model, underscoring the necessity of understanding forecasts simulated from NWP to enhance power prediction accuracy.This study aims to enhance the quality of wind power forecasts in complex terrains, focusing on identifying and processing appropriate wind features of vertical layers simulated by NWP. In complex terrains with significant terrain variability, it is crucial to meticulously analyze and select the optimal vertical layer for each site or turbine individually, as simulated wind speeds at higher vertical layers than those used in previous research could potentially yield stronger correlations. Furthermore, we introduce a data processing technique that integrates wind characteristics across vertical layers, utilizing Principal Component Analysis (PCA). This approach not only provides physically intuitive results but also demonstrates enhanced performance compared to other feature selection methods. By selecting the appropriate vertical layers and applying the proper feature extraction, for wind farms situated in complex terrains in Korea, the annual normalized mean absolute error can be reduced by up to 1.2 %.

Wind power forecasting: A temporal domain generalization approach incorporating hybrid model and adversarial relationship-based training

Accurate forecasting of wind power is of significance for the operation and scheduling of the power grid in which wind power is integrated. However, distribution shifts caused by time-varying data would damage the stable generalization of forecasting models in future testing. In this paper, a temporal domain generalization approach incorporating adversarial relationship-based training and hybrid model is proposed to address wind power forecasting from the distribution perspective. Two joint modules, the temporal domain split and domain-adversarial hybrid model, are employed to function as a forecasting framework. The former splits training data into temporal domains with the maximum distribution differences, while the latter aims to learn shared knowledge from the above domains. The shared knowledge is independent of distribution shifts, which can be generalized to future testing well. Specifically, the domain-adversarial hybrid model, which is jointly optimized by minimizing the loss of power prediction and maximizing the loss of temporal domain classification, can be divided into a base model incorporating temporal convolutional network and gated recurrent unit, and a domain classifier. The data from actual wind turbines are performed to validate the proposed approach. The experimental results show that the proposed approach has superior performance to benchmark models and the significant improvement in accuracy reached 39.48% using adversarial relationship-based training.