Australian Energy Market Operator Research Articles

Constructing Interval Forecasts for Solar and Wind Energy Using Quantile Regression, ARCH and Exponential Smoothing Methods

The research reported in this article focuses on a comparison of two different approaches to setting error bounds, or prediction intervals, on short-term forecasting of solar irradiation as well as solar and wind farm output. Short term in this instance relates to the time scales applicable in the Australian National Electricity Market (NEM), which operates on a five-minute basis throughout the year. The Australian Energy Market Operator (AEMO) has decided in recent years that, as well as point forecasts of energy, it is advantageous for planning purposes to have error bounds on those forecasts. We use quantile regression as one of the techniques to construct the bounds. This procedure is compared to a method of forecasting the conditional variance by use of either ARCH/GARCH or exponential smoothing, whichever is more appropriate for the specific application. The noise terms for these techniques must undergo a normalising transformation before their application. It seems that, for certain applications, quantile regression performs better, and the other technique for some other applications.

Australia’s gas markets: similar problems…similar solutions?

Declining supply and entrenched demand in all three of Australia’s domestic gas markets – the East Coast, Western Australia (WA) and the Northern Territory (NT) – is creating similar issues. We tend to think of these as essentially separate, disconnected and divided by vast swathes of land. But the reality is that scarce local supply has been sanctioned in all three markets – despite around 16 tcf being sanctioned for export as liquefied natural gas (LNG) – and domestic demand remains resilient, as per both Australian Energy Market Operator (AEMO) and Wood Mackenzie forecasts. Starting with the largest, the increasingly tight East Coast market continues to hit headlines as it deals with supply crises, government interventions and new regulations. Undersupplied, underinvested and underexplored, it is also divided by activism and state and federal politics. Traditionally more sedate, the WA market has slowly drifted in a similar direction. New forecasts highlighting potential gas shortages have sparked more state intervention in the market, including banning LNG exports by certain suppliers. The NT market might be the smallest, but it’s ahead of the curve, already experiencing emergency upstream shortages that sparked increased domestic supply from LNG exporters. While there is a less diverse range of supply sources in the NT, the answer to current shortages matches the options currently being looked at by regulators on both coasts.

The carbon tax and the crisis in Australia’s National Electricity Market

Australia faced a severe energy crisis in 2022, which prompted the Australian Energy Market Operator to halt in National Electricity Market operations from June 15 to June 24. In this study, which examines half-hourly data from Australia’s National Electricity Market between 2010 and 2016, we aim to demonstrate the notable and intended consequences of the carbon tax, which was enacted in 2012 and repealed in 2014. Our findings show that this tax had a significant impact on wholesale prices. It also induced substitution effects in the generation mix. If these effects had persisted, the electricity generation landscape in 2022 would have been different. Specifically, the economic feasibility of coal would have decreased and that of gas would have increased. As a result, the composition of electricity generating capacity leading up to the 2022 crisis would have been altered.

Fast and Accurate Short-Term Load Forecasting with a Hybrid Model

Short-term electric load forecasting (STLF) plays a pivotal role in modern power system management, bolstering forecasting accuracy and efficiency. This enhancement assists power utilities in formulating robust operational strategies, consequently fostering economic and social advantages within the systems. Existing methods employed for STLF either exhibit poor forecasting performance or require longer computational time. To address these challenges, this paper introduces a hybrid learning approach comprising variational mode decomposition (VMD) and random vector functional link network (RVFL). The RVFL network, serving as a universal approximator, showcases remarkable accuracy and fast computation, owing to the randomly generated weights connecting input and hidden layers. Additionally, the direct links between hidden and output layers, combined with the availability of a closed-form solution for parameter computation, further contribute to its efficiency. The effectiveness of the proposed VMD-RVFL was assessed using electric load datasets obtained from the Australian Energy Market Operator (AEMO). Moreover, the effectiveness of the proposed method is demonstrated by comparing it with existing benchmark forecasting methods using two performance indices such as root mean square error (RMSE) and mean absolute percentage error (MAPE). As a result, our proposed method requires less computational time and yielded accurate and robust prediction performance when compared with existing methods.

NEMSEER: A Python package for downloading and handling historical National Electricity Market forecast data produced by the Australian Energy Market Operator

NEMSEER: A Python package for downloading and handling historical National Electricity Market forecast data produced by the Australian Energy Market Operator

Spatial optimality and temporal variability in Australia’s wind resource

To meet electricity demand using renewable energy supply, wind farm locations should be chosen to minimise variability in output, especially at night when solar photovoltaics cannot be relied upon. Wind farm location must balance grid-proximity, resource potential, and wind correlation between farms. A top-down planning approach for farm locations can mitigate demand unmet by wind supply, yet the present Australian wind energy market has bottom-up short-term planning. Here we show a computationally tractable method for optimising farm locations to maximise total supply. We find that Australia’s currently operational and planned wind farms produce less power with more variability than a hypothetical optimal set of farms with equivalent capacity within 100 km of the Australian Energy Market Operator grid. Regardless of the superior output, this hypothetical set is still subject to variability due to large-scale weather correlated with climate modes (i.e. El Niño). We study multiple scenarios and highlight several internationally transferable planning implications.

A hybrid model for deep learning short-term power load forecasting based on feature extraction statistics techniques

Accurate and reliable load forecasting can ensure the safety and economy of power system operation. To improve the accuracy of short-term power load forecasting, this paper adopts feature extraction combined with a variety of machine learning methods,(Empirical wavelet decomposition (EWT), Convolutional neural network (CNN) model, Recurrent neural network (RNN) model, Long short-term memory network (LSTM) model, Bayesian optimization (BOA) algorithm). A new hybrid forecasting model, the EWT-CNN-S-RNN + LSTM model is proposed to predict power consumption. In this method, the fixed mode of EWT decomposition is used to extract statistical features, and the LSTM/RNN model is selected according to the statistical features. This method uses the Bayesian optimization (BOA) algorithm to optimize the parameters to solve the model gradient explosion problem. Based on the electricity consumption data of 2007 (Australian Energy Market Operator) and a total of 336 power load data, a good load forecasting result is obtained, which can provide indirect support for power regulation and sustainable development of electricity.

Potential Domestic Energy System Vulnerabilities from Major Exports of Green Hydrogen: A Case Study of Australia

Australia has clear aspirations to become a major global exporter of hydrogen as a replacement for fossil fuels and as part of the drive to reduce CO2 emissions, as set out in the National Hydrogen Strategy released in 2019 jointly by the federal and state governments. In 2021, the Australian Energy Market Operator specified a grid forecast scenario for the first time entitled “hydrogen superpower”. Not only does Australia hope to capitalise on the emerging demand for zero-carbon hydrogen in places like Japan and South Korea by establishing a new export industry, but it also needs to mitigate the built-in carbon risk of its export revenue from coal and LNG as major customers, such as Japan and South Korea, move to decarbonise their energy systems. This places hydrogen at the nexus of energy, climate change mitigation and economic growth, with implications for energy security. Much of the published literature on this topic concentrates on the details of what being a major hydrogen exporter will look like and what steps will need to be taken to achieve it. However, there appears to be a gap in the study of the implications for Australia’s domestic energy system in terms of energy security and export economic vulnerability. The objective of this paper is to develop a conceptual framework for the implications of becoming a major hydrogen exporter on Australia’s energy system. Various green hydrogen export scenarios for Australia were compared, and the most recent and comprehensive was selected as the basis for further examination for domestic energy system impacts. In this scenario, 248.5 GW of new renewable electricity generation capacity was estimated to be required by 2050 to produce the additional 867 TWh required for an electrolyser output of 2088 PJ of green hydrogen for export, which will comprise 55.9% of Australia’s total electricity demand at that time. The characteristics of comparative export-oriented resources and their interactions with the domestic economy and energy system are then examined through the lens of the resource curse hypothesis, and the LNG and aluminium industries. These existing resource export frameworks are reviewed for applicability of specific factors to export-oriented green hydrogen production, with applicable factors then compiled into a novel conceptual framework for exporter domestic implications from large-scale exports of green hydrogen. The green hydrogen export superpower (2050) scenario is then quantitatively assessed using the established indicators for energy exporter vulnerability and domestic energy security, comparing it to Australia’s 2019 energy exports profile. This assessment finds that in almost all factors, exporter vulnerability is reduced, and domestic energy security is enhanced by the transition from fossil fuel exports to green hydrogen, with the exception of an increase in exposure of the domestic energy system to international market forces.

Robust provision of demand response from thermostatically controllable loads using lagrangian relaxation

This paper presents an end-user privacy and thermal comfort preserving approach for residential thermostatically controllable loads (TCLs) to provide demand response (DR) in grid services under uncertainties. Unlike the standard approach whereby a central utility-level control is used to control end-users loads, this paper splits the control effort between a central coordinating controller and local robust controllers. The Lagrangian relaxation (LR) method is applied by relaxing the power tracking constraint to obtain a hierarchical control framework with a local controller at each household level and a coordinating controller at the central utility level. Household-level local controllers are based on robust model predictive control (MPC) that relies on minimum household-specific information while accounting for thermal model parameter uncertainties and forecast errors associated with exogenous inputs. Considering inverter-type air conditioners as the TCL, the approach is validated using a practical signal corresponding to the Australian Energy Market Operator. The results demonstrate that accurate tracking of the load set-point signal can be achieved under a considerable range of uncertainties while preserving thermal comfort for end-users. Furthermore, the proposed DR scheme is computationally tractable and robust for real-world implementation.

Dynamic Pricing Based on Demand Response Using Actor–Critic Agent Reinforcement Learning

Eco-friendly technologies for sustainable energy development require the efficient utilization of energy resources. Real-time pricing (RTP), also known as dynamic pricing, offers advantages over other pricing systems by enabling demand response (DR) actions. However, existing methods for determining and controlling DR have limitations in managing an increasing demand and predicting future pricing. This paper presents a novel approach to address the limitations of existing methods for determining and controlling demand response (DR) in the context of dynamic pricing systems for sustainable energy development. By leveraging actor–critic agent reinforcement learning (RL) techniques, a dynamic pricing DR model is proposed for efficient energy management. The model’s learning framework was trained using DR and real-time pricing data extracted from the Australian Energy Market Operator (AEMO) spanning a period of 17 years. The efficacy of the RL-based dynamic pricing approach was evaluated through two predicting cases: actual-predicted demand and actual-predicted price. Initially, long short-term memory (LSTM) models were employed to predict price and demand, and the results were subsequently enhanced using the deep RL model. Remarkably, the proposed approach achieved an impressive accuracy of 99% for every 30 min future price prediction. The results demonstrated the efficiency of the proposed RL-based model in accurately predicting both demand and price for effective energy management.

Real-Time Pricing-Enabled Demand Response Using Long Short-Time Memory Deep Learning

Sustainable energy development requires environment-friendly energy-generating methods. Pricing system constraints influence the efficient use of energy resources. Real-Time Pricing (RTP) is theoretically superior to previous pricing systems for allowing demand response (DR) activities. The DR approach has been useful for correcting supply–demand imbalances as technology has evolved. There are several systems for determining and controlling the DR. However, most of these solutions are unable to control rising demand or forecast prices for future time slots. This research provides a Real-Time Pricing DR model for energy management based on deep learning, where the learning framework is trained on demand response and real-time pricing. The study data in this article were taken from the Australian Energy Market Operator (AEMO), and the learning framework was trained over 17 years of data to forecast the real future energy price and demand. To investigate the suggested deep learning-based dynamic pricing strategy, two prediction instances are addressed: actual–predicted demand and actual–predicted price. We estimated pricing and demand outcomes using long short-term memory (LSTM), which were then greatly improved by architectural changes in the model. The findings showed that the suggested model is suitable for energy management in terms of demand and price prediction.

Probabilistic Wind Power Forecasting Using Optimized Deep Auto-Regressive Recurrent Neural Networks

Wind power forecasting is very crucial for power system planning and scheduling. Deep neural networks (DNNs) are widely used in forecasting applications due to their exceptional performance. However, the DNNs’ architectural configuration has a significant impact on their performance, and the selection of proper hyper-parameters determines the success or failure of these models. Therefore, one of the challenging issues in DNNs is how to assess their hyper-parameter values effectively. Most of the previous researches in the literature have tuned the DNNs’ hyper-parameters manually, which is a weak and time-consuming task. Using optimization/evolutionary algorithms is an effective way to obtain the optimal values of DNNs’ hyper-parameters automatically. In this article, we propose a novel evolutionary algorithm that is based on the grasshopper optimization algorithm (GOA) improved by adding two evolutionary operators, opposition-based learning and chaos theory, to the optimization process. Overall, a novel probabilistic wind power forecasting model named neural GOA deep auto-regressive (NGOA-DeepAr) is proposed based on an auto-regressive recurrent neural network in which the proposed evolutionary algorithm has optimized its hyper-parameters. The performance of the proposed NGOA-DeepAr model is tested on two different datasets: One is the publicly available GEFCom-2014 dataset and the other is the Australian Energy Market Operator dataset. The prediction interval coverage probability and pinball loss for the two datasets are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$[0.902, 0.320]$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$[0.933, 1.4885]$</tex-math></inline-formula> , respectively. According to the experimental findings, our proposed NGOA-DeepAr is much faster in learning and outperforms the benchmark DNNs and the other neuroevolutionary models.

Electricity Load and Price Forecasting Using a Hybrid Method Based Bidirectional Long Short-Term Memory with Attention Mechanism Model

The accuracy of forecasting short- or medium-term electricity loads and prices is critical in the energy market. The amplitude and duration of abnormally high prices and load spikes can be detrimental to retailers and production systems. Therefore, predicting these spikes to effectively manage risk is critical. In this paper, a novel hybrid method that combines ensemble empirical mode decomposition (EEMD) algorithm and a bidirectional long short-term memory with attention mechanism (BiLSTM-AM) model is proposed to predict electricity loads and prices. A simple approach is proposed to determine the number of intrinsic mode functions (IMFs) that decompose raw data using EEMD to avoid overdecomposition, irrelevant components, and high computational cost. Each selected mode is then modeled with BiLSTM-AM to obtain a predicted sequence. These sequences are summed and then reverted to obtain the final predicted value. The proposed method is validated using two datasets (PMJ and Australian Energy Market Operator) with different time intervals to demonstrate the generality and robustness of the forecasts, especially in temporal valley or peak forecasting. The results show that the proposed method outperforms other methods in prediction accuracy and spike-capturing ability, with EEMD reducing the mean absolute percentage error (MAPE) by 53%, 54%, and 60%, respectively. In the three forecast periods, the average MAPE and R ̲ 2 are 0.097 and 0.92, respectively. Furthermore, we use Kolmogorov-Smirnov predictive accuracy (KSPA) test and model confidence set (MCS) test to validate the superiority of the proposed model. The results demonstrate its suitability, reliability, and performance in short- and medium-term forecasting.

Cooperative ensemble learning model improves electric short-term load forecasting

Efficient models for short-term load forecasting (STLF) plays a crucial role in establishing the companies’ energetic planning due to their importance in electric power distribution and generation systems. An ensemble learning model based on dual decomposition approach, which combines two signals decomposition techniques, machine learning models and hyperparameters optimization based on metaheuristics, is applied to electric STLF. The seasonal and trend decomposition using locally-weighted regression (STL) decompose the time series into seasonal, trend, and residual components. Moreover, Variational Mode Decomposition (VMD) is applied to decompose the STL residual into different frequencies. Also, seasonal Naïve is adopted to handle the seasonal patterns. Moreover, due to the nonlinearities of the remaining components, an eXtreme gradient boosting model with hyperparameters tuned by a coyote optimization algorithm, extreme learning machines, ridge regression, and support vector regression models are employed to handle the STL trend and VMD components. The datasets from five regions and four seasons of the year for the Australian energy market operator are used to test the effectiveness of the proposed model for STLF with a multi-step-ahead horizon (12-hours-ahead and 24-hours-ahead). The performance analysis is based on the mean absolute error, symmetric mean absolute percentage error (sMAPE), overall weighted average, and Diebold–Mariano statistical test. The results of the study were divided into four comparative experiments: comparisons with (i) single decomposed models, (i) dual decomposed models, (iii) non-decomposed models, and (iv) state-of-art models. Regarding the sMAPE performance criterion, the proposed models achieved errors between 0.75%–3.18% and 1.56%–7.72% for STLF 12 and 24-hours-ahead. In the comparative scenarios, the proposed model improved the forecasting results up to 99% regarding compared models in terms of overall weighted average. Lastly, the proposed cooperative ensemble learning model outperformed 71.25% of the state-of-art models regarding forecasting error reduction in terms of sMAPE. From a broader perspective, the proposed cooperative ensemble learning model showed to be an efficient tool based of forecasting erros reduction for STLF due to its capability to improve forecasting accuracy and achieve reliable forecasting results compared with observed ones.

Open-source project feasibility tools for supporting development of the green ammonia value chain

Ammonia plays a vital role in feeding the world through fertilizer production, as well as having other industrial uses. However, current ammonia production processes rely heavily on fossil fuels, mostly natural gas, to generate hydrogen as a feedstock. There is an urgent need to re-design and decarbonise the production process to reduce greenhouse emissions and avoid dependence on volatile gas markets and a depleting resource base. Renewable energy driven electrolysis to generate hydrogen provides a viable pathway for producing carbon-free or green ammonia. However, a key challenge associated with producing green ammonia is managing low cost but highly variable wind and solar renewable energy generation for hydrogen electrolysis while maintaining reliable operation of the less flexible ammonia synthesis unit. To date, green ammonia production has only been demonstrated at pilot scale, and optimising plant configurations and scaling up production facilities is an urgent task. Existing feasibility studies have demonstrated the ability to model and cost green ammonia production pathways that can overcome the technical and economic challenges. However, these existing approaches are context specific, demonstrating the ability to model and cost green ammonia production for defined locations, with set configurations. In this paper we present a modelling framework that consolidates the array of configurations previously studied into a single framework that can be tailored to the location of interest. Our open-source green ammonia modelling and costing tool dynamically simulates the integration of renewable energy with a wide range of balancing power and storage options to meet the flexible demands of the green ammonia production process at hourly time resolution over a year or more. Unlike existing models, the open-source implementation of our tool allows it to be used by a potentially wide range of stakeholders to explore their own projects and help guide the upscaling of green ammonia as a pathway for decarbonisation. Using Gladstone in Australia as a case study, a 1 million tonne per annum (MMTPA) green ammonia plant is modelled and costed using price assumptions for major equipment in 2030 provided by the Australian Energy Market Operator (AEMO). Using a hybrid (solar PV and wind) renewable energy source and Battery Energy Storage System as balancing technology, we estimate a levelized cost of ammonia (LCOA) between 0.69 and 0.92 USD kgNH3-1. While greater than historical ammonia production costs from natural gas, falling renewables costs and emission reduction imperatives suggest a major future role for green ammonia.

Interactive Grid Synchronization-Based Virtual Synchronous Generator Control Scheme on Weak Grid Integration

While high volume of power electronic converters with renewable energy are being integrated into the grid, low system strength of grid causes reduction in system inertia and loss of synchronization with power converters. This paper proposes a novel interactive control scheme of voltage source converter (VSC) connected to weak grid. Firstly, a grid synchronization approach is performed using VSC side voltage transformed on rotating reference frame with respect to grid voltage, which is carried out based on lowest short circuit ratio (SCR) as approved by utilities such as Australian Energy Market Operator’s system strength report. The transformed voltage is then fed into the active power and reactive power control through PLL. Secondly, the grid synchronization approach through PLL is made interactive by adding virtual synchronous generator (VSG) control into the outer loop, which generates virtual inertia to mimic the synchronous generators of a power system. The design of the VSG control parameters is realized based on small signal stability analysis. Furthermore, the control method is evaluated as per the National Electricity Rules clauses S5.2.5.1-S5.2.14 under AEMO and National Service Providers of Australian strict Industry guidelines. The time-domain simulation results are conducted on PSCAD platform.

How renewables are transforming electricity grids: an analysis of Australia’s integrated system plan

Most countries will require mega programmes of investment to support the transition to higher shares of renewable energy. Large increases in roof-top solar and other grid-scale renewables are changing the way that electricity grids operate and are designed. This paper focuses on the case of Australia, where the Australian Energy Market Operator develops a bi-annual Integrated System Plan (ISP) to provide advice on the development of the electricity grid. The ISP is based on modelling of the future energy system and, accordingly, accounts for a rise in distributed energy generation and other technologies, such as batteries and electric drive vehicles. There are many lessons that can be learned from regions with high shares of renewables, including the importance of grid interconnectivity and storage. This example of rapid change has occurred in a country that did not have a robust national decarbonisation policy, but does have excellent wind and solar resources.

False Data Injection Attack Detection in Smart Grid Using Energy Consumption Forecasting

Supervisory Control and Data Acquisition (SCADA) systems are essential for reliable communication and control of smart grids. However, in the cyber-physical realm, it becomes highly vulnerable to cyber-attacks like False Data Injection (FDI) into the measurement signal which can circumvent the conventional detection methods and interfere with the normal operation of grids, which in turn could potentially lead to huge financial losses and can have a large impact on public safety. It is imperative to have an accurate state estimation of power consumption for further operational decision-making.This work presents novel forecasting-aided anomaly detection using an CNN-LSTM based auto-encoder sequence to sequence architecture to combat against false data injection attacks. We further present an adaptive optimal threshold based on the consumption patterns to identify abnormal behaviour. Evaluation is performed on real-time energy demand consumption data collected from the Australian Energy Market Operator. An extensive experiment shows that the proposed model outperforms other benchmark algorithms in not only improving the data injection attack (95.43%) but also significantly reducing the false positive rate.

Random vector functional link neural network based ensemble deep learning for short-term load forecasting

Electric load forecasting is essential for the planning and maintenance of power systems. However, its un-stationary and non-linear properties impose significant difficulties in predicting future demand. This paper proposes a novel ensemble deep Random Vector Functional Link (edRVFL) network for electricity load forecasting. The weights of hidden layers are randomly initialized and fixed during the training process. The hidden layers are stacked to enforce deep representation learning. Then, the model generates the forecasts using the ensemble of the outputs of each layer. Moreover, we also propose to augment the random enhancement features by empirical wavelet transformation (EWT). The raw load data are decomposed by EWT in a walk-forward approach without introducing future data leakage problems in the decomposition process. Finally, all the sub-series generated by the EWT, including raw data, are fed into the edRVFL for forecasting purposes. The proposed model is evaluated on sixteen publicly available time series from the Australian Energy Market Operator of the year 2020. The simulation results demonstrate the proposed model’s superior performance over eleven forecasting methods in two error metrics and statistical tests on electricity load forecasting tasks.

Concurrent 3. Presentation for: The impact of new supply from the Perth Basin on the supply of natural gas in Western Australia

Presented on Tuesday 17 May: Session 3 The Australian Energy Market Operator (AEMO) forecasts demand for natural gas (gas) in Western Australia will remain steady through the 2020s, but that the local production will drop below anticipated demand in the mid-2020s and again from 2030 onwards. If this situation eventuates, lack of gas production will impact the competitiveness of WA’s energy-consuming industries. This paper demonstrates that the Perth Basin has the potential to meet this supply–demand gap and augment existing sources of gas over the next 20+ years. A subsurface-led assessment of the Perth Basin’s Reserves, Contingent Resources and Prospective Resources incorporating all recent exploration well results shows that the unrisked resource potential of the Perth Basin is 7052 bcfe (7.475 PJ) with production potential up to 1100 TJ/day. The basis for these estimates will be illustrated with examples, alongside a discussion of the variables that affect the resource assessment, the impact of capacity constraints on the timing of development of Perth Basin gas, and the dynamics of the WA gas market. To access the presentation click the link on the right. To read the full paper click here