Reliable Forecasts Research Articles

A Standardized Sky Condition Classification Method for Multiple Timescales and Its Applications in the Solar Industry

The deployment of photovoltaic (PV) systems has increased globally to meet renewable energy targets. Intermittent PV power generated due to cloud-induced variability introduces reliability and grid stability issues at higher penetration levels. Variability in power generation can induce voltage fluctuations within the distribution system and cause adverse effects on power quality. Therefore, it is essential to quantify resource variability to mitigate an intermittent power supply. In this study, we propose a new scheme to classify the sky conditions that are based on two common variability metrices: daily clear-sky index and normalized aggregate ramp rates. The daily clear-sky index estimates the cloudiness in the sky, and ramp rates account for the variability introduced in the system generation due to sudden cloud movements. This classification scheme can identify clear-sky, highly variable, low intermittent, high intermittent and overcast days. By performing a Chi-square test on the training and test sets, we obtain Chi-square statistic values greater than 3 with p-value > 0.05. This indicates that the distribution of the training and test clusters are similar, indicating the robustness of the proposed sky classification scheme. We have demonstrated the applicability of the scheme with diverse datasets to show that the proposed classification scheme can be homogenously applied to any dataset globally despite their temporal resolution. Using various case studies, we demonstrate the potential applications of the scheme for understanding resource allocation, site selection, estimating future intermittency due to climate change, and cloud enhancement effects. The proposed sky classification scheme enhances the precision and reliability of solar energy forecasts, optimizing system performance and maximizing energy production efficiency. This improved accuracy is crucial for variability control and planning, ensuring optimal output from PV plants.

A Fully Connected Neural Network (FCNN) Model to Simulate Karst Spring Flowrates in the Umbria Region (Central Italy)

In the last decades, climate change has led to increasingly frequent drought events within the Mediterranean area, creating an urgent need of a more sustainable management of groundwater resources exploited for drinking and agricultural purposes. One of the most challenging issues is to provide reliable simulations and forecasts of karst spring discharges, whose reduced information, as well as the hydrological processes involving their feeding aquifers, is often a big issue for water service managers and researchers. In order to plan a sustainable water resource exploitation that could face future shortages, the groundwater availability should be assessed by continuously monitoring spring discharge during the hydrological year, using collected data to better understand the past behaviour and, possibly, forecast the future one in case of severe droughts. The aim of this paper is to understand the factors that govern different spring discharge patterns according to rainfall inputs and to present a model, based on artificial neural network (ANN) data training and cross-correlation analyses, to evaluate the discharge of some karst spring in the Umbria region (Central Italy). The model used is a fully connected neural network (FCNN) and has been used both for filling gaps in the spring discharge time series and for simulating the response of six springs to rainfall seasonal patterns from a 20-year continuous daily record, collected and provided by the Regional Environmental Protection Agency (ARPA) of the Umbria region.

An integrated, automated and modular approach for real-time weather monitoring of surface meteorological variables and short-range forecasting using machine learning

Weather monitoring and forecasting plays a vital role in a great variety of human activities such as agriculture, transportation, and extreme weather phenomena. This study presents the first outcomes of the development of a fully automated system regarding the real-time recording of basic meteorological parameters and their short-range forecasting (nowcasting). The system itself is divided into five core components: a hardware system for monitoring atmospheric conditions (Commercial Off-The-Shelf structures), a system for storing and managing data, a module for distributing data to support applications, a machine learning algorithm for nowcasting, and a user-friendly interface, all made by modern tools and methods, described analytically. Finally, the nowcasting procedure along with the relative accuracy results, is presented. The nowcasting procedure is based on a Long Short-Term Memory (LSTM) model scheme which is parametrized in such a way that reliable forecasts, up to 2 h ahead of time, can be provided.

Improving sub-seasonal extreme precipitation forecasts over China through a hybrid statistical-dynamical framework

Skillful and reliable sub-seasonal extreme precipitation forecasts are crucial for disaster prevention and mitigation. In this study, we introduce a hybrid statistical-dynamical framework to predict monthly maximum one-day precipitation (Rx1D) and monthly maximum five-day precipitation (Rx5D) over China from May to October. In the hybrid statistical-dynamical framework, the ECMWF forecasts of precipitation and boreal summer intraseasonal oscillation (BSISO) indices are used as predictors to establish calibration model and bridging models, separately. The calibration model and bridging models are then merged to generate probabilistic forecasts of Rx1D and Rx5D. Our results suggest that the bridging models show better performance in predicting Rx1D and Rx5D than calibration model in May, June, and July when the BSISO indices are used as predictors. The forecast skill of calibration model is higher compared to bridging models in August, September, and October. The BMA merged forecasts take advantage of both calibration model and bridging models, and can provide skilful and reliable forecasts for both Rx1D and Rx5D prediction. To have a more comprehensive assessment, we also evaluate the prediction skill of the occurrence of extreme precipitation events with exceedance probabilities of 50%, 20%, and 5% for both Rx1D and Rx5D. The Brier skill score of merged forecasts indicates that the hybrid statistical-dynamical framework can also provide skilful forecasts for the occurrence of extreme precipitation events greater than one-in-5-year return value of Rx1D (5Rx1D) and one-in-5-year return value of Rx5D (5Rx5D) in comparison to long-term climatology. These findings demonstrate the great potential of combining dynamical models and statistical models in improving sub-seasonal extreme precipitation forecasts.

SDE-HNN: Accurate and Well-calibrated Forecasting using Stochastic Differential Equations

It is crucial yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. Heteroscedastic neural networks (HNNs) are promising methods that capture data uncertainty for forecasting problems, while existing HNNs have difficulties in conjoining calibrated uncertainty estimation and satisfactory predictive performance due to the failure to construct an explicit interaction between the prediction and its associated uncertainty. This paper develops SDE-HNN, an improved HNN equipped with stochastic differential equations (SDE), to characterize the interaction between the predictive mean and variance inside HNNs for accurate and reliable forecasting. The existence and uniqueness of the solution to the devised neural SDE are guaranteed. Moreover, based on the bias-variance trade-off for the optimization in SDE-HNN, we design an enhanced numerical SDE solver to improve learning stability. Finally, we present two new diagnostic uncertainty metrics to systematically evaluate the predictive uncertainty. Experiments on various challenging datasets show that our method significantly outperforms state-of-the-art baselines on both predictive performance and uncertainty quantification, delivering well-calibrated and sharp prediction intervals in time-series forecasting.

The Fogees system for forecasting particulate matter concentrations in urban areas

Air quality forecasting requires appropriate models and data sources. The Fogees system presented in this paper enables mapping, evaluation and forecasting of the level of PMx pollution in the air, for urban and suburban areas. Input data were downloaded from the Meteoblue service, the GIS database and – in the case of integration with existing measurement systems – also from local air quality monitoring stations. The system uses a diagnostic model to determine the velocity field and a Lagrange model to describe pollution dispersion. The system has an autocalibration model and uses proprietary algorithms to estimate emissions from domestic heating, transport and background pollution levels. Modular design and parallel computation facilitate simultaneous calculation of forecasts for existing emission conditions and calculations for alternative emission conditions. The system permits forecasting for the next hour and for several consecutive hours. The validation results confirm that the system reliable forecasting of PM concentrations.

Long Short-Term Memory vs Gated Recurrent Unit: A Literature Review on the Performance of Deep Learning Methods in Temperature Time Series Forecasting

Temperature forecasting is a crucial aspect of meteorology and climate change studies, but challenges arise due to the complexity of time series data involving seasonal patterns and long-term trends. Traditional methods often fall short in handling this variability, necessitating more advanced solutions to enhance prediction accuracy. LSTM and GRU models have emerged as promising alternatives for modeling temperature data. This study is a literature review comparing the effectiveness of LSTM and GRU based on previous research in temperature forecasting. The goal of this review is to evaluate the performance of both models using various evaluation metrics such as MSE, RMSE, and MAE to identify gaps in previous research and suggest improvements for future studies. The method involves a comprehensive analysis of previous studies using LSTM and GRU for temperature forecasting. Assessment is based on RMSE values and other metrics to compare the accuracy and consistency of both models across different conditions and temperature datasets. The analysis results show that LSTM has an RMSE range of 0.37 to 2.28. While LSTM demonstrates good performance in handling long-term dependencies, GRU provides more stable and accurate performance with an RMSE range of 0.03 to 2.00. This review underscores the importance of selecting the appropriate model based on data characteristics to improve the reliability of temperature forecasting.

Toroidal modified Miller-Turner CME model in EUHFORIA: Validation and comparison with flux rope and spheromak

Rising concerns about the impact of space-weather-related disruptions demand modelling and reliable forecasting of coronal mass ejection (CME) impacts. In this study, we demonstrate the application of the modified Miller-Turner (mMT) model implemented within EUropean Heliospheric FORecasting Information Asset (EUHFORIA) in forecasting the geo-effectiveness of observed coronal mass ejection (CME) events in the heliosphere. Our goal is to develop a model that not only has a global geometry, in order to improve overall forecasting, but is also fast enough for operational space-weather forecasting. We test the original full torus implementation and introduce a new three-fourths Torus version called the Horseshoe CME model. This new model has a more realistic CME geometry, and overcomes the inaccuracies of the full torus geometry. We constrain the torus geometrical and magnetic field parameters using observed signatures of the CMEs before, during, and after the eruption. We perform EUHFORIA simulations for two validation cases -- the isolated CME event of 12 July 2012 and the CME--CME interaction event of 8-10 September 2014. We performed an assessment of the model's capability to predict the most important $B_z$ component using the advanced dynamic time-warping (DTW) technique. The Horseshoe model predictions of CME arrival time and geo-effectiveness for both validation events compare well with the observations and are weighed against the results obtained with the spheromak and FRi3D models, which were already available in EUHFORIA. The runtime of the Horseshoe model simulations is close to that of the spheromak model, which is suitable for operational space weather forecasting. However, the capability of the magnetic field prediction at 1 AU of the Horseshoe model is close to that of the FRi3D model. In addition, we demonstrate that the Horseshoe CME model can be used for simulating successive CMEs in EUHFORIA, overcoming a limitation of the FRi3D model.

Improving Air Quality Data Reliability through Bi-Directional Univariate Imputation with the Random Forest Algorithm

Forecasting the future levels of air pollution provides valuable information that holds importance for the general public, vulnerable populations, and policymakers. High-quality data are essential for precise and reliable forecasts and investigations of air pollution. Missing observations arise when the sensors utilized for assessing air quality parameters experience malfunctions, which result in erroneous measurements or gaps in the dataset and hinder the data quality. This research paper presents a novel approach for imputing missing values in air quality data in a univariate approach. The algorithm employs the random forest (RF) algorithm to impute missing observations in a bi-directional (forward and reverse in time) manner for air quality (particulate matter less than 2.5 μm (PM2.5)) data from the Republic of Serbia. The algorithm was evaluated against simple methods, such as the mean and median imputation methods, for missing observations over durations of 24, 48, and 72 h. The results indicate that our algorithm yielded comparable error rates to the median imputation method for all periods when imputing the PM2.5 data. Ultimately, the algorithm’s higher computational complexity proved itself as not justified considering the minimal error decrease it achieved compared with the simpler methods. However, for future improvement, additional research is needed, such as utilizing low-code machine learning libraries and time-series forecasting techniques.

A reliable ensemble forecasting modeling approach for complex time series with distributionally robust optimization

Accurate ensemble forecast results provide strong support for management decisions. While the existing ensemble forecasting model ignores the requirement of directional accuracy making its prediction direction is usually error-prone. To address this issue, we develop a convex directional prediction error measure and subsequently construct a reliable ensemble forecasting model that minimizes the horizontal prediction error with the bounded directional prediction error. Mathematically, we provided the proper range of the required directional error level. Furthermore, we find the classical ensemble forecasting model is a special case of our study when the directional error level is larger than the upper bound we gave in this study. To deal with the possible deterioration of the individual model over time, we also considered its worst possible prediction performance by introducing the distributionally robust optimization (DRO) technique into the proposed reliable ensemble forecasting model. Technically, we showed that the DRO-based reliable ensemble forecasting model is convex and can be reformulated into a second-order cone problem which can be readily solved by off-the-shelf solvers. Finally, the effectiveness of the proposed reliable ensemble forecasting model and the DRO-based reliable ensemble forecasting model were validated on three different datasets, e.g., the exchange rate, the oil price, and the country risk index. To sum up, we construct a reliable ensemble forecasting model to simultaneously control the horizontal prediction error and directional prediction error of ensemble forecasting, and thus enhance the reliability of ensemble forecasting.

APPROACH TO CREATION OF GEOLOGICAL AND RESERVOIR SIMULATION MODEL AND METHODS FOR HISTORY MATCHING OF RESERVOIR ENERGY STATE

Under conditions of complex geological structure and substantial share of hard-to-recover oil reserves, reservoir simulation is a key tool in reservoir management. Direct problems solved by reservoir simulation include forecasting of the production performance, control and management of reservoir development, and localization of residual oil reserves. Inverse problem solution is useful for updating the knowledge of geological structure, reservoir quality, and historical production data by means of history matching. The quality of history matching is fundamental to ensure reliable future forecasts and localization of residual oil zones. The main approaches to appropriate history matching rely on analysis and adjustment of well logging and geological parameters, as well as checking and correction of errors, inadequate historical data, application of physics-based and reasonable approaches and tools. The prerequisite for adequate history matching and predictive ability of the resultant reservoir simulation model is the compliance of simulated and actual energy state of the reservoir. In this paper, the initial stage of reservoir simulation is performed, the approach to history matching of reservoir energy state aimed at improving the quality of further predictions is described as exemplified by one of the production areas of Russian oil field. The main causes of poor history match of formation and bottomhole pressures are revealed. These causes are grouped as follows: — geological causes/reservoir quality, which include permeability field distribution, reservoir connectivity, and determination of rock and fluid compressibility;— technical causes (during reservoir simulation), that is model limitations in case of larger areas, the need to use sector models, consideration of buffer region;— technological causes, including inadequate historical data or lack of data, detection of behind-the-casing flows and production casing leaks in wellbore, etc. For each group of factors that have a substantial impact on reservoir pressure, recommendations and methods of energy state history matching on geological and reservoir simulation models are provided, examples are given.

Propose and implement a Rule-Based System to Predict Crop Yield Production

Accurate meteorological forecasting plays a pivotal role in agricultural decision-making, particularly in determining crop yield potential and optimizing agricultural practices. This study investigates the application of data mining techniques in meteorological forecasting to enhance crop yield prediction accuracy Preliminary findings suggest that data mining techniques, including machine learning algorithms, neural networks, and ensemble methods, offer significant potential for improving the accuracy and reliability of meteorological forecasts for crop yield prediction. In conclusion, this study underscores the potential of data mining techniques in improving meteorological forecasting for crop yield potential assessment.

Establish an Effective Framework for Accurate Prediction of Employee Mental Health for Public Health

Employee mental health is a primary feature of workplace efficiency and overall public health. With rising mental health problems impaired by variables contains stress, burnout, and isolation, there is an urgent need for effective prediction models that can identify at-risk employees. This study proposes a robust predictive model utilizing dynamic lion optimized VGG16 (DLO-VGG16), for employee mental health prediction represents a significant advancement in public healthcare. The use of these sophisticated prediction models can improve medical results for groups at risk and is essential in the battle against heart disease. Employee’s mental health data collected from Kaggle website. The data preprocessing phase employed Z-score normalization to standardize input values, thereby addressing data inconsistencies and errors. The Sequential Forward Floating Search Algorithm (SFFS) was utilized for extracting the employee’s mental health. This enhanced heuristic search method begins with an empty feature set, incrementally adding features until the optimal set is identified. The proposed DLO-VGG16 framework to balance accuracy, ROC curve, and f1-score ensuring reliable detection and forecasting of employee mental health issues, thereby contributing to the broader public health landscape.

An extension to ensemble forecast of conditional nonlinear optimal perturbation considering nonlinear interaction between initial and model parametric uncertainties

Initial and model uncertainties are the main sources of forecast errors, making the single and deterministic forecasts unreliable. To estimate these uncertainties, a growing consensus shifts towards ensemble forecasting, aiming to provide the probability density distribution of the atmosphere. However, current ensemble methods either focus on single-source uncertainties or employ a simple superposition of the two, neglecting the nonlinear interaction between them, and thus fail to reflect the real forecast uncertainty. Motivated by this, this study extends the CNOP approach, defined as the optimal growth considering nonlinear interaction between initial and model parameters, to the scenario of ensemble forecasts and proposes an orthogonal CNOPs method (O-CNOP-IPs). This method concerns the nonlinear effect of initial and model parametric uncertainties through a joint optimization strategy and enhances the estimation of this effect by providing diversity and independent CNOPs (via orthogonality). To evaluate the performance of O-CNOP-IPs, extensive experiments are conducted for North Atlantic Oscillation (NAO) ensemble forecasts in the realistically configured Community Earth System Model (CESM). Our findings reveal that the O-CNOP-IPs method outperforms existing methods in forecast skill and reliability, improving deterministic skill by 17.5 % and probabilistic skill by 52 %–63 %. Our dynamic analysis also unveils that this method undergoes rapid development in the early stage and effectively neutralizes errors in control forecasts, significantly enhancing the reliability of ensemble forecasts. It is expected that O-CNOP-IPs plays a significant role in accurately representing the forecast uncertainty of other high-impact weather and climate phenomena.

How dependent are quantitative volcanic ash concentration and along‐flight dosage forecasts to model structural choices?

AbstractProducing quantitative volcanic ash forecasts is challenging due to multiple sources of uncertainty. Careful consideration of this uncertainty is required to produce timely and robust hazard warnings. Structural uncertainty occurs when a model fails to produce accurate forecasts, despite good knowledge of the eruption source parameters, meteorological conditions and suitable parameterizations of transport and deposition processes. This uncertainty is frequently overlooked in forecasting practices. Using a Lagrangian particle dispersion model, simulations with varied output spatial resolution, temporal averaging period and particle release rate are performed to quantify the impact of these structural choices. This experiment reveals that, for the 2019 Raikoke eruption, structural choices give measurements of peak ash concentration spanning an order of magnitude, significantly impacting decision‐relevant thresholds used in aviation flight planning. Conversely, along‐flight dosage estimates exhibit less sensitivity to structural choices, suggesting it is a more robust metric to use in flight planning. Uncertainty can be reduced by eliminating structural choices that do not result in a favourable level of agreement with a high‐resolution reference simulation. Reliable forecasts require output spatial resolution 80 km, temporal averaging periods 3 h and particle release rates 5000 particles/h. This suggests that simulations with relatively small numbers of particles could be used to produce a large ensemble of simulations without significant loss of accuracy. Comparison with previous Raikoke simulations indicates that the uncertainty associated with these constrained structural choices is smaller than those associated with satellite constrained eruption source parameter and internal model parameter uncertainties. Thus, given suitable structural choices, other epistemic sources of uncertainty are likely to dominate. This insight is useful for the design of ensemble methodologies which are required to enable a shift from deterministic to probabilistic forecasting. The results are applicable to other long‐range dispersion problems and to Eulerian dispersion models.

Modeling financial market dynamics with the use of fuzzy

In the modern world, financial markets play an important role in the economy and people’s lives. They provide access to financial resources and are also a source of profit for many companies. However, instability in the financial markets can lead to serious consequences such as financial crises and loss of investor confidence. In this regard, modelling the financial market dynamics becomes increasingly relevant. This work considered the use of fuzzy mathematics for this purpose. Fuzzy mathematics is a branch of mathematics that studies methods and algorithms for dealing with fuzzy data and fuzzy objects. It allows to consider uncertainty and incompleteness of information, which is especially important in the financial markets where data is often incomplete and inaccurate. The purpose of this research is to establish the relationship between financial asset prices while using behavioural factors (investor sentiment), fundamental (market returns), and microstructural ones (company size, ratio of book and market values of the company). The application of fuzzy mathematics in financial modelling will improve the accuracy and reliability of forecasts as well as increase the stability of the model to various sources of uncertainty.

Predictive Modelling of Network Capacity Demands: A Machine Learning Approach for Global Enterprise Backbone Networks

In the dynamic landscape of global enterprise networks, accurate capacity forecasting is paramount for ensuring optimal resource allocation and preventing service disruptions. This paper presents a hybrid machine learning methodology that combines Autoregressive Integrated Moving Average (ARIMA) models with additional techniques to enhance the accuracy and reliability of network capacity forecasts. By leveraging historical traffic data and incorporating external factors, we develop a predictive model that outperforms traditional methods and adapts to the evolving demands of modern networks. The effectiveness of our approach is validated through rigorous testing against established benchmarks, demonstrating significant improvements in forecasting accuracy.

An advanced hybrid deep learning model for accurate energy load prediction in smart building

In smart cities, sustainable development depends on energy load prediction since it directs utilities in effectively planning, distributing and generating energy. This work presents a novel hybrid deep learning model including components of the Improved-convolutional neural network (CNN), bidirectional long short-term memory (Bi-LSTM), Graph neural network (GNN), Transformer and Fusion Layer architectures for precise energy load forecasting. Better feature extraction results from the Improved-CNN's dilated convolution and residual block accommodation of wide receptive fields reduced the vanishing gradient problem. By capturing temporal links in both directions, Bi-LSTM networks help to better grasp complicated energy use patterns. Graph neural networks improve predictive capacities across linked systems by characterizing the spatial relationships between energy-consuming units in smart cities. Emphasizing critical trends to guarantee reliable forecasts, transformer models use attention methods to manage long-term dependencies in energy consumption data. Combining CNN, Bi-LSTM, Transformer and GNN component predictions in a Fusion Layer synthesizes numerous data representations to increase accuracy. With Root Mean Square Error of 5.7532 Wh, Mean Absolute Percentage Error of 3.5001%, Mean Absolute Error of 6.7532 Wh and R2 of 0.9701, the hybrid model fared better than other models on the ‘Electric Power Consumption’ Kaggle dataset. This work develops a realistic model that helps informed decision-making and enhances energy efficiency techniques, promoting energy load forecasting in smart cities.

Predicting Financial Performance in the IT Industry with Machine Learning: ROA and ROE Analysis

IT is recognized as the engine of the digital world. The fact that this technology has multiple sub-sectors makes it the driving force of the economy. With these characteristics, the sector is becoming the center of attention of investors. Considering that investors prioritize profitability, it becomes a top priority for managers to make accurate and reliable profitability forecasts. The aim of this study is to estimate the profitability of IT sector firms traded in Borsa Istanbul using machine learning methods. In this study, the financial data of 13 technology firms listed in the Borsa Istanbul Technology index and operating between March 2000 and December 2023 were used. Return on assets (ROA) and return on equity (ROE) were estimated using machine learning methods such as neural networks, multiple linear regression and decision tree regression. The results obtained reveal that the performance of artificial neural networks (ANN) and multiple linear regression (MLR) are particularly effective.

Physics-informed reinforcement learning for probabilistic wind power forecasting under extreme events

With wind power penetration increases, accurate and reliable wind power forecasting is becoming gradually critical, and data driven model is a promising solution to implement this task. However, limited by deficient data samples under extreme conditions, scarcity of feature measurements fails to meet the number of training samples required, making the forecasting model exhibits low adaptability. This paper proposes a physics-informed reinforcement learning based method for probabilistic forecasting. Analytical physical expression of wind power output under extreme event is established to construct the error evaluation function for abrupt feature. Deep deterministic policy gradient-based quantile fitting model is then proposed, with abrupt feature embedded as the auxiliary input data for neural network. On this basis, parameter training technique under small data set is proposed to deal with the effect of extreme conditions on correction process of network. It extracts key historical transitions from experience replay pool to establish effective training samples, and model parameters are updated through the small-batch learning strategy to minimize long-term error feedback. Test result on the practical cold wave event of wind farms in China shows effectiveness of the proposed method.