Individual Forecasts Research Articles

Collective intelligence improves probabilistic diagnostic assessments.

Collective intelligence, the "wisdom of the crowd," seeks to improve the quality of judgments by aggregating multiple individual inputs. Here, we evaluate the success of collective intelligence strategies applied to probabilistic diagnostic judgments. We compared the performance of individual and collective intelligence judgments on two series of clinical cases requiring probabilistic diagnostic assessments, or "forecasts". We assessed the quality of forecasts using Brier scores, which compare forecasts to observed outcomes. On both sets of cases, the collective intelligence answers outperformed nearly every individual forecaster or team. The improved performance by collective intelligence was mediated by both improved resolution and calibration of probabilistic assessments. In a secondary analysis looking at the effect of varying number of individual inputs in collective intelligence answers from two different data sources, nearly identical curves were found in the two data sets showing 11-12% improvement when averaging two independent inputs, 15% improvement averaging four independent inputs, and small incremental improvements with further increases in number of individual inputs. Our results suggest that the application of collective intelligence strategies to probabilistic diagnostic forecasts is a promising approach to improve diagnostic accuracy and reduce diagnostic error.

Taylor expansion of the correlation metric for an individual forecast evaluation and its application to East Asian sub-seasonal forecasts

This study develops a skill evaluation metric for an individual forecast by applying a Taylor expansion to the commonly-used temporal correlation skill. In contrast to other individual forecast evaluation metrics, which depend on the amplitude of forecasted and observed anomalies, the so-called “association strength (AS) skill” is less affected by the anomaly amplitude and mainly depends on the degree of similarity between the forecasted and the observed values. Based on this newly developed index, the forecast skill is evaluated for an individual case, then, a group is categorized with respect to the AS skill. The cases with the highest AS skill exhibit the highest correlation skill than any group randomly selected, indicating that the AS skill is a powerful metric to evaluate the non-dimensionalized forecast skill. This strategy is adopted for the subseasonal East Asian summer precipitation forecasts produced by the UK Met Office’s ensemble Global Seasonal forecast system version 5 (GloSea5). In the group with the highest AS skill of the East Asian summer precipitation index (i.e., highest AS cases), the geopotential height anomalies showed quasi-stationary Rossby waves from the North Atlantic to East Asia. The spatial distribution of the dominant subseasonal anomalies for cases with the highest AS is distinct from the cases or groups with the lowest AS skill. Furthermore, the dominant pattern with the highest AS is not solely explained by any well-known typical subseasonal climate patterns, such as the Madden–Julian Oscillation, circumglobal teleconnection pattern, Pacific-Japan pattern, or the Summer North Atlantic Oscillation. This implies that the excitation of well-known climate patterns only partly contributes to increasing the mid-latitude climate predictability in the GloSea5.

Is Single Forecast Method Better than Combined Forecast Method?

Many studies have been done to prove that combining forecast methods gives a better predictive performance relative to individual forecasts. This paper compared the single forecast method and the combined methods in predicting time series data. The study used annual oil revenue for the period 1981–2019 from the Central Bank of Nigeria (CBN), which were divided into two sets: the Training Set (TS) which covered the period 1981–2010 and the Test Set (VS) which covered 2011–2019. The study adopted autoregressive integrated moving average (ARIMA), simple exponential smoothing (SES), and Holt’s linear trend (Holt) as the individual forecast methods; it also adopted outperformance of forecasts (OPF) and weighted mean (WM) as weight selection methods. The forecast methods were applied to the Training Set after which they were combined. Two combined methods CM1 (ARIMA + SES) and CM2 (ARIMA + SES + Holt) were obtained. The result of this study showed that simple exponential smoothing (SES) as an individual forecast method is better and less risky than the combined methods for forecasting time series.

A Hybrid Short-Term Load Forecasting Approach for Individual Residential Customer

This article proposes a hybrid method (HM) to improve the accuracy of short-term individual residential load forecasting. The HM includes an ensemble model (EM), deep ensemble model (DEM), and thermal dynamic model expressed by resistance-capacitance (RC). The EM consists of three predictors of support vector machine (SVM), back propagation neural network (BPNN), and generalized regression neural network (GRNN). The genetic algorithm (GA) is used to optimize SVM and BPNN to enhance their performance. The DEM includes multiple bi-directional long-short term memory (Bi-LSTM) networks. The Bayesian algorithm (BA) is used to optimize the hyperparameters of the Bi-LSTM. The outputs of individual predictors are aggregated using an optimal trimmed algorithm. At first, the total load is separated into the heater and air conditioning (HAC), and non-HAC loads. Then, the RC model is presented to predict the indoor temperature, which integrates outdoor weather and less HAC historical data as the input of the EM to forecast the HAC load. After that, non-HAC loads are further divided into electric lighting and other loads. A daylight equation is used to calculate the illuminance, which is combined with less lighting historical data as the input of DEM to predict electric lights usage. Then, other loads are captured by DEM through less historical data. Finally, the total load is obtained by combining the predicted HAC and non-HAC loads. The datasets from the UMass Smart Microgrid and Flexhouse projects are used to test the proposed method. The comparison with existing models proves that the presented model can provide accurate short-term individual load forecasting.

Maximizing the forecasting skill of an ensemble model

SUMMARYAn ensemble model integrates forecasts of different models (or different parametrizations of the same model) into one single ensemble forecast. This procedure has different names in the literature and is approached through different philosophies in theory and practice. Previous approaches often weighted forecasts equally or according to their individual skill. Here we present a more meaningful strategy by obtaining weights that maximize the skill of the ensemble. The procedure is based on a multivariate logistic regression and exposes some level of flexibility to emphasize different aspects of seismicity and address different end users. We apply the ensemble strategy to the operational earthquake forecasting system in Italy and demonstrate its superior skill over the best individual forecast model with statistical significance. In particular, we highlight that the skill improves when exploiting the flexibility of fitting the ensemble, for example using only recent and not the entire historical data.

Adaptive individual residential load forecasting based on deep learning and dynamic mirror descent

With a growing penetration of renewable energy generation in the modern power networks, it has become highly challenging for network operators to balance electricity supply and demand. Residential load forecasting nowadays plays an increasingly important role in this aspect and facilitates various interactions between power networks and electricity users. While numerous research works have been proposed targeting at aggregate residential load forecasting, only a few efforts have been made towards individual residential load forecasting. The issue of volatility of individual residential load has never been addressed in forecasting. Thus, to fill this gap, this paper presents a deep learning method empowered with dynamic mirror descent for adaptive individual residential load forecasting. The proposed method is evaluated on a real-life Irish residential load dataset, and the experimental results show that it improves the prediction accuracy by 9.1% and 11.6% in the aspects of RMSE and MAE respectively in comparison with a benchmark method.

Do Analysts Learn from Each Other? Evidence from Analysts’ Location Diversity

Consistent with the idea that some of the noise in analysts' earnings forecasts originates in their geographic locations, we find that when analysts' locations are geographically more dispersed, the consensus forecast is more accurate, suggesting a diversification effect. Importantly, analysts' individual forecasts are also more accurate, implying that analysts incorporate idiosyncratic (private) information in their peer's forecasts when generating their own forecasts. Moreover, in line with efficient weighted average forecasting behavior, the weights assigned to peer forecasts vary with measures of the precision of the analyst's signal and those of the peers. Overall, we find strong evidence of analyst learning.

A Scalable Ensemble Approach to Forecast the Electricity Consumption of Households

Long-term individual household forecasting is useful in various applications, e.g., to determine customers’ advance payments. However, the literature on this type of forecasting is limited; existing methods either focus on short-term predictions for individual households, or long-term prediction at an aggregated level (e.g., neighborhood). To fill this gap, we present a method that predicts the monthly consumption of individual households over the next year, given only a few months of consumption data during the current year. Utility companies can exploit this method to predict the consumption of any customer for the next year even with incomplete data. The method consists of three steps: clustering the data using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${k}$ </tex-math></inline-formula> -means, prediction using an ensemble of forecasts based on the historical median distribution among similar households, and smoothing the predictions to remove weather-dependent patterns. The method is highly accurate as it finished third in the IEEE-CIS competition (and ranks first when leveraging insights from another team), focused on forecasting long-term household consumption with incomplete data. It is also very scalable thanks to its low computational complexity and weak data requirements: the method only requires a few months of historical data and no household-specific or weather information.

A hybrid-extreme learning machine based ensemble method for online dynamic security assessment of power systems

• A hybrid model is developed, blending ELM and Levenberg-Marquardt. • Particle swarm optimization-based weighted averaging ensemble is developed. • Energy margin and time margin are calculated for better application of techniques. • Hybrid ELM is found most suited technique for online DSA application. This manuscript develops a new hybrid-extreme learning machine (ELM) based ensemble model for real-time dynamic security assessment (DSA) of power systems. In order to boost the forecasting accuracy of ELM algorithm, a Levenberg-Marquardt (LM) backpropagation algorithm is used. The Ensemble strategy takes advantage of complementary information to improve the generalization capability of model. To realize an accurate ensemble model, particle swarm optimization (PSO) based weighted averaging is used to combine the individual forecasts. Transient energy function (TEF) terms, generator real and reactive power outputs and fault location are taken as input features for classification as well as prediction of transient energy margin (TEM) and time margin (TM) for a given operating condition and fault location. The potential of the proposed technique is validated on New England 39-bus and 68-bus test power systems. The results demonstrate that the developed model outperforms other state-of-the-art methods.

A Forecast Cycle–Based Evaluation for Tropical Cyclone Rapid Intensification Forecasts by the Operational HWRF Model

Abstract An evaluation framework for tropical cyclone rapid intensification (RI) forecasts is introduced and applied to evaluate the performance of RI forecasts by the operational Hurricane Weather Research and Forecasting (HWRF) Model. The framework is based on the performance of each 5-day forecast cycle, while the conventional RI evaluation is based on the statistics of successful or false RI forecasts at individual lead times. The framework can be used to compare RI forecasts of different cycles, which helps model developers and forecasters to characterize RI forecasts under different scenarios. It also can provide the evaluation of statistical performance in the context of 5-day forecast cycles. The RI forecast of each cycle is assessed using a modified probability-based approach that takes the absolute errors in intensity changes into account. The overall performance of RI forecasts during a given period is assessed based on the fractions of the individual forecast cycles during which RI events are successfully or falsely predicted. The framework is applied to evaluate the performance of RI forecasts by the HWRF Model for the whole life cycle of a single hurricane, as well as for each of the hurricane seasons from 2009 to 2021. The metric based on the probabilities of detection and false alarm rate of RI is compared with that based on the absolute errors in the intensity and intensity change during RI events. Significance Statement An evaluation framework for tropical cyclone rapid intensification (RI) forecasts is introduced, focusing on the performance of RI forecasts in each 5-day forecast cycle. The cycle-based approach can help to characterize RI forecasts under different conditions such as certain synoptic scenarios, initial conditions, or vortex structures. It also can be used to assess the overall performance of RI forecasts in terms of the percentages of individual forecast cycles that successfully or falsely predict RI events.

Forecasting VaR and ES using the joint regression combined forecasting model in the Chinese stock market

PurposeThis paper aims to better jointly estimate Value at Risk (VaR) and expected shortfall (ES) by using the joint regression combined forecasting (JRCF) model.Design/methodology/approachCombining different forecasting models in financial risk measurement can improve their prediction accuracy by integrating the individual models’ information. This paper applies the JRCF model to measure VaR and ES at 5%, 2.5% and 1% probability levels in the Chinese stock market. While ES is not elicitable on its own, the joint elicitability property of VaR and ES is established by the joint consistent scoring functions, which further refines the ES’s backtest. In addition, a variety of backtesting and evaluation methods are used to analyze and compare the alternative risk measurement models.FindingsThe empirical results show that the JRCF model outperforms the competing models. Based on the evaluation results of the joint scoring functions, the proposed model obtains the minimum scoring function value compared to the individual forecasting models and the average combined forecasting model overall. Moreover, Murphy diagrams’ results further reveal that this model has consistent comparative advantages among all considered models.Originality/valueThe JRCF model of risk measures is proposed, and the application of the joint scoring functions of VaR and ES is expanded. Additionally, this paper comprehensively backtests and evaluates the competing risk models and examines the characteristics of Chinese financial market risks.

Benchmark of Electricity Consumption Forecasting Methodologies Applied to Industrial Kitchens

Even though Industrial Kitchens (IKs) are among the highest energy intensity spaces, very little work has been done to forecast their consumption. This work explores the possibility of increasing the accuracy of the consumption forecast in an IK by forecasting disaggregated appliance consumption and comparing these results with the forecast of the total consumption of these appliances (Virtual Aggregate—VA). To do so, three different methods are used: the statistical method (Prophet), classic Machine Learning (ML) method such as random forest (RF), and deep learning (DL) method, namely long short-term memory (LSTM). This work uses individual appliance electricity consumption data collected from a Portuguese restaurant over a period of four consecutive weeks. The obtained results suggest that Prophet and RF are the more viable options. The former achieved the best performance in aggregated data, whereas the latter showed better forecasting results for most of the individual loads. Regarding the performance of the VA against the sum of individual appliance forecasts, all models perform better in the former. However, the very small difference across the results shows that this is a viable alternative to forecast aggregated consumption when only individual appliance consumption data are available.

Predicting power ramps from joint distributions of future wind speeds

Abstract. Power ramps are sudden changes in turbine power and must be accurately predicted to minimize costly imbalances in the electrical grid. Doing so requires reliable wind speed forecasts, which can be obtained from ensembles of physical numerical weather prediction (NWP) models through statistical postprocessing. Since the probability of a ramp event depends jointly on the wind speed distributions forecasted at multiple future times, these postprocessing methods must not only correct each individual forecast but also estimate the temporal dependencies among them. Typically though, crucial dependencies are adopted directly from the raw ensemble, and the postprocessed forecast is limited to the tens of members computationally feasible for an NWP model. We extend statistical postprocessing to include temporal dependencies using novel multivariate Gaussian regression models that forecast 24-dimensional distributions of next-day hourly wind speeds at three offshore wind farms. The continuous joint distribution forecast is postprocessed from an NWP ensemble using flexible generalized additive models for the components of its mean vector μ and for parameters defining the forecast error covariance matrix Σ. Modeling these parameters on predictors which characterize the empirical joint distribution of the NWP ensemble allows forecasts for each hour and their temporal dependencies to be adjusted in one step. Wind speed ensembles of any size can be simulated from the postprocessed joint distribution and transformed into power for computing high-resolution ramp predictions that outperform state-of-the-art reference methods.

Predicting the future with humans and AI

AbstractWe review the classic clinical versus statistical prediction debate as well as related modern work on humans versus. algorithms. Despite the successes of statistical prediction over clinical prediction, there is still widespread resistance to algorithms. We discuss recent attempts to understand that resistance. Current research focuses on when people use algorithmic predictions, how people perceive algorithms, and how algorithms can be made more appealing. We also examine attempts to boost human forecasting accuracy, either by spotting talent, cultivating talent via training, or developing algorithms that aggregate individual forecasts. We hypothesize that hybrid models with both human and algorithmic predictions may encounter less resistance than algorithms alone, especially when the algorithm is “humanized” (with anthropomorphic features) and the human is “algorithmized” (by reducing noise, decreasing bias and increasing signal).

A multi-scale forecasting model for CPI based on independent component analysis and non-linear autoregressive neural network

According to excellent performance on forecasting future trends based on historical observations, time series prediction has attracted considerable attention of many researchers. Consumer price index (CPI), as one of the indispensable indicators of macro economy, occupies a significant position in the national economic system. Arguably, the nonlinearity and nonstationarity of CPI series lead to poor accuracy of traditional prediction methods, from which the improvement of prediction performance is perceived as a challenging task. In this paper, a multi-scale forecasting technique for CPI is proposed, which is the hybrid of complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), hierarchical agglomerative clustering (HAC), independent component analysis (ICA) and non-linear autoregressive (NAR) neural network. Firstly, the original time series is decomposed into several adaptive modes via the CEEMDAN. Secondly, HAC is introduced to cluster the modes according to their similarity aiming at performing dimensionality reduction. Thirdly, ICA is utilized to separate independent components (ICs) which implies the hidden information of time series. Finally, NAR neural network is applied to the ICs to obtain the forecasting sequences, and the final forecasting result is the combination of these individual forecasts. Comparative experiment results verify the superiority of the proposed hybrid methodology from the aspect of the performance criterion in CPI forecasting.

Nonadditive tourism forecast combination using grey relational analysis

PurposeForecasting tourism demand accurately can help private and public sector formulate strategic planning. Combining forecasting is feasible to improving the forecasting accuracy. This paper aims to apply multiple attribute decision-making (MADM) methods to develop new combination forecasting methods.Design/methodology/approachGrey relational analysis (GRA) is applied to assess weights for individual constituents, and the Choquet fuzzy integral is employed to nonlinearly synthesize individual forecasts from single grey models, which are not required to follow any statistical property, into a composite forecast.FindingsThe empirical results indicate that the proposed method shows the superiority in mean accuracy over the other combination methods considered.Practical implicationsFor tourism practitioners who have no experience of using grey prediction, the proposed methods can help them avoid the risk of forecasting failure arising from wrong selection of one single grey model. The experimental results demonstrated the high applicability of the proposed nonadditive combination method for tourism demand forecasting.Originality/valueBy treating both weight assessment and forecast combination as MADM problems in the tourism context, this research investigates the incorporation of MADM methods into combination forecasting by developing weighting schemes with GRA and nonadditive forecast combination with the fuzzy integral.

Optimal forecast combination based on PSO-CS approach for daily agricultural future prices forecasting

Forecasting agricultural commodity prices accurately is a challenging task due to the complexity of the trading market and the variability of influencing factors. Many studies have proven that forecast combination is an effective strategy for improving forecast performance relative to individual forecasting. In the field of forecast combination, how to determine the reasonable weights for combination is still an open question. This study proposed an optimal forecast combination framework for agricultural commodity prices forecasting, which integrates the decomposition–reconstruction–ensemble methodology with an improved nature-inspired global optimization algorithm. The update mechanism of particle swarm optimization (PSO) is introduced to improve cuckoo search (CS), in order to reduce the searching blindness in the huge exploration space. The framework consists of four steps. First, data decomposition using empirical wavelet transform (EWT), singular spectral analysis (SSA), and variational mode decomposition (VMD); Second, component reconstruction via a modified reconstruction approach based on the largest comprehensive grey correlation degree clustering (CGCD); Third, individual forecasting using autoregressive integrated moving average regression (ARIMA), exponential smoothing (ETS), back propagation neural network (BPNN) and extreme learning machine (ELM); Fourth, forecast combination via PSO-CS weight assignment method. Using corn and wheat future prices as research samples, the experimental results demonstrated that: (a) the PSO-CS weight assignment approach is superior to other combination approaches in most cases; (b) the CGCD approach can effectively reduce the computational cost of forecasting and improve the prediction performance; (c) the Full-PSO-CS model provides the most accurate forecast due to the diversity of individual forecasts, it reduces MAPE by 43.66% and improves directional accuracy by 30.80% on average compared with the best single model.

ℓ 2 -Relaxation: With Applications to Forecast Combination and Portfolio Analysis

Abstract We propose ℓ2-relaxation, which is a novel convex optimization problem, to tackle forecast combination with many forecasts or minimum variance portfolio with many assets. ℓ2-relaxation minimizes the squared Euclidean norm of the weight vector subject to a set of relaxed linear inequalities to balance the bias and variance. It delivers optimality with approximately equal within-group weights when latent block equicorrelation patterns dominate the high dimensional sample variance-covariance matrix of the individual forecast errors or the assets. Its wide applicability is highlighted in three real data examples in microeconomics, macroeconomics, and finance.

Using shapely values to define subgroups of forecasts for combining

AbstractThis paper proposes an algorithm based on the Shapley value method to select a superior subset from a group of multiple individual forecasts. The Shapley value of a forecast in a combination is an indicator that describes the contribution of the forecast to the combination, which is expressed as the function of four factors in this paper: The accuracy of the forecast, the average accuracy of other forecasts in the combination, the average degree of diversity between the forecast and other forecasts in the combination, and the average degree of diversity between any other two forecasts in the combination. Based on the Shapley value method, the order of different forecasts entering the screening process is determined and the selection algorithm is continuously executed until all the redundant forecasts have been removed from the combination. The effectiveness, superiority, and robustness of the proposed algorithm have been verified via three experiments. The results of empirical studies indicate that the combination of individual forecasts from the selected group markedly outperforms the best individual forecast and the combination of all available forecasts.

Observer-based power forecast of individual and aggregated offshore wind turbines

Abstract. Due to the increasing share of wind energy in the power system, minute-scale wind power forecasts have gained importance. Remote-sensing-based approaches have proven to be a promising alternative to statistical methods and thus need to be further developed towards an operational use, aiming to increase their forecast availability and skill. Therefore, the contribution of this paper is to extend lidar-based forecasts to a methodology for observer-based probabilistic power forecasts of individual wind turbines and aggregated wind farm power. To do so, lidar-based forecasts are combined with supervisory control and data acquisition (SCADA)-based forecasts that advect wind vectors derived from wind turbine operational data. After a calibration, forecasts of individual turbines are aggregated to a probabilistic power forecast of turbine subsets by means of a copula approach. We found that combining the lidar- and SCADA-based forecasts significantly improved both forecast skill and forecast availability of a 5 min ahead probabilistic power forecast at an offshore wind farm. Calibration further increased the forecast skill. Calibrated observer-based forecasts outperformed the benchmark persistence for unstable atmospheric conditions. The aggregation of probabilistic forecasts of turbine subsets revealed the potential of the copula approach. We discuss the skill, robustness and dependency on atmospheric conditions of the individual forecasts, the value of the observer-based forecast, its calibration and aggregation, and more generally the value of minute-scale power forecasts of offshore wind. In conclusion, combining different data sources to an observer-based forecast is beneficial in all regarded cases. For an operational use one should distinguish between and adapt to atmospheric stability.