Analysis based on statistical distributions: A practical approach for stochastic solvers using discrete and continuous problems

Abstract

This paper proposes an approach for the analysis and comparison of stochastic solvers based on the statistical distribution of their variables. The observed variables of the stochastic solvers are the runtime and number of function evaluations required to reach a (sub)-optimal solution. These variables were measured using a target approach. We extended the conventional approach, which usually predicts the average value of variables, by predicting their statistical distributions. If possible, we can predict, not only the average value, but also all the values of the observed variables according to the identified distribution and given probability. We can also predict the probability of reaching (sub)-optimal solutions according to the variable's values. The approach was empirically validated by comparing solvers for discrete and continuous problems. In our experiment, the differences between predicted and measured values of runtime, number of function evaluations and probability are 10% or less. This proves that the proposed approach is useful and can be used for the analysis and comparison of stochastic solvers. Although, the approach has some limitations, it can mitigate the issue of stochastic solvers being unable to provide (sub)-optimal solutions, and can be used to determine the stopping optimization criteria.