Stochastic Optimization Methods Research Articles

RETRACTED: Pumped hydro energy storage arbitrage in the day-ahead market in smart grid using stochastic p-robust optimization method

• Individual participation of pumped hydro energy storage (PHES) in day-ahead market. • Use of stochastic optimization to model the day-ahead market price uncertainty. • Stochastic p-robust optimization (SPRO) is proposed to measure the financial risks. • Risk-guaranteed strategy is obtained in comparison with risk-neutral strategy. • Robust bidding and offering strategies for PHES are obtained with SPRO method. Individual operation of pumped hydro energy storage (PHES) as large-scale energy storage needs the bidding and offering curves to participate, as a prosumer, in the day-ahead electricity market. Therefore, in this paper, by taking advantage of stochastic and robust optimization, a new stochastic p-robust optimization (SPRO) method is proposed to deal with the financial risks of the day-ahead price uncertainty. The proposed method compares the profit of PHES with the relative regret to get a risk-controlled strategy. According to obtained results, reducing the robustness parameter (p) from the p=+∞ to p=0.21, the profit, and the maximum relative regret are reduced by $ 2611 and 15.02 %, respectively. In other words, with a 19.11 % decline in profit, the maximum amount of relative regret decreased by 41.30 %. Therefore, it can be shown that by a few reductions in profit, the maximum amount of relative regret decreases significantly. At the same time, the robustness of PHES is guaranteed against the day-ahead market price uncertainty. Finally, the risk-based bidding and offering curves are derived using the results of the proposed SPRO method.

Computer-Aided Design of a Perfluorinated Sulfonic Acid Proton Exchange Membrane Using Stochastic Optimization and Molecular Dynamic Method

In this paper, a computer-aided polymer design (CAPD) framework with a stochastic optimization model is proposed for the design of perfluorinated sulfonic acid proton exchange membrane (PFSA-PEM) w...

Expected complexity analysis of stochastic direct-search

This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It introduces an algorithm designed to optimize differentiable objective functions f whose values can only be computed through a stochastically noisy blackbox. The proposed stochastic directional direct-search (SDDS) algorithm accepts new iterates by imposing a sufficient decrease condition on so called probabilistic estimates of the corresponding unavailable objective function values. The accuracy of such estimates is required to hold with a sufficiently large but fixed probability \(\beta\). The analysis of this method utilizes an existing supermartingale-based framework proposed for the convergence rates analysis of stochastic optimization methods that use adaptive step sizes. It aims to show that the expected number of iterations required to drive the norm of the gradient of f below a given threshold \(\epsilon\) is bounded in \({\mathcal {O}}\left( \epsilon ^{\frac{-p}{\min (p-1,1)}}/(2\beta -1)\right)\) with \(p>1\). Unlike prior analysis using the same aforementioned framework such as those of stochastic trust-region methods and stochastic line search methods, SDDS does not use any gradient information to find descent directions. However, its convergence rate is similar to those of both latter methods with a dependence on \(\epsilon\) that also matches that of the broad class of deterministic directional direct-search methods which accept new iterates by imposing a sufficient decrease condition.

Perceptual hashing method for video content authentication with maximized robustness

Perceptual video hashing represents video perceptual content by compact hash. The binary hash is sensitive to content distortion manipulations, but robust to perceptual content preserving operations. Currently, boundary between sensitivity and robustness is often ambiguous and it is decided by an empirically defined threshold. This may result in large false positive rates when received video is to be judged similar or dissimilar in some circumstances, e.g., video content authentication. In this paper, we propose a novel perceptual hashing method for video content authentication based on maximized robustness. The developed idea of maximized robustness means that robustness is maximized on condition that security requirement of hash is first met. We formulate the video hashing as a constrained optimization problem, in which coefficients of features offset and robustness are to be learned. Then we adopt a stochastic optimization method to solve the optimization. Experimental results show that the proposed hashing is quite suitable for video content authentication in terms of security and robustness.

Modeling a robust multi-objective locating-routing problem with bounded delivery time using meta-heuristic algorithms

Purpose This study aims to investigate a locating-routing-allocating problems and the supply chain, including factories distributor candidate locations and retailers. The purpose of this paper is to minimize system costs and delivery time to retailers so that routing is done and the location of the distributors is located. Design/methodology/approach The problem gets closer to reality by adding some special conditions and constraints. Retail service start times have hard and soft time windows, and each customer has a demand for simultaneous delivery and pickups. System costs include the cost of transportation, non-compliance with the soft time window, construction of a distributor, purchase or rental of a vehicle and production costs. The conceptual model of the problem is first defined and modeled and then solved in small dimensions by general algebraic modeling system (GAMS) software and non-dominated sorting genetic algorithm II (NSGAII) and multiple objective particle swarm optimization (MOPSO) algorithms. Findings According to the solution of the mathematical model, the average error of the two proposed algorithms in comparison with the exact solution is less than 0.7%. Also, the algorithms’ performance in terms of deviation from the GAMS exact solution, is quite acceptable and for the largest problem (N = 100) is 0.4%. Accordingly, it is concluded that NSGAII is superior to MOSPSO. Research limitations/implications In this study, since the model is bi-objective, the priorities of decision makers in choosing the optimal solution have not been considered and each of the objective functions has been given equal importance according to the weighting methods. Also, the model has not been compared and analyzed in deterministic and robust modes. This is because all variables, except the one that represents the uncertainty of traffic modes, are deterministic and the random nature of the demand in each graph is not considered. Practical implications The results of the proposed model are valuable for any group of decision makers who care optimizing the production pattern at any level. The use of a heterogeneous fleet of delivery vehicles and application of stochastic optimization methods in defining the time windows, show how effective the distribution networks are in reducing operating costs. Originality/value This study fills the gaps in the relationship between location and routing decisions in a practical way, considering the real constraints of a distribution network, based on a multi-objective model in a three-echelon supply chain. The model is able to optimize the uncertainty in the performance of vehicles to select the refueling strategy or different traffic situations and bring it closer to the state of certainty. Moreover, two modified algorithms of NSGA-II and multiple objective particle swarm optimization (MOPSO) are provided to solve the model while the results are compared with the exact general algebraic modeling system (GAMS) method for the small- and medium-sized problems.

Online convex combination of ranking models

As a task of high importance for recommender systems, we consider the problem of learning the convex combination of ranking algorithms by online machine learning. First, we propose a stochastic optimization algorithm that uses finite differences. Our new algorithm achieves close to optimal empirical performance for two base rankers, while scaling well with an increased number of models. In our experiments with five real-world recommendation data sets, we show that the combination offers significant improvement over previously known stochastic optimization techniques. The proposed algorithm is the first effective stochastic optimization method for combining ranked recommendation lists by online machine learning. Secondly, we propose an exponentially weighted algorithm based on a grid over the space of combination weights. We show that the algorithm has near-optimal worst-case performance bound. The bound provides the first theoretical guarantee for non-convex bandits using limited number of evaluations under very general conditions.

RETRACTED: Integration of hydrogen storage system and wind generation in power systems under demand response program: A novel p-robust stochastic programming

Penetration of renewable energy sources (RESs) in power systems increase all over the world to overcome current challenges, most importantly environmental issues. Beside advantages of RESs, their integration into the power systems have imposed various challenges considering uncertain and intermitted power output. To cope with these challenges, utilizing energy storage systems with renewable energy sources alongside the demand response (DR) programs are considered as reliable solutions. On the other hand, in an uncertain environment, minimizing worst-case cost or regret is counted as an important criterion to evaluate operation of any system under uncertain parameters. Therefore, in this paper, optimal operation of power systems is solved under penetration of wind turbines, hydrogen storage system, and DR programs in an uncertain environment. To guarantee robust operation of the system under the worst-case scenario, a novel stochastic p-robust optimization method (SPROM) is proposed which combines both stochastic programming and robust optimization approaches where minimizes the worst-case cost or regret level. The performance of the developed model is evaluated considering a six-bus test system under two cases as stochastic optimization (SO) and SPROM. Obtained results show that the maximum regret level is reduced considerably using the proposed SPROM comparing with pure stochastic method.

Security-aware stochastic optimization method for operating active distribution networks with resilience enhancement

This paper proposes a stochastic optimal operation method for active distribution networks to improve system resilience. The source–network–load uncertainties are handled with a set of typical scenarios, and the system post-contingency security level is quantitatively evaluated by security distance. The formulated problem determines the on-load tap-changer tap position, the distributed generator (DG) output power, the DG power factor angle, the demand side management participation rate and the switch status to achieve the economic operation solution. The off-the-shelf solver is deployed to solve the proposed mixed-integer nonlinear programming model. Numerical simulations studied on a modified 104-bus distribution network validate the effectiveness and high performance of the presented method. The solutions indicate that the security-aware operation scheme can simultaneously improve system economy and resilience compared with the existing operation model. Moreover, the operation solution accommodates more renewable DGs.

Gaussian process latent variable model factorization for context-aware recommender systems

• We propose a novel algorithm named GPLVMF to achieve a Bayesian factorization for Context-aware recommendation. • We model the non-zero mean in Gaussian processes to capture the bias and provide a generalized variational inference. • Our method can flexibly deal with both real-valued and categorical contexts. • Both scaled conjugate gradient and stochastic gradient descent optimization methods are applied to the model. • Experiment results show that GPLVMF not only improves accuracy but can automatically infer the influence of context. Context-aware recommender systems (CARS) have gained increasing attention due to their ability to utilize contextual information. Compared to traditional recommender systems, CARS are, in general, able to generate more accurate recommendations. The latent factors approach accounts for a large proportion of CARS. Recently, a non-linear Gaussian Process (GP) based factorization method was proven to outperform the state-of-the-art methods in CARS. Despite its effectiveness, standard GP model-based methods can suffer from over-fitting and may not be able to determine the impact of each context automatically. In order to address such shortcomings, we propose a Gaussian Process Latent Variable Model Factorization (GPLVMF) method, where we apply an appropriate prior to the original GP model. Our work is primarily inspired by the Gaussian Process Latent Variable Model (GPLVM), which is a non-linear dimensionality reduction method. Since the traditional maximum likelihood approach for standard GP cannot be applied to our model, we adopt a variational inference method to solve the GPLVMF model. As a result, we improve the results on the real datasets significantly as well as capturing the importance of each context. In addition to the general advantages, our method flexibly provides two contributions regarding recommender system settings: (1) addressing the influence of bias by setting a non-zero mean function, and (2) utilizing real-valued contexts by fixing the latent space with real values.

Stochastic Model and Method of Optimization of Operation Modes of a Group of Artesian Wells in Water Supply Systems

Groups of spatially distributed wells interconnected by sections of pipelines (a common collector) supplying water to a clean water reservoir are used in water supply systems in order to provide drinking water to settlements. A group of wells together with a clean water reservoir is called a water intake facility of the water supply system. Further, water is supplied from the clean water reservoir to consumers in settlements by means of second stage pumping stations. In terms of each well the operating mode of the submersible pumping unit, i.e. the position of its operating point at a fixed moment in time, depends on the actual values ​​of many internal and external factors: dynamic water level in the well; the actual characteristics of the pumping unit; the position of the final control elements (drive speed, opening degree of the regulating valve); hydraulic resistances of pipeline sections of the water intake facility; the water level in the clean water reservoir; the number and operating modes of pumping units in other wells of the water intake facility.
 The paper describes a deterministic model of a water intake well. At the same time, the processes of water consumption are random processes, since they depend on many uncontrolled and unmanageable factors. The parameters of the technological equipment of the water intake facility are also random values, since they are estimated from experimental data of finite length.
 Therefore, a stochastic model of a water intake well is also proposed. A model of a quasi-stationary operation mode of a water intake facility, which is used to optimize the operation of a group of artesian wells operating on a common collector, is considered.

TOFFEE: Task Offloading and Frequency Scaling for Energy Efficiency of Mobile Devices in Mobile Edge Computing

As an emerging computing paradigm, mobile edge computing (MEC) can improve users’ service experience by provisioning the cloud resources close to the mobile devices. With MEC, computation-intensive tasks can be processed on the MEC servers, which can greatly decrease the mobile devices’ energy consumption and prolong their battery lifetime. However, the highly dynamic task arrival and wireless channel states pose great challenges on the computation task allocation in MEC. This paper jointly investigates the task allocation and CPU-cycle frequency, to achieve the minimum energy consumption while guaranteeing that the queue length is upper bounded. We formulate it as a stochastic optimization problem, and with the aid of stochastic optimization methods, we decouple the original problem into two deterministic optimization subproblems. An online Task Offloading and Frequency Scaling for Energy Efficiency (TOFFEE) algorithm is proposed to obtain the optimal solutions of these subproblems concurrently. TOFFEE can obtain the close-to-optimal energy consumption while bounding the applications’ queue length. Performance evaluation is conducted which verifies TOFFEE’s effectiveness. Experiment results indicate that TOFFEE can decrease the energy consumption by about 15 percent compared with the RLE algorithm, and by about 38 percent compared with the RME algorithm.

A zeroth order method for stochastic weakly convex optimization

In this paper, we consider stochastic weakly convex optimization problems, however without the existence of a stochastic subgradient oracle. We present a derivative free algorithm that uses a two point approximation for computing a gradient estimate of the smoothed function. We prove convergence at a similar rate as state of the art methods, however with a larger constant, and report some numerical results showing the effectiveness of the approach.

Examination of the Chance Constrained Optimal WT Penetration Level in Distorted Distribution Network with Wind Speed and Load Uncertainties

Harmonic penetration can be problematic by the growing interconnection of Wind Turbines (WTs) in distribution networks. Since the active power outputs of WTs and loads in the distribution system have uncertainties, the optimal WT penetration level problem can be considered to have a stochastic nature. In this study, this problem is taken into account in the stochastic optimization method with the consideration of uncertainties in wind speed and distribution network load profile. Chance constraint programming is taken into account in the determination of optimal WT penetration levels by applying the Genetic Algorithm (GA) along with Monte Carlo Simulation (MCS). The harmonic power flow analysis based on the decoupled harmonic load flow approach is employed in the distorted distribution network. Chance constraints are considered for the harmonic issues such as the Total Harmonic Distortion of Voltage (VTHD), Individual Harmonic Distortion of Voltage (VIHDh), and Root Mean Square of Voltage (VRMS).

On the modelling of the anhysteretic magnetization of homogeneous soft magnetic materials

The analysis and optimization of devices with soft magnetic cores, such as inductive sensors and electric machines, rely on a simple and precise modelling of their nonlinear anhysteretic magnetization. The Langevin-Weiss equation of state has the advantages of being physically-based and capable of accurately describing isotropic materials. In fact, it has even been adopted as the backbone of the popular Jiles-Atherton model to describe magnetic hysteresis. However, many studies show the difficulty of retrieving the modelling parameters given an experimental anhysteretic curve, for which stochastic optimization methods are generally used.In this work, we present a simple and deterministic method for finding the parameters of the Langevin-Weiss model: the saturation magnetization, the temperature dependent a parameter, and the molecular field constants. To this end, we optimize a problem with a nonlinear solver where the fitting variables are instrinsically bounded. We validated the approach by analyzing three widely used soft magnets; a grain-oriented silicon steel, a Mn-Zn soft ferrite, and a Fe-based nanocrystalline alloy with induced anisotropy.We demonstrate that, contrary to what has previously been assumed, the Langevin-Weiss function can provide a good description of the anhysteretic magnetization of any homogeneous material, even materials with strong transverse uniaxial anisotropy. The key, which seems to have been overlooked, is to allow the Weiss molecular field to be negative.

Global optimisation in Hilbert spaces using the survival of the fittest algorithm

Global optimisation problems in high-dimensional and infinite-dimensional spaces arise in various real-world applications such as engineering, economics and finance, geophysics, biology, machine learning, optimal control, etc. Among stochastic approaches to global optimisation, biology-inspired methods are currently very popular in the literature. Bio-inspired approaches imitate natural ecological and evolutionary processes and are reported to be efficient in a large number of practical study cases. However, many bio-inspired methods can possess some vital drawbacks. For example, due to their semi-empirical nature, convergence to the globally optimal solution cannot always be guaranteed. Another major obstacle is that the existing methods often struggle with higher dimensionality of the space of parameters, which results in a slow convergence. Moreover, it is often difficult to adjust the dimensionality of the space of parameters in the corresponding computer code for a practical realisation of the optimisation method. Here, we present a bio-inspired global stochastic optimisation method applicable in Hilbert function spaces. The proposed method is an evolutionary algorithm inspired by the Darwin’s famous idea of survival of the fittest and is, therefore, referred to as the ‘Survival of the Fittest Algorithm’ (SoFA). Mathematically, the convergence of SoFA is a consequence of the fundamental property of localisation of probabilistic measure in a Hilbert space and we rigorously prove the convergence of the introduced algorithm for a generic class of functionals. The approach is simple in terms of practical coding. As an insightful, real-world problem, we apply our method to find the globally optimal trajectory for the daily vertical migration of zooplankton in the ocean and lakes, this phenomenon is considered to be the largest synchronised movement of biomass on Earth. We maximise fitness in a function space derived from a von-Foerster stage-structured population model with biologically realistic parameters. We show that for problems of fitness maximisation in high-dimensional spaces, SoFA provides better performance as compared to some other stochastic global optimisation algorithms. We highlight the links between the new optimisation algorithm and natural selection processes in ecosystems occurring within a population via gradual exclusion of competitive con-specific strains.

Uniform error estimates for artificial neural network approximations for heat equations

Abstract Recently, artificial neural networks (ANNs) in conjunction with stochastic gradient descent optimization methods have been employed to approximately compute solutions of possibly rather high-dimensional partial differential equations (PDEs). Very recently, there have also been a number of rigorous mathematical results in the scientific literature, which examine the approximation capabilities of such deep learning-based approximation algorithms for PDEs. These mathematical results from the scientific literature prove in part that algorithms based on ANNs are capable of overcoming the curse of dimensionality in the numerical approximation of high-dimensional PDEs. In these mathematical results from the scientific literature, usually the error between the solution of the PDE and the approximating ANN is measured in the $L^p$-sense, with respect to some $p \in [1,\infty )$ and some probability measure. In many applications it is, however, also important to control the error in a uniform $L^\infty $-sense. The key contribution of the main result of this article is to develop the techniques to obtain error estimates between solutions of PDEs and approximating ANNs in the uniform $L^\infty $-sense. In particular, we prove that the number of parameters of an ANN to uniformly approximate the classical solution of the heat equation in a region $ [a,b]^d $ for a fixed time point $ T \in (0,\infty ) $ grows at most polynomially in the dimension $ d \in {\mathbb {N}} $ and the reciprocal of the approximation precision $ \varepsilon> 0 $. This verifies that ANNs can overcome the curse of dimensionality in the numerical approximation of the heat equation when the error is measured in the uniform $L^\infty $-norm.

OC-0420 A novel stochastic optimization method for handling uncertainty in combined proton-photon treatments

OC-0420 A novel stochastic optimization method for handling uncertainty in combined proton-photon treatments

A reconsideration on the resolution of phase stability analysis using stochastic global optimization methods: Proposal of a reliable set of benchmark problems

Phase stability analysis is an important thermodynamic calculation in the context of process systems engineering of multiphase units (e.g., separation systems, reactors). It implies the minimization of the Tangent plane distance function (TPDF), which has been recognized as a challenging global optimization problem. Despite the importance of this thermodynamic calculation, the characterization and classification of resolution complexity of phase stability problems that are commonly used to test and compare global optimization methods have not been addressed in literature. This paper reports the first step to propose a set of benchmark TPDF problems to assess and identify the advantages and limitations of stochastic optimization methods used in phase stability calculations. A detailed review of TPDF problem reported in literature was performed and the most representative ones were studied and classified in terms of their resolution complexity via the analysis of several performance metrics associated to both reliability and efficiency to find the global TPDF minimum. Differential Evolution, Simulated Annealing, Harmony Search, Tabu Search, Particle Swarm Optimization and Genetic Algorithm were selected as the metaheuristics to perform the global minimization of selected TPDF problems. 35 phase stability problems were classified in low, medium and high difficulty TPDF problems. Results showed that, although a wide variety of phase stability problems has been reported and used for phase stability calculations, a significant number of them corresponded to global optimization problems of easy resolution. Therefore, some authors have reported biased conclusions on the performance of stochastic optimization methods. A set of 23 TPDF was proposed as benchmark problems to obtain a reliable analysis of the numerical performance of metaheuristics employed in phase stability calculations. This benchmark set contains problems with different resolution difficulties and characteristics that are considered appropriate to study and assess new methods to resolve the global TPDF optimization, as well as to improve the existing methods. This paper highlights the relevance of a continuous development and improvement of stochastic global optimizers for solving, robustly and efficiently, the phase stability analysis in multicomponent systems. TPDF minimization can be still considered a challenging problem to be faced in the context of applied thermodynamics.

Stochastic model and method of optimization of operating modes groups of artesian wells in water supply systems

Water supply of large cities is carried out, as a rule, from open reservoirs - lakes, reservoirs, large rivers, ie from sources with significant water reserves. In this case, the water supply scheme includes: pumping stations of the first rise, which take water directly from the water; water treatment system (purification); pumping stations of the second and third rise, pumping clean water over long distances and supplying it to settlements [1]. If the settlement is located in an area where there are no large open water sources, or the geographical location of the settlement allows the use of groundwater, in which case artesian wells are used as a source of water supply. Usually small settlements are supplied with water from artesian wells. Artesian wells are widely used for water supply of villages, industrial water supply, for irrigation (irrigation systems), in the mining industry, to reduce the level of groundwater. The advantage of using artesian wells to supply the population with drinking water is that the water is extracted from great depths and purified using natural filters [2]. Thus, the cost of water treatment is significantly reduced. The disadvantage of using artesian wells is usually low productivity of the well, the strong dependence of water volumes in the well on climatic and weather conditions, depletion of the well as a result of large-scale construction or extraction of natural minerals such as shale gas [3].

Stochastic security-constrained optimal power flow for a microgrid considering tie-line switching

With the rapid development of microgrid, its tie-line switching from grid-connected to islanded mode is a topic worth discussing for considering both main grid resilience and microgrid security. In this paper, a stochastic security-constrained optimal power flow (OPF) method is proposed to deal with these conditions under high uncertainties. Firstly, a linear load flow model and a backward forward sweep algorithm are applied to present microgrid power flow with reduced computing burdens. Secondly, with consideration of tie-line switching from grid-connected to islanded operation mode, a security-constrained OPF problem for a microgrid is proposed to minimize operating cost and by optimizing microturbine setpoints and load shedding coefficient. To promise stable islanded operation after disconnection from the main grid, a Benders decomposition method is developed to decouple the OPF problem into a grid-connected master problem and an islanded sub-problem and then solve them iteratively with Benders cuts to guarantee microgrid security after tie-line switching. Last, a stochastic optimization method with probabilistic modelling is adopted to address the uncertainty issue caused by renewable energy sources and loads. The proposed stochastic security-constrained OPF method has been verified with high computing efficiency and robust security via comprehensive numerical simulations.