Stochastic Optimization Methods Research Articles

Bayesian statistics approach to chess engines optimization

We develop a new method for stochastic optimization using the Bayesian statistics approach. More precisely, we optimize parameters of chess engines as those data are available to us, but the method should apply to all situations where we want to optimize a certain gain/loss function which has no analytical form and thus cannot be measured directly but only by comparison of two parameter sets. We also experimentally compare the new method with the famous SPSA method.

Local search enhanced Aquila optimization algorithm ameliorated with an ensemble of Wavelet mutation strategies for complex optimization problems

Aquila Optimization Algorithm (AQUILA) is a newly emerged metaheuristic optimizer for solving global optimization problems, which is based on intrinsic hunting behaviors of the foraging aquila individuals. However, this stochastic optimization method suffers from some algorithm-specific drawbacks, such as premature convergence to the local optimum points over the search hyperspace due to the lack of solution diversity in the population. To conquer this algorithmic deficiency, an ensemble of Wavelet mutation operators has been implemented into the standard AQUILA to enhance the explorative capabilities of the algorithm by diversifying the search domain as much as possible. Furthermore, a brand-new local search scheme empowered by the synergetic interactions of elite opposition-based learning and a simple-yet-effective exploitative manipulation equation is introduced into the base AQUILA to intensify on the previously visited promising regions. The proposed learning schemes are stochastically applied to the obtained solutions from the base Aquila algorithm to refine the overall solution quality and amend the premature convergence problem. It is also aimed to investigate whether the collective application of Wavelet mutation operators with different types entails a significant improvement in the general search effectivity of the algorithm rather than their individual efforts. Numerical experiments made on a suite of unconstrained unimodal and multimodal benchmark functions reveal that this hybridization with AQUILA has improved the general solution accuracy and stability to very high standards, outperforming its contemporary counterparts in the comparative statistical analysis. Furthermore, an exhaustive benchmark analysis has been performed on fourteen constrained real-world complex engineering problems.

Cascade refrigeration system synthesis based on hybrid simulated annealing and particle swarm optimization algorithm

Cascade refrigeration system (CRS) can meet a wider range of refrigeration temperature requirements and is more energy efficient than single-refrigerant refrigeration system, making it more widely used in low-temperature industry processes. The synthesis of a CRS with simultaneous consideration of heat integration between refrigerant and process streams is challenging but promising for significant cost saving and reduction of carbon emission. This study presented a stochastic optimization method for the synthesis of CRS. An MINLP model was formulated based on the superstructure developed for the CRS, and an optimization framework was proposed, where simulated annealing algorithm was used to evolve the numbers of pressure/temperature levels for all sub-refrigeration systems, and particle swarm optimization algorithm was employed to optimize the continuous variables. The effectiveness of the proposed methodology was verified by a case study of CRS optimization in an ethylene plant with 21.89% the total annual cost saving.

Risk analysis framework for the optimum remediation of a contaminated aquifer under uncertainty: application in Lake Karla aquifer, Thessaly, Greece

A risk analysis framework is proposed for the optimum remediation of a contaminated aquifer under hydrogeological uncertainty. The limited information and the spatial variation of hydraulic conductivity in a real-world large-scale aquifer create uncertain conditions for decision-making when remediation schemes ought to be accompanied by the minimum possibility of failure. The primary concern is focused on safeguarding public health when groundwater is used for urban drinking purposes from a contaminated aquifer. The proposed framework is based on the conjunctive use of stochastic simulation–optimization modelling followed up by a risk analysis application on remediation trade-offs. The framework includes three main steps/procedures: (i) the model formulation of multiple realizations of groundwater flow and contaminant transport, (ii) the optimal positioning and operation of the clean-up wells determined by the method of stochastic optimization, and (iii) the risk analysis of the optimum remediation strategies through a proposed decision model, so as the one with the minimum cost and risk of failure is chosen as the most appropriate. The proposed framework is tested for two scenarios of nitrogen fertilizer application in the cultivated areas. The strategic target is the groundwater nitrate concentration minimization in an area where exceedances of nitrate concentrations have been observed and water supply wells have been operating for the last twenty years satisfying domestic needs. The results demonstrate that, when decision-making is under hydrogeological uncertainty, the combined use of stochastic optimization and risk-based decision analysis can commend the remediation strategy with the minimum cost and the highest possibility of success.

SABRINA: A Stochastic Subspace Majorization-Minimization Algorithm

A wide class of problems involves the minimization of a coercive and differentiable function F on $${\mathbb {R}}^N$$ whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustness to errors in the gradient is not necessarily guaranteed. This work is dedicated to investigating the convergence of Majorization-Minimization (MM) schemes when stochastic errors affect the gradient terms. We introduce a general stochastic optimization framework, called StochAstic suBspace majoRIzation-miNimization Algorithm SABRINA that encompasses MM quadratic schemes possibly enhanced with a subspace acceleration strategy. New asymptotical results are built for the stochastic process generated by SABRINA. Two sets of numerical experiments in the field of machine learning and image processing are presented to support our theoretical results and illustrate the good performance of SABRINA with respect to state-of-the-art gradient-based stochastic optimization methods.

Reducing external container trucks’ turnaround time in ports: A data-driven approach under truck appointment systems

A longer turnaround time of container trucks in ports are commonly related to a higher energy cost and more carbon emissions, hindering the green development goal of ports. In this context, the implementation of Truck appointment system (TAS) is widely recognized as a very effective way to reduce external container trucks’ total turnaround time in ports. However, the number of appointment quotas allocated for each time window are the key to determine the efficiency of a TAS. This study proposed a data-driven approach to address the appointment quota optimization problem, which combined data mining technologies with mathematical optimization modeling methods, and an empirical study in YT port, China was given. Specifically, the smart gate data of the port were mined to explore the causal relationship between the number of truck arrivals at each time window and their total turnaround time within the port. The results exhibited a non-deterministic quadratic function relationship, and the range of changes regarding total turnaround time were measured by boundary functions obtained through a stepwise approaching algorithm. Considering the obtained causal relationship and its uncertainties, a robust optimization model was built to find the optimal appointment quota plan by minimizing the total turnaround time of external container trucks and the total deviations from the trucks’ preferred arrival time in the worst case. A numerical experiment was designed to examine the validity of the proposed data-driven approach, and compare the effectiveness of the proposed approach with a conventional stochastic optimization method. The results indicated the proposed approach was more effective on reducing the external container trucks’ total turnaround time in ports under a Truck appointment system.

Biomechanical Trajectory Optimization of Human Sit-to-Stand Motion With Stochastic Motion Planning Framework

Trajectory optimization has been an important approach in biomechanics for the analysis and prediction of the limb movement. Such approaches have paved the way for the motion planning of biped and quadruped robots as well. Most of these methods are deterministic, utilizing first-order iterative gradient-based algorithms incorporating the constrained differentiable objective functions. However, the limitation of prevailing methods concerning differentiability hinders the implementation of non-differentiable objective functions such as metabolic energy expenditure (MEE) function, which is highly relevant for physiological systems and can even be implemented across the muscular space. This paper consolidates the implementation of the prevalent direct collocation-based optimal control method with the stochastic trajectory optimization method based on Policy Improvement with Path Integral (PI2) for comprehending the human sit-to-stand (STS) motion. PI2 method, which utilizes reinforcement learning of Dynamic Movement Primitive (DMP) to learn a goal-based trajectory is implemented and validated by comparing with the experimental result in joint-space and muscle-space.

Two-Stage stochastic optimization for operating a Renewable-Based Microgrid

• A conventional MILP algorithm does not consider the uncertainty in operating a renewable-based Microgrid. • The two-Stage stochastic optimization algorithm takes into consideration the potential future PV profile. • The proposed method improves the Microgrid reliability by a 19.4% decrease in the Energy not supplied. • The proposed method decreases the operation cost by 18.8%. Carbon emissions are increasing as a result of urbanization and population growth all over the world. Scientists agree that these emissions are one of the main causes of climate change and global warming. The power industry is shifting to renewable energy sources (RES) such as solar power (PV) and employing different Energy Storage Systems (ESS) to enable a clean energy future and compensate for the scarcity of fossil fuel. However, the intermittent behaviour of renewable resources causes some obstacles such as power fluctuation, committing extra reserve units, and load shedding. Microgrid (MG) technology is introduced as a promising solution for integrating different RES and loads into the grid. Operating the MG in islanded mode with a limited ESS capacity requires a sophisticated scheduling method. Previous studies on MG addressed the operation issue, neglecting the supply–demand uncertainty or adopting the worst-case scenario. Uncertainty is an inherent characteristic in power systems; on the other hand, considering the worst-case scenario may unnecessarily increase the operation and planning costs. To address the uncertainty and fluctuant characteristics of RES-based MG, this paper proposes a two-stage stochastic optimization integrated with a novel ANN-based prediction model. A new model for PV power prediction is proposed by which the predicted data is in a probability density function (PDF) form. A stochastic optimization (SO) method is proposed to minimize the operation cost and load shedding during the islanding mode. In the proposed SO, the optimal scheduling decision is made in the current moment taking into account the probability of potential supply, load, and ESS capacities in the near future. For this purpose, an ANN-based prediction model is developed to represent the PV output uncertainty in the SO problem. The proposed prediction model proves efficient in the prediction with nRMSE of 9.7% and nMAE of 9.1%. The proposed method is applied to a real Microgrid designed by the Natural Energy Laboratory of Hawaii Authority (NELHA) and compared to a conventional optimization method. The proposed scheduling method reduces both the average Energy Not Supplied (ENS) and operation costs by 19.4% and 18.8%, respectively, with no additional investment cost. A sensitivity study is also conducted to assess the performance of the proposed method in terms of ENS, cost, and simulation time.

Distributionally robust offering strategy of the aggregator integrating renewable energy generator and energy storage considering uncertainty and connections between the mid-to-long-term and spot electricity markets

As an emerging entity, the aggregator integrating renewable energy generator and energy storage (REG-ES aggregator) can promote the consumption of renewable energy by participating in electricity market, which is an important way to deal with global climate change and achieve the “carbon neutrality” target. To solve the problems of multiple market connection and REG output uncertainty faced by the aggregator in the offering process, this paper develops a comprehensive offering strategy decision-making model for REG-ES aggregator, and the Wasserstein based two-stage distributionally robust optimization (DRO) technique is proposed to deal with the uncertainty in the model. Linear affine technique and strong duality theory are applied to reconstruct the DRO model into an easy-to-solved form. The simulation results and further discussions show that (1) the aggregator can effectively suppress the output uncertainty through ES system, but the adjustment effect in certain periods is not significant due to ES's power balance constraint. (2) Considering market connection, aggregator will offer over 90% of the output in mid-to-long-term market and tend to reduce ES capacity to gain more stable profits. Therefore, market regulator need to formulate necessary ES configuration requirements and market incentive mechanisms to guarantee aggregators' orderly participation in multiple markets. (3) When ignoring uncertainty, aggregator's revenue will increase by 3.46% while its profit will decrease by 1.85% due to higher default risk and deviation penalty costs. (4) By comparing the results of stochastic optimization, robust optimization and DRO methods, the DRO model is proved to have significant advantages in data-driven, low computational complexity and result conservativeness.

An online framework for survival analysis: reframing Cox proportional hazards model for large data sets and neural networks.

In many biomedical applications, outcome is measured as a "time-to-event" (e.g., disease progression or death). To assess the connection between features of a patient and this outcome, it is common to assume a proportional hazards model and fit a proportional hazards regression (or Cox regression). To fit this model, a log-concave objective function known as the "partial likelihood" is maximized. For moderate-sized data sets, an efficient Newton-Raphson algorithm that leverages the structure of the objective function can be employed. However, in large data sets this approach has two issues: (i) The computational tricks that leverage structure can also lead to computational instability; (ii) The objective function does not naturally decouple: Thus, if the data set does not fit in memory, the model can be computationally expensive to fit. This additionally means that the objective is not directly amenable to stochastic gradient-based optimization methods. To overcome these issues, we propose a simple, new framing of proportional hazards regression: This results in an objective function that is amenable to stochastic gradient descent. We show that this simple modification allows us to efficiently fit survival models with very large data sets. This also facilitates training complex, for example, neural-network-based, models with survival data.

Two stochastic optimization methods for shift design with uncertain demand

The purpose of this paper is to investigate the shift design problem with a probability constraint on demand satisfaction and to design the corresponding stochastic model so that the staffing for each shift can cope with stochastic demand. To solve this stochastic model, we propose two solution methods: a method involving average sample approximation and a two-stage heuristic algorithm based on statistics with a greedy strategy. Numerical results show that both methods can solve the stochastic model of this paper well, and the sample average approximation method outperforms the two-stage heuristic algorithm in terms of cost optimization. However, as the number of scenarios used to approximate realistic situations increases, the superiority of the two-stage heuristic algorithm in terms of solution speed becomes progressively more significant.

A multi-time scale dispatching optimal model for rural biomass waste energy conversion system-based micro-energy grid considering multi-energy demand response

Aiming at utilizing straw, garbage, domestic sewage and other biomass waste resources in rural areas, this study designed a rural biomass wastes energy conversion system (BWs)-based micro energy grid (BWs-MEG), and the mathematical modeling of BWs-MEG is carried out including multi-energy transforms system (METS) and multi-energy demand response (MEDR). Then, a two-stage optimal framework for rural BWs-MEG multi-time scale dispatching is designed. In the day-ahead stage, a multi-objective dispatching optimal model was constructed with the objectives of minimum operating costs and maximum eco-environment benefits. In the intra-day stage, the robust optimization theory was utilized to characterize the uncertainties of wind power plant (WPP) and photovoltaic power generation (PV) and a rolling dispatching optimal model was constructed with the objective of minimum deviation costs. After that, the above dispatching model was fuzzified and linearized, and then converted into a mixed integer linear programming model. Finally, a micro-energy grid in northern China was selected as an example for case study. The results showed: (1) BWs can utilize rural biomass waste resources for pyrolysis power generation (PG) and gas production to achieve energy utilization and provide power, heating and gas output. In the islanded operation mode and grid-connected operation mode, when MEG is configured with BWs, the deviation costs decrease by 25.6 % and 26.33 %, while the operating costs increases only by 7.60 % and 13.52 %, respectively. The power output of WPP and PV increase by 1.03 % and 2.19 % in the grid-connected operation mode, indicating that BWs is conducive to realizing the multi-dimension supply and demand balance of power, heating and gas loads. (2) Multi-time scale dispatching model can give play to the regulation ability of BWs, METS and MEDR and connect the day-ahead dispatching plan with the intra-day dispatching strategy to formulate the optimal dispatching strategy. When there is deviation in the day-ahead dispatching strategy, METS and MEDR could maintain the supply and demand balance of power load. AHP can change the heating period and BWs could maintain the supply and balance of heating and gas loads. Compared with the day-ahead dispatching plan, the output of PG and power-to-gas device in intra-day dispatching strategy increase by 21.22 % and 9.78 %. (3) Robust stochastic optimization method can characterize the uncertainties of WPP and PV and formulate the dispatching decision schemes with different risk attitudes. With the robust coefficient Г increasing, MEG operating costs and eco-environment benefits increase and deviation adjustment costs decreases, but the robustness of the dispatching scheme is improved. When 0.25≤Г≤0.75, the increase of the uncertainty parameter will have a direct impact on the formulation of dispatching scheme. And the decision maker belongs to risk preference type, who is willing to take certain risks to win excess benefits. Besides, if MEG operates in the grid-connected mode, the uncertainty risks will be weakened and the operation mode shall be reasonably selected according to the demand for dispatching decision. Overall, the proposed optimization model can promote the energy utilization of rural biomass waste resources, which is conducive to the realization of a clean and low-carbon transformation of the overall energy structure.

A Slime Mould Algorithm Programming for Solving Single and Multi-Objective Optimal Power Flow Problems with Pareto Front Approach: A Case Study of the Iraqi Super Grid High Voltage

Optimal power flow (OPF) represents one of the most important issues in the electrical power system for energy management, planning, and operation via finding optimal control variables with satisfying the equality and inequality constraints. Several optimization methods have been proposed to solve OPF problems, but there is still a need to achieve optimum performance. A Slime Mould Algorithm (SMA) is one of the new stochastic optimization methods inspired by the behaviour of the oscillation mode of slime mould in nature. The proposed algorithm is characterized as easy, simple, efficient, avoiding stagnation in the local optima and moving toward the optimal solution. Different frameworks have been applied to achieve single and conflicting multi-objective functions simultaneously (Bi, Tri, Quad, and Quinta objective functions) for solving OPF problems. These objective functions are total fuel cost of generation units, real power loss on transmission lines, total emission issued by fossil-fuelled thermal units, voltage deviation at load bus, and voltage stability index of the whole system. The proposed algorithm SMA has been developed by incorporating it with Pareto concept optimization to generate a new approach, named the Multi-Objective Slime Mould Algorithm (MOSMS), to solve multi-objective optimal power flow (MOOPF) problems. Fuzzy set theory and crowding distance are the proposed strategies to obtain the best compromise solution and rank and reduce a set of non-dominated solutions, respectively. To investigate the performance of the proposed algorithm, two standard IEEE test systems (IEEE 30 bus IEEE 57 bus systems) and a practical system (Iraqi Super Grid High Voltage 400 kV) were tested with 29 case studies based on MATLAB software. The optimal results obtained by the proposed approach (SMA) were compared with other algorithms mentioned in the literature. These results confirm the ability of SMA to provide better solutions to achieve the optimal control variables.

Improved spherical search with local distribution induced self-adaptation for hard non-convex optimization with and without constraints

Most metaheuristic optimizers rely heavily on precisely setting their control parameters and search operators to perform well. Considering the complexity of real-world problems, it is always preferable to adjust control parameter values automatically rather than clamping them to a fixed value. In recent years, Spherical Search (SS) has emerged as a population-based stochastic optimization method that exploits the concepts of random projection matrices in linear algebra. As a result of the success of SS in solving non-convex, real-parameter optimization problems of various complexity, we have significantly extended SS in this paper by introducing a set of new algorithms, collectively known as Self Adaptive Spherical Search (SASS). Our proposal aims to enhance the performance of SS by using different projection matrix schemes in conjunction with improved search-direction calculations and an adaptive modification of parameter values. In our proposed adaptation scheme, parameters are modified to relevant values by applying a self-adaptive process that does not rely upon prior knowledge of the correlation between the parameter values and characteristics of the problem space. Consequently, we may apply the algorithms to bound and nonlinearly constrained optimization problems. For the benchmark suites derived from the most recent IEEE Congress on Evolutionary Computation (CEC) competitions, simulation results indicate that the SASS family of algorithms performs better than or is comparable to state-of-the-art algorithms from the other paradigms concerning robustness and convergence.

Adaptive Evolution Strategies for Stochastic Zeroth-Order Optimization

We consider solving a class of unconstrained optimization problems in which only stochastic estimates of the objective functions are available. Existing stochastic optimization methods are mainly extended from gradient-based methods, faced with the challenges of noisy function evaluations, hardness in choosing step-sizes, and probably ill-conditioned landscapes. This paper presents a stochastic evolution strategy (SES) framework and several adaptation schemes to avoid these challenges. The SES framework combines the ideas of population sampling and minibatch sampling in exploiting the zeroth-order gradient information, efficiently reducing the noise in both data selection and gradient approximation. In addition, it admits approximating the gradients using a non-isotropic Gaussian distribution to better capture the curvature information of the landscapes. Based on this framework, we implement a step-size adaptation rule and two covariance matrix adaptation rules, where the former can automatically tune the step-sizes and the latter are intended to cope with ill-conditioning. For SES with certain fixed step-sizes, we establish a nearly optimal convergence rate over smooth landscapes. We also show that using the adaptive step-sizes allows convergence at a slightly slower rate but without the need to know the smoothness constant. Several numerical experiments on machine learning problems verify the above theoretical results and suggest that the adaptive SES methods show much promise.

3D printable bone-mimicking functionally gradient stochastic scaffolds for tissue engineering and bone implant applications

A number of designs have been used in the field of tissue engineering for the purposes of bone biomimicry (e.g.) triply periodic minimal surfaces (TPMS), honeycomb lattice, and face centered cubic (FCC). While the usage of these structures was met with some success, the results were not optimal due to factors such as lack of localized porosity control and stress concentration at nodes of gradient scaffolds. This led to a push for novel designs, such as, stochastic scaffolds with locally tunable porosity as well as anisotropically varying mechanical properties. A number of methods exist for generating these stochastic structures, such as, random set method, stochastic optimization method, iterative non-linear transformative method and others, but while these techniques led to a greater approximation of the bone microstructure, they did not help in the creation of a structure with controlled modulus along different directions. The novelty of the method used in this paper is that it not only helps in generating stochastic structures in a computationally inexpensive and non-complex way, but also generates a porosity gradient along three different axes in a controlled manner. Various additive manufacturing techniques have been successfully utilized for fabricating these structures at different scales, using different materials, thus, proving their versatility. The proposed design approach was implemented in the case of a 3D reconstructed rabbit model. Magics software was used to extract the internal microstructure of the rabbit model and our design approach was used for generating a stochastic scaffold which accurately mimicked the porosity gradient of the rabbit model in all the three directions. On comparing the mechanical properties of the rabbit microstructure and the gradient mimicking scaffold, the variations were significant as the properties depend not only on the porosity gradient but also on the internal microstructure, which warrants further multi-property optimization.

Bi-layer optimal secondary frequency control approach for energy storage clusters considering wind power uncertainty

Increasing penetration of wind power with intermittency and variability threatens the stability of the power system frequency. The fast response capability of the energy storage system (ESS) makes it an effective measure to improve frequency regulation performance. However, designing an optimal control for numerous energy storage units (ESUs) with different power and energy characteristics is challenging. To solve the dilemma that the distributed control methods cannot achieve optimality over time horizons while the centralized optimization methods would cause high computational burdens, a bi-layer optimal control approach is proposed in this study. In the upper layer, a multi-area secondary frequency control (SFC) problem is established to determine control policies for ESS clusters under continuous wind power fluctuations and is solved by the Itô theory-based stochastic optimization (ITB-SO) method with high computational efficiency in a rolling-horizon manner. In the lower layer, the power output of ESUs is dispatched and coordinated using the distributed algorithm (DA) method via communication networks, considering different characteristics and the current state-of-charge (SoC) levels. Simulation studies carried out in a dual-region test system evidence the improvement of system frequency stability by compensating for rapid wind power fluctuations immediately via ESSs. The results show that the proposed ITB-SO method without scenario generation is suitable for real-time SFC for the ESS. The effectiveness of the DA method and its robustness in encountering loss of communication links are also verified.

Regularized quasi-monotone method for stochastic optimization

We adapt the quasi-monotone method, an algorithm characterized by uniquely having convergence quality guarantees for the last iterate, for composite convex minimization in the stochastic setting. For the proposed numerical scheme we derive the optimal convergence rate of $$\text{ O }\left( \frac{1}{\sqrt{k+1}}\right)$$ in terms of the last iterate, rather than on average as it is standard for subgradient methods. The theoretical guarantee for individual convergence of the regularized quasi-monotone method is confirmed by numerical experiments on $$\ell _1$$ -regularized robust linear regression.

Enhancements to explicit stochastic reservoir operation optimization method

The Fletcher-Ponnambalam (FP) method is an explicit stochastic optimization method for design and operations management of real-world storage systems including surface water reservoir and groundwater management problems. The FP method faces no curse of dimensionality and no need for scenario generation. The paper introduces a novel implementation for the FP method, named FP-2022 here for clarity, by removing the need for nonlinear constraints and by decreasing the number of decision variables to just one third of its original value, significantly reducing solving time (∼27 times faster than the original formulation). Additionally, new expressions derived for the first and second moments of both reservoir release deficit and surplus variables and the already-derived expression for the second moments of reservoir storages are incorporated into the FP-2022 formulation enabling the method to reach an improved optimality for a nonlinear objective function. The enhanced procedure is applied to solving a reservoir operation optimization problem for a major dam in Brazil. The result comparisons made with other methods along with a thorough analysis of release operation policies prove the optimality of this highly numerically efficient and convenient-to-use FP-2022 method. Finally, a multi-reservoir application of the model is also tested with corrective simulations for improved estimates of some additional variables of interest. A specific constraint-handling approach regarding reservoir release lower and upper bounds is also presented. Satisfactory results are obtained for solving the Parambikulam-Aliyar reservoir system, a real world five-reservoir operation optimization problem from India.

Disentangled Representation Learning and Generation With Manifold Optimization.

Disentanglement is a useful property in representation learning, which increases the interpretability of generative models such as variational autoencoders (VAE), generative adversarial models, and their many variants. Typically in such models, an increase in disentanglement performance is traded off with generation quality. In the context of latent space models, this work presents a representation learning framework that explicitly promotes disentanglement by encouraging orthogonal directions of variations. The proposed objective is the sum of an autoencoder error term along with a principal component analysis reconstruction error in the feature space. This has an interpretation of a restricted kernel machine with the eigenvector matrix valued on the Stiefel manifold. Our analysis shows that such a construction promotes disentanglement by matching the principal directions in the latent space with the directions of orthogonal variation in data space. In an alternating minimization scheme, we use the Cayley ADAM algorithm, a stochastic optimization method on the Stiefel manifold along with the Adam optimizer. Our theoretical discussion and various experiments show that the proposed model is an improvement over many VAE variants in terms of both generation quality and disentangled representation learning.