Gradient-based Optimization Research Articles

Life-Cycle Gradient-Based Production Optimization Including Well-Shutoff Option with Least-Squares Support Vector Regression

Summary In this work, we focus on well-shutoff/well-control optimization, which enables shutting off a production well or injection well to be part of the well-control optimization within a net present value (NPV) formulation that includes the operation expenditures (OPEX) if the well is not economical to produce or inject. For this purpose, we formulate an objective function for the NPV by introducing a production time fraction as the design variable to shut off a production well over its total production life, divided into a fixed number of cycles. Unlike the previous studies, we use a shutoff period within each cycle instead of shutting it off with a bottomhole pressure (BHP) control, where the BHPs for each producer are part of the optimization variables. When BHPs are considered control optimization variables with an NPV having OPEX, the NPV could be discontinuous for BHPs. To avoid this problem, we built a continuously differentiable proxy function for NPV using least-square support vector regression (LS-SVR). We use linear equality constraints so that the sums of the lengths of the cycles at each producer and injector are equal to the life of the production. Thus, we do not need to truncate the size of the last cycle, as in the previous studies, which may lead to suboptimal solutions. We use a simulator-based optimization method with stochastic simplex approximate gradient (StoSAG) and a machine learning-based (LS-SVR) optimization method to solve such an optimization problem. We update the LS-SVR proxy during optimization so that the updated proxy remains predictive toward promising regions of search space during the optimization. We compare the performance of the proposed LS-SVR-based iterative sampling refinement (ISR) method with the StoSAG-based and the finite difference (FD)-based optimization methods. To demonstrate the applicability of our proposed methodologies, we consider a synthetic example of a waterflooding process in a tight oil reservoir with two water injectors and four producers. Results show that the LS-SVR-based optimization method is at least three to seven times more computationally efficient than the StoSAG-based optimization method using a high-fidelity numerical simulator. However, we observe that the size and sampling of the training data, as well as the selection of bound constraints for the well controls, influence the performance of the LS-SVR-based optimization method. Designing multiple shutoffs and making cycle lengths unknown are found to be ineffective compared to single shutoff cases, as they yield lower NPVs.

Shape Optimization for Lithium-Ion Battery with Porous Electrodes

In this presentation, we introduce a gradient-based shape optimization framework to design porous electrodes for maximum energy storage. Lithium-ion batteries are becoming an increasingly important energy source and storage device. Large surface area that advocates high reaction rates often leads to small porosities that deteriorate electrochemical transport, which significantly restricts ion penetration depth, limiting the material utilization in planar electrodes. Consequently, architected electrodes are required to optimize performance.We consider a model with two materials, namely porous electrode and pure electrolyte. Density-based topology optimization with continuous material volume fraction has previously been implemented on lithium-ion battery system for both half-cell and full-cell model to maximizes its energy storage. Despite the capability of producing complex interdigitated structures, an appropriate penalization scheme for intermediate materials is required for the optimizer to provide near binary designs. Setting up such a penalization is often time-consuming due to the large number of design-dependent variables that affect the forward model differently. Alternatively, the shape optimization approach used in this work produces binary designs naturally by morphing conformal meshes, circumventing inaccuracies associated with fuzzy interfaces. Shape sensitivities are computed via the adjoint method and leveraging automatic differentiation. The optimization problem is solved using a nonlinear programming technique.In this presentation, we perform shape optimization on a half-cell model to maximize its energy storage when a fixed charging current is applied. We study multiple arrangements of physical parameters and compare their performance against a monolithic electrode. Both 2D and 3D results are presented.

An Analytical Model for Predicting the Rate Performance of Battery Cells Under Mixed Kinetic Control

The prediction of battery performance conventionally relies on resource-intensive numerical simulations based on the porous electrode theory. For computational speedup, simplified physics-based models have proven attractive as they still capture the underlying reaction mechanisms to a certain degree. In this class of models, physical complexity is usually reduced by assuming a single rate-limiting reaction. For example, the single particle model describes battery cells limited only by the solid diffusion pathway [1]. On the flipside, our previous analytical model exploited inefficiency of electrolyte transport, particularly in thick electrodes [2]. With advancement in both electrode and electrolyte technologies, however, it becomes challenging for these models to handle more realistic batteries.In this work, we propose an improved physics-based analytical model that enables mixed control of both kinetic processes, reflecting both salt depletion and particle size effect. An extension to our uniform reaction (UR) model, the new model uses salt distribution in electrolyte to probe the accessible lithium in the electrode level. The added solid diffusion module calculates the equilibrium potential at particle surface, lithium concentration at particle surface (Cs), and in turn the particle-level depth of discharge. We refer to the model as the URCs model, recognizing its two components. The model features higher tunability as it now accepts function-valued electrode and electrolyte properties. It also allows for prediction of discharge voltage curves and energy output of the cell.The model is compared with numerical simulation over a wide range of cell parameters. It shows good agreement with simulated results in rate performance test, specific energy output test, and discharge voltage curves. Better alignment is observed for the NMC/Li half cell than the NMC/Gr full cell, likely due to mixed reaction type in the graphite anode. Figure 1 shows the comparison of voltage curves across various C rates for a half cell with 70um NMC electrode and particles of 4um radius. Additionally, the cells are optimized for their specific capacity against electrode thickness and porosity. The optimal parameters predicted by the model lie closely to the global optimum from numerical simulation. Figure 2 shows the optimization results for a full cell with particles of 4um radius, overlaid on contour lines generated from the URCs model.The model offers a speed up of over 500 times compared with state-of-art P2D solvers [3]. Furthermore, its compatibility with gradient-based optimization algorithms opens possibilities for more efficient global optimization schemes, in which the number of function calls can be reduced. The high efficiency and the analytical nature of the URCs model render it a powerful tool for both on-board applications and battery cell design.[1] Doyle, M. and Newman, J., 1997. Analysis of capacity–rate data for lithium batteries using simplified models of the discharge process. Journal of Applied Electrochemistry, 27, pp.846-856.[2] Wang, F. and Tang, M., 2020. A quantitative analytical model for predicting and optimizing the rate performance of battery cells. Cell Reports Physical Science, 1(9).[3] Sulzer, V., Marquis, S.G., Timms, R., Robinson, M. and Chapman, S.J., 2021. Python battery mathematical modelling (PyBaMM). Journal of Open Research Software, 9(1). Figure 1

Topology Optimization of Hard-Magnetic Soft Phononic Structures for Wide Magnetically Tunable Band Gaps

Abstract Hard-magnetic soft materials, which exhibit finite deformation under magnetic loading, have emerged as a promising class of soft active materials for the development of phononic structures with tunable elastic wave band gap characteristics. In this paper, we present a gradient-based topology optimization framework for designing the hard-magnetic soft materials-based two-phase phononic structures with wide and magnetically tunable anti-plane shear wave band gaps. The incompressible Gent hyperelastic material model, along with the ideal hard-magnetic soft material model, is used to characterize the constitutive behavior of the hard-magnetic soft phononic structure phases. To extract the dispersion curves, an in-house finite element model in conjunction with Bloch’s theorem is employed. The method of moving asymptotes is used to iteratively update the design variables and obtain the optimal distribution of the hard-magnetic soft phases within the phononic structure unit cell. Analytical sensitivity analysis is performed to evaluate the gradient of the band gap maximization function with respect to each one of the design variables. Numerical results show that the optimized phononic structures exhibit a wide band gap width in comparison to a standard hard-magnetic soft phononic structure with a central circular inclusion, demonstrating the effectiveness of the proposed numerical framework. The numerical framework presented in this study, along with the derived conclusions, can serve as a valuable guide for the design and development of futuristic tunable wave manipulators.

Shape and size optimization framework of stiffened shell using isogeometric analysis

A shape and size optimization framework is developed to improve the structural performance of the stiffened shell using isogeometric analysis. To accurately model the stiffened shell, the Lagrange multiplier method is employed to establish the coupling relationship of isogeometric degenerated shell elements and isogeometric degenerated beam elements. Due to the advantages of the non-uniform rational B-spline (NURBS), the unit normal direction vectors of the shell can be defined analytically, which guarantees stiffeners perpendicular to the skin. Additionally, the stiffeners can be easily generated based on the mapping relationship of the parametric space and the physical space of the shell. Analytical sensitivity is derived in detail, and the gradient-based optimization method can be used to solve the minimum compliance optimization problem owing to the high order continuity of NURBS. Three numerical illustrative examples are presented to verify the effectiveness of the proposed framework for the shape and size optimization of stiffened shells, consisting of a cylindrical shell patch, a hyperbolic shell, and a complex engineering segment. It is worth noting that the proposed framework is especially suitable for optimization problems of the stiffened shell, eliminating the need of complex feature extraction.

An improved peridynamics topology optimization formulation for compliance minimization

This work proposes an improved peridynamics density-based topology optimization framework for compliance minimization. One of the main advantages of using a peridynamics discretization relies in the fact that it provides a consistent regularization of classical continuum mechanics into a nonlocal continuum, thus containing an inherent length scale called the horizon. Furthermore, this reformulation allows for discontinuities and is highly suitable for treating fracture and crack propagation. Partial differential equations are rewritten as integrodifferential equations and its numerical implementation can be straightforwardly done using meshfree collocation, inheriting its advantages. In the optimization formulation, Solid Isotropic Material with Penalization (SIMP) is used as interpolation for the design variables. To improve the peridynamic formulation and to evaluate the objective function in a energetically consistent manner, surface correction is implemented. Moreover, a detailed sensitivity analysis reveals an analytical expression for the objective function derivatives, different from an expression commonly used in the literature, providing an important basis for gradient-based topology optimization with peridynamics. The proposed implementation is studied with two examples illustrating different characteristics of this framework. The analytical expression for the sensitivities is validated against a reference solution, providing an improvement over the referred expression in the literature. Also, the effect of using the surface correction is evidenced. An extensive analysis of the horizon size and sensitivity filter radius indicates that the current method is mesh-independent, i.e. a sensitivity filter is redundant since peridynamics intrinsically filters length scales with the horizon. Different optimization methods are also tested for uncracked and cracked structures, demonstrating the capabilities and robustness of the proposed framework.

Enhancing source separation quality via optimal sensor placement in noisy environments

The paper aims to bridge a part of the gap between source separation and sensor placement studies by addressing a novel problem: “Predicting optimal sensor placement in noisy environments to improve source separation quality”. The structural information required for optimal sensor placement is modeled as the spatial distribution of source signal gains and the spatial correlation of noise. The sensor positions are predicted by optimizing two criteria as measures of separation quality, and a gradient-based global optimization method is developed to efficiently address this optimization problem. Numerical results exhibit superior performance when compared with classical sensor placement methodologies based on mutual information, underscoring the critical role of sensor placement in source separation with noisy sensor measurements. The proposed method is applied to actual electroencephalography (EEG) data to separate the P300 source components in a brain-computer interface (BCI) application. The results show that when the sensor positions are chosen using the proposed method, to reach a certain level of spelling accuracy, fewer sensors are required compared with standard sensor locations.

Gradient-Based Aero-Stealth Optimization of a Simplified Aircraft

Modern fighter aircraft increasingly need to conjugate aerodynamic performance and low observability. In this paper, we showcase a methodology for a gradient-based bidisciplinary aero-stealth optimization. The shape of the aircraft is parameterized with the help of a CAD modeler, and we optimize it with the SLSQP algorithm. The drag, computed with the help of a RANS method, is used as the aerodynamic criterion. For the stealth criterion, a function is derived from the radar cross-section in a given cone of directions and weighed with a function whose goal is to cancel the electromagnetic intensity in a given direction. Stealth is achieved passively by scattering back the electromagnetic energy away from the radar antenna, and no energy is absorbed by the aircraft, which is considered as a perfect conductor. A Pareto front is identified by varying the weights of the aerodynamic and stealth criteria. The Pareto front allows for an easy identification of the CAD model corresponding to a chosen aero-stealth trade-off.

Self-Organizing Optimization Based on Caputo’s Fractional Order Gradients

This paper analyses the condition necessary to guarantee no divergence for Caputo’s fractional order gradient descent (C-FOG) algorithm on multivariate functions. C-FOG is self-organizing, computationally efficient, simple, and understandable. It converges faster than the classical gradient-based optimization algorithms and converges to slightly different points when the order of the fractional derivative is different. The additional freedom of the order is very useful in situations where the diversity of convergence is required, and it also allows for more precise convergence. Comparative experiments on a typical poor conditioned function and adversarial sample generation frameworks demonstrate the convergence performance of C-FOG, showing that it outperforms currently popular algorithms in terms of convergence speed, and more excitingly, the diversity of convergence allows it to exhibit stronger and more stable attack capability in adversarial sample generation procedures (The code for experiments is available at: https://github.com/mulertan/self_optimizing/tree/main, accessed on 30 April 2024).

A gradient-based optimization algorithm to solve optimal control problems with conformable fractional-order derivatives

In this paper, we solve numerically a class of fractional optimal control problems in the conformable sense. We first propose a nonlinear fractional optimal control problem subject to general equality and inequality constraints. Then, based on a novel numerical integration scheme, we present a discretization method for the governed fractional-order system as well as the cost and constraint functionals, which yields a discretized approximate problem. We further derive the gradients of the cost and constraint functionals in regard to the discretized controls. On the basis of this result, a numerical optimization approach to solve the approximate problem is developed. Finally, three non-trivial example problems are solved to illustrate the applicability and effectiveness of the developed approach.

The Role of Mathematics in Artificial Intelligence and Machine Learning

Mathematics serves as the foundational backbone of artificial intelligence (AI) and machine learning (ML), providing the essential tools and frameworks for developing sophisticated algorithms and models. the pivotal role of various mathematical disciplines, including linear algebra, calculus, probability theory, and optimization, in advancing AI and ML technologies. We begin by examining how linear algebra facilitates the manipulation and transformation of high-dimensional data, which is crucial for techniques such as principal component analysis (PCA) and singular value decomposition (SVD). Next, we delve into the applications of calculus in training neural networks through gradient-based optimization methods, highlighting the importance of differentiation and integration in backpropagation and loss function minimization. the role of probability theory in handling uncertainty and making predictions, emphasizing its application in Bayesian networks, Markov decision processes, and probabilistic graphical models. Additionally, we discuss optimization techniques, both convex and non-convex, that are fundamental to finding optimal solutions in machine learning tasks, including support vector machines (SVMs) and deep learning architectures.

Optimization-based stacked machine-learning method for seismic probability and risk assessment of reinforced concrete shear walls

Efficient seismic risk assessment aids decision-makers in formulating citywide risk mitigation plans, providing insights into building performance and retrofitting costs. The complexity of modeling, analysis, and post-processing of the results makes it hard to fast-track the seismic probabilities, and there is a need to optimize the computational time. This research addresses seismic probability and risk assessment of reinforced concrete shear walls (RCSWs) by introducing stacked machine learning (Stacked ML) models based on Bayesian optimization (BO), genetic algorithm (GA), particle swarm optimization (PSO), and gradient-based optimization (GBO) algorithms. The study investigates 4-, to 15-Story RCSWs assuming different bay lengths and soil types to build a comprehensive database based on the incremental dynamic analysis (IDA) subjected to 56 near-field pulse-like and no-pulse records. Having 227,200 and 63,384 data points for a median of IDA curve (MIDA) and seismic probability curve, respectively, the proposed Stacked ML models have shown good performance on curve fitting ability by accuracy of 99.1% and 99.4% for MIDA and seismic fragility curves, respectively. In addition, the proposed models can estimate the mean annual frequency, λ, which is a key parameter in seismic risk assessment of buildings. To provide the results of the study for general buildings, a user-friendly GUI is proposed that facilitates result utilization, offering insights into seismic performance levels, providing the estimated MIDA and seismic failure probability curves, and mean annual frequency calculations for specific performance levels and seismic hazard curves.

Computational synthesis of locomotive soft robots by topology optimization.

Locomotive soft robots (SoRos) have gained prominence due to their adaptability. Traditional locomotive SoRo design is based on limb structures inspired by biological organisms and requires human intervention. Evolutionary robotics, designed using evolutionary algorithms (EAs), have shown potential for automatic design. However, EA-based methods face the challenge of high computational cost when considering multiphysics in locomotion, including materials, actuations, and interactions with environments. Here, we present a design approach for pneumatic SoRos that integrates gradient-based topology optimization with multiphysics material point method (MPM) simulations. This approach starts with a simple initial shape (a cube with a central cavity). The topology optimization with MPM then automatically and iteratively designs the SoRo shape. We design two SoRos, one for walking and one for climbing. These SoRos are 3D printed and exhibit the same locomotion features as in the simulations. This study presents an efficient strategy for designing SoRos, demonstrating that a purely mathematical process can produce limb-like structures seen in biological organisms.

Multi-objective meta-learning

Meta-learning has arisen as a powerful tool for many machine learning problems. With multiple factors to be considered when designing learning models for real-world applications, meta-learning with multiple objectives has attracted much attention recently. However, existing works either linearly combine multiple objectives into one objective or adopt evolutionary algorithms to handle it, where the former approach needs to pay high computational cost to tune the combination coefficients while the latter approach is computationally heavy and incapable to be integrated into gradient-based optimization. To alleviate those limitations, in this paper, we aim to propose a generic gradient-based Multi-Objective Meta-Learning (MOML) framework with applications in many machine learning problems. Specifically, the MOML framework formulates the objective function of meta-learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta-learner. Different from those existing works, in this paper, we propose a gradient-based algorithm to solve the MOBLP. Specifically, we devise the first gradient-based optimization algorithm by alternately solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence property and provide a non-asymptotic analysis of the proposed gradient-based optimization algorithm. Empirically, extensive experiments justify our theoretical results and demonstrate the superiority of the proposed MOML framework for different learning problems, including few-shot learning, domain adaptation, multi-task learning, neural architecture search, and reinforcement learning. The source code of MOML is available at https://github.com/Baijiong-Lin/MOML.

Human-Exoskeleton Coupling Simulation for Lifting Tasks with Shoulder, Spine, and Knee-Joint Powered Exoskeletons.

In this study, we introduce a two-dimensional (2D) human skeletal model coupled with knee, spine, and shoulder exoskeletons. The primary purpose of this model is to predict the optimal lifting motion and provide torque support from the exoskeleton through the utilization of inverse dynamics optimization. The kinematics and dynamics of the human model are expressed using the Denavit-Hartenberg (DH) representation. The lifting optimization formulation integrates the electromechanical dynamics of the DC motors in the exoskeletons of the knee, spine, and shoulder. The design variables for this study include human joint angle profiles and exoskeleton motor current profiles. The optimization objective is to minimize the squared normalized human joint torques, subject to physical and task-specific lifting constraints. We solve this optimization problem using the gradient-based optimizer SNOPT. Our results include a comparison of predicted human joint angle profiles, joint torque profiles, and ground reaction force (GRF) profiles between lifting tasks with and without exoskeleton assistance. We also explore various combinations of exoskeletons for the knee, spine, and shoulder. By resolving the lifting optimization problems, we designed the optimal torques for the exoskeletons located at the knee, spine, and shoulder. It was found that the support from the exoskeletons substantially lowers the torque levels in human joints. Additionally, we conducted experiments only on the knee exoskeleton. Experimental data indicated that using the knee exoskeleton decreases the muscle activation peaks by 35.00%, 10.03%, 22.12%, 30.14%, 16.77%, and 25.71% for muscles of the erector spinae, latissimus dorsi, vastus medialis, vastus lateralis, rectus femoris, and biceps femoris, respectively.

Semi-Intrusive Stochastic Galerkin Finite Element Method for Adjoint-Based Optimization Under Uncertainty

The stochastic Galerkin method for the propagation of probabilistically modeled uncertainties can be difficult to apply in practice due to its formulation and the challenge of creating a computational infrastructure to support it. To address these challenges, this work proposes a sampling-based stochastic Galerkin method that leverages existing deterministic analysis and adjoint-based derivative implementations. The proposed formulation is semi-intrusive since it is implemented using an existing deterministic framework, requiring only the numerical sampling of the deterministic residuals, Jacobians, boundary conditions, and adjoint implementations at nodes in the probabilistic domain. The software architectures to support stochastic generalizations of the deterministic finite element frameworks are presented. This proposed approach is demonstrated using a finite element framework for flexible multibody dynamics problems. Finally, the semi-intrusive implementation of the stochastic Galerkin method is used to demonstrate gradient-based optimizations of flexible multibody dynamics systems in the presence of probabilistically modeled uncertainties.

ZeroGrads: Learning Local Surrogates for Non-Differentiable Graphics

Gradient-based optimization is now ubiquitous across graphics, but unfortunately can not be applied to problems with undefined or zero gradients. To circumvent this issue, the loss function can be manually replaced by a "surrogate" that has similar minima but is differentiable. Our proposed framework, ZeroGrads , automates this process by learning a neural approximation of the objective function, which in turn can be used to differentiate through arbitrary black-box graphics pipelines. We train the surrogate on an actively smoothed version of the objective and encourage locality, focusing the surrogate's capacity on what matters at the current training episode. The fitting is performed online, alongside the parameter optimization, and self-supervised, without pre-computed data or pre-trained models. As sampling the objective is expensive (it requires a full rendering or simulator run), we devise an efficient sampling scheme that allows for tractable run-times and competitive performance at little overhead. We demonstrate optimizing diverse non-convex, non-differentiable black-box problems in graphics, such as visibility in rendering, discrete parameter spaces in procedural modelling or optimal control in physics-driven animation. In contrast to other derivative-free algorithms, our approach scales well to higher dimensions, which we demonstrate on problems with up to 35k interlinked variables.

Combined deep-learning optimization predictive models for determining carbon dioxide solubility in ionic liquids

This study explores the development of predictive models for carbon dioxide (CO2) solubility in ionic liquids based on a compiled dataset of 10,116 experimentally measured data points involving four input variables: pressure (P), temperature (T), cation type, and anion type. The deep-learning (DL) predictive models evaluated are standalone and hybrid versions of convolutional neural network (CNN) and long short-term memory (LSTM) algorithms with cuckoo optimization algorithm (COA) and gradient-based optimization (GBO). The laboratory-measured data was separated into training and test categories, and each category was normalized separately to improve the performance of the deep learning algorithms. The Mahalanobis distance-based quantile method was utilized to identify any outliers in the training data. Once identified, the outlier data points were eliminated from the training dataset. The control parameters of the deep learning algorithms were optimized using COA to enhance their efficiency, and the algorithms were hybridized with optimization algorithms to further improve their performance. The resulting models were analyzed to assess their accuracy, degree of overfitting, and the importance of input features. The study found that using 80% of the data for training and 20% for testing results in more accurate and generalizable models. Using the outlier detection method on the training data led to 307 data points being eliminated as outliers. Developing CO2-solubility predictive model showed that, the CNNCOA model had the lowest RMSE and highest R2 among the developed models, indicating high generalizability for data unseen by the trained model. The analysis revealed that using optimization algorithms increased the CO2-solubility prediction performance of DL algorithms and reduced overfitting. T and cation type were the most and least important input features, respectively. Simultaneous changes in cation and anion type on CO2-solubility predictions displayed no systematic pattern. For increases in T, CO2 solubility typically decreased, whereas for increases in P CO2 solubility always increased but at variable rates. The results of this study can be used to develop accurate and generalizable CO2-solubility predictive models for various applications.

Random coordinate descent: A simple alternative for optimizing parameterized quantum circuits

Variational quantum algorithms rely on the optimization of parameterized quantum circuits in noisy settings. The commonly used back-propagation procedure in classical machine learning is not directly applicable in this setting due to the collapse of quantum states after measurements. Thus, gradient estimations constitute a significant overhead in a gradient-based optimization of such quantum circuits. This paper introduces a random coordinate descent algorithm as a practical and easy-to-implement alternative to the full gradient descent algorithm. This algorithm only requires one partial derivative at each iteration. Motivated by the behavior of measurement noise in the practical optimization of parameterized quantum circuits, this paper presents an optimization problem setting that is amenable to analysis. Under this setting, the random coordinate descent algorithm exhibits the same level of stochastic stability as the full gradient approach, making it as resilient to noise. The complexity of the random coordinate descent method is generally no worse than that of the gradient descent and can be much better for various quantum optimization problems with anisotropic Lipschitz constants. Theoretical analysis and extensive numerical experiments validate our findings. Published by the American Physical Society 2024

Gradient-based optimization for quantum architecture search

Quantum Architecture Search (QAS) has shown significant promise in designing quantum circuits for Variational Quantum Algorithms (VQAs). However, existing QAS algorithms primarily explore circuit architectures within a discrete space, which is inherently inefficient. In this paper, we propose a Gradient-based Optimization for Quantum Architecture Search (GQAS), which leverages a circuit encoder, decoder, and predictor. Initially, the encoder embeds circuit architectures into a continuous latent representation. Subsequently, a predictor utilizes this continuous latent representation as input and outputs an estimated performance for the given architecture. The latent representation is then optimized through gradient descent within the continuous latent space based on the predicted performance. The optimized latent representation is finally mapped back to a discrete architecture via the decoder. To enhance the quality of the latent representation, we pre-train the encoder on a substantial dataset of circuit architectures using Self-Supervised Learning (SSL). Our simulation results on the Variational Quantum Eigensolver (VQE) indicate that our method outperforms the current Differentiable Quantum Architecture Search (DQAS).