Derivative-free Optimization Methods Research Articles

An optimal interpolation set for model-based derivative-free optimization methods

This paper demonstrates the optimality of an interpolation set employed in derivative-free trust-region methods. This set is optimal in the sense that it minimizes the constant of well-poisedness in a ball centred at the starting point. It is chosen as the default initial interpolation set by many derivative-free trust-region methods based on underdetermined quadratic interpolation, including NEWUOA, BOBYQA, LINCOA, and COBYQA. Our analysis provides a theoretical justification for this choice.

Q&A With Dennis Denney, JPT Technology Focus Creator

_ In 1997, JPT debuted a monthly feature that represented an unconventional approach in meeting SPE’s goal of disseminating technical information to its members. Soon titled “Technology Focus,” the feature brought together subject-matter experts (SMEs) in a range of important industry topics to review technical-paper abstracts gathered from the previous year of SPE meetings, as well as the Offshore Technology Conference. Its architect, engineer, and pilot was Technology Editor Dennis Denney, who would become a familiar presence to SPE members and staff alike over a 17-year career, although his industry roots extended to his college days. Dennis helped recruit SMEs for review service; sent them hundreds of abstracts and full papers (monthly, that is; the cumulative count of papers he handled is likely a six-digit figure); answered their questions and concerns with prompt, supportive guidance; and then, once papers had been selected, synopsized them for publication in the magazine with the longtime help of Assistant Technology Editor Karen Bybee. As this writer well knows, condensing a 10,000-word paper on, say, distributed quasi-Newton derivative-free optimization methods for field development optimization into a 1,500-word summary that captures the novelty and technical sophistication of its authors’ work can be a daunting task. Applying his own background as a petroleum engineer to methodically analyze and trim each chosen paper, Dennis did just that dozens of times a year, transforming these texts into svelte packages that included only one table or figure to deliver the highest possible quantity of quality to JPT readers. When Dennis retired in 2013, I initially split his eyebrow-raising workload with fellow Technology Editor Adam Wilson (now JPT’s Special Publications Editor). It was immediately clear from the way that reviewers spoke about Dennis, and his contributions to the magazine and SPE itself, that he was held in the same high regard outside of the organization as he was within it, a testament to the reputation he had earned while working with so many members and SMEs. It was entirely fitting that, for JPT’s 75th anniversary celebration, we caught up with Dennis to reflect on his achievements in the industry and with the groundbreaking Technology Focus feature he created. Dennis lives with his wife Linda in Rockwall, Texas—not too far from SPE headquarters in Dallas, but not too close, either!—enjoying his retirement and his family. (Note: For more on the development and history of the Technology Focus features, read the companion piece to this article, “Technology Focus Topics Reflect Industry Growth, Evolution Over 25+ Years.”) Why did you decide to go into the field of petroleum engineering? I worked several years as an engineering tech, testing gas wells, tracking well and reservoir pressures, and writing computer programs to analyze pressure data. The department manager encouraged me to finish my degree and become a petroleum engineer. So, after I received a scholarship from J. Harold Dunn (at age 28), my wife and I sold our house, packed our things into a U-Haul truck, and loaded two young kids into the car to move to Bryan/College Station to attend Texas A&M University.

Comparative analysis of a mining truck motor with permanent magnets in a rotor and a synchronous homopolar motor without magnets in a drive

The relevance. Increasing need for using mining trucks with a diesel-electric (hybrid) drive for development of minerals. Improving operational and cost characteristics of the electric drive of mining trucks helps to reduce costs in the development of minerals. The main aim. To compare theoretically the performance of synchronous traction motors of various designs (a conventional design with permanent magnets inside a rotor and a homopolar design without permanent magnets with an excitation winding on a stator), optimized by the same method, in the drive of a mining truck. To optimize the motor design to reduce power loss and required inverter power, as well as to limit torque ripple and reduce the risk of permanent magnet demagnetization. Objects. Design of twelve-pole nine-phase synchronous AC motors with a rated power of 370 kW of various designs: a homopolar motor without permanent magnets with an excitation winding on the stator and a motor of a traditional design with permanent magnets in the rotor. Methods. Derivative-free optimization method; equivalent circuit method; mathematical modeling; two-dimensional finite element method. Results. Based on the analysis, the advantages and disadvantages of the considered motors were revealed. The advantage of the motor with permanent magnets in the rotor is reduction in active part length by 30 %. The advantage of the homopolar motor with an excitation winding on the stator are 4.6 times lower cost of active materials. In addition, the homopolar motor has a more reliable design, without the risk of overheating, demagnetization or deterioration of the properties of permanent magnets over time

A hybrid direct search and projected simplex gradient method for convex constrained minimization

We propose a new Derivative-free Optimization (DFO) approach for solving convex constrained minimization problems. The feasible set is assumed to be the non-empty intersection of a finite collection of closed convex sets, such that the projection onto each of these individual convex sets is simple and inexpensive to compute. Our iterative approach alternates between steps that use Directional Direct Search (DDS), considering adequate poll directions, and a Spectral Projected Gradient (SPG) method, replacing the real gradient by a simplex gradient, under a DFO approach. In the SPG steps, if the convex feasible set is simple, then a direct projection is computed. If the feasible set is the intersection of finitely many convex simple sets, then Dykstra's alternating projection method is applied. Convergence properties are established under standard assumptions usually associated to DFO methods. Some preliminary numerical experiments are reported to illustrate the performance of the proposed algorithm, in particular by comparing it with a classical DDS method. Our results indicate that the hybrid algorithm is a robust and effective approach for derivative-free convex constrained optimization.

A Derivative-free Trust-region Method for Optimization on the Ellipsoid

Optimization methods play a crucial role in various fields and applications. In some optimization problems, the derivative information of the objective function is unavailable. Such black-box optimization problems need to be solved by derivative-free optimization methods. At the same time, optimization problems with ellipsoidal constraints are important and have widespread applications in various fields as well. Following the development of the late professor M. J. D. Powell’s efficient derivative-free trust-region optimization methods, this paper considers solving derivative-free optimization problems on the ellipsoid. Our new optimization solver EC-NEWUOA for problems on the ellipsoid in ℜ n is designed based on Powell’s derivative-free software NEWUOA for unconstrained optimization problems. The proposed techniques for our new method mainly include using the Courant penalty function, the augmented Lagrangian method, and the projection technique. Details about the method and theoretical analysis are included in this paper. We also compare our new method with other algorithms by solving test problems and then show the numerical advantages of our new method.

A modified approach on modeling-design-optimization procedure for cutting of pure titanium using wire electric discharge machining (WEDM)

The main goal of this study is to introduce a modified approach to the modeling-design-optimization triple procedure for cutting pure titanium. The manufacturing process is selected as wire electric discharge machining. To achieve a smooth cutting operation of the pure titanium plate (grade 2), the independent cutting parameters, peak current, pulse off time, pulse on time, spark gap voltage, wire feed, and wire tension are taken into consideration. Their effects on machining rate, surface roughness, dimensional deviation, and wire wear ratio are also examined. The mathematical model of the process is determined with the multiple nonlinear neuroregression approach. Stability study that measures the ability to realize the cutting parameters was conducted during the modeling phase. This type of step has not been used as a criterion for model determination before in modeling studies. Comparisons were accomplished with the traditional and nontraditional models, and it is shown that the nontraditional models’ accuracy was more reasonable. Besides, this study contains the design optimization of the manufacturing process parameters using modified version of different stochastic and derivative-free optimization methods: differential evaluation, random search, simulated annealing, and Nelder-Mead algorithm, (MDE, MRS, MSA and MNM) simultaneously.

A Hybrid Direct Search and Model-Based Derivative-Free Optimization Method with Dynamic Decision Processing and Application in Solid-Tank Design

A derivative-free optimization (DFO) method is an optimization method that does not make use of derivative information in order to find the optimal solution. It is advantageous for solving real-world problems in which the only information available about the objective function is the output for a specific input. In this paper, we develop the framework for a DFO method called the DQL method. It is designed to be a versatile hybrid method capable of performing direct search, quadratic-model search, and line search all in the same method. We develop and test a series of different strategies within this framework. The benchmark results indicate that each of these strategies has distinct advantages and that there is no clear winner in the overall performance among efficiency and robustness. We develop the Smart DQL method by allowing the method to determine the optimal search strategies in various circumstances. The Smart DQL method is applied to a problem of solid-tank design for 3D radiation dosimetry provided by the UBCO (University of British Columbia—Okanagan) 3D Radiation Dosimetry Research Group. Given the limited evaluation budget, the Smart DQL method produces high-quality solutions.

Neural Inverse Design of Nanostructures (NIDN)

In the recent decade, computational tools have become central in material design, allowing rapid development cycles at reduced costs. Machine learning tools are especially on the rise in photonics. However, the inversion of the Maxwell equations needed for the design is particularly challenging from an optimization standpoint, requiring sophisticated software. We present an innovative, open-source software tool called Neural Inverse Design of Nanostructures (NIDN) that allows designing complex, stacked material nanostructures using a physics-based deep learning approach. Instead of a derivative-free or data-driven optimization or learning method, we perform a gradient-based neural network training where we directly optimize the material and its structure based on its spectral characteristics. NIDN supports two different solvers, rigorous coupled-wave analysis and a finite-difference time-domain method. The utility and validity of NIDN are demonstrated on several synthetic examples as well as the design of a 1550 nm filter and anti-reflection coating. Results match experimental baselines, other simulation tools, and the desired spectral characteristics. Given its full modularity in regard to network architectures and Maxwell solvers as well as open-source, permissive availability, NIDN will be able to support computational material design processes in a broad range of applications.

Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models

The ensemble Kalman inversion (EKI) is a particle based method which has been introduced as the application of the ensemble Kalman filter to inverse problems. In practice it has been widely used as derivative-free optimization method in order to estimate unknown parameters from noisy measurement data. For linear forward models the EKI can be viewed as gradient flow preconditioned by a certain sample covariance matrix. Through the preconditioning the resulting scheme remains in a finite dimensional subspace of the original high-dimensional (or even infinite dimensional) parameter space and can be viewed as optimizer restricted to this subspace. For general nonlinear forward models the resulting EKI flow can only be viewed as gradient flow in approximation. In this paper we discuss the effect of applying a sample covariance as preconditioning matrix and quantify the gradient flow structure of the EKI by controlling the approximation error through the spread in the particle system. The ensemble collapse on the one side leads to an accurate gradient approximation, but on the other side to degeneration in the preconditioning sample covariance matrix. In order to ensure convergence as optimization method we derive lower as well as upper bounds on the ensemble collapse. Furthermore, we introduce covariance inflation without breaking the subspace property intending to reduce the collapse rate of the ensemble such that the convergence rate improves. In a numerical experiment we apply EKI to a nonlinear elliptic boundary-value problem and illustrate the dependence of EKI as derivative-free optimizer on the choice of the initial ensemble.

Field-Development Optimization Method Benchmarked, Field Tested

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 206267, “Benchmarking and Field Testing of the Distributed Quasi-Newton Derivative-Free Optimization Method for Field Development Optimization,” by Faruk Alpak, SPE, Yixuan Wang, SPE, and Guohua Gao, SPE, Shell, et al. The paper has not been peer reviewed. _ Recently, a novel distributed quasi-Newton (DQN) derivative-free optimization (DFO) method was developed for generic reservoir-performance optimization problems, including well-location optimization (WLO) and well-control optimization (WCO). DQN is designed to locate multiple local optima of highly nonlinear optimization problems effectively. However, its performance has been neither validated by realistic applications nor compared with other DFO methods. Field-testing results reinforce the auspicious computational attributes of DQN. Background An optimization problem is posed as the minimization or maximization of an objective function by modifying the control variables (x) within a search domain. The objective function is a highly nonlinear function of x and may have multiple local optima. In this paper, the objective function is assumed to be twice differentiable. DFO methods can be classified into local search methods and global search methods. In the complete paper, the authors’ goal is to locate multiple local optima of the objective function. Their focus is on local search DFO optimization methods, which include direct search methods and model-based methods. Current DFO methods reviewed in the complete paper have a common feature: Only one best approximation to the solution is updated in each iteration, and only one optimal solution is identified in the last iteration. Therefore, they are referred to as single-thread DFO methods. They do not represent an efficient approach because simulation results obtained by one optimization task starting from one initial guess are not shared with other optimization tasks that start from different initial guesses. The distributed Gauss-Newton method and the DQN method benchmarked in the paper are referred to as multiple-thread optimization methods. The authors have integrated DQN into a versatile field-development optimization platform designed specifically for iterative work flows enabled through distributed-parallel flow simulations.

Taking Care of the Discretization Problem: A Comprehensive Study of the Discretization Problem and a Black-Box Adversarial Attack in Discrete Integer Domain

Neural network based (NN-based) classifiers are known vulnerable against adversarial examples, namely, adding slight perturbations to a benign image cause a classifier to make a false prediction. To evaluate the robustness of NN-based classifiers against adversarial examples, numerous adversarial attacks with high success rates have been proposed recently. NN-based image classifiers usually normalize valid images (e.g., RGB image where the value at each coordinate is an integer between 0 and 255) into a real continuous domain (e.g., 3-dimensional matrix where the value at each coordinate is a real number between 0 and 1) and make classification decisions on the normalized images. However, adversarial examples crafted in a real continuous domain may become benign once they are denormalized back into the corresponding discrete integer domain, known as the discretization problem. This problem has been mentioned in some prior works but received relatively limited attention. In this work, we report the first comprehensive study of existing works to understand the impacts of the discretization problem. By analyzing 35 representative methods and empirically studying 20 representative open source tools, we found 29/35 (theoretically) and 14/20 (empirically) are affected by the discretization problem, e.g., the success rate could dramatically drop from 100 to 10 percent after the domain transformation. As the first step towards addressing this problem in a black-box scenario, we propose a novel derivative-free optimization method, which can directly craft adversarial examples in the discrete integer domain. Experimental results show that the method achieves nearly 100 percent attack success rates for both targeted and untargeted attacks, comparable to the most popular white-box methods (FGSM, BIM and C&W), and significantly outperforms representative black-box methods (ZOO, AutoZOOM, NES-PGD, Bandits, FD, FD-PSO and GenAttack). Our results suggest that the discretization problem should be treated more seriously, and the discrete optimization algorithms show a promising future in crafting effective black-box attacks.

Technology Focus: Field Development (September 2022)

In this month’s Field Development feature, SPE conference authors place an emphasis not only on original research and findings but also on validation and refinement of existing techniques that may not have received thorough attention in the literature. As operators strive to achieve the greatest returns possible from technically, environmentally, and economically challenging plays, they are driven to innovate new approaches and to fine-tune existing ones. In paper SPE 206267, the authors investigate the performance of a distributed quasi-Newton (DQN) derivative-free optimization (DFO) method, which, they write, had not yet been validated by realistic applications or compared with other DFO methods. They integrate DQN into a versatile field-development optimization platform designed for iterative work flows enabled through distributed parallel flow simulations. The method was field-tested on two realistic applications and identified the global optimum with the least number of simulations and the shortest run time on a synthetic problem with a known solution. The authors of paper SPE 207146 create and evaluate a development plan for an oil field discovered in a remote offshore environment in the Niger Delta. Because the oil in place was uncertain, a probabilistic approach was used to estimate the stock-tank oil originally in place using low, mid, and high cases. Keeping in mind governmental regulations to maintain the reservoir pressure above bubblepoint, the authors propose a development plan that is marginally base modeled and, despite the uncertainty of oil prices in the market, able to cover any unforeseen situations. Finally, the authors of paper SPE 208882, addressing the challenge posed by cumbersome integrated technical work flows for multiwell fracture modeling and reservoir simulation, use tools existing in the literature to quantify the effect of changing well spacing on well productivity for a given completion design using a simple, intuitive empirical equation. After describing and qualifying the use of their approach, they apply it to a case study from the Permian Basin. Recommended additional reading at OnePetro: www.onepetro.org. OTC 31151 - Autonomous Subsea Field Development—Value Proposition, Technology Needs, and Gaps for Future Advancement by Giorgio Arcangeletti, Saipem, et al. SPE 206533 - Field Development Optimization Using Machine-Learning Methods To Identify the Optimal Waterflooding Regime by Alexey Vasilievich Timonov, Consultant, et al. URTeC-2021-5301 - Unconventional Reservoir Development Performance Reviews—the Northern Midland Basin Case Study by Hongjie Xiong, University Lands, et al.

Novel formulations of flexibility index and design centering for design space definition

Design space definition is one of the key issues in pharmaceutical research and development. Flexibility index and design centering are two complementary ways to estimate a candidate design space. In this study, we first propose a novel formulation of flexibility index based on a direction search method, which is applied to any shape of the design space. Next, we propose two design centering methods. The vertex direction search method is first developed as a single-level optimization model, which is rigorous for convex regions. Next, based on the proposed flexibility index model, a derivative-free optimization (DFO) method is developed for solving the bi-level optimization models involved in design centering problems, which is applicable not only to convex but also to nonconvex problems. In order to find near global solutions, latin hypercube sampling (LHS) is used to generate multiple starting points for the DFO solver. The solution yields the optimal nominal point, which is the candidate point with the largest flexibility index. Several case studies demonstrate the performance of the proposed methods.

Complete vehicle energy management using adaptive primal–dual operator splitting

Complete vehicle energy management (CVEM) aims at minimizing the energy consumption of all subsystems inside a vehicle. When the subsystems consist of energy buffers with linear dynamics and/or power converters with quadratic power losses, the CVEM becomes a large-scale and ill-conditioned optimal control problem for which well-known solution approaches (e.g. Pontryagin’s maximum principle, dynamic programming, convex and derivative-free optimization methods) are not effective. In this paper, we design an efficient solution algorithm for this problem by leveraging the framework of operator splitting methods. To facilitate its implementation, we propose a spectral method that automatically tunes the step sizes at every iteration and, in practice, accelerates convergence. Finally, we consider a case study for a parallel-hybrid vehicle and we illustrate the numerical performance of our method by showing that we can efficiently solve a CVEM problem with very large horizons.

A simplified, efficient approach to hybrid wind and solar plant site optimization

Abstract. Wind plant layout optimization is a difficult, complex problem with a large number of variables and many local minima. Layout optimization only becomes more difficult with the addition of solar generation. In this paper, we propose a parameterized approach to wind and solar hybrid power plant layout optimization that greatly reduces problem dimensionality while guaranteeing that the generated layouts have a desirable regular structure. Thus far, hybrid power plant optimization research has focused on system sizing. We go beyond sizing and present a practical approach to optimizing the physical layout of a wind–solar hybrid power plant. We argue that the evolution strategy class of derivative-free optimization methods is well-suited to the parameterized hybrid layout problem, and we demonstrate how hard layout constraints (e.g., placement restrictions) can be transformed into soft constraints that are amenable to optimization using evolution strategies. Next, we present experimental results on four test sites, demonstrating the viability, reliability, and effectiveness of the parameterized evolution strategy approach for generating optimized hybrid plant layouts. Completing the tool kit for parameterized layout generation, we include a brief tutorial describing how the parameterized evolutionary approach can be inspected, understood, and debugged when applied to hybrid plant layouts.

An Efficient Bi-Objective Optimization Workflow Using the Distributed Quasi-Newton Method and Its Application to Well-Location Optimization

SummaryAlthough it is possible to apply traditional optimization algorithms to determine the Pareto front of a multiobjective optimization problem, the computational cost is extremely high when the objective function evaluation requires solving a complex reservoir simulation problem and optimization cannot benefit from adjoint-based gradients. This paper proposes a novel workflow to solve bi-objective optimization problems using the distributed quasi-Newton (DQN) method, which is a well-parallelized and derivative-free optimization (DFO) method. Numerical tests confirm that the DQN method performs efficiently and robustly.The efficiency of the DQN optimizer stems from a distributed computing mechanism that effectively shares the available information discovered in prior iterations. Rather than performing multiple quasi-Newton optimization tasks in isolation, simulation results are shared among distinct DQN optimization tasks or threads. In this paper, the DQN method is applied to the optimization of a weighted average of two objectives, using different weighting factors for different optimization threads. In each iteration, the DQN optimizer generates an ensemble of search points (or simulation cases) in parallel, and a set of nondominated points is updated accordingly. Different DQN optimization threads, which use the same set of simulation results but different weighting factors in their objective functions, converge to different optima of the weighted average objective function. The nondominated points found in the last iteration form a set of Pareto-optimal solutions. Robustness as well as efficiency of the DQN optimizer originates from reliance on a large, shared set of intermediate search points. On the one hand, this set of searching points is (much) smaller than the combined sets needed if all optimizations with different weighting factors would be executed separately; on the other hand, the size of this set produces a high fault tolerance, which means even if some simulations fail at a given iteration, the DQN method’s distributed-parallel information-sharing protocol is designed and implemented such that the optimization process can still proceed to the next iteration.The proposed DQN optimization method is first validated on synthetic examples with analytical objective functions. Then, it is tested on well-location optimization (WLO) problems by maximizing the oil production and minimizing the water production. Furthermore, the proposed method is benchmarked against a bi-objective implementation of the mesh adaptive direct search (MADS) method, and the numerical results reinforce the auspicious computational attributes of DQN observed for the test problems.To the best of our knowledge, this is the first time that a well-parallelized and derivative-free DQN optimization method has been developed and tested on bi-objective optimization problems. The methodology proposed can help improve efficiency and robustness in solving complicated bi-objective optimization problems by taking advantage of model-based search algorithms with an effective information-sharing mechanism.NOTE: This paper is also published as part of the 2021 SPE Reservoir Simulation Conference Special Issue.

Multiphysics Optimization for First Wall Design Enhancement in Water-Cooled Breeding Blankets

The commercial feasibility of the first fusion power plant generation adopting D-T plasma is strongly dependent upon the self-sustainability in terms of tritium fuelling. Within such a kind of reactor, the component selected to house the tritium breeding reactions is the breeding blanket, which is further assigned to heat power removal and radiation shielding functions. As a consequence of both its role and position, the breeding blanket is heavily exposed to both surface and volumetric heat loads and, hence, its design requires a typical multiphysics approach, from the neutronics to the thermo-mechanics. During last years, a great deal of effort has been put in the optimization of the breeding blanket design, with the aim of maximizing the tritium breeding and heat removal performances without undermining its structural integrity. In this paper, a derivative-free optimization method named “Complex method” is applied for the design optimization of the European DEMO Water-Cooled Lithium Lead breeding blanket concept. To this purpose, a potential performances-based objective function, focusing on the maximization of the tritium breeding, is defined and a multiphysics numerical model of the blanket is developed in order to solve the coupled thermo-mechanical problem, while the optimization algorithm leads the design towards a minimum optimum point compliant with the prescribed requirements. Once the optimized design is obtained, its nuclear and thermo-structural performances are assessed by means of specific neutron transport and multiphysics simulations, respectively. Finally, the structural integrity is verified by means of the application of the RCC-MRx design criteria. • A tool for First Wall optimization in water-cooled breeding blankets is developed. • The optimization is performed using the gradient-free method named “Complex method”. • An objective function aiming at tritium breeding maximization is defined. • A FE model is used to check the compliance with the thermo-structural requirements. • The optimized design is finally assessed by proper, more detailed numerical analyses.

Anomaly detection for data accountability of Mars telemetry data

The Mars Curiosity rover is frequently sending engineering and science data that goes through a pipeline of systems before reaching its final destination at the mission operations center making it prone to volume loss and data corruption. A ground data system analysis (GDSA) team is charged with the monitoring of this flow of information and the detection of anomalies in that data to request a re-transmission when necessary. So far, this task relied primarily on the expertise of GDSA analysts who, especially with new missions launching, require the assistance of an autonomous and effective detection tool. Variational autoencoders are powerful deep neural networks that can learn to isolate such anomalies yet, as any deep network, they require an architectural search and fine-tuning to yield exploitable performance. Furthermore, this process needs to be repeated periodically to adjust to the changing flow of transmissions. This work presents Δ-MADS, a hybrid derivative-free optimization method designed to quickly produce efficient variational autoencoders in order to assist the GDSA team in their mission.

Distributed quasi-Newton derivative-free optimization method for optimization problems with multiple local optima

Distributed quasi-Newton derivative-free optimization method for optimization problems with multiple local optima

Derivative-free optimization adversarial attacks for graph convolutional networks.

In recent years, graph convolutional networks (GCNs) have emerged rapidly due to their excellent performance in graph data processing. However, recent researches show that GCNs are vulnerable to adversarial attacks. An attacker can maliciously modify edges or nodes of the graph to mislead the model’s classification of the target nodes, or even cause a degradation of the model’s overall classification performance. In this paper, we first propose a black-box adversarial attack framework based on derivative-free optimization (DFO) to generate graph adversarial examples without using gradient and apply advanced DFO algorithms conveniently. Second, we implement a direct attack algorithm (DFDA) using the Nevergrad library based on the framework. Additionally, we overcome the problem of large search space by redesigning the perturbation vector using constraint size. Finally, we conducted a series of experiments on different datasets and parameters. The results show that DFDA outperforms Nettack in most cases, and it can achieve an average attack success rate of more than 95% on the Cora dataset when perturbing at most eight edges. This demonstrates that our framework can fully exploit the potential of DFO methods in node classification adversarial attacks.