AbstractThis paper presents the development of an optimization and modeling method for the objective functions of output power, efficiency and weight of an outer rotor permanent magnet brushless DC (BLDC) motor based on radial basis function (RBF) approximation technique. The proposed RBF‐based Pareto optimization method requires less knowledge about electric/magnetic formulas and can replace conventional optimizations based on these equations with higher accuracy. To apply the proposed optimization method, the initial design should be developed using such equations. Therefore, RBFs are used to model and predict engine behavior. To optimize the objective functions, we used a genetic algorithm optimization technique with nonlinear electric and magnetic constraints to find the Pareto front set. The design obtained by the proposed radial basis function Pareto optimization (RBFPO) method was finally verified by Ansoft Maxwell. The results of optimal design using the RBFPO method have higher output power and efficiency. Also, in addition to the advantage of a favorable accuracy, RBF‐based models are significantly faster than models available in simulation tools.