Abstract Aiming to optimize the mode transition control schedule of the adaptive cycle engine (ACE), an optimization method based on a gradient algorithm was proposed. During the mode transition, the compressor surge margin, the total turbine inlet temperature and the combustion chamber fuel-air ratio are not exceeded. The integration of thrust by time was selected as the optimization objective. The geometric parameters were updated via the optimization target derivative. The speed limit of the geometric adjustment mechanism was also considered in the optimization process. The control schedules of two mode transition processes, from triple to double bypass mode and from double to triple-bypass mode were optimized. The optimization method presented in this paper can be applied to various mode transitions and the geometric adjustment speed meets all the constraints.