This paper proposes an optimization method for solving fractional variational problems depending on indefinite integrals, where the fractional derivative is described in the Caputo sense. The method is based on the new basis functions consisting of the transcendental Bernstein series (TBS) and their operational matrices. In the first step, we derive an approximate solution for the problem using TBS with the free coefficients and control parameters. In the second step, we use the fractional operational matrix, with the help of the Lagrange multipliers technique, for converting the fractional variational problem into an easier one, described by a system of nonlinear algebraic equations. The convergence analysis of the method, will be guaranteed by proving a new theorem concerning TBS. Finally, for illustrating the efficiency and accuracy of the proposed technique, several numerical examples are analyzed and the results compared with the analytical solutions or the approximation obtained by other techniques.