We study the Taylor expansion for the solution of a differential equation driven by a multi-dimensional Hölder path with exponent $H> 1/2$. We derive a convergence criterion that enables us to write the solution as an infinite sum of iterated integrals on a non empty interval. We apply our deterministic results to stochastic differential equations driven by fractional Brownian motions with Hurst parameter $H > 1/2$. We also study the convergence in L2 of the stochastic Taylor expansion by using L2 estimates of iterated integrals and Borel-Cantelli type arguments.