We study the splitting of invariant manifolds of whiskered tori with two or three frequencies in nearly-integrable Hamiltonian systems, such that the hyperbolic part is given by a pendulum. We consider a 2-dimensional torus with a frequency vector $\omega=(1,\Omega)$, where $\Omega$ is a quadratic irrational number, or a 3-dimensional torus with a frequency vector $\omega=(1,\Omega,\Omega^2)$, where $\Omega$ is a cubic irrational number. Applying the Poincaré--Melnikov method, we find exponentially small asymptotic estimates for the maximal splitting distance between the stable and unstable manifolds associated to the invariant torus, and we show that such estimates depend strongly on the arithmetic properties of the frequencies. In the quadratic case, we use the continued fractions theory to establish a certain arithmetic property, fulfilled in 24 cases, which allows us to provide asymptotic estimates in a simple way. In the cubic case, we focus our attention to the case in which $\Omega$ is the so-called cubic golden number (the real root of $x^3+x-1=0$), obtaining also asymptotic estimates. We point out the similitudes and differences between the results obtained for both the quadratic and cubic cases.