The action of a finite group G on a compact Riemann surface X naturally induces another action of G on its Jacobian variety J(X). In many cases, each component of the group algebra decomposition of J(X) is isogenous to a Prym varieties of an intermediate covering of the Galois covering πG:X→X/G; in such a case, we say that the group algebra decomposition is affordable by Prym varieties. In this article, we present an infinite family of groups that act on Riemann surfaces in a manner that the group algebra decomposition of J(X) is not affordable by Prym varieties; namely, affine groups Aff(Fq) with some exceptions: q=2, q=9, q a Fermat prime, q=2n with 2n−1 a Mersenne prime and some particular cases when X/G has genus 0 or 1. In each one of this exceptional cases, we give the group algebra decomposition of J(X) by Prym varieties.