We study two-dimensional (2D) matter-wave gap solitons trapped in an elliptically deformed concentric lattice potential, within the framework of the Gross-Pitaevskii equation (GPE) with self-attraction or self-repulsion. For a fixed eccentricity of the lattice, soliton families are found in both the repulsive and attractive models. In the former case, the analysis reveals two kinds of gap solitons trapped in the first oval trough (the ring-shaped potential minimum closest to the center): elliptic annular solitons (EASs), and double solitons (DSs), which are formed by two tightly localized density peaks located at diametrically opposite points of the trough, with zero phase difference between them. With the decrease of the norm, the density distribution in the EAS along the azimuthal direction changes from nearly uniform to double-peaked and, eventually, to the DS. In the attractive model, there exist only DSs in the oval trough, while EASs are not found. All such solitons without the angular momentum ($l=0$) are fully stable. For $l\ensuremath{\ne}0$, vortical solitons---both EASs with a sufficiently large norm (in the repulsive model) and DSs (in models with both signs of the nonlinearity)---are quasistable, exhibiting rocking motion in the elliptic trough (we consider the cases of $l=1 \mathrm{and} l=2$). At smaller values of the norm, the vortical annular solitons (in the repulsive model) are unstable. Stable fundamental solitons trapped in the central potential well are investigated, too, in both the attractive and repulsive models, by means of the variational approximation and numerical methods.