A framework for residual-based a posteriori error estimation and adaptive mesh refinement and polynomial chaos expansion for general second order linear elliptic PDEs with random coefficients is presented. A parametric, deterministic elliptic boundary value problem on an infinite-dimensional parameter space is discretized by means of a Galerkin projection onto finite generalized polynomial chaos (gpc) expansions, and by discretizing each gpc coefficient by a FEM in the physical domain.An anisotropic residual-based a posteriori error estimator is developed. It contains bounds for both contributions to the overall error: the error due to gpc discretization and the error due to Finite Element discretization of the gpc coefficients in the expansion. The reliability of the residual estimator is established.Based on the explicit form of the residual estimator, an adaptive refinement strategy is presented which allows to steer the polynomial degree adaptation and the dimension adaptation in the stochastic Galerkin discretization, and, embedded in the gpc adaptation loop, also the Finite Element mesh refinement of the gpc coefficients in the physical domain. Asynchronous mesh adaptation for different gpc coefficients is permitted, subject to a minimal compatibility requirement on meshes used for different gpc coefficients.Details on the implementation with the open-source software framework ALEA are presented; it is generic, and is based on available stiffness and mass matrices of a FEM for the deterministic, nonparametric nominal problem evaluated in the FEniCS environment.Preconditioning of the resulting matrix equation and iterative solution are discussed. Numerical experiments in two spatial dimensions for membrane and plane stress boundary value problems on polygons are presented. They indicate substantial savings in total computational complexity due to FE mesh coarsening in high gpc coefficients.