We present a framework for modeling gliomas growth and their mechanical impact on the surrounding brain tissue (the so-called, mass-effect). We employ an Eulerian continuum approach that results in a strongly coupled system of nonlinear Partial Differential Equations (PDEs): a reaction-diffusion model for the tumor growth and a piecewise linearly elastic material for the background tissue. To estimate unknown model parameters and enable patient-specific simulations we formulate and solve a PDE-constrained optimization problem. Our two main goals are the following: (1) to improve the deformable registration from images of brain tumor patients to a common stereotactic space, thereby assisting in the construction of statistical anatomical atlases; and (2) to develop predictive capabilities for glioma growth, after the model parameters are estimated for a given patient. To our knowledge, this is the first attempt in the literature to introduce an adjoint-based, PDE-constrained optimization formulation in the context of image-driven modeling spatio-temporal tumor evolution. In this paper, we present the formulation, and the solution method and we conduct 1D numerical experiments for preliminary evaluation of the overall formulation/methodology.