Residential sector has a large underutilized potential for demand response in its electricity consumption optimization. In this study residential electric hot water heaters (EHWH) are considered as distributed energy storage providing demand response in electricity markets. Heating optimization is conducted with a dynamic optimization model describing the water heating characteristics in Finland. Cost and CO2 emissions minimizing optimal heating controls are solved for an individual household and for a pool of EHWHs controlled by an aggregator. Water heating costs under fully dynamic and static electricity pricing contracts are compared. Our main contribution is to provide important insights of complex market dynamics related to the demand response resource aggregation, different electricity pricing contracts and increasing wind power generation. There are three main results. First, households benefit from adopting dynamic electricity pricing and allowing load control in terms of heating cost savings. Secondly, the optimal number of controlled EHWHs depend on the viewpoint. Lastly, cost minimization yields relatively good solution from environmental perspective when compared with the emissions minimization target.