In this work, we present a generic approach to optimize the design of a parametrized robot gripper including both selected gripper mechanism parameters, and parameters of the finger geometry. We suggest six gripper quality indices that indicate different aspects of the performance of a gripper given a CAD model of an object and a task description. These quality indices are then used to learn task-specific finger designs based on dynamic simulation. We demonstrate our gripper optimization on a parallel finger type gripper described by twelve parameters. We furthermore present a parametrization of the grasping task and context, which is essential as an input to the computation of gripper performance. We exemplify important aspects of the indices by looking at their performance on subsets of the parameter space by discussing the decoupling of parameters and show optimization results for two use cases for different task contexts. We provide a qualitative evaluation of the obtained results based on existing design guidelines and our engineering experience. In addition, we show that with our method we achieve superior alignment properties compared to a naive approach with a cutout based on the “inverse of an object”. Furthermore, we provide an experimental evaluation of our proposed method by verifying the simulated grasp outcomes through a real-world experiment.