The detailed implementation and analysis of a finite element multigrid scheme for the solution of elliptic optimal control problems is presented. A particular focus is in the definition of smoothing strategies for the case of constrained control problems. For this setting, convergence of the multigrid scheme is discussed based on the BPX framework. Results of numerical experiments are reported to illustrate and validate the optimal efficiency and robustness of the performance of the present multigrid strategy.