The source of high–brilliance, hard x-ray radiation in light sources and free electron lasers are undulator insertion devices. Recently, REBCO-based superconductors have been demonstrated to be a potential candidate to enhance the performance of the undulator insertion devices. Epoxy impregnation remained as one of the steps which is crucial to fully realize this technology. Epoxy impregnation of magnets is required because it prevents the motion of the wire and provides conduction cooling to the winding layers. Up until now, most of the impregnated REBCO coils/magnets showed some degree of degradation after cold cycles. We developed a vacuum impregnation technique that does not degrade the performance of the short REBCO prototype undulator magnets after cold cycles. The results showed that in order to prevent degradation, a bumper layer between the magnet winding stacks and the epoxy/powder mixture is required. Microstructure images of vacuum impregnated coil packs showed homogenously-spaced REBCO winding layers and very thin epoxy fillings between the layers, which is important for the overall performance of the device.