Electromagnetic tracking (EMT) has been researched for brachytherapy applications, showing a great potential for automating implant reconstruction, and overcoming image-based limitations such as contrast and spatial resolution. One of the challenges of this technology is that it does not intrinsically share the same reference frame as the patient's medical imaging. To present a novel phantom that can be used for a comprehensive quality assurance (QA) program of brachytherapy EMT systems and use this phantom to validate a novel applicator-based registration method of EMT and image reference frames for gynecological (GYN) interstitial brachytherapy. Eleven 6F-catheters (20cm long), one 6F round tip catheter (29.4cm long) and a tandem and ring gynecological applicator (Elekta, CT/MR 60°, 40mm long tandem, 30mm diameter ring) were placed in a rigid custom-made phantom (Elekta Brachytherapy, Veenendaal, The Netherlands) to reconstruct their geometry using a five-degree of freedom EMT sensor attached to an afterloader's check cable. All EMT reconstructions were done in three different environments: disturbance free (no metal nearby), computed tomography (CT)-on-rails brachytherapy suite and magnetic resonance imaging (MRI) brachytherapy suite. Implants were placed parallel to a magnetic field generatorand were reconstructed using two different acquisition methods: step-and-record and continuous motion. In all cases, the acquisition is performed at a rate of approximately 40Hz. A CT scan of the phantom inside a water cube was obtained. In the treatment planning system (TPS), all catheters in the CT images were manually reconstructed and the applicator reconstruction was achieved by manually placing its solid 3D model, found in the applicator library of the TPS. The Iterative Closest Point and the Coherent Point Drift algorithms were used, initialized with four known points, to register both EMT and CT scan reference frames using corresponding points from the EMT and CT based reconstructions of the phantom, following three approaches: one gynecological applicator, four interstitial catheters inside four calibration plates having an S-shaped path, and four 5mm diameter ceramic marbles found in each of the four calibration plates. Once registered, the registration error (perpendicular distance) was computed. The absolute median deviation from the expected value for EMT measurements in the disturbance free environment, CT-on-rails brachytherapy suite, and MRI-brachytherapy suite are 0.41, 0.23, and 0.31mm, respectively, while for the CT scan it is 0.18mm. These values significantly lie below the sensor's expected accuracy of 0.70mm (p<0.001), suggesting that the environment did not have a significant impact on the measurements, given that care is taken in the immediate surroundings. In all three environments, the two acquisitions and three registration approaches have mean and median registration errors that lie at or below 1mm, which is lower than the clinical acceptable threshold of 2mm. The novel phantom allowed to successfully evaluate the accuracy of EMT-based reconstructions of catheters and a GYN tandem and ring applicator in different clinical environments. A registration method based only on the applicator geometry, reconstructed withan EMT sensor and the TPS solid applicator library, was validated and shows clinically acceptable accuracy, comparable to CT-based reconstruction but within a few minutes. Since the applicator is also visible in MRI, this method could potentially be used in clinics in an EMT-MR interstitial GYN brachytherapy workflow.