Highlight safety considerations in intravascular brachytherapy (IVBT) programs, provide relevant quality assurance (QA) and safety measures, and establish their effectiveness. Radiation oncologists, medical physicists, and cardiologists from three institutions performed a failure modes and effects analysis (FMEA) on the radiation delivery portion of IVBT. We identified 40 failure modes and rated the severity, occurrence, and detectability before and after consideration of safety practices. Risk priority numbers (RPN) and relative risk rankings were determined, and a sample QA safety checklist was developed. We developed a process map based on multi-institutional consensus. Highest-RPN failure modes were due to incorrect source train length, incorrect vessel diameter, and missing prior radiation history. Based on these, we proposed QA and safety measures: ten of which were not previously recommended. These measures improved occurrence and detectability: reducing the average RPN from 116 to 58 and median from 84 to 40. Importantly, the average RPN of the top 10% of failure modes reduced from 311 to 172. With QA considered, the highest risk failure modes were from contamination and incorrect source train length. We identified several high-risk failure modes in IVBT procedures and practical safety and QA measures to address them.