Background Umbilical arterial catheterization is an extremely useful tool in the premature, sick neonate for intensive monitoring and as a source of arterial blood sampling. We describe the case of an inadvertent breakage of an umbilical artery catheter (UAC), to replace it, and the potential complications that can arise. Case Report A premature 33-week infant weighing 1.965 kg was admitted to the neonatal intensive care unit (NICU) and had umbilical catheterization. An abdominal X-ray (AXR) was done following the procedure to confirm the position, and the UAC was found to be too low, between T11 and T12. In attempting to replace it, the UAC was transected, with the umbilical stump bleeding, and the internal tip not seen. The patient underwent laparotomy and examination under anesthesia revealed a UAC that was palpable in the right lower quadrant and fluoroscopy revealed a tip in the external iliac artery. This migrated with each subsequent palpation. Meticulous dissection revealed a catheter which was palpable in the right internal iliac artery. The bifurcation was cleared, and vascular slings applied, getting proximal and distal control. The vessel was opened, and the catheter retrieved in its 15-cm entirety. The vessel was repaired with 7/0 Prolene suture. Conclusions This case is intended to highlight that even a trivial procedure of UAC removal must be done carefully with appropriate instrumentation to prevent this complication. For retrieval of this type of vascular foreign body, an open technique can be used with the assistance of fluoroscopy. Endovascular retrieval techniques are also an option but are not available in our setting for this age group.