Background: Residual Pulmonary Vascular Obstruction (RPVO) is an area of increasing focus in patients with acute pulmonary embolism (PE) due to its association with long-term morbidity and mortality. The predictive factors and the effect catheter-directed therapies (CDT) have on RPVO are still under investigation. Methods: This is a single-center retrospective review between April 2017 and July 2021. Patients with intermediate risk of PE were included. Patient variables associated with RPVO were analyzed and the degree of clot burden was quantified using the Qanadli score. Results: A total of 551 patients with acute PE were identified, 288 were intermediate risk and 53 had RPVO based on CT or V/Q scan three months post-PE. Baseline clot burden was higher in patients who received CDT compared to those who received anticoagulation alone (Qanadli score 45.88% vs. 31.94% p < 0.05). In univariate analysis, treatment with CDT showed a HR of 0.32 (95% CI 0.21-0.50, p < 0.001) when compared with anticoagulation alone. Patient variables including intermediate-high risk, sPESI ≥ 1, elevated biomarkers, RV dysfunction on imaging, malignancy, history of or concurrent DVT were also significantly associated with development of RPVO in univariate analysis. In multivariable analysis, only baseline Qanadli score (HR 13.88, 95% CI 1.42-135.39, p = 0.02) and concurrent DVT (HR 2.53, 95% CI 1.01-6.40, p = 0.04) were significantly associated with the development of RPVO. Conclusions: Catheter-directed therapy may be associated with a reduced risk of RPVO at 3 months; however, quantitative clot burden scores, such as the Qanadli score, may be stronger predictors of the risk of developing RPVO at 3 months. Further prospective studies are required.