Parking is a necessary component of traditional last-mile delivery practices, but finding parking can be difficult. Yet, the routing literature largely does not account for the need to find parking. In this paper, we address this challenge of finding parking through the Capacitated Delivery Problem with Parking (CDPP). Unlike other models in the literature, the CDPP accounts for parking time in the objective and minimizes the completion time of the delivery tour. Parking time represents the process of the delivery person searching for a parking spot and then parking the vehicle at the chosen location. When we restrict the customer geography to a complete grid, we identify conditions for when a Traveling Salesman Problem (TSP) solution that parks at each customer is an optimal solution to the CDPP. We then determine when the parking time is large enough for the CDPP optimal solution to differ from this TSP solution. We introduce a heuristic for the CDPP that quickly finds high quality solutions to large instances. Computational experiments show that parking matters in last-mile delivery optimization. The CDPP outperforms industry practice and models in the literature showing the greatest advantage when the parking time is high. This analysis provides immediate ways to improve routing in last-mile delivery.