Due to the lack of inertial support from photovoltaic (PV) systems, large-scale PV integration into the existing grid poses a major threat to frequency stability. To address this, numerous techniques for incorporating energy storage systems have been proposed in the existing literature, but they incur significant operational costs. PV systems with deloading mechanisms have been researched as an alternative efficient strategy in many studies as a key frequency response support after a disturbance. However, deloaded PV systems incur a cost burden due to idle reserve capacity during regular operation. Consequently, an optimum deloading of PV systems is required to enhance the grid's frequency stability while minimizing its total operating cost. To that aim, this paper proposes a gradient descent-based optimization method to determine the optimum deloading of PV systems. The optimization considers both frequency response constraints (frequency nadir and rate of change of frequency – ROCOF) and operating cost constraints (generation, up-regulation, and value of lost load cost). The proposed technique is employed in a modified IEEE 39 bus New England System for varying levels of PV integration under two distinct cases. The optimization and relevant simulations are performed in Python and DIgSILENT PowerFactory platforms. In the case of a major generator outage with PV penetrations of 30%, 40%, and 50%, the optimal deloading percentages are 6.96%, 5.931%, and 4.968%, respectively, with no load shedding. Moreover, in the interconnection outage case, the optimal deloading percentages increase to 14.92%, 13.3%, and 12.42%, respectively, with load shedding. Nevertheless, in each case, deloaded PV systems consistently preserve frequency stability in the grid while incurring minimized operating costs. Furthermore, the performance and cost of the proposed solution are compared to two other primary frequency response support systems, viz. Battery Energy Storage System (BESS) and Synchronous Condenser (SC). The optimally deloaded PV system outperforms both these devices considering both technical and economic aspects.