The decay rate of cosmological gravitational potential measures the deviation from Einstein-de Sitter universe and can put strong constraints on the nature of dark energy and gravity. Usual method to measure this decay rate is through the integrated Sachs-Wolfe (ISW) effect-large scale structure (LSS) cross correlation. However, the interpretation of the measured correlation signal is complicated by the galaxy bias and matter power spectrum. This could bias and/or degrade its constraints to the nature of dark energy and gravity. But, combining the lensing-LSS cross correlation measurements, the decay rate of gravitational potential can be isolated. For any given narrow redshift bin of LSS, the ratio of the two cross correlations directly measures $[d\ln D_{\phi}/d\ln a]H(z)/W(\chi,\chi_s)$, where $D_{\phi}$ is the linear growth factor of the gravitational potential, $H$ is the Hubble constant at redshift $z$, $W(\chi,\chi_s)$ is the lensing kernel and $\chi$ and $\chi_s$ are the comoving angular diameter distance to lens and source, respectively. This method is optimal in the sense that (1) the measured quantity is essentially free of systematic errors and is only limited by cosmic variance and (2) the measured quantity only depends on several cosmological parameters and can be predicted from first principles unambiguously. Though fundamentally limited by inevitably large cosmic variance associated with the ISW measurements, it can still put useful independent constraints on the amount of dark energy and its equation of state. It can also provide a powerful test of modified gravity and can distinguish the Dvali-Gabadadze-Porrati model from $\Lambda$CDM at $>2.5\sigma$ confidence level.