Harvesting low-density ambient microwave power as an alternative power source for small ubiquitous wireless nodes has been proposed in recent papers discussing emerging technologies like the Internet of Things and Smart Cities. However, a literature review of the state-of-the-art Schottky diode based microwave rectifiers shows that a maximum efficiency has been reached for such devices operating in the low-power regime, as is the case for ambient microwave power-harvesters. This work examines the underlying physical mechanisms responsible for this RF-to-dc power conversion efficiency limitation, and explores a high I-V curvature backward tunnel diode to overcome this efficiency limitation. Measurements of the 2.4 GHz RF-to-dc power conversion efficiency at $-$ 40 dBm input power demonstrates that the backward tunnel diode outperforms the HSMS-285B Schottky diode by a factor of 10.5 and the Skyworks SMS7630 by a factor of 5.5 in a lossless matching network scenario. A prototype built using a new GSG probe embedded with a matching circuit showed a total power conversion efficiency of 3.8% for $-$ 40 dBm input power and 18.2% for $-$ 30 dBm input power at 2.35 GHz.