Graphene is one of the most promising candidates for integrated circuits due to its robustness against short-channel effects, inherent high carrier mobility and desired gapless nature for Ohmic contact, but it is difficult to achieve satisfactory on/off ratios even at the expense of its carrier mobility, limiting its device applications. Here, we present a strategy to realize high back-gate switching ratios in a graphene monolayer with well-maintained high mobility by forming a vertical heterostructure with a black phosphorus multi-layer. By local current annealing, strain is introduced within an established area of the graphene, which forms a reflective interface with the rest of the strain-free area and thus generates a robust off-state via local current depletion. Applying a positive back-gate voltage to the heterostructure can keep the black phosphorus insulating, while a negative back-gate voltage changes the black phosphorus to be conductive because of hole accumulation. Then, a parallel channel is activated within the strain-free graphene area by edge-contacted electrodes, thereby largely inheriting the intrinsic carrier mobility of graphene in the on-state. As a result, the device can provide an on/off voltage ratio of >103 as well as a mobility of ∼8000cm2 V-1 s-1 at room temperature, meeting the low-power criterion suggested by the International Roadmap for Devices and Systems.