Electronic synapses with both long-term and short-term plasticity are considered as significant components for constructing brain-inspired computing systems. Research progress on electrical synapses have proved that memristors possess huge similarities with biological synapses. Nevertheless, an effective mean of manipulating the biological properties of memristors is still unclear. In this letter, we propose a memristor and reveal that the compliance current of electroforming plays an active role in tuning short-term and long-term plasticity of the memristors. The results may provide a useful guideline for manipulating memristor as electronic synapses in the hardware implementation of artificial neural networks.