Piezoelectric stick-slip actuators (PSSAs) utilize sliding friction between the mover and stator to convert and transmit motion. However, the phenomenon of backward displacement often hinders the output performance of PSSAs. This paper proposes a method to mitigate backward displacement and enhance output performance by modifying the overall flexibility of the actuator. The key idea of this approach is to propose a novel flexible hinge structure and apply it to PSSA. Numerical calculations and finite element analysis confirm that the flexibility and output performance of the PSSA are significantly improved. The method's feasibility is supported by comparing experiments. The experimental results show that under the same locking force, the optimal excitation frequency of perforated Elliptical Flexure Hinge (EFH) is significantly lower than the non-perforated EFH and the speed is increased over 53 %. Furthermore, the PSSA has a maximum load capacity of 190 g, which is 31.7 times its own weight (6 g). The proposed PSSA can provide valuable insights for its application in precision motion control systems in the foreseeable future.