Fracture toughness, which is the resistance of a material to crack propagation, is a critical material property for ensuring the mechanical reliability of damage-tolerant design. Recently, damage-tolerant design is introduced to flexible electronics by adopting micro-cracked ultra-thin nanocrystalline (NC) gold films as stretchable electrodes in a plane stress state. However, experimental investigation of the plane stress fracture toughness of those films remains challenging due to the intrinsic fragility from their sub-100nm thicknesses. Here, a quantitative method for systematically evaluating the plane stress fracture toughness of freestanding ultra-thin NC gold film on water surface platform is presented. After effectively fabricating single-edge-notched-tension samples with femtosecond laser, mode I stress intensity factors are measured in the plane stress state on water surface. Moreover, investigation regarding the effect of notch length, notch sharpness, and notch tip plasticity validates this method based on linear elastic fracture mechanics theory. As a demonstration, the thickness-dependent plane stress fracture toughness of ultra-thin NC gold films is qualitatively unveiled. It is revealed that the thickness confinement effect on grain boundary sliding induces a transition in fracture behavior. This method is expected to further clarify the fracture-related properties of various ultra-thin films for next-generation electronics.