Recently, several ownership protection schemes which combine encryption and secret sharing technology have been proposed. To reveal the original message, however, they exploited XOR operation which is similar to a one-time pad. It is fairly losing the reconstruction simplicity due to the human visual system (HVS). It should be noted that it is completely different from the original concept of visual cryptography proposed by Naor and Shamir. To decrypt the secret message, Naor and Shamir’s concept stacked k transparencies together. The operation solely does a visual OR of the shares rather than XOR, the way HVS does. In this paper, we, consequently, adopt Naor and Shamir’s concept to apply correct theory of visual cryptography. Furthermore, audio copyright protection schemes which exploit chaotic modulation or watermark integration into frequency components have been widely proposed. Nevertheless, security issue against intentional distortions has not been addressed yet. In this paper, we aim to construct a resilient audio ownership protection scheme to enhance the security by integrating the discrete wavelet transform and discrete cosine transform, visual cryptography, and digital timestamps. In the proposed scheme, the watermark does not require to be embedded within the original audio but is used to generate a secret image and a public image. The watermark is then acquired by performing OR between the secret and public image. We can alleviate the trade-off expenses between the capacity of data payload and two other important properties such as imperceptibility and robustness without modifying the original audio signals. The experiments against a variety of audio signals processing provided by StirMark confirm superior robustness of the proposed scheme. We also demonstrate the intentional distortion by modifying the original content via experiments, it reveals comparable reliability. The proposed scheme can be widely applied to the area of audio ownership protection.