Robustness, imperceptibility, and watermark capacity are three indispensable and contradictory properties for any image watermarking systems. It is a challenging work to achieve the balance among the three important properties. In this paper, by using the bivariate Beta mixture model (Bivariate BMM), we present a statistical image watermark scheme in nonsubsampled Contourlet transform (NSCT)-fractional order Jacobi Fourier moments (FJFMs) amplitude hybrid domain. The whole watermarking algorithm includes two parts: watermark embedding and detection. NSCT is firstly performed on host image to obtain the frequency subbands, and the NSCT subbands are divided into non overlapping blocks. Then, the significant NSCT domain blocks are selected using local binary patterns (LBP). Meanwhile, for each selected NSCT coefficient block, FJFMs are calculated to obtain the NSCT-FJFMs amplitude. Finally, watermark signals are inserted into the amplitude hybrid domain of NSCT-FJFMs. In order to detect accurately watermark signal, the statistical characteristics of NSCT-FJFMs magnitudes are analyzed in detail. Then, NSCT-FJFMs magnitudes are described statistically by Bivariate BMM, which can simultaneously capture the marginal distribution and strong dependencies of NSCT-FJFMs magnitudes. Also, Bivariate BMM parameters are estimated accurately by the rough-enhanced-Bayes mixture estimation & expectation–maximization (REBMIX&EM) approach. Finally, a statistical watermark detector based on the locally optimum (LO) decision rule and Bivariate BMM is developed in NSCT-FJFMs magnitude hybrid domain. Also, the closed-form expressions on the LO statistic is derived and the receiver operating characteristic (ROC) about our detector is analyzed. Extensive experimental results show the superiority of the proposed image watermark detector over some state-of-the-art statistical watermarking methods.