Due to the wavelength-dependent absorption and scattering, underwater images are often degraded by color distortion, haze, and low contrast. Despite the great advancements achieved in improving the quality of underwater image, the loss of contour and texture details in the restored results remains a major obstacle for subsequent high-level visual tasks. To this end, we develop a novel variational model, including one data term and two fidelity terms, for underwater image restoration. Technically, according to Retinex theory, we first extend the traditional underwater image formation (UIF) model by decomposing the total radiance into the reflectance component and illumination component. Based on the extended UIF model, two low-rank penalty regularization terms are designed to constrain the gradients of reflectance and illumination. Furthermore, we innovatively incorporate the weighted fusion bright-dark channel algorithm and the fusion upper-middle channel prior algorithm into the proposed variational model for estimating the local background light (ambient illumination) and transmission map, respectively. Simultaneously, based on the alternating direction multiplier method (ADMM), we develop a fast optimization algorithm to speed up the iterative process. Extensive experiments demonstrate that the proposed method is effective in removing haze, enhancing contrast and sharpness, as well as preserving the image contour and texture details. Comprehensive comparisons on two large-scale underwater image datasets further validate the superiority of our method against several state-of-the-art methods in both qualitative and quantitative evaluations. The code of the proposed method will be available soon.