We present a systematic theoretical investigation of second-harmonic generation (SHG) in a subwavelength lithium niobate (LN) single-crystal thin-film waveguide with birefringence in permittivity and refractive index and strong anisotropy in the nonlinear susceptibility ${\ensuremath{\chi}}^{(2)}$. In a macroscopic birefringence LN crystal, the birefringence phase matching scheme that makes use of the refractive index management between the extraordinary light (EL) and ordinary light (OL) is dominantly used to ensure high-efficiency SHG. Yet, this scenario needs significant modification in a subwavelength LN thin film, where one should instead consider the waveguide modes of fundamental wave (FW) and second-harmonic wave (SHW) to participate in the SHG process. To this end, we first analytically solve and calculate the dispersion and modal profile of optical waveguide modes supported in the birefringence LN planar waveguide in a broad bandwidth ranging from ultraviolet to infrared regimes for TM (EL-like) and TE (OL-like) polarization states and at different waveguide thicknesses. Then we systematically examine the phase matching conditions that are necessary for high-efficiency energy conversion from FW waveguide mode to SHW waveguide mode via SHG. We find several modal phase matching schemes by exploiting the versatile freedoms of the FW and SHW waveguide mode number and polarization, and the waveguide thickness and symmetry. Yet modal phase matching only takes place at very limited discrete wavelengths of FW pump light. Under the modal phase matching condition, we derive the nonlinear coupled mode theory from Maxwell's equations and calculate the conversion efficiency of SHG, and find that the efficiency of SHG is determined by the nonlinear coupling coefficients that are closely associated with the modal profile overlap extent and mode transition strength between FW mode and SHW mode, and the involved nonlinear coefficient components of ${\ensuremath{\chi}}^{(2)}$. Finally, we have analytically derived the explicit formula that allows for direct calculation of the SHG efficiency as a function of the interaction length of FW mode and SHW mode, the nonlinear coupling coefficient, and the FW pump intensity, for both the small signal and general large signal situations. Our analytical theory and formulation can greatly facilitate the deep understanding of nonlinear optical interaction physics and provide a convenient tool for accurate quantitative evaluation of the SHG energy conversion efficiency of a pump laser in nanoscale LN single-crystal thin films. This analytical theory can also benefit for the design of nanoscale nonlinear optical devices based on the material platform of LN and other nonlinear crystal thin films.