A planar optical waveguide mode solver is established based on a finite-element (FE) formulation for determining the guided and leaky modes that exist on waveguides made of anisotropic materials with an arbitrary permittivity tensor, for example, with arbitrary optic-axis orientation in the uniaxially anisotropic material case. Correct numerical determination of the complex effective index, especially its imaginary part that gives the modal leakage, is particularly emphasized referring to available analytical solutions. For the situation when the optic axis changes its direction only in the plane parallel to the waveguide interface planes, analytical characteristic equations for solving purely guided and leaky modes are separately derived in a more systematic manner compared with prior analytical formulae reported more than three decades ago, with the obtained complex effective indices agreeing excellently with FE solutions. It is found that in the FE analysis of leaky modes, the thickness of the perfectly matched layer (PML) and the PML theoretical reflection coefficient should be properly chosen. The FE formulation is based on either the three electric-field components or the three magnetic-field components using quadratic nodal bases, resulting in a quadratic eigenvalue equation that is then solved by the shift-and-invert Arnoldi method.