We propose and analyze a unified structure-preserving parametric finite element method (SP-PFEM) for the anisotropic surface diffusion of curves in two dimensions ( d = 2 ) (d=2) and surfaces in three dimensions ( d = 3 ) (d=3) with an arbitrary anisotropic surface energy density γ ( n ) \gamma (\boldsymbol {n}) , where n ∈ S d − 1 \boldsymbol {n}\in \mathbb {S}^{d-1} represents the outward unit vector. By introducing a novel unified surface energy matrix G k ( n ) \boldsymbol {G}_k(\boldsymbol {n}) depending on γ ( n ) \gamma (\boldsymbol {n}) , the Cahn-Hoffman ξ \boldsymbol {\xi } -vector and a stabilizing function k ( n ) : S d − 1 → R k(\boldsymbol {n}):\ \mathbb {S}^{d-1}\to {\mathbb R} , we obtain a unified and conservative variational formulation for the anisotropic surface diffusion via different surface differential operators, including the surface gradient operator, the surface divergence operator and the surface Laplace-Beltrami operator. A SP-PFEM discretization is presented for the variational problem. In order to establish the unconditional energy stability of the proposed SP-PFEM under a very mild condition on γ ( n ) \gamma (\boldsymbol {n}) , we propose a new framework via local energy estimate for proving energy stability/structure-preserving properties of the parametric finite element method for the anisotropic surface diffusion. This framework sheds light on how to prove unconditional energy stability of other numerical methods for geometric partial differential equations. Extensive numerical results are reported to demonstrate the efficiency and accuracy as well as structure-preserving properties of the proposed SP-PFEM for the anisotropic surface diffusion with arbitrary anisotropic surface energy density γ ( n ) \gamma (\boldsymbol {n}) arising from different applications.