The smoothed finite element method (S-FEM) is known for its outstanding performance for solid mechanics problems, and working effectively with triangular or tetrahedral mesh that can be generated automatically for complicated geometries. In this work, a framework of S-FEM for modeling anisotropic crystalline plasticity is presented to simulate the mechanical behavior with rate-independence. The strain smoothing technique is extended to deal with finite strains in a nonlinear incremental integration procedure based on the Newton–Raphson scheme. The constitutive model utilizes a hyperelastic-based multiplicative plasticity method, which involves a local multiplicative decomposition of the deformation gradient into an elastic and a plastic part. The stress updates for a planar double-slip model exploit the return-mapping method with exponential map algorithm. The capability of the simulations to capture the strain localization and to handle plastic incompressibility of single crystal are demonstrated in representative examples. The proposed formulations and algorithms are also implemented to explore the mesoscopic and macroscopic elasto-plastic behavior of polycrystalline aggregates through modeling the synthetic microstructure constructed by Voronoi tessellation technique.