Abstract The main objective of the present study is to develop a hexahedral C 1 continuous element for the three-dimensional nonlinear free vibration analysis of rectangular nano-plates with circular cutout based on the strain gradient elasticity theory. In the proposed element, in addition to the values of displacement components, the corresponding first, mixed second and mixed third order derivatives are taken into account as nodal values. Since arbitrary-shaped hexahedral elements are considered, spatial derivatives are presented using isoparametric formulation. To derive the nonlinear governing equations, the matrix form of kinetic and strain energies are written based on the three-dimensional strain gradient elasticity theory which can be reduced to the modified strain gradient theory (MSGT) and the modified couple stress theory (MCST). The accuracy and convergence of the numerical results are first investigated. In the parametric study, the nonlinear free vibration response of nano-plate is studied for different length scale parameters, geometrical factors and boundary conditions.