Based on two-temperature model and free-electron gas model, a new calculation method, finite element model is fabricated, which provides a new perspective into studying the energy transport process in silicon film irradiated by ultrashort laser pulses. By choosing suitable thickness of silicon films, the 2D spatial and temporal evolutions of the electron temperature as well as carrier density in silicon film irradiated by IR and visible lasers are obtained. The evolutions of complex refractive index and plasma reflectivity are also calculated, through analysis, results show that they are dominated by carrier density. By depicting the electron heat flux evolutions, the energy transport process is analyzed. The distributions of lattice and carrier density are depicted, results show that the lattice temperature is stayed well down below the melting point, and the critical density of carrier density is the dominated factor of ablation. The calculated threshold influences are validated by comparing study and experimental data. The predicted crater shapes are obtained, which are waiting for experimental validating.