Hydrogen-enhanced localized plasticity (HELP) has been recognized as an important mechanism for hydrogen embrittlement. Two recognized explinations have been proposed for HELP. The first one is the elastic shielding theory, in which the elastic stress field of solute hydrogen weakens the interactions between dislocations. Another one is the Gibbs theory of absorption isotherm: the lowered dislocation line energy by segregated hydrogen is considered as the main reason. In this work, a dislocation-density based crystal plasticity framework concerning the explicit incorporation of the dislocation line energy is proposed. The explicit consideration of dislocation line energy leads the way to the modelling of HELP mechanism according to the Gibbs theory of absorption isotherm. It is shown that the experimentally observed hydrogen-reduced activation volume and total activation free energy in thermally activated forest intersection in face-centered cubic (FCC) metals can be easily attributed to the reduction of dislocation line energy in the hydrogen environment. Finite element calculations of tensile tests for polycrystalline Pd-H and Ni-H alloys capture several important hydrogen-affected plasticity behaviors: hydrogen-increased flow stress, hydrogen-enhanced dislocation multiplication, hydrogen-promoted heterogeneity of plastic strain and hydrogen-delayed exhaustion of mobile dislocations, which can be all attributed to hydrogen reduced dislocation line energy in the present model. Besides, the influences of hydrogen-reduced vacancy formation free energy and stacking fault energy on thermal annihilation are also considered. However, the simulation results show that they are less important compared with hydrogen-reduced dislocation line energy. The present work is essential for further physically-based modeling of hydrogen distribution in polycrystals and hydrogen-induced damage and failure.