Laser peening can introduce compressive residual stresses on the surface of various materials, thereby effectively prolonging their fatigue lives. In this study, the effects of laser peening with pulse energies of 20 and 70 mJ on residual stress and fatigue life were investigated on two types of HT780 box-welded specimens of different sizes, considering the realization of a portable laser peening system equipped with a small laser device with low pulse energy. As a result of such low pulse energy laser peening, it was found that the depth of compressive residual stress becomes shallower compared to the current laser peening with a pulse energy of 200 mJ, while the residual stress on the surface remains at the same level. Bending fatigue tests were conducted with the stress ranges at 100 and 150 MPa, which revealed that the fatigue life of the specimens with low pulse energy laser peening was at least 50 times and eight times longer than that of the as-welded specimens, respectively. Comparing the fatigue test results of different size specimens, it is clear that laser peening with low pulse energy is effective in extending the fatigue life of HT780 box-welded specimen regardless of its size.