We discuss universal features on the electron spin resonance (ESR) of a temperature-induced Tomonaga-Luttinger liquid phase in a wide class of weakly coupled $S=1/2$ antiferromagnetic spin chains such as spin ladders, spin tubes and three-dimensionally coupled spin chains. We show that the ESR linewidth of various coupled chains increases with lowering temperature while the linewidth of a single spin chain is typically proportional to temperature. This broadening with lowering temperature is attributed to anisotropic interchain interactions and has been indeed observed in several kinds of three-dimensional (3D) magnets of weakly coupled spin chains above the 3D ordering temperature. We demonstrate that our theory can account for anomalous behaviors of the linewidths in an $S=1/2$ four-leg spin tube compound Cu$_2$Cl$_4 \cdot $H$_8$C$_4$SO$_2$ (abbreviated to Sul-Cu$_2$Cl$_4$) and a three-dimensionally coupled $S=1/2$ spin chain compound CuCl$_2\cdot 2$NC$_5$H$_5$.