The strain dependence of the Johari-Goldstein (JG)-β relaxation time, as well as the directional dependence, was systematically investigated for stretched cross-linked polybutadiene using time-domain interferometry. We found that the strain dependence of the JG-β relaxation time is directionally dependent, contrary to expectation: the relaxation time of the JG-β motion, whose displacement is perpendicular to the stretching direction, decreases with stretching, whereas the relaxation time of the parallel JG-β motion changes little. This result is distinct from the previously reported strain dependence of the α relaxation time, where the relaxation time increases isotropically with stretching. Thus, the difference in the strain dependence of the relaxation time between the α and JG-β processes suggests a microscopic origin and requires the modification of the conventional dynamic picture for stretched polymers.