The rate- and temperature-dependence of the macro-yield behavior, as one of the important concerns for the investigation and application of amorphous glassy polymers, is very significant below and near the glass transition temperature (Tg). Two molecular chain microstructures, i.e., sub-entanglement (SE) and topological entanglement (TE) and their evolution, are introduced as the inherent micro-mechanisms for the rate- and temperature-dependent macro-yield performance. With the introduced entanglement microstructures-based internal variables, an elasto-viscoplastic finite deformation constitutive model of amorphous glassy polymers is proposed to describe its rate- and temperature-dependent macro-yield response. The rate-dependence of Tg and its effect on the microstructures’ thermo-mechanical state and evolution are also taken into account. Comparing with the experimental results of PMMA in literature, the proposed model shows a good capability to simulate and predict the macro-yield behavior at various strain rates (from 0.0003 s−1 to 0.1 s−1) and temperatures (from room temperature to Tg). This work illustrated that the introduction of SE and TE as the inherent micro-mechanisms is a reasonable interpretation for the macro-yield behavior of amorphous glassy polymers.