The accurate characterization of the anisotropic thermal-mechanical constitutive properties of structural sheet metals at elevated temperatures and under nonuniform stress/strain states is crucial for the precise hot plastic forming and structural behavior evaluation of an engineering sheet part. Traditional thermal-mechanical testing methods rely on the assumption of states homogeneity, leading to a large number of tests required for the characterization of material anisotropy and nonlinearity at various high temperatures. In this work, a highly efficient identification method is proposed that allows the simultaneous characterization of the anisotropic yielding, strain hardening and elasto-plasticity thermal softening material properties using the minimum number of tests, releasing the limitations of traditional identification methods. This is implemented by performing a digital image correlation and infrared thermography assisted heterogeneous high temperature test and processing the full-field measurement data based on the principle of virtual work. A double-notched tensile specimen configuration combined with a center-to-periphery temperature gradient is designed first to enable the high heterogeneity of stress/temperature states. Simulations of the heterogeneous tests are then performed to supply idealized reference data that can be used to evaluate the identification sensitivity of the proposed identification algorithm. After the numerical validation of the methodology, it is then applied to the TC4 titanium alloy sheet specimen using the self-developed experimental setup. The experimental identification results verify that the multiple anisotropic thermal-mechanical elasto-plasticity constitutive parameters can be accurately identified from the heterogeneous test with high computation efficiency, significantly simplifying the testing process in comparison to applying multiple traditional homogeneous tests. The current work provides an effective and convenient alternative to high temperature identification strategies used by the material processing community.