Lateral confinement and high temperature affect the load-carrying capacity of a concrete member significantly. In a fire condition, a concrete member exhibits not only mechanical deformation but also thermal deformation. Therefore, accurately modeling the fire behavior of confined structural concrete is challenging. In this paper, a new three-dimensional thermo-plasticity model for concrete is developed, taking both high temperature and confinement into account. A temperature-dependent yield surface and a non-associate flow rule are formulated. The hardening rules for concrete are proposed as a function of confinement and temperature. The load-induced strain of concrete at elevated temperatures is calculated by using a stress-triaxiality factor to consider the confinement effect. The proposed concrete model is implemented in the user-subroutine (UMAT) of Abaqus, in which Runge-Kutta-England scheme with sub-stepping is used to update stress increment. The accuracy of the proposed model is validated by experiments on concrete specimens, circular concrete-filled steel tubular (CFST) columns, double-skin CFST (DCFST) columns, and concrete-filled double steel tubular (CFDST) columns. It is shown that the developed thermo-plasticity model of concrete accurately captures the behavior of confined concrete at ambient and elevated temperatures and can be incorporated in nonlinear procedures for simulating the fire resistance of composite columns exposed to fire.