This paper explores the important roles of concrete filled steel tube (CFST) columns and inter-module connections on the structural robustness of composite modular buildings. Various numerical models were developed for concrete filled steel tube (CFST) columns, semi-rigid frames, and conventional steel buildings under column removal scenarios to verify the validation of the present study. A numerical model of a 10-storey composite modular building was then developed, in which conventional hollow steel section columns were replaced by CFST columns to improve resistance against buckling of columns. To examine the behavior and force transmission mechanism of composite modular buildings under various module removal situations, nonlinear dynamic and nonlinear static pushover analyses was conducted. The introduction of CFST columns provided resistance against buckling of columns under high axial forces. It was observed that a pin-joint inter-module connection approach is a conservative approach to model the connections between modules. It was discovered that the dynamic amplification factor (DAF) recommended by the general service administration (GSA) for the nonlinear static analysis overestimates the displacement response of the modular building structure under module loss situations. The modular building was deemed safe against progressive collapse without excessive failure of the component members. However, progressive failure of the modular building under corner module removal scenario was observed in pushover analysis due to shear failure of horizontal inter-module connections. It was observed that the corner module removal scenario is more critical as compared to the column removal scenario for modular buildings. Based on the location of module removal, the DAF values of 1.65 and 1.2 were recommended for corner module removal and internal and edge module removal, respectively.