This paper proposes an assembled beam-to-double skin composite wall (DSCW) joint embedded with insert diaphragms. The insert diaphragms are designed to be welded to the slotted boundary column and bolted on-site to the steel beam, functioning as a corbel beam. This joint presents a suite of advantages such as eliminating on-site welding, easy decoration, and non-protruding from the column profile. To evaluate the hysteretic performance and failure model of the proposed joint, a full-scale quasi-static low-cycle hysteresis test on an individual specimen was conducted. The experimental results demonstrate that the insert-diaphragm joint has good performance in high ductility, favorable plastic rotation capacity, and energy dissipating capacity. A refined finite element model (FEM) was established and verified through the test results. The parametric analysis was conducted to analyze the influences of axial compression ratio, geometrical dimensions of the insert diaphragm, and column thickness on the stiffness and yield strength of the joint. Based on the experimental results and FEM analyses, an alternative mechanical model with two potential failure mechanisms was proposed, in which the connection strength was contributed by the insert diaphragm, the faceplate and the side plate of the column. An optimized plastic hinge relocation configuration was also investigated by FEM to further improve the seismic performance of the joint. Finally, the design method and configurational suggestions were proposed based on the Chinese specifications to facilitate practical applications/design of the proposed insert-diaphragm connection.