Due to the specific gravity of recycled brick aggregate (RBA) is less than that of natural aggregate (NA), the traditional mix proportion design of concrete can not be used. The purpose of this study is to propose a new mix proportion design method, which is suitable for recycled brick aggregate concrete (RBAC) or lightweight aggregate concrete. In this test, RBA volume fraction, sand volume fraction and water/cement ratio were considered; calculation formula of paste thickness on coated aggregate was established; calculation model of RBAC compressive strength was established, in which RBA volume fraction, the compressive strength of cement mortar and water/cement ratio were considered; mix proportion design method of RBAC based on aggregate skeleton theory was established. The results show that: when the volume fraction of RBA is between 44% and 47%, and the total volume fraction of RBA and sand is about 70%, the compressive strength of RBAC reaches the maximum; when the volume fraction of RBA is between 40% and 50%, the skeleton contribution strength of RBA is greater than that of cement mortar. When sand/cement ratio is about between 0.35 and 2.1, the sand skeleton contribution strength is higher. When the volume fraction of RBA is between 40% and 50%, the actual average paste thickness on coated coarse aggregate (APTCCA) is between 10.04 mm and 5.86 mm and the APTCCA simulated by PFC3D software is between 9.21 mm and 5.75 mm. In the compressive strength calculation model of RBAC, the relative errors of the predicted values are concentrated within 10%, the average relative error is 0.53%, and the correlation coefficient between the predicted values and the test values is 0.96; Compared with ACI 211.2 and ACI 211.6, the mix proportion design method proposed in this paper is to mix more RBA and less sand and cement in concrete mix proportion under the condition of ensuring concrete strength and slump, which is beneficial to protect the environment.