For the purpose of investigating the modified flame-retardant epoxy resin (FREP) with the low smoke density release during combustion, the flame retardant containing P/N/B elements named DBT was synthesized with the raw materials of 4-Acetylphenylboronic acid, 3,5-diaminotriazole and DOPO. The DBT was added as a co-curing agent to an amine-cured epoxy resin system, and based on the DSC results of the resin, it was revealed that -NH- in the structure of the DBT was capable of facilitating EP curing. With the introduction of DBT, the transparency of FREP samples was slightly affected. On account of the excellent flame retardancy exerted by the DBT in the FREP system, the FREP samples reached the V-0 grade in UL-94 testing with an LOI of 35.9% at 5 wt% DBT addition. Meanwhile, the results of the cone calorimetry test demonstrated that in comparison with the epoxy resin, the PHRR, THR and av-EHC of the EP/DBT7.5 sample decreased by 34.0%, 35.4% and 18.68%, respectively. The DBT was effective in reducing the smoke density of EP, and the EP/DBT7.5 sample attained the HL1 level for DS (4) and VOF4. The chemical analyses for residual char revealed that DBT was mainly employed for flame retardancy and smoke suppression by forming P/B-containing chars in the condensed phase. There was no loss of mechanical properties of the FREP samples as the rigid groups were present in the DBT structure. Furthermore, it was noted that the FREP samples exhibited a decrease in dielectric loss and dielectric constant as the content increased.