Carbon fiber-reinforced polymer (CFRP) cables are anticipated to be employed in larger, longer, and more durable structures in the engineering field. However, its anchorage devices and mechanism should be appropriately developed and improved. At present, mainly relying on the adhesive force, most anchorage devices may lose their efficiency because of adhesive aging and failure or the slip of an individual tendon. A friction-based composite anchorage device with an integrated bearing of inner cone filler (i.e., load transfer media (LTM)) bonding and a single extruding anchor is proposed, and the anchorage mechanism is examined for Φ7 CFRP cables of strength 2400 MPa. Firstly, sufficient conditions for anti-slip failure of CFRP tendons in the anchorage zone are derived by assuming uniform LTM bonding. The obtained results reveal that the smaller inner pore size of the barrel leads to higher efficiency. Additionally, the maximum efficiency depends on the friction coefficient of the contact surface, the inner cone angle of the barrel, and the diameter and quantity of the CFRP tendons. The necessary conditions for the safety of the CFRP tendon anchorage zone are carefully obtained based on the Tsai–Wu failure criterion. It is concluded that the compressive stress of CFRP tendons in the anchorage zone should gradually increase from the load-bearing end to the no-loading end. Additionally, the relations among the anchorage efficiency coefficient and the CFRP tendon diameter d, the anchorage length l, the dip angle of LTM external conical surface α, and the friction angle β are derived based on the equivalent failure principle. The CFRP cables of four specifications (i.e., with Φ12, Φ19, Φ37, and Φ121 tendons) are designed under theoretical guidance, and eight static tests are carried out for more verification studies. The test results indicate that the anchorage efficiency coefficient of designed anchorage devices can be over 90%, and even up to 96.8%. Further, the failure modes are divergent destruction, which verifies the reliability of friction-based anchorage devices and provides a solid theoretical foundation for the design and engineering applications of CFRP cables.