Asphalt binder is a viscoelastic material that works as a glue to bind the aggregate particles to form an asphalt concrete composite. Fatigue cracking is a predominant form of distress in the asphalt pavements that is primarily dictated by the ability of the asphalt binder to resist tensile loads. The motivation of this study was to quantify the tensile strength of the asphalt binder under a confined state of stress that is similar to what the binder experiences in an asphalt concrete mixture. A poker chip geometry was used to simulate the confined state. Laboratory tests were performed on binders of different grades by varying the film thickness, rate of loading and modes of loading. These three factors influence the stress–strain behavior and failure stress of the confined asphalt binder films. Within the linear viscoelastic limit and before damage is induced, the differences due to the rate and mode of loading could be successfully explained and unified using modified correspondence principle. Beyond this limit, the differences due to rate and mode of loading could be successfully explained and unified using a power law damage evolution model following the methodology referred to as the viscoelastic continuum damage model. Examination of the failure surfaces and true internal stresses within the poker chip geometry reveals that tensile failure occurs in the form of cavitation as long as a certain degree of confinement is achieved.