The linear stability characteristics of pressure-driven two-layer channel flow are considered, wherein a Newtonian fluid layer overlies a layer of a Herschel-Bulkley fluid. A pair of coupled Orr-Sommerfeld eigenvalue equations are derived and solved using an efficient spectral collocation method for cases in which unyielded regions are absent. An asymptotic analysis is also carried out in the long-wave limit, the results of which are in excellent agreement with the numerical predictions. Our analytical and numerical results indicate that increasing the dimensionless yield stress, prior to the formation of unyielded plugs below the interface, is destabilizing. Increasing the shear-thinning tendency of the lower fluid is stabilizing.