The output feedback $H_\infty$ control is addressed for a class of continuous-time Markov jump linear systems with the Markov process taking values in an infinite countable set $\mathcal{S}$. We consider that only an output and the jump parameters are available to the controller. Via a certain bounded real lemma, together with some extensions of Schur complements and of the projection lemma, a theorem which characterizes whether there exists a full-order solution to the disturbance attenuation problem is devised in terms of two different linear matrix inequality (LMI) feasibility problems. This result connects a certain projection approach to an LMI problem which is more amenable to computer solution, and hence for design. We conclude the paper with some design algorithms for the construction of such controllers and an illustrative example.