A coupled higher-order nonlinear Schrodinger system, which describes the ultrashort pulses in a birefringent optical fiber, is analytically investigated. For the complex envelopes of the electric field in the fiber, we construct a Lax pair which is different from that in the existing literature, and derive out the corresponding first- and second-order breather solutions. We present the first- and second-order breather-to-soliton conversion conditions, related to the strength of the higher-order linear and nonlinear effects. We find that the strength affects the peak numbers of solitons, and see the multi-peak soliton, W-shaped soliton, M-shaped soliton, anti-dark soliton and two kinds of periodic waves. From the second-order breather solutions under the first-order breather-to-soliton conversion condition, interactions between the breather and a W-shaped soliton, an M-shaped soliton or periodic waves are given. From the second-order breather solutions under the second-order breather-to-soliton conversion conditions, we present the interactions between the two M-shaped solitons, the two anti-dark solitons, a W-shaped soliton and an M-shaped soliton, or a W-shaped soliton and two kinds of the periodic waves. Those results might provide certain assistance for the studies on the birefringent optical fibers.