We study critical behavior in gravitational collapse of a general spherically symmetric Yang-Mills field coupled to the Einstein equations. Unlike the magnetic ansatz used in previous numerical work, the general Yang-Mills connection has two degrees of freedom in spherical symmetry. This fact changes the phenomenology of critical collapse dramatically. The magnetic sector features both type I and type II critical collapse, with universal critical solutions. In contrast, in the general system type I disappears and the critical behavior at the threshold between dispersal and black hole formation is always type II. We obtain values of the mass scaling and echoing exponents close to those observed in the magnetic sector, however we find some indications that the critical solution differs from the purely magnetic discretely self-similar attractor and exact self-similarity and universality might be lost. The additional "type III" critical phenomenon in the magnetic sector, where black holes form on both sides of the threshold but the Yang-Mills potential is in different vacuum states and there is a mass gap, also disappears in the general system. We support our dynamical numerical simulations with calculations in linear perturbation theory; for instance, we compute quasi-normal modes of the unstable attractor (the Bartnik-McKinnon soliton) in type I collapse in the magnetic sector.