Current-induced magnetization switching by spin-orbit torque (SOT) is of great importance for the energy-efficient operation of spin-based memory and logic devices. However, the requirement of an external in-plane magnetic field to deterministically switch the perpendicular magnetization of a device is a bottleneck for device application. There have been many efforts to realize field-free SOT switching using interlayer/exchange coupling, the spin valve structure, or materials with lateral symmetry breaking. However, limitations of material selection or layer structure modification hinder the application of these methods in practice. Here, we demonstrate the field-free SOT switching of a GdCo ferrimagnet with lateral symmetry breaking by He ion irradiation. Local control of the magnetic property with different He ion irradiation conditions induces a lateral magnetic gradient orthogonal to the current flow direction in the ferrimagnet. We also observe out-of-plane-SOT generation due to lateral symmetry breaking, which is essential for field-free switching. Since the He ion irradiation technique is utilized for the fabrication of complementary−metal−oxide−semiconductors, and resolution can reach the nanometer level, our findings have the potential to serve as the basis for new developments in the fabrication of wafer-scale spintronic memory and logic devices with high energy efficiency and high density.