This study aimed to compensate for the impairments in structural load-carrying capacity and stiffness caused by conventional reduced web section beams, and to meet the strength design requirements for engineering. A new hybrid high-strength steel composite cellular beam (HHS-CCB) using high-strength steel in the beam flange and ordinary steel in the web was proposed. A composite cellular beam between the rigid joint and inflection point in the frame structure was selected as the research object, and low cyclic loading tests were conducted on one homogeneous ordinary steel composite cellular beam (HOS-CCB) and four HHS-CCB specimens. The effects of the hybrid use of high-strength steel and ordinary steel on the failure modes, load-carrying capacities, strength and stiffness degradation, ductility, and energy-dissipation capacities of the composite cellular beams were investigated. The test results show that with reasonable values for the diameter of the web opening and for the distance from the first opening at the beam end to the column surface (abbreviated as the hole edge distance), the HHS-CCB specimens exhibit local plastic damage at the first opening at the end of the beam but maintain a good ductility, stiffness degradation, strength degradation, and energy dissipation capacity, while significantly improving the structural load-carrying capacity loss owing to the web opening. Compared with HHS-CCB specimens using HHS in both the upper and lower flanges, the HHS-CCB specimens using HHS only in the lower flange show superior mechanical properties. Considering the poor plastic deformation capacity of HSS, the steel plate strength of the flange should not be much higher than that of the web to ensure the full development of plasticity in the cross-section, and Q460 steel or Q550 steel is generally recommended. A nonlinear finite element (FE) model was developed using ABAQUS software to predict the hysteresis behavior of specimens. Numerical simulation shows that the simplified FE model can be used to simulate the hysteresis performance of HOS-CCB and HHS-CCB.