The set-top boxes based on the digital video broadcasting-cable (DVB-C) standard have been deployed worldwide for decades, and the square-root raised cosine filter is specified as its receive filter. Recent years, plenty of the preferable square-root Nyquist filters have been developed. It is eagerly desired to upgrade the filter in DVB-C system. However, because a great number of the set-top boxes are still in operation, the upgraded new set-top boxes are required to coexist with the already deployed old ones. Theoretically, it requires the receive filters in both the old and new set-top boxes satisfy the Nyquist criterion with the same transmit filter. Up to now, there is no acceptable solution to this problem. This paper proposed a backward compatible square-root Nyquist filter design method, which enables the old set-top boxes to coexist with the upgraded new one. Theoretical analysis shows that the proposed filter enables both the old and new set-top boxes to work properly, in terms of satisfying the Nyquist criterion, achieving the required stop-band attenuation and satisfying other requirements defined by the DVB-C standard. The experimental tests and simulation results have also demonstrated that the proposed filter has increased the spectrum utilization by 14% compared with the DVB-C standard, and with much lower computational complexity.