Signed-digit number systems support carry-free, constant time addition. By introducing the signed-digit number arithmetic into a residue number system (RNS), arithmetic operations can be performed efficiently. In this paper, a new algorithm for residue-to-binary conversion for four moduli set {2n-1, 2n + 1, 22n + 1, 2n} that only requires modulo 24n - 1 SD number addition is proposed. This moduli set has 5n-bit dynamic range. Based on the proposed algorithm, the converters are designed with a two-level binary tree structure formed by the modulo 24n - 1 SD number residue adders. Moreover, we simplify the residue adders in converters to obtain more area and time efficiency. The comparison of the converters proposed with the converter using binary arithmetic using 0.18 μm CMOS gate array technology yields reductions in delays of 44%, 60% and 75% for n = 4, n = 8 and n = 16, respectively.