Modulation filter banks can be obtained by modulating one or two prototype filters, which reduce the design complexity of the filter banks. Furthermore, this kind of filter can be realized by using DFT or FFT with very high computationally efficiency. The proposition of a modulation filter bank reduces the computation complexity of designing the filter bank consumedly, which has very high computation efficiency. However, with the increasing of channels, the request for transition of the prototype filter is very high. This makes the length of the prototype filter longer, which decreases the computation efficiency. Frequency-response masking technique, which was proposed by Lim in 1986, is a very efficient way for designing the sharp transition filter. It is composed of three sub-filters and a delay chain. The basic thought is designing a low complexity filter before compressing its frequency response, and then eliminating the useless periodic images of the prototype filter and its compensation filter by two masking filters. The complexity of the FRM filter can also be reduced by modifying its architecture. According to it, the paper deduces a Novel FRM low-pass filter based on Complex Exponential Modulation (CEM). The deduced filter is very suitable to the multi-rate digital signal process, apart from applying a single rate digital signal process. In the application of software defined radio, the most bandwidth needs to be separated because it may has many narrowband channels. The need is very stringent for the filter to attenuate the closer channels. The filter banks encounter a main problem when it is used to separate channels, which is that the prototype filter must be a sharp transition bandwidth filter. Furthermore, the length of the prototype filter becomes longer in the DFT filter banks when the number of sub-filters is large and the transition band is very sharp. So it is very important to use a high efficiency digital filter, which can reduce the number of coefficient observably. Based on the advantage of deduced novel low-pass filter, the paper proposes a new analysis filter bank architecture, which makes use of the high efficiency low-pass filter as a prototype filter. So it can decrease the computation complexity consumedly when the transition bandwidth of prototype filter is especially very sharp. It is the same with the alterable bandwidth system. At last, an eight channels filter bank design example is used to illustrate its feasibility. It reduces the computationally complexity of the sharp transition filter by using a CEM FRM prototype filter. At last, an application example has been given to illustrate the design of analysis filter banks, showing that the resulting filter bank reduces the computation complexity, which adapt to use in engineering implementation.