To the banknote sorting system, application of large-scale high-performance FPGA can help to conduct high speed capture and processing of banknote color images. In line with the detection algorithm, the real-time-captured image upon color-space conversion needs to be compared in the four color channels of LGAB with the template images so as to work out the checking results. With Xilinx soft IP core MPMC, two sets of multiport video interface controllers have been designed, each of which embraces four ways of video data read-write operation; the controller interface parameters can be configured via the DSP interface, with the process flow control based on the image frame synchronizing signal. Tests conducted show that the controllers with flexible configurations boast of reliable performance and now are reputed for proven application to large banknote sorting equipment.