This paper designs a video surveillance system that uses TMS320DM6467 dual-core processor as the control core, and puts forward an improved motion detection algorithm named background frame differential method to realize the video image of moving target detection, which uses mathematical morphology filtering method to fill a void in the motion area and remove isolated noise, using high real-time line segment labeled area growth method for detecting the target connected domain so as to realize the identification of moving vehicle. The system has stable performance and good real-time detection effect.