We propose an image processing system which searches the moving human and vehicles from the long-term surveillance video for intruder detection or parking lot monitoring, by comparing a series of retrieval queries like passing a certain area, moving direction, duration time and so on. In such a system, not only on-line detection from real-time video by pre-defined query but also quick re-search going back to the past when the user changed the query, and post-retrieval by interactive query change are required. In proposed method, we simplified the image retrieval meta-data to be described in a small and fixed length data however complicated the trajectory of moving object is. Moreover, the matching core function of image retrieval process is realized by a simple comparator. That enables fast image retrieval however complicated the series of the retrieval queries is. To test the performance of the system, we used a long-term parking lot monitoring sequence. About 34 hours after the on-line image processing started when the total number of moving objects reached to 10,000, we changed the retrieval query and restarted matching process and confirmed that the new result is listed up in less than 0.2 seconds.