This paper develops a large scale surveillance video data unsupervised segmentation technique regarding whether there is a presence of independent motion. We propose a holistic, in-compression approach to efficient video segmentation. By efficient, we mean that the processing speed is close to or even faster than real-time in normal platforms (we do not assume using special hardware or any parallel machines) while still maintaining a good quality segmentation. Theoretical and experimental analyses demonstrate and validate the holistic, in-compression approach to solving for video segmentation problem for independent motion detection.