This study has been undertaken to investigate and implement multiple detection systems into a single surveillance system and check whether the input videos may comprise and capture a variety of realistic anomalies or not. In this paper, we propose to learn various anomalies by exploiting both normal and anomalous videos and implemented it to new model. Real time object detection is a vast, vibrant and sophisticated area of computer vision aimed towards object identification and recognition. Object detection detects the semantic objects of a class objects using Open source Computer Vision, which is a library of programming functions mainly trained towards real time computer vision in digital images and videos. The main aim behind this real time object detection is to help the peoples to overcome their difficulty. Real time object detection finds its uses in the areas like tracking objects, video surveillance, pedestrian detection, people counting, self-driving cars, face detection, tracking in sports and many more. This is achieved using Convolution, Probabilistic Neural Networks, etc. which are a representative tool of Deep learning. This project acts as an aiding tool for peoples who wants to take care of everything inside, outside, and around their house just for their full security expectations. Surveillance is a must for small houses to large-scale industries as they fulfil our safety aspects because theft and burglary have always been a problem. By combining this Surveillance idea to IoT and some Machine Learning stuff this will be a major product. The proposed project is a single autonomous surveillance system, based on analysis and detection technology. The proposed system is capable of monitoring all actions at once and alerts the concerned officials immediately and precisely.