The need for a monitoring system has grown as a result of rising crime and anomalous activity. To avoid unusual incidents, the common man initiated video surveillance of important areas, which was then passed on to the government. In typical surveillance operations, surveillance devices create a vast volume of data that must be manually analysed. Manually handling huge data sets in real time results in information loss. To prevent abnormal incidents, the actions in sensitive areas can be properly monitored, evaluated, and alerted to the appropriate authorities. Previous deep learning-based activity identification methods have appeared, but the findings are inaccurate, and the proposed Hybrid Machine Learning Algorithms (HMLA) incorporate two detection methods for surveillance videos like as Transfer Learning (TL) and Continual Learning (CL). As a result, the suspicious activity in the video may be missed. Consequently, numerous image processing and computer vision technologies were used in activity detection to decrease human effort and mistakes in surveillance operations. Activities in sensitive areas can be properly monitored and evaluated to avoid unusual incidents, and the appropriate authorities may be alerted. Hence, in order to decrease human error and effort in surveillance operations, activity recognition embraced a variety of image processing and computer vision technologies. In this present work, the capacity has constraints that impact recognition accuracy. Consequently, this research paper presents a HMLA based technique that uses feature extraction using multilayer (Long Short Term Memory) LSTM, Convolutional Neural Networks (CNN), and Temporal feature extraction using multilayer LSTM to improve identification accuracy by 96% while requiring minimal execution time. To show the superior performance of the proposed hybrid machine learning technique, a standard UCF crime dataset was utilised for experimental analysis and compared to existing deep learning algorithms.