Vision-based Surveillance System Research Articles

Integrating vision-based AI and large language models for real-time water pollution surveillance.

Water pollution has become a major concern in recent years, affecting over 2 billion people worldwide, according to UNESCO. This pollution can occur by either naturally, such as algal blooms, or man-made when toxic substances are released into water bodies like lakes, rivers, springs, and oceans. To address this issue and monitor surface-level water pollution in local water bodies, an informative real-time vision-based surveillance system has been developed in conjunction with large language models (LLMs). This system has an integrated camera connected to a Raspberry Pi for processing input frames and is further linked to LLMs for generating contextual information regarding the type, causes, and impact of pollutants on both human health and the environment. This multi-model setup enables local authorities to monitor water pollution and take necessary steps to mitigate it. To train the vision model, seven major types of pollutants found in water bodies like algal bloom, synthetic foams, dead fishes, oil spills, wooden logs, industrial waste run-offs, and trashes were used for achieving accurate detection. ChatGPT API has been integrated with the model to generate contextual information about pollution detected. Thus, the multi-model system can conduct surveillance over water bodies and autonomously alert local authorities to take immediate action, eliminating the need for human intervention. PRACTITIONER POINTS: Combines cameras and LLMs with Raspberry Pi for processing and generating pollutant information. Uses YOLOv5 to detect algal blooms, synthetic foams, dead fish, oil spills, and industrial waste. Supports various modules and environments, including drones and mobile apps for broad monitoring. Educates on environmental healthand alerts authorities about waterpollution.

Surveillance using Ml

There is a strong demand for smart vision based surveillance systems owing to the increase in crime at a frightening rate at various public places. As crime increases at a disturbing rate, public security violations and high cost of security personnel have motivated the strategic survey of existing vision and image processing based techniques. It discusses the system to analyze human’s normal and abnormal activities recognition along with various feature selectors and detectors.Deep learning approach is used to detect suspicious or normal activity in an academic environment, and which sends an alert message to the corresponding authority, in case of predicting a suspicious activity.

Robust Pedestrian Detection and Path Prediction using Improved YOLOv5

In vision-based surveillance systems, pedestrian recognition and path prediction are critical concerns. Advanced computer vision applications, on the other hand, confront numerous challengesdue to differences in pedestrian postures and scales, backdrops, and occlusion. To tackle these challenges, we present a YOLOv5-based deep learning-based pedestrian recognition and path prediction method. The updated YOLOv5 model was first used to detect pedestrians of various sizes and proportions. The proposed path prediction method is then used to estimate the pedestrian's path based on motion data. The suggested method deals with partial occlusion circumstances to reduce object occlusion-induced progression and loss, and links recognition results with motion attributes. After then, the path prediction algorithm uses motion and directional data to estimate the pedestrian movement's direction. The proposed method outperforms the existing methods, according to the results of the experiments. Finally, we come to a conclusion and look into future study.

Computer Vision based Surveillance System

Abstract: Automated surveillance systems are surveillance systems that employ cameras to monitor the area. The observed setting is analyzed using motion detection, crowd behavior, individual behavior, and interactions between people, crowds, and their surroundings. These automated systems are capable of a wide range of tasks, including detection, interpretation, understanding, recording, and alert creation based on the results of the study. Researchers have been able to increase monitoring performance while limiting human errors by changing various features of these systems. This research looks into video surveillance systems and the components that go with them in depth. The designs that were used, as well as the most important studies in these systems, are presented. Existing surveillance systems were compared in terms of to provide a wider picture and a more holistic understanding of the system.

Deep vision-based surveillance system to prevent train–elephant collisions

Animal conservation is imperative, and technology can certainly assist in different ways. The extinction of endangered species like tigers and elephants has boosted the necessity for such efforts. Human–elephant collision (HEC) has been an active area of research for years. Apart from deforestation, the roads and rail tracks laid down through forest areas intervene a lot in wildlife. Collisions and tragedies are every day, especially in green belts in India and other Asian countries. Therefore, it is crucial to develop vision-based, automated, warning-generating systems to identify the animal/elephant near-site. In the proposed work, different deep learning-based models are proposed to identify elephants in image/video. Several convolutional neural network (CNN)-based models and three transfer learning (TL)-based models, i.e., ResNet50, MobileNet, Inception V3, have been experimented with and tuned for elephant detection. All the models are tested on a synthesized dataset having about 4200 images built using two public datasets, i.e., ELPephant and RailSem19. Two accurate CNN and transfer learning-based models are presented in detail. These highly accurate and precise models can alarm the trains and generate warning signals on site. The proposed CNN and inception network demonstrated high accuracy of 99.53% and 99.91%, respectively, and are remarkable in identifying elephants and hence preventing HEC. The same model can be trained for other animals for their preservation in similar scenarios.

A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system

There is a strong demand of smart vision based surveillance system owing to the increase in crime at a frightening rate at various public places like Banks, Airport, Shopping malls and its application in human activity recognition ranges from patient fall detection, irregular pattern recognition or Human computer Interaction. As the crime increases at a disturbing rate, public security violations and high cost of security personals have motivated the author to do the strategic survey of existing vision and image processing based techniques in the past literature. The paper begins with discussing the common approach towards suspicious activity detection and recognition followed by summarizing the supervised and unsupervised machine learning methodologies mainly based on SVM, HMM and ANN classifiers, which were adopted by the researchers previously varying from single human behavior modeling to crowded scenes. Next, this paper discusses system model for human’s normal and abnormal activities recognition along with various feature selectors and detectors used in previous literature. This was followed by conducting a review of benchmark researches which covered a comprehensive state of art methodologies in the related fields, key points owned, feature learning and applications. At last experimental aspects of various papers have been discussed with essential performance matrices like accuracy along with the major issues, common problems, challenges and future scope in the related field.

Real-Time Detection of Fall From Bed Using a Single Depth Camera

Toward the medical and living healthcare for the elderly and patients, fall from bed is a critical accident that may lead to serious injuries. To alleviate this, an essential problem is to detect this event in time for earning the rescue time. Although some efforts that resort to the wearable devices and smart healthcare room have already been paid to address this problem, the performance is still not satisfactory enough for the practical applications. In this paper, a novel fall from a bed detection method is proposed. In particular, the depth camera is used as the visual sensor due to its insensitivity to illumination variation and capacity of privacy protection. To characterize the human activity well, an effective human upper body detection approach able to extract human head and upper body center is proposed using random forest. Compared with the existing widely used human body parsing methods (e.g., Microsoft Kinect SDK or OpenNI SDK), our proposition can still work reliably when human–bed interaction happens. According to the motion information of human upper body, the fall from bed detection task is formulated as a two-class classification problem. Then, it is solved using the large margin nearest neighbor classification approach. Our method can meet the real-time running requirement with the normal computer. In experiments, we construct a fall from bed detection data set that contains the samples from 42 volunteers (26 males and 16 females) for test. The experimental results demonstrate the effectiveness and efficiency of our proposition. Note to Practitioners—This paper aims to develop the vision-based surveillance system to automatically detect fall from bed in real time under the unconstrained conditions. This cannot be well handled by using wearable devices. Our proposition can be used to construct a smart healthcare room. It is worth noting that the proposed system is insensitive to illumination variation and capable of protecting privacy due to the use of depth camera. The proposed human body extraction method is able to address the human–bed interaction well. An effective motion pattern categorization approach is applied to ensure the good separation between fall from bed and the other activities. According to the extensive experiments under the laboratory and sickroom environments, our fall from bed detection method generally can achieve acceptable result with high efficiency. Nevertheless, it is still somewhat sensitive to noise within the depth frames. In addition, when serious human–quilt interaction happens, the performance of human body extraction procedure is not satisfactory enough.

An integrated highly synchronous, high resolution, real time eye tracking system for dynamic flight movement

Electronic surveillance systems are being used rapidly today, ranging from a simple video camera to a complex biometric surveillance system for facial patterns and intelligent computer vision based surveillance systems, which are applied in many fields such as home monitoring, security surveillance of important places and mission critical tasks like air traffic control surveillance. Such systems normally involve a computer system and a human surveillance operator, who looks at the dynamic display to perform his surveillance tasks. Exploitation of shared information between these physical heterogeneous data capture systems with human operated functions is one emerging aspect in electronic surveillance that has yet to be addressed deeply. Hence, an innovative interaction interface for such knowledge extraction and representation is required. Such an interface should establish a data activity register frame which captures information depicting various surveillance activities at a specified spatial and time reference.This paper presents a real time eye tracking system, which integrates two sets of activity data in a highly dynamic changing and synchronous manner in real-time with respect to both spatial and time frames, through the “Dynamic Data Alignment and Timestamp Synchronisation Model”. This model matches the timestamps of the two data streams, aligns them to the same spatial reference frame before fusing them together into a data activity register frame. The Air Traffic Control (ATC) domain is used to illustrate this model, where experiments are conducted under simulated radar traffic situations with participants and their radar input data. Test results revealed that this model is able to synchronise the timestamp of the eye and dynamic display data, align both of these data spatially, while taking into account dynamic changes in space and time on a simulated radar display. This system can also distinguish and show variations in the monitoring behaviour of participants. As such, new knowledge can be extracted and represented through this innovative interface, which can then be applied to other applications in the field of electronic surveillance to unearth monitoring behaviour of the human surveillance operator.

Surveillance system with motion and face detection using histograms of oriented gradients

<span>With the rapidly increasing crime rate in recent years, community safety issues aroused a wide concern among public community. Various security technologies had been invented and carried out, for example password door lock, alarm system, and closed-circuit televisions (CCTVs). Although the installation of CCTVs is common in most premises, they require extensive man power to manually monitor the videos. Moreover, the reliability of human operator greatly deteriorates when they are in fatigue condition. In view of this, our project aims to develop an automated computer vision based surveillance system. Unlike ordinary CCTV system that requires human operator to manually observe and detect intruder, a computer vision based surveillance system automatically monitor the security of premises and trigger actions once an intrusion is detected. Basically, it is a simple surveillance camera system that will be setup at the entrance of the house. The reliability is being enhanced by applying the motion detection and face recognition algorithm, using histogram of oriented gradients that could detect the existence of people at the main entrance and try to validate the user. Apart from recognizing the user, the propose system also support mobile interaction whereby user can monitor the camera, activate alarm, and even received notification when a stranger was being detected at the entrance of the house. By including such functionalities, proposed system had highly surpassed the existing surveillance system by not only support monitoring, but also try to recognize the people and inform the user at the exact moment when stranger detected, so that user could take immediate action about it, for example activating the alarm or report to police. The project was executed with expected outcome and objectives had been accomplished.</span>

Efficient Deep Network Architecture for Vision-Based Vehicle Detection Keyvan Kasiri,

With the progress in intelligent transportation systems in smartcities, vision-based vehicle detection is becoming an important issuein the vision-based surveillance systems. With the advent ofthe big data era, deep learning methods have been increasinglyemployed in the detection, classification, and recognition applicationsdue to their performance accuracy, however, there are stillmajor concerns regarding deployment of such methods in embeddedapplications. This paper offers an efficient process leveragingthe idea of evolutionary deep intelligence on a state-of-the-art deepneural network. Using this approach, the deep neural network isevolved towards a highly sparse set of synaptic weights and clusters.Experimental results for the task of vehicle detection demonstratethat the evolved deep neural network can achieve a substantialimprovement in architecture efficiency adapting for GPUacceleratedapplications without significant sacrifices in detectionaccuracy. The architectural efficiency of ~4X-fold and ~2X-folddecrease is obtained in synaptic weights and clusters, respectively,while the accuracy of 92.8% (drop of less than 4% compared to theoriginal network model) is achieved. Detection results and networkefficiency for the vehicular application are promising, and opensthe door to a wider range of applications in deep learning.

Vision-based surveillance system for monitoring traffic conditions

With the rapid advancement of sensing technologies, it has been feasible to collect various types of traffic data such as traffic volume and travel times. Vision-based approach is one of the major scheme actively used for the automated traffic data collection, and continues to gain traction to a broader utilization. It collects video streams from cameras installed near roads, and processes the video streams frame by frame using image processing algorithms. The widely used algorithms include vehicle detection and vehicle tracking which recognize every vehicle in the camera view and track it in the consecutive frames. Vehicle counts and speed can be estimated from the detection and tracking results. Continuous efforts have been made for the performance improvement of the algorithms and for their effective applications. However, little research has been found on the application to the various view settings of highway CCTV cameras as well as the reliability of the speed estimation. This paper proposes a vision-based system that integrates vehicle detection, vehicle tracking, and field of view calibration algorithms to obtain vehicle counting data and to estimate individual vehicle speed. The proposed system is customized for the video streams collected from highway CCTVs which have various settings in terms of focus and view angles. The system detects and tracks every vehicle in the view unless it is occluded by other vehicles. It is also capable of handling occlusions that occurs frequently depending on the view angles. The system has been tested on the several different views including congested scenes. Vehicle counts and speed estimation results are compared to the manual counting and GPS data, respectively. The comparison signifies that the system has a high potential to extract reliable information about highway traffic conditions from highway CCTVs.

Vanishing point-based image transforms for enhancement of probabilistic occupancy map-based people localization.

The widespread use of vision-based surveillance systems has inspired many research efforts on people localization. In this paper, a series of novel image transforms based on the vanishing point of vertical lines is proposed for enhancement of the probabilistic occupancy map (POM)-based people localization scheme. Utilizing the characteristic that the extensions of vertical lines intersect at a vanishing point, the proposed transforms, based on image or ground plane coordinate system, aims at producing transformed images wherein each standing/walking person will have an upright appearance. Thus, the degradation in localization accuracy due to the deviation of camera configuration constraint specified can be alleviated, while the computation efficiency resulted from the applicability of integral image can be retained. Experimental results show that significant improvement in POM-based people localization for more general camera configurations can indeed be achieved with the proposed image transforms.

Full-Body Occlusion Handling and Density Analysis in Traffic Video-Surveillance Systems

Vision-based traffic surveillance systems are among the most reliable, inexpensive, and highly applicable methodologies for surveying traffic conditions. The implementation of these strategies, however, is limited under certain conditions, such as the presence of vehicle occlusions or poor illumination conditions that lead to either overcounted or under-counted traffic data. The proposed motion-based methodology is intended to overcome these limitations by using a new technique for full-body occlusion handling of vehicles. The methodology is based on five main steps: calibration, detection, tracking, counting, and occlusion handling. The proposed methodology was tested with various 30–min videos and 452 preidentified cases of occlusion. Preliminary results indicated that the proposed methodology was reliable and robust in providing traffic density analysis. Future work may rely on the extension of the proposed methodology to deal with the detection of vehicles moving toward multiple directions.

Gait identification using shadow biometrics

We propose a novel biometrics method based on shadows (shadow biometrics, SB) and introduce a SB-based person identification method for a vision-based surveillance system. Conventional biometric identification based on body movements, as is the case in gait recognition, uses cameras that provide a good view of entire human body. Aerial search and surveillance systems only see the human body from top view with a smaller cross-section and with less details than seen in side views, which is further aggravated by the lower resolution associated with this imagery. Shadows, i.e. body projections due to the Sun, or artificial lights at night, can offer body biometrics information that cannot be directly seen in body top view. In this paper we use SB for person identification, automatically extracting shadows in captured video images, and processing them to extract gait features, further analyzed by spherical harmonics. We demonstrate shadow-based person identification in experiments inside a building using artificial light and outside under the Sun. The introduced method using spherical harmonics outperforms methods based on Fourier transform, gait energy image, and active energy image. Furthermore, we show that the combination of body and shadow areas, as seen from an oblique camera on an upper floor of a building, has better performance than using body only or shadow only information.

A Block-Based Multi-Scale Background Extraction Algorithm

Problem statement: To extract the moving objects, vision-based surveil lance systems subtract the current image from a predefined backgr ound image. The efficiency of these systems mainly depends on accuracy of the extracted backgro und image. It should be able to adapt to the changes continuously. In addition, especially in re al-time applications the time complexity of this adaptation is a critical matter. Approach: In this study, to extract an adaptive background, a combination of blocking and multi-scale methods is presented. Because of being less sensitive to local movements, block-based techniques are proper to control the non-stationary objects' movements, especially in outdoor applications. They can be use ful to reduce the effect of these objects on the extracted background. We also used the blocking method to intelligently select the regions which the temporal filtering has to be applied on. In additio n, an amended multi-scale algorithm is introduced. This algorithm is a hybrid algorithm, a combination of some nonparametric and parametric filters. It uses a nonparametric filter in the spatial domain t o initiate two primary backgrounds. In continue two adapted two-dimensional filters will be used to ext ract the final background. Results: The qualitative and quantitative results of our experiments certify not only the quality of the final extracted background is acceptable, but also its time consump tion is approximately half in compare to the simila r methods. Conclusion: Using Multi scaling filtering and applying the filt ers just to some selected non- overlapped blocks reduce the time consumption of the extracting background algorithm.

Goal-based trajectory analysis for unusual behaviour detection in intelligent surveillance

In a typical surveillance installation, a human operator has to constantly monitor a large array of video feeds for suspicious behaviour. As the number of cameras increases, information overload makes manual surveillance increasingly difficult, adding to other confounding factors such as human fatigue and boredom. The objective of an intelligent vision-based surveillance system is to automate the monitoring and event detection components of surveillance, alerting the operator only when unusual behaviour or other events of interest are detected. While most traditional methods for trajectory-based unusual behaviour detection rely on low-level trajectory features such as flow vectors or control points, this paper builds upon a recently introduced approach that makes use of higher-level features of intentionality. Individuals in the scene are modelled as intentional agents, and unusual behaviour is detected by evaluating the explicability of the agent's trajectory with respect to known spatial goals. The proposed method extends the original goal-based approach in three ways: first, the spatial scene structure is learned in a training phase; second, a region transition model is learned to describe normal movement patterns between spatial regions; and third, classification of trajectories in progress is performed in a probabilistic framework using particle filtering. Experimental validation on three published third-party datasets demonstrates the validity of the proposed approach.

Unsupervised and simultaneous training of multiple object detectors from unlabeled surveillance video

Object detection is an essential component in automated vision-based surveillance systems. In general, object detectors are constructed using training examples obtained from large annotated data sets. The inevitable limitations of typical training data sets make such supervised methods unsuitable for building generic surveillance systems applicable to a wide variety of scenes and camera setups. In our previous work we proposed an unsupervised method for learning and detecting the dominant object class in a general dynamic scene observed by a static camera. In this paper, we investigate the possibilities to expand the applicability of this method to the problem of multiple dominant object classes. We propose an idea on how to approach this expansion, and perform an evaluation of this idea using two representative surveillance video sequences.

Seeing is believing: A common sense theory of the adoption of perception-based beliefs

In this paper we present a formal common sense theory of the adoption of perception-based beliefs. We begin with a logical analysis of perception and then consider when perception should lead to belief change. Our theory is intended to apply to perception in humans and to perception in artificial agents at the level of the symbolic interface between a vision system and a belief system. In order to provide a context for our work we relate it to the emerging field of cognitive robotics, give an abstract architecture for an agent which is both embodied and capable of reasoning, and relate this to the concrete architectures of two vision-based surveillance systems.