Surveillance Camera System Research Articles

Pose-driven attention-guided image generation for person re-Identification

Person re-identification (re-ID) concerns the matching of subject images across different camera views in a multi camera surveillance system. One of the major challenges in person re-ID is pose variations across the camera network, which significantly affects the appearance of a person. Existing development data lack adequate pose variations to carry out effective training of person re-ID systems. To solve this issue, in this paper we propose an end-to-end pose-driven attention-guided generative adversarial network, to generate multiple poses of a person. We propose to attentively learn and transfer the subject pose through an attention mechanism. A semantic-consistency loss is proposed to preserve the semantic information of the person during pose transfer. To ensure fine image details are realistic after pose translation, an appearance discriminator is used while a pose discriminator is used to ensure the pose of the transferred images will exactly be the same as the target pose. We show that by incorporating the proposed approach in a person re-identification framework, realistic pose transferred images and state-of-the-art re-identification results can be achieved.

Real-time masked face classification and head pose estimation for RGB facial image via knowledge distillation

Recently, human head pose estimation and masked face classification are two essential problems in facial analysis. It is necessary to design a compact model to resolve both tasks in order to reduce the computational cost when deploying face recognition-based applications such as camera surveillance systems, AI cameras while maintaining accuracy. In this work, we proposed a lightweight multi-task model called MHPNet that simultaneously addresses both head pose estimation and masked face classification problems. Because of the lack of datasets with available labels for both tasks, we first train teacher models independently on two labelled datasets 300 W-LPA and MAFA to extract the head pose and masked soft label. After that, we design architecture with ResNet18 backbone and two branches for two tasks and train our proposed model with the predictions of teacher models on joint datasets via the knowledge distillation process. To evaluate the effectiveness of our model, we use AFLW2000 and BIWI datasets for head pose estimation problems and MAFA datasets for masked face classification problems. Experiment results show that our proposed model significantly improves the accuracy compared to the state-of-the-art head pose estimation methods and achieves remarkable performance on masked face dataset. Furthermore, our model has the real-time speed of ∼400 FPS when inferring on a Tesla V100 GPU device. Source codes and datasets are available at https://github.com/chientv99/maskpose.

A Two-Mode Underwater Smart Sensor Object for Precision Aquaculture Based on AIoT Technology.

Monitoring the status of culture fish is an essential task for precision aquaculture using a smart underwater imaging device as a non-intrusive way of sensing to monitor freely swimming fish even in turbid or low-ambient-light waters. This paper developed a two-mode underwater surveillance camera system consisting of a sonar imaging device and a stereo camera. The sonar imaging device has two cloud-based Artificial Intelligence (AI) functions that estimate the quantity and the distribution of the length and weight of fish in a crowded fish school. Because sonar images can be noisy and fish instances of an overcrowded fish school are often overlapped, machine learning technologies, such as Mask R-CNN, Gaussian mixture models, convolutional neural networks, and semantic segmentation networks were employed to address the difficulty in the analysis of fish in sonar images. Furthermore, the sonar and stereo RGB images were aligned in the 3D space, offering an additional AI function for fish annotation based on RGB images. The proposed two-mode surveillance camera was tested to collect data from aquaculture tanks and off-shore net cages using a cloud-based AIoT system. The accuracy of the proposed AI functions based on human-annotated fish metric data sets were tested to verify the feasibility and suitability of the smart camera for the estimation of remote underwater fish metrics.

Self-Updatable Database System Based on Human Motion Assessment Framework

Recently, human motion-centric videos have been attracting attention in the field of computer vision. Observing and detecting human motion in intelligent surveillance camera systems is essential for understanding the intentions of target subjects. However, these videos have vast amounts of disparate and complex information, and hence they are difficult to process and label automatically. As a result, building and maintaining a database using motion-centric videos requires considerable labor in trimming and classifying the videos. Therefore, we propose a self-updatable motion database system based on a human motion assessment framework for evaluating complex human movements. The framework quantifies three primitive motion properties: stability, liveliness, and attention. This assessment highlights the semantics of human motion in the input video. The semantic motion sequence obtained after the motion assessment is compared with a similarity motion database to determine whether the database needs to be updated; for efficient comparison, we introduce a sequential autoencoder model with a long short-term memory neural network. The proposed system maintains the database within a surveillance camera system using a motion update algorithm; unseen motions in the database are updated using a camera-based surveillance system. In addition, this framework combines state-of-art action recognition methods to improve performance by up to 11% via the self-update of motion.

Disentangled Feature Learning Network and a Comprehensive Benchmark for Vehicle Re-Identification

Vehicle Re-Identification (ReID) is of great significance for public security and intelligent transportation. Large and comprehensive datasets are crucial for the development of vehicle ReID in model training and evaluation. However, existing datasets in this field have limitations in many aspects, including the constrained capture conditions, limited variation of vehicle appearances, and small scale of training and test set, etc. Hence, a new, large, and challenging benchmark for vehicle ReID is urgently needed. In this paper, we propose a large vehicle ReID dataset, called VERI-Wild 2.0, containing 825,042 images. It is captured using a city-scale surveillance camera system, consisting of 274 cameras covering a very large area over 200 km2. Specifically, the samples in our dataset present very rich appearance diversities thanks to the long time span collecting settings, unconstrained capturing viewpoints, various illumination conditions, diversified background environments, and different weather conditions. Furthermore, to facilitate more practical benchmarking, we define a challenging and large test set containing about 400K vehicle images that do not have any camera overlap with the training set. VERI-Wild 2.0 is expected to be able to facilitate the design, adaptation, development, and evaluation of different types of learning models for vehicle ReID. Besides, we also design a new method for vehicle ReID. We observe that orientation is a crucial factor for feature matching in vehicle ReID. To match vehicle pairs captured from similar orientations, the learned features are expected to capture specific detailed differential information for discriminating the visually similar yet different vehicles. In contrast, features are desired to capture the orientation invariant common information when matching samples captured from different orientations. Thus a novel disentangled feature learning network (DFNet) is proposed. It explicitly considers the orientation information for vehicle ReID, and concurrently learns the orientation specific and orientation common features that thus can be adaptively exploited via an adaptive matching scheme when dealing with matching pairs from similar or different orientations. The comprehensive experimental results show the effectiveness of our proposed method.

Towards Proactive Surveillance through CCTV Cameras under Edge-Computing and Deep Learning

Weapons, usually a handgun, a revolver, or a pistol, are used in the majority of criminal acts. The traditional closed-circuit television (CCTV) surveillance and control system requires human intervention to detect such crime incidents. The purpose of this research is to develop a real-time automatic weapon carrier detection system that may be used with CCTV cameras and surveillance systems. The goal is to alarm and alert the security officials to take proactive action to prevent violent activities. In deep learning literature, region-based classifiers (R-FCN and Faster R-CNN) and regression-based detectors (Yolo invariant) are being used as promising object detection methods. Although region-based classifiers are accurate, they lack the speed of detection required for real-time detection, whereas regression-based detectors (for example, YoloV4 invariant) are fast enough for real-time detection, but lack accuracy. The method applied in this study relies on Yolov4 to quickly detect anomalies, followed by R-FCN to boost detection accuracy by filtering out any false positives. A weapon dataset comprising 4430 locally and internationally available weapon photos with a 70–30 split ratio is used to train and test the system, which is subsequently evaluated using a live surveillance camera system. This hybrid system achieved a 90% accuracy with a low false positive rate, as well as 94% precision, 86% recall, and 89% F1 score. Our results prove that the proposed hybrid system is useful for proactive real-time surveillance to alarm the existence of a suspicious weapon carrier in a surveillance area.

TransRisk

Human mobility data may lead to privacy concerns because a resident can be re-identified from these data by malicious attacks even with anonymized user IDs. For an urban service collecting mobility data, an efficient privacy risk assessment is essential for the privacy protection of its users. The existing methods enable efficient privacy risk assessments for service operators to fast adjust the quality of sensing data to lower privacy risk by using prediction models. However, for these prediction models, most of them require massive training data, which has to be collected and stored first. Such a large-scale long-term training data collection contradicts the purpose of privacy risk prediction for new urban services, which is to ensure that the quality of high-risk human mobility data is adjusted to low privacy risk within a short time. To solve this problem, we present a privacy risk prediction model based on transfer learning, i.e., TransRisk, to predict the privacy risk for a new target urban service through (1) small-scale short-term data of its own, and (2) the knowledge learned from data from other existing urban services. We envision the application of TransRisk on the traffic camera surveillance system and evaluate it with real-world mobility datasets already collected in a Chinese city, Shenzhen, including four source datasets, i.e., (i) one call detail record dataset (CDR) with 1.2 million users; (ii) one cellphone connection data dataset (CONN) with 1.2 million users; (iii) a vehicular GPS dataset (Vehicles) with 10 thousand vehicles; (iv) an electronic toll collection transaction dataset (ETC) with 156 thousand users, and a target dataset, i.e., a camera dataset (Camera) with 248 cameras. The results show that our model outperforms the state-of-the-art methods in terms of RMSE and MAE. Our work also provides valuable insights and implications on mobility data privacy risk assessment for both current and future large-scale services.

Bi-Directional Style Adaptation Network for Person Re-Identification

Large-scale surveillance camera system can provide multiple visual information. Nevertheless, the images exhibit diverse appearance features due to different parameters and installation positions of the cameras. This camera style variation deteriorates its benefit from capturing identity features in person re-identification (Re-Id). The existing methods for filtering shallow appearance information through the Instance Normalization (IN) layer are extremely unfavorable to supervised tasks. To mitigate this problem, a simplified and valid Bidirectional Style Adaptation Network (BSA-Net) is presented to incorporate a new branch containing IN layer to learn invariant features with changing appearance. For these two branches, the structure is completely independent and the parameters are partially shared. BAS-Net is able to focus on the extraction of identity information while preserving appearance features. Specially, this new branch is removed during the testing phase, which significantly facilitates performance without introducing new computation. The superiority of the model is confirmed in extensive experiments on widely used benchmarks.

WatchDog- A Smart Surveillance System

In the modern world, security and privacy are the major concerns. Due to this, security cameras are installed in all public places to monitor any suspicious activities. But the general public does not know who is behind the screen watching them, monitoring their every move. It raises both securities as well as privacy issues. Another major issue regarding security is CCTV cameras are installed in public places only. Everyone cannot afford to hire someone to monitor their homes and alert them. This makes people at risk even in their homes. These issues lead to the development of WatchDog. WatchDog is a computer vision-based smart camera surveillance system that can monitor any suspicious activity and warn the owner about it. It uses python as its base language and uses OpenCV and skimage.metric library to capture any movement within its scope of vision. It is a simple yet effective program that can resolve many issues regarding security and privacy. Key Words: Python, OpenCV, Smart-Surveillance System

Approximating the Pareto frontier for a challenging real-world bi-objective covering problem

We study a bi-objective covering problem stemming from a real-world application concerning the design of camera surveillance systems for large-scale outdoor areas. It is in this application prohibitively costly to surveil the entire area, and therefore necessary to be able to present a decision-maker with trade-offs between total cost and the portion of the area that is surveilled. The problem can be stated as a set covering problem with two objectives, describing cost and portion of covering constraints that are fulfilled. Finding the Pareto frontier for these objectives is very computationally demanding and we therefore derive a method for finding a good approximate frontier in a practically feasible computing time. The method is based on the ϵ-constraint reformulation, an established heuristic for set covering problems, and subgradient optimization.

Pedestrian Traffic Light Control with Crosswalk FMCW Radar and Group Tracking Algorithm.

The increased mobility requirements of modern lifestyles put more stress on existing traffic infrastructure, which causes reduced traffic flow, especially in peak traffic hours. This calls for new and advanced solutions in traffic flow regulation and management. One approach towards optimisation is a transition from static to dynamic traffic light intervals, especially in spots where pedestrian crossing cause stops in road traffic flow. In this paper, we propose a smart pedestrian traffic light triggering mechanism that uses a Frequency-modulated continuous-wave (FMCW) radar for pedestrian detection. Compared to, for example, camera-surveillance systems, radars have advantages in the ability to reliably detect pedestrians in low-visibility conditions and in maintaining privacy. Objects within a radar’s detection range are represented in a point cloud structure, in which pedestrians form clusters where they lose all identifiable features. Pedestrian detection and tracking are completed with a group tracking (GTRACK) algorithm that we modified to run on an external processor and not integrated into the used FMCW radar itself. The proposed prototype has been tested in multiple scenarios, where we focused on removing the call button from a conventional pedestrian traffic light. The prototype responded correctly in practically all cases by triggering the change in traffic signalization only when pedestrians were standing in the pavement area directly in front of the zebra crossing.

A computer vision framework using Convolutional Neural Networks for airport-airside surveillance

Modern airports often have large and complex airside environments featuring multiple runways, with changing configurations, numerous taxiways for effective circulation of flights and tens, if not hundreds, of gates. With inherent uncertainties in gate push-back and taxiway routing, efficient surveillance and management of airport-airside operations is a highly challenging task for air traffic controllers. An increase in air traffic may lead to gate delays, taxiway congestion, taxiway incursions as well as significant increase in the workload of air traffic controllers. With the advent of Digital Towers, airports are increasingly being equipped with surveillance camera systems. This paper proposes a novel computer vision framework for airport-airside surveillance, using cameras to monitor ground movement objects for safety enhancement and operational efficiency improvement. The framework adopts Convolutional Neural Networks and camera calibration techniques for aircraft detection and tracking, push-back prediction, and maneuvering monitoring. The proposed framework is applied on video camera feeds from Houston Airport, USA (for maneuvering monitoring) and Obihiro Airport, Japan (for push-back prediction). The object detection models of the proposed framework achieve up to 73.36% average precision on Houston airport and 87.3% on Obihiro airport. The framework estimates aircraft speed and distance with low error (up to 6 meters), and aircraft push-back is predicted with an average error of 3 min from the time an aircraft arrives with the error-rate reducing until the aircraft’s actual push-back event.

A Social Distance Estimation and Crowd Monitoring System for Surveillance Cameras.

Social distancing is crucial to restrain the spread of diseases such as COVID-19, but complete adherence to safety guidelines is not guaranteed. Monitoring social distancing through mass surveillance is paramount to develop appropriate mitigation plans and exit strategies. Nevertheless, it is a labor-intensive task that is prone to human error and tainted with plausible breaches of privacy. This paper presents a privacy-preserving adaptive social distance estimation and crowd monitoring solution for camera surveillance systems. We develop a novel person localization strategy through pose estimation, build a privacy-preserving adaptive smoothing and tracking model to mitigate occlusions and noisy/missing measurements, compute inter-personal distances in the real-world coordinates, detect social distance infractions, and identify overcrowded regions in a scene. Performance evaluation is carried out by testing the system’s ability in person detection, localization, density estimation, anomaly recognition, and high-risk areas identification. We compare the proposed system to the latest techniques and examine the performance gain delivered by the localization and smoothing/tracking algorithms. Experimental results indicate a considerable improvement, across different metrics, when utilizing the developed system. In addition, they show its potential and functionality for applications other than social distancing.

Accurate and Privacy-Preserving Person Localization Using Federated-Learning and the Camera Surveillance Systems of Public Places

In this paper we propose an accurate and privacy-preserving scheme that enables a law enforcement agency to locate persons of interest using the camera surveillance systems of public places. Comparing to the existing schemes that measure the Euclidean distance to locate persons using their embedding vectors storing facial features, we use a more accurate approach by training a machine learning model. Moreover, to avoid leaking sensitive information by sharing the images of the public places’ visitors to train the model, we use a federated learning technique to compute the model in a privacy-preserving way. The model is designed in such a way that makes executing it over encrypted data efficient. Specifically, the model is executed by three parties as follows. Each public place computes an embedding vector for each visitor’s image and inputs it to a neural network and encrypts the output using a modified inner product encryption scheme and sends the ciphertext to a cloud server. The law enforcement agency does the same steps on the images of persons of interest. Finally, the server uses these ciphertexts to evaluate the last layer of the model by computing the inner product of the two vectors over encrypted data. The cryptosystem enables the server to compute the inner product of two vectors using their ciphertexts without being able to learn the vectors. We have modified an encryption cryptosystem that is designed for a single public place and a single law enforcement agency to make it more efficient in our application that has multiple public places. To evaluate our scheme, we have conducted extensive experiments and the results confirm that our model is accurate in locating persons of interest with low communication and computation overhead. A formal proof and analysis are used to demonstrate the ability of our scheme to preserve privacy.

Vickrey-Clarke-Groves Auction-Based Storage Allocation for Distributed Camera Systems

Video surveillance systems are critical infrastructures and that growing in size and complexity. Storage space is the prime resource in such systems but current surveillance setups are centralized and limited in resources due to security and cost constraints. Allocating the correct amount of storage to each camera considering their large differences in characteristics and video content is challenging. In this paper we propose a game theoretic approach to storage allocation for video surveillance camera systems based on the Vickrey-Clarke-Groves (VCG) auction mechanism.

Desain Robot Pengintai Segala Medan dengan Kendali Wireless PS2

A mobile robots is one of the solution to overcome the deficiency of security surveillance camera systems that are static in nature, which are vulnerable to blind spots on the results of their observations. In this research, the surveillance robot is made to resemble the shape of a mars rover robot that is applied to be able to move more freely to stalk areas and be able to navigate on uneven surfaces. The movement of the surveillance robot wheel uses a wireless PS 2 remote control and is driven by a DC motor rotation as its main drive. The connection between the PS2 Remote Control and the surveillance robot is by using the Arduino Uno. In addition, this robot will be equipped with a camera that will send visual surveillance results from the robot to the smartphone.

An Expert System for Weapon Identification and Categorization Using Machine Learning Technique to Retrieve Appropriate Response

In response to any terrorist attack on hospitals, airports, shopping malls, schools, universities, colleges, railway stations, passport offices, bus stands, dry ports and the other important private and public places, a proper plan will need to be developed effective response. In normal moments, security guards are deployed to prevent criminals from doing anything wrong. For example, someone is moving around with a weapon, and security guards are watching its movement through closed circuit television (CCTV). Meanwhile, they are trying to identify his weapon in order to plan an appropriate response to the weapon he has. The process of manually identifying weapons is man-made and slow, while the security situation is critical and needs to be accelerated. Therefore, an automated system is needed to detect and classify the weapon so that appropriate response can be planned quickly to ensure minimal damage. Subject to previous concerns, this study is based on the Convoluted Neural Network (CNN) model using datasets that are assembled on the YOLO and you only see once. Focusing on real-time weapons identification, we created a data collection of images of multiple local weapons from surveillance camera systems and YouTube videos. The solution uses parameters that describe the rules for data generation and problem interpretation. Then, using deep convolutional neural network models, an accuracy of 97.01% is achieved.

Research of Distorted Vehicle Magnetic Signatures Recognitions, for Length Estimation in Real Traffic Conditions.

Reliable cost-effective traffic monitoring stations are a key component of intelligent transportation systems (ITS). While modern surveillance camera systems provide a high amount of data, due to high installation price or invasion of drivers’ personal privacy, they are not the right technology. Therefore, in this paper we introduce a traffic flow parameterization system, using a built-in pavement sensing hub of a pair of AMR (anisotropic magneto resistance) magnetic field and MEMS (micro-electromechanical system) accelerometer sensors. In comparison with inductive loops, AMR magnetic sensors are significantly cheaper, have lower installation price and cause less intrusion to the road. The developed system uses magnetic signature to estimate vehicle speed and length. While speed is obtained from the cross-correlation method, a novel vehicle length estimation algorithm based on characterization of the derivative of magnetic signature is presented. The influence of signature filtering, derivative step and threshold parameter on estimated length is investigated. Further, accelerometer sensors are employed to detect when the wheel of vehicle passes directly over the sensor, which cause distorted magnetic signatures. Results show that even distorted signatures can be used for speed estimation, but it must be treated with a more robust method. The database during the real-word traffic and hazard environmental condition was collected over a 0.5-year period and used for method validation.

Efforts to Prevent Railway Suicides in Denmark.

Background: Reviews of camera surveillance systems have demonstrated ambivalent behaviors among people who die by railway suicide. Yet, only few preventive measures have been evaluated. Aims: We aimed to review incidents of suicidal behavior at a Danish railway station, install preventive measures, and monitor subsequent calls to a telephone helpline and reports of suicidal incidences. Method: Suicide incidents at Valby Station during 2012-2018 were reviewed to identify options for preventive measures. Based on these findings, signs encouraging help-seeking and other measures were implemented. Calls to the Danish helpline for suicide prevention and suicidal events at the station were subsequently monitored. Results: The review revealed locations where measures were meaningful and signs, physical barriers, and motion-sensitive lights were installed. Over the following 14 months, no suicide deaths occurred, and the signs were mentioned in 14 calls to the helpline, some of which were made by callers who were evaluated to be at high risk of suicide. Limitations: No direct link between implemented measures and observed outcomes could be established. Conclusion: Installing measures, including signs, at appropriate locations at railway platforms may encourage people in crisis to seek support.

Application of one-stage instance segmentation with weather conditions in surveillance cameras at construction sites

On-site surveillance camera systems have been utilized for safety monitoring remotely to reduce occupational accidents and injuries. Even though previous studies for workspace safety monitoring using computer vision have been undertaken, weather-related conditions such as raindrops and snows affecting pan-tilt-zoom camera visibility have not been considered in depth. Developing a robust detector that is reliable in different weather conditions is a challenge for vision-based monitoring outdoor works in construction sites. This study proposes a deep learning-based real time one-stage instance segmentation model for monitoring systems and data augmentation methods to improve the detector's performance under five weather conditions: brightness, darkness, rainy, snowy, and foggy. Experimental results indicate that the proposed model with weather augmentation outperformed the baseline model without augmentation by around 0.05 mAP05:95. Therefore, this work verified the applicability of the model for detecting and segmenting instances robustly in construction sites. This weather augmentation approach could be applicable to any type of vision-based monitoring system such as quality inspection, productivity assessment, schedule monitoring, and other work management as well as safety management affected by the different weather conditions. There are still room for the future researches in the development of fully autonomous vision-based monitoring systems in the applications mentioned.