Static Camera Research Articles

VEHICLE TRACKING AND SPEED ESTIMATION UNDER MIXED TRAFFIC CONDITIONS USING YOLOV4 AND SORT: A CASE STUDY OF HANOI

This paper presents a method to estimate vehicle speed automatically, including cars and motorcycles under mixed traffic conditions from video sequences acquired with stationary cameras in Hanoi City of Vietnam. The motion of the vehicle is detected and tracked along the frames of the video sequences using YOLOv4 and SORT algorithms with a custom dataset. In the method, the distance traveled by the vehicle is the length of virtual point-detectors, and the travel time of the vehicle is calculated using the movement of the centroid over the entrance and exit of virtual point-detectors (i.e., region of interest), and then the speed is also estimated based on the traveled distance and the travel time. The results of two experimental studies showed that the proposed method had small values of MAPE (within 3%), proving that the proposed method is reliable and accurate for application in real-world mixed traffic environments like Hanoi, Vietnam.

Sonotoki

This film takes inspiration from the work of Kon Wajiro,- an architect, ethnologist and sociologist who created detailed sketches of everyday life on the streets of Tokyo in the 1920s. Shot in Aomori City, Japan in the depths of winter, Sonotoki captures fleeting movements through the snowy streets as they play out before a static camera. Humans are made of generations of cycles; each leaving subtle indications that are part of an inexpressible whole.

Exploring the effects of stationary camera spots on inferences drawn from real-time crash severity models

This study combined crash reports, land use, real-time traffic, and weather data to form an integrated database to analyze the severity of crashes taking place on rural highways. As the traffic cameras are placed at fixed locations, there is a wide range of measured distances between crashes and the selected nearest camera for extracting traffic variables. This may change the significance of traffic variables. For the first time, spacing was introduced as the distance around the detectors in which traffic characteristics are inferred to crashes. Classification and Regression Tree (CART) was employed as an interpretable tool to explore how spacing affects model performance and the significance of traffic variables. Twelve spacing scenarios from 250 to 3000 m were evaluated. Except for short spacings suffering from the low sample size issue, each model has a good predictive performance based on overall accuracy and F2 score in the 1000–3000 m spacings. In this range, three dominant rules emerged: (1) high deviations of speed on the roads surrounded by wastelands are associated with severe crashes; (2) faded markings in residential zones increase the likelihood of severe outcomes; (3) installation of barriers decrease the probability of severe crashes. Comparing the Variable Importance Measure (VIM) reveals that the total importance of traffic variables reduces as the spacing increases. Also, results indicate that average speed is significant until 1750 m; but speed deviation, traffic flow, and percent of heavy vehicles are more stable variables for further spacings. In conclusion, for the first time, spacing scenarios were evaluated systematically and proved that they have a remarkable impact on the significance of variables. This novel research provides guidance not only on the spacing but also on which real-time traffic variables have a greater impact on crash severity, along with design, land use, and environmental variables.

Fish surveys on the move: Adapting automated fish detection and classification frameworks for videos on a remotely operated vehicle in shallow marine waters

Mobile underwater cameras, diver-operated or on underwater vehicles, have become popular for monitoring fisheries. Replacing divers with cameras has clear advantages, such as creating permanent records and accessing waters unavailable to divers. The use of cameras, however, typically produces large quantities of video that are time-consuming to process manually. Automated analysis of underwater videos from stationary cameras using deep learning techniques has advanced considerably in recent years, but the use of mobile cameras potentially raises new challenges for existing methods. We tested how well three automation procedures for stationary underwater cameras, taking an object-centric rather than background-centric approach, performed on surveys of fish using a mobile camera. We analyzed underwear drone videos from reef and seagrass habitat to detect and count two marine fisheries species, luderick (Girella tricuspidata) and yellowfin bream (Acanthopagrus australis). Three convolutional neural network (CNN) frameworks were compared: Detectron Faster R-CNN, Detectron2 Faster R-CNN (using a Regional Proposal Network, RPN), and YOLOv5 (a single-stage detector, SSD). Models performed well overall. Per frame, overall F1 scores ranged 81.4 - 87.3%, precision 88.2 – 96.0%, and recall 73.2 - 88.2%. For quantifying MaxN per video, overall F1 ranged 85.9 – 91.4%, precision 81.9 – 95.3%, and recall 87.1 – 91.1%. For luderick, F1 was > 80% for all frameworks per frame and 89% or higher for MaxN. For yellowfin bream, F1 scores were lower (35.0 - 73.8% for frames, 43.4 - 73.0% for MaxN). Detectron2 performed poorly, and YOLOv5 and Detectron performed similarly with advantages depending on metrics and species. For these two frameworks, performance was as good as in videos from stationary cameras. Our findings show that object detection technology is very useful for extracting fish data from mobile underwater cameras for the system tested here. There is a need now to test performance over a wider range of environments to produce generalizable models. The key steps required area to test and enhance performance: 1. for suites of species in the same habitats with different water clarity, 2. in other coastal environments, 3. trialing cameras moving at different speeds, and 4. using different frame-rates.

A Multi-Agent System Using Decentralized Decision-Making Techniques for Area Surveillance and Intruder Monitoring

A decentralized swarm of quadcopters designed for monitoring an open area and detecting intruders is proposed. The system is designed to be scalable and robust. The most important aspect of the system is the swarm intelligent decision-making process that was developed. The rest of the algorithms essential for the system to be completed are also described. The designed algorithms were developed using ROS and tested with SITL simulations in the GAZEBO environment. The proposed approach was tested against two other similar surveilling swarms and one approach using static cameras. The addition of the real-time decision-making capability offers the swarm a clear advantage over similar systems, as depicted in the simulation results.

Thin structures retrieval using anisotropic neighborhoods of superpixels: application to shape-from-focus

Shape-from-focus (SFF) refers to the challenging inverse problem of recovering the scene depth from a given set of focused images using a static camera. Standard approaches model the interactions between neighboring pixels to get a regularized solution. Nevertheless, isotropic regularization is known to introduce undesired artifacts and to remove early thin structures. These structures have a small size in at least one dimension and are more numerous when considering superpixel preprocessing. This paper addresses the improvement of SFF regularization through the estimation of the presence of such structures and the construction of anisotropic neighborhoods sticking along image edges and proposes a flexible formulation over pixels or superpixels. A thoroughly study comparing different strategies for constructing these neighborhoods in terms of accuracy and running time for the targeted application is provided. Notably, experiments performed on a reference dataset show the overall superiority of the approach, e.g. a decrease of the RMSE value by about 20%, and its robustness against generated superpixels.

Comparative Analysis of Background Subtraction Models Applied on a Local Dataset Using a New Approach for Ground-truth Generation

Abstract— Background subtraction is the dominant approach in the domain of moving object detection. Lots of research have been done to design or improve background subtraction models. However, there is a few well known and state of the art models which applied as a benchmark. Generally, these models are applied on different dataset benchmarks. Most of the time Choosing appropriate dataset is challenging due to the lack of datasets availability and the tedious process of creating the ground-truth frames for the sake of quantitative evaluation.
 Therefore, in this article we collected local video scenes for street and river taken by stationary camera focusing on dynamic background challenge. We presented a new technique for creating ground-truth frames using modelling, composing, tracking, and rendering each frame. Eventually we applied nine promising benchmark algorithms used in this domain on our local dataset. Results obtained by quantitative evaluations exposed the effectiveness of our new technique for generating the ground-truth scenes to be benchmarked with the original scenes using number of statistical metrics. Furthermore, results shows the outperformance of SuBSENSE model against other tested models.

An underwater Serengeti: Seagrass‐mediated effects on intake and cultivation grazing behavior of a marine megaherbivore

AbstractPopulations of green turtles (Chelonia mydas), a megaherbivore that consumes seagrasses via cultivation grazing, are recovering worldwide. Information on plant‐mediated effects on herbivore foraging behavior is critical to understanding plant–herbivore interactions and sustainability of grazing as ecosystems continue to change. In a Caribbean seagrass ecosystem, we use stationary cameras and benthic surveys to evaluate the effects of seagrass morphology and leaf nitrogen content on green turtle grazing behavior.Thalassia testudinumleaf morphology has significant effects on forage intake (in milligrams of dry mass [DM] per minute) for green turtles, whereas leaf nitrogen content has no effect. Intake increases in grazed areas with shorter leaves and higher leaf biomass concentration (in milligrams of DM per cubic centimeter), indicating more efficient foraging under these conditions. Bite rate (in bites per minute) increases in grazed areas with short leaves, a result of reduced search time. Bite size (in milligrams of DM per bite) increases in grazed areas with short but dense canopies, because a turtle crops more shoots with each bite. Increased foraging efficiency and reduced search time in grazed areas with high biomass concentrations collectively maximize intake. Ingested leaves are shorter than the mean height of all available leaves in grazed areas, indicating herbivore selection for shorter leaves. Our estimate for daily intake is 86.1 g DM day−133‐kg turtle−1. Our study provides a novel contribution on the effects of plant‐level cues on the grazing behavior of a marine megaherbivore, and how cultivation grazing behavior optimizes the green turtle foraging strategy by maximizing foraging efficiency and intake.

UAV with Vision to Recognise Vehicle Number Plates

The trend of using Unmanned Aerial Vehicles (UAVs) in industries is rapidly growing. Nowadays, they are used in many applications, from surveillance to disaster assistance. Almost all applications of UAVs require cameras either to perform specific vision tasks such as facial recognition and vehicle number plate identification or to avoid obstacles in the flight path of the UAV. The most emerging application of UAVs today is to provide security and surveillance, and they are mostly used for vehicle number identification. Typical Automatic Number Plate Recognition (ANPR) uses static high-resolution cameras mounted in specific places to identify the vehicle’s number plates. Identifying the characters on the number plate becomes a very crucial task when the number plate is at an arbitrary angle to the drone camera. The camera gimble angle, the height of the UAV from the vehicle, and the relative speed of the UAV w.r.t vehicle play a very key role in identifying the vehicle license plate correctly. This study explains how Automatic Number Plate Recognition (ANPR) is performed on real-time images using MATLAB with the help of a UAV to analyze the effect of the above-mentioned key factors. The process is completed in three steps: collecting visual data from the drone, processing that data, and obtaining the recognized number plate.

StaticPigDet: Accuracy Improvement of Static Camera-Based Pig Monitoring Using Background and Facility Information.

The automatic detection of individual pigs can improve the overall management of pig farms. The accuracy of single-image object detection has significantly improved over the years with advancements in deep learning techniques. However, differences in pig sizes and complex structures within pig pen of a commercial pig farm, such as feeding facilities, present challenges to the detection accuracy for pig monitoring. To implement such detection in practice, the differences should be analyzed by video recorded from a static camera. To accurately detect individual pigs that may be different in size or occluded by complex structures, we present a deep-learning-based object detection method utilizing generated background and facility information from image sequences (i.e., video) recorded from a static camera, which contain relevant information. As all images are preprocessed to reduce differences in pig sizes. We then used the extracted background and facility information to create different combinations of gray images. Finally, these images are combined into different combinations of three-channel composite images, which are used as training datasets to improve detection accuracy. Using the proposed method as a component of image processing improved overall accuracy from 84% to 94%. From the study, an accurate facility and background image was able to be generated after updating for a long time that helped detection accuracy. For the further studies, improving detection accuracy on overlapping pigs can also be considered.

VirtuaLot—A Case Study on Combining UAS Imagery and Terrestrial Video with Photogrammetry and Deep Learning to Track Vehicle Movement in Parking Lots

This study investigates the feasibility of applying monoplotting to video data from a security camera and image data from an uncrewed aircraft system (UAS) survey to create a mapping product which overlays traffic flow in a university parking lot onto an aerial orthomosaic. The framework, titled VirtuaLot, employs a previously defined computer-vision pipeline which leverages Darknet for vehicle detection and tests the performance of various object tracking algorithms. Algorithmic object tracking is sensitive to occlusion, and monoplotting is applied in a novel way to efficiently extract occluding features from the video using a digital surface model (DSM) derived from the UAS survey. The security camera is also a low fidelity model not intended for photogrammetry with unstable interior parameters. As monoplotting relies on static camera parameters, this creates a challenging environment for testing its effectiveness. Preliminary results indicate that it is possible to manually monoplot between aerial and perspective views with high degrees of transition tilt, achieving coordinate transformations between viewpoints within one deviation of vehicle short and long axis measurements throughout 70.5% and 99.6% of the study area, respectively. Attempted automation of monoplotting on video was met with limited success, though this study offers insight as to why and directions for future work on the subject.

Vehicle Counting and Classification for Traffic Surveillance

This work was carried out to determine the performance of image processing algorithms in classifying and counting moving vehicles in video streams of traffic scenes recorded by stationary cameras. The method for detection and tracking approach is as follows. The moving vehicles are first extracted from the traffic scene by applying the adoptive background substraction in order detect and count the vehicles using Gaussian Mixture Model. Isolated picture blobs are identified as individual vehicles after background subtraction using threshold and median filters. After the blobs have been identified, the vehicles in a given location are counted and classified. The preliminary results demonstrate that the developed system can track vehicles efficiently and reliably when a clear view of the traffic scene is available. For optimal camera calibration, an accuracy better than 80% in counting vehicles was recorded. The results of the developed system demonstrate that with additional enhancements, it may be utilized in real-time to count and classify vehicles on busy traffic routes

A cognitive assistance system with augmented reality for manual repair tasks with high variability based on the digital twin

Cognitive assistance systems with augmented reality are a promising solution to the increasing complexity in all industries. Our proposed assistance system is geared towards manual repair tasks with high variability in the maintenance, repair and overhaul industry. We use the digital twin for object-specific information and a human–machine interface that uses a web application for digital work instructions and augmented reality for spatial information. Context awareness of the assistance system is achieved by using two stationary 3D cameras and a barcode scanner. Camera images and point cloud data are used for precise markerless pose estimation and body tracking.

A New Method for Validating and Generating Vehicle Trajectories From Stationary Video Cameras

Image processing based vehicle tracking is a powerful tool for monitoring traffic, but it is error prone. Relatively small errors that impede measuring time-series speed and acceleration can be hard to detect, e.g., 1 m positioning error in a 100 m long trajectory. This paper presents an efficient approach to separate the positioning errors from vehicle travel for evaluating image processing based vehicle trajectories. The approach starts with a spatiotemporal slice, STS, which is effectively a visual time-space diagram sampled from the video. This work skews the STS to flatten a given trajectory, eliminating the vehicle travel recorded in the trajectory. Positioning errors that were imperceptible relative to the distance traveled become readily apparent in the flattened track. Thus, providing a means to quickly assess reported trajectories from almost any image processing system against the true vehicle positions in the original video data. Recognizing that the flattening process works both ways, if errors are evident in a given trajectory, the STS method can also be used to quickly fix them. Thereby providing a path to accurate instantaneous speed and acceleration throughout the given trajectory. Alternatively, one can use this process to generate vehicle trajectories directly from the STS. While the main focus is longitudinal tracking, the process can also be used to assess (extract) the lateral position of a given vehicle. The method is evaluated using the NGSIM, Cityflow and UA-DETRAC datasets, in each case it is shown how this work can increase the fidelity of the given dataset.

“Can I go into the artwork?” Material–relational situations with abstract art

The purpose of this article is to study museum educational situations, where 5–7-year-old children encounter abstract art, from a new materialist perspective. The children visit an exhibition curated by me, where abstract modernist art is mediated, with an emphasis on multisensory experiences and experimentation. The visits were recorded with stationary and action cameras. By focusing on material–relational situations, I investigate how learning takes place when the children engage with the museum educational setting. A girl asks a surprising question that challenges the takenfor-granted beliefs about what art can be. The children break rational and logical patterns by creating abstract art, and “aesthetic-intuitive order” takes place in the compositions. A child and an adult relate differently to the agential “teaching matter.” Embodied and material pedagogy with abstract art indicates how “making sense” of the world is not only done in verbal and logical ways but also by experimenting with bodies and senses with teaching matter.

A Fully Automatic Feature-Based Real-Time Traffic Surveillance System Using Data Association in the Probabilistic Framework

Multi-object tracking involves maintaining several trajectories of different objects moving in the scene throughout the video. With this objective, in this article, a fully automatic and real-time tracking algorithm to track multiple vehicles in a video is proposed. The proposed method specifically tries to address the challenges of occlusion and fast-motion in traffic surveillance. The algorithm begins with automatic detection of the moving targets by an adaptive GMM-based background subtraction method. The trajectories of these detected targets are then built using a three-level multi-motion modeled particle filter framework which allows to deal with the challenges of occlusion and fast-motions of the target. The likelihood model for targets is based on their color distribution and edge oriented histogram features. It is contended that the color distribution feature, which can represent the target appearance and the edge oriented histogram, which can describe the target structure are sufficient to represent it in an unique feature space. Based on the similarity of target likelihood, the locations of the targets are filtered. These filtered locations are then associated with the most likely detections using the proposed low-cost and fast data association algorithm based on Euclidean distance and prevailing motion vector. The performance evaluation of the proposed scheme is carried out based on the six measures: Multi Object Tracking Accuracy, Multi Object Tracking Precision, Mostly Tracked trajectories, Mostly Lost trajectories, Identity Switches and Frames Per Second. The results evaluated on stationary camera shot sequences from benchmark datasets as well as real-time shot videos indicate that the proposed algorithm ensures robust tracking and can be used effectively for real-time surveillance of highways.

Survey On Real Time Multiple Object Detection Using MobileNet-SSD with Opencv

The field of computer vision known as real-time object detection is vast, dynamic, and difficult. Real-time object recognition and tracking are critical and difficult problems in many computer vision applications, including video surveillance, robot navigation, and vehicle navigation. Detecting an item in a video sequence is what object detection is all about. Every tracking technique necessitates an object detection mechanism, either in each frame or whenever an item appears for the first time in the video sequence. The practise of identifying one object or numerous objects using a static or dynamic camera is known as object tracking. The availability of powerful computers, high-quality, low-cost video cameras will enhance the need for automated video analysis. Image Localization is used when there is only one object to distinguish in an image, and Object Detection is used when there are multiple objects in an image. The most often used strategies for contemporary deep learning models to perform various tasks on embedded devices are mobile networks and binary neural networks. In this research, we propose a method for distinguishing an item based on the MobileNet deep learning pre- prepared model for Single Shot Multi-Box Detector (SSD). This technique is used to recognise objects in a video stream in real time and for webcam broadcasting. Following that, we use an object detection module to determine what is in the video stream. To complete the module, we combine the MobileNet and SSD frameworks to create a fast and efficient deep learning-based item detection technique.

Vehicle Counting Method Based On Gaussian Mixture Models And Blob Analysis

Combination of a Gaussian Mixture Models and Entropy to detect vehicles in motion from any sort of video sequences. The proposed method is composed of four major steps: background subtraction, improved GMM, Blob Analysis, Voting Scheme processing. An implementation of proposed technique has been performed using MATLAB R2017a. Vehicle detection process provides relevant information about traffic patterns, crash occurrences and traffic peak times in roadways. Detection of number of vehicles and traffic monitoring will be helpful by implementing motion and object detection using background subtraction, Monitoring traffic from video footages will provide a greater reality in terms of vehicle count. Gaussian Mixture Model is most advanced model in motion detection due to the reliability that it has shown in the background extraction and foreground segmentation process. Here we are going to find motion of object in video sequences under both static camera arrangement and dynamic background sequences by using background subtraction method and blob analysis approach and analyzing them

Prospects for the monitoring of the great cormorant (Phalacrocorax carbo sinensis) using a drone and stationary cameras

Great cormorants (Phalacrocorax carbo sinensis) are piscivorous predators that cause high losses of fish in both aquaculture facilities and natural water bodies. Their monitoring (direct counting) in Central European carp (Cyprinus carpio) ponds is currently the only possible source of information for the evaluation of the extent of losses they cause to fish farmers. This method of monitoring is based on regular (optimally daily) counting by farmers, who are the applicants for compensation; however, this method can be easily biased by irregular observations and mistakes made in the counting of individual birds, particularly when cormorant flocks consist of tens or even hundreds of individuals. Such monitoring is not supported by any kind of record, except for the observer's notes on counting and can, therefore, be considered subjective to some extent. We introduce a new objective methodology for cormorant monitoring which eliminates the mistakes caused by the human factor, automatizes bird counting, and provides a record for the evaluation of the monitoring quality. The methodology is based on the use of drones equipped with a thermal camera and/or stationary 360° cameras placed on the monitored pond. The system automatically detects the arrival of cormorants to the pond and counts the individuals in the drone images. The methodology can be adapted to many other bird species on the ponds or appropriate water bodies.

Learning-Based Non-rigid Video Depth Estimation Using Invariants to Generalized Bas-Relief Transformations

We present a method to locally reconstruct dense video depth maps of a non-rigidly deformable object directly from a video sequence acquired by a static orthographic camera. The estimation of depth is performed locally on spatiotemporal patches of the video, and then, the full depth video is recovered by combining them together. Since the geometric complexity of a local spatiotemporal patch of a deforming non-rigid object is often simple enough to be faithfully represented with a parametric model, we artificially generate a database of small deforming rectangular meshes rendered with different material properties and light conditions, along with their corresponding depth videos, and use such data to train a convolutional neural network. Since the database images are rendered with an orthographic camera model, linear deformations along the optical axis cannot be recovered from the training images. These are known in the literature as generalized bas-relief (GBR) transformations. We address this ambiguity problem by employing the invariant-theoretic normalization procedure in order to obtain complete invariants with respect to this group of transformations, and use them in the loss function of a neural network. We tested our method on both synthetic and Kinect data and experimentally observed that the reconstruction error is significantly lower than the one obtained using conventional non-rigid structure from motion approaches and state-of-the-art video depth estimation techniques.