Pedestrians In Real Time Research Articles

Real time pedestrian and objects detection using enhanced YOLO integrated with learning complexity-aware cascades

Real time pedestrian and objects detection using enhanced YOLO integrated with learning complexity-aware cascades

AI on the Road: NVIDIA Jetson Nano-Powered Computer Vision-Based System for Real-Time Pedestrian and Priority Sign Detection

Advances in information and signal processing, driven by artificial intelligence techniques and recent breakthroughs in deep learning, have significantly impacted autonomous driving by enhancing safety and reducing the dependence on human intervention. Generally, prevailing ADASs (advanced driver assistance systems) incorporate costly components, making them financially unattainable for a substantial portion of the population. This paper proposes a solution: an embedded system designed for real-time pedestrian and priority sign detection, offering affordability and universal applicability across various vehicles. The suggested system, which comprises two cameras, an NVIDIA Jetson Nano B01 low-power edge device and an LCD (liquid crystal system) display, ensures seamless integration into a vehicle without occupying substantial space and provides a cost-effective alternative. The primary focus of this research is addressing accidents caused by the failure to yield priority to other drivers or pedestrians. Our study stands out from existing research by concurrently addressing traffic sign recognition and pedestrian detection, concentrating on identifying five crucial objects: pedestrians, pedestrian crossings (signs and road paintings separately), stop signs, and give way signs. Object detection was executed using a lightweight, custom-trained CNN (convolutional neural network) known as SSD (Single Shot Detector)-MobileNet, implemented on the Jetson Nano. To tailor the model for this specific application, the pre-trained neural network underwent training on our custom dataset consisting of images captured on the road under diverse lighting and traffic conditions. The outcomes of the proposed system offer promising results, positioning it as a viable candidate for real-time implementation; its contributions are noteworthy in advancing the safety and accessibility of autonomous driving technologies.

Robot Real-time Pedestrian Tracking Algorithm based on Deep Learning

In recent years, deep learning models have greatly improved the performance of pedestrian detection and tracking. Although the accuracy of neural network-based modeling methods is high, they often require large computational and storage resource overheads, which makes them difficult to apply to robots with high resource requirements. Pedestrian tracking for robots still faces great challenges in the problems of occlusion, multi-target, and target loss. In this thesis, we focus on solving the problem of real-time pedestrian tracking by lightweight fusion of deep learning target detection models for robots, firstly, through the lightweight YOLO network, we perform pedestrian detection and feature extraction, and then we propose a Gaussian mixture model based feature matching method to construct the target pedestrian tracker, and finally, we use the PID-based control algorithm for real-time control of the robot's motion, and finally realize the real-time pedestrian Tracking. In this thesis, we validate the feature matching method based on the Gaussian mixture model on the ETH dataset, and at the same time, combined with the motion control algorithm, we carry out the actual validation, and the experimental results show that our proposed method can realize the real-time pedestrian tracking.

Hold surrounding’s key-you only look once version 7: a real-time pedestrian and vehicle detection algorithm in the low-signal-to-noise ratio infrared image

Hold surrounding’s key-you only look once version 7: a real-time pedestrian and vehicle detection algorithm in the low-signal-to-noise ratio infrared image

EfficientLiteDet: a real-time pedestrian and vehicle detection algorithm

EfficientLiteDet: a real-time pedestrian and vehicle detection algorithm

Diminished reality system with real-time object detection using deep learning for onsite landscape simulation during redevelopment

Landscape simulation is necessary for stakeholders to discuss future landscapes with new designs in order to preserve good landscapes. Augmented reality can be used to study the future landscape on a large scale by adding a three-dimensional design model to the real world. On the other hand, diminished reality (DR) can simulate the virtual demolition and removal of structures in redevelopment. However, it has not been possible to visually remove moving landscape objects such as vehicles and pedestrians in real time for accurate landscape simulation. This research develops a DR system that can virtually remove moving landscape objects by implementing real-time object detection using deep learning with a game engine, as well as immobile objects such as structures. In addition to evaluating the performance of detecting the size of moving landscape objects, the developed DR system is applied to large-scale landscape simulation at two sites, and its utility is validated.

A Pedestrian Detection and Tracking Method for Robot Equipped with Laser Radar

In order to detect and track pedestrians in complex indoor backgrounds, a pedestrian detection and tracking method for indoor robots equipped with Laser radar is proposed. Firstly, The SLAM (Simultaneous Location and Mapping) technology is applied to obtain 2D grid map for a strange environment; then, Monte Carlo localization is employed to obtain the posterior pose of the robot in the map; then, an improved likelihood field background subtraction algorithm is proposed to extract the interesting foreground in changeable environment; then, the hierarchical clustering algorithm combining with an improved leg model is proposed to detect the objective pedestrian; at last, an improved tracking intensity formula is designed to track and follow the objective pedestrian. Experimental results in some complex environments show that our method can effectively reduce the impact of confusing scenarios which are challenges for other algorithms, such as the motion of the chair, the suddenly passing by person and when the objective pedestrian close to the wall and so on, and can detect, track and follow pedestrians in real time with high accuracy.

Priority and collision avoidance system for traffic lights

In this paper, a collision avoidance system is presented to detect red light running and warn nearby vehicles and pedestrians in real time in order to prevent possible accidents. No complex infrastructure-based solution such as those based on radars or cameras is here required. Instead, a new solution based on smartphones carried by drivers and pedestrians is proposed so that it is the device inside the vehicle violating a traffic light, the one that self-reports the offence in order to generate alerts and warn nearby vehicles and pedestrians to prevent accidents. The proposal could also be used by road authorities to collect data on traffic lights that are most frequently violated in order to define an action plan to investigate causes and look for solutions. It includes a classifier for learning and estimating driver behaviour based on collected data, which is used to predict whether he/she is about to run a red light or detect whether that has already happened. In the first case, the system broadcasts warnings directly to close vehicles and pedestrians through Wi-Fi, while in the second case, the proposal warns vehicles and pedestrians in the neighbourhood through a server. The solution also includes a prioritization system based on changing traffic lights at intersections according to the needs and characteristics of the traffic at all times, giving the top priority to emergency vehicles. Furthermore, the proposal involves the use of cryptographic schemes to protect authenticity and integrity of messages sent from traffic lights, smartphones and servers, and privacy and anonymity to promote the use of the system. A beta version with some parts of the proposal has been implemented and the obtained results are promising.

Lidar-Based Gait Analysis and Activity Recognition in a 4D Surveillance System

This paper presents new approaches for gait and activity analysis based on data streams of a rotating multibeam (RMB) Lidar sensor. The proposed algorithms are embedded into an integrated 4D vision and visualization system, which is able to analyze and interactively display real scenarios in natural outdoor environments with walking pedestrians. The main focus of the investigations is gait-based person reidentification during tracking and recognition of specific activity patterns, such as bending, waving, making phone calls, and checking the time looking at wristwatches. The descriptors for training and recognition are observed and extracted from realistic outdoor surveillance scenarios, where multiple pedestrians are walking in the field of interest following possibly intersecting trajectories; thus, the observations might often be affected by occlusions or background noise. Since there is no public database available for such scenarios, we created and published a new Lidar-based outdoor gait and activity data set on our website that contains point cloud sequences of 28 different persons extracted and aggregated from 35-min-long measurements. The presented results confirm that both efficient gait-based identification and activity recognition are achievable in the sparse point clouds of a single RMB Lidar sensor. After extracting the people trajectories, we synthesized a free-viewpoint video, in which moving avatar models follow the trajectories of the observed pedestrians in real time, ensuring that the leg movements of the animated avatars are synchronized with the real gait cycles observed in the Lidar stream.

Nighttime Foreground Pedestrian Detection Based on Three-Dimensional Voxel Surface Model.

Pedestrian detection is among the most frequently-used preprocessing tasks in many surveillance application fields, from low-level people counting to high-level scene understanding. Even though many approaches perform well in the daytime with sufficient illumination, pedestrian detection at night is still a critical and challenging problem for video surveillance systems. To respond to this need, in this paper, we provide an affordable solution with a near-infrared stereo network camera, as well as a novel three-dimensional foreground pedestrian detection model. Specifically, instead of using an expensive thermal camera, we build a near-infrared stereo vision system with two calibrated network cameras and near-infrared lamps. The core of the system is a novel voxel surface model, which is able to estimate the dynamic changes of three-dimensional geometric information of the surveillance scene and to segment and locate foreground pedestrians in real time. A free update policy for unknown points is designed for model updating, and the extracted shadow of the pedestrian is adopted to remove foreground false alarms. To evaluate the performance of the proposed model, the system is deployed in several nighttime surveillance scenes. Experimental results demonstrate that our method is capable of nighttime pedestrian segmentation and detection in real time under heavy occlusion. In addition, the qualitative and quantitative comparison results show that our work outperforms classical background subtraction approaches and a recent RGB-D method, as well as achieving comparable performance with the state-of-the-art deep learning pedestrian detection method even with a much lower hardware cost.

ROBOT MÓVIL CON NAVEGACIÓN BASADA EN VISIÓN Y DETECCIÓN DE PEATONES

<p>Este artículo propone el diseño e implementación de un robot móvil con navegación basada en visión, de bajo costo, que sigue la trayectoria de peatones en tiempo real usando una cámara IP a bordo. El propósito de este prototipo es la navegación basada en el seguimiento de personas conservando una distancia segura a través de controladores PID y $on-off$. Para la puesta en marcha se evalúan dos algoritmos de detección de peatones: cascada de clasificadores HOG y cascada de clasificadores LBP, tanto fuera de línea como a bordo del robot. Adicionalmente, se implantó un sistema de comunicación entre el robot y una estación de tierra. Las métricas de evaluación para las propuestas de detección de personas fueron la precisión y sensibilidad, obteniendo mejores resultados con HOG. Al final, se evaluó el sistema de comunicación, calculando el retraso de la respuesta del controlador. Los resultados mostraron que el sistema trabaja adecuadamente para una tasa de transmisión de 115200 baudios.</p>

Temporal Search Algorithm for Multiple-Pedestrian Tracking

In this paper, we provide a trajectory-generation algorithm that can identify pedestrians in real time. Typically, the contours for the extraction of pedestrians from the foreground of images are not clear due to factors including brightness and shade; furthermore, pedestrians move in different directions and interact with each other. These issues mean that the identification of pedestrians and the generation of trajectories are somewhat difficult. We propose a new method for trajectory generation regarding multiple pedestrians. The first stage of the method distinguishes between those pedestrian-blob situations that need to be merged and those that require splitting, followed by the use of trained decision trees to separate the pedestrians. The second stage generates the trajectories of each pedestrian by using the point-correspondence method; however, we introduce a new point-correspondence algorithm for which the A* search method has been modified. By using fuzzy membership functions, a heuristic evaluation of the correspondence between the blobs was also conducted. The proposed method was implemented and tested with the PETS 2009 dataset to show an effective multiple-pedestrian-tracking capability in a pedestrian-interaction environment.

A Multi-Stage Approach for Real Time Speed and Direction Estimations of Pedestrian

In autonomous vehicle system, the direction and speed of pedestrian must be estimated in order to guide the automatic breaking system. It can be used to protect the pedestrian walking on road. This paper presents a new algorithm for speed and direction estimations of pedestrian in real time. In the first step, the HOG features are extracted. The features are then fed to SVM for classification. Given the bounding box, the optical flow for each bounding box is calculated. Therefore, direction and speed are estimated from the optical flow. We achieved a very low time computation by using CPU-GPU co-design which is performed based on parallel processing. Experiments were performed to evaluate our system and compare several methods for time computation. Performance of our algorithm will also be demonstrated.

A Real-Time Active Pedestrian Tracking System Inspired by the Human Visual System

Pedestrian detection and tracking play a significant role in surveillance. Despite the numerous detection and tracking methods proposed in the literature, when the pedestrian is too small to recognize, which is a common case in modern surveillance systems, all methods fail. In order to deal with such case, we propose an active pedestrian tracking system inspired by the human visual system. A coarse-to-fine pedestrian detection algorithm is proposed for the small pedestrian detection by combining the Gaussian mixture model background subtraction with the histogram of oriented gradient detection. In addition, a three-dimensional pan–tilt–zoom control model is presented, which requires no calibration and is more accurate than other control models. In order to actively track a pedestrian in real time, we utilize an active control algorithm and a tracking–learning–detection tracker. Experimental results demonstrate that our active tracking system is both efficient and effective.

An Efficient Methodology for Calibrating Traffic Flow Models Based on Bisection Analysis

As urban planning becomes more sophisticated, the accurate detection and counting of pedestrians and cyclists become more important. Accurate counts can be used to determine the need for additional pedestrian walkways and intersection reorganization, among other planning initiatives. In this project, a camera-based approach is implemented to create a real-time pedestrian and cyclist counting system which is regularly accurate to 85% and often achieves higher accuracy. The approach retasks a state-of-the-art traffic camera, the Autoscope Solo Terra, for pedestrian and bicyclist counting. Object detection regions are sized to identify multiple pedestrians moving in either direction on an urban sidewalk and bicyclists in an adjacent bicycle lane. Collected results are processed in real time, eliminating the need for video storage and postprocessing. In this paper, results are presented for a pedestrian walkway for pedestrian flow up to 108 persons/min and the limitations of the implemented system are enumerated. Both pedestrian and cyclist counting accuracy of over 90% is achieved.

Top-View-Based Guidance for Blind People Using Directional Ellipse Model

The guidance system proposed in this paper aims to complement the white cane by monitoring road conditions in a medium range for blind pedestrians in real time. The system prototype employs only one webcam fixed at the waist of user. One of the main difficulties of using a single camera in outdoor obstacle detection is the discrimination of obstacles from a complex background. To solve this problem, this paper re-formulates top-view mapping as an inhomogeneous re-sampling process, so that background edges are sub-sampled while obstacle edges are oversampled in the top-view domain. Morphology filters are then used to enhance obstacle edges as edge-blobs, which are further represented using a directional ellipse as a new model for obstacle classification. Based on the identified obstacles, safe walking area is estimated by tracking a polar edge-blob histogram. To transfer the information obtained from image domain to language domain, this paper proposes a verbal message generation scheme based on fuzzy logic. The efficiency of the system is confirmed by testing the system with visually impaired people on outdoor pedestrian paths.

An Efficient Tree Classifier Ensemble-Based Approach for Pedestrian Detection

Classification-based pedestrian detection systems (PDSs) are currently a hot research topic in the field of intelligent transportation. A PDS detects pedestrians in real time on moving vehicles. A practical PDS demands not only high detection accuracy but also high detection speed. However, most of the existing classification-based approaches mainly seek for high detection accuracy, while the detection speed is not purposely optimized for practical application. At the same time, the performance, particularly the speed, is primarily tuned based on experiments without theoretical foundations, leading to a long training procedure. This paper starts with measuring and optimizing detection speed, and then a practical classification-based pedestrian detection solution with high detection speed and training speed is described. First, an extended classification/detection speed metric, named feature-per-object (fpo), is proposed to measure the detection speed independently from execution. Then, an fpo minimization model with accuracy constraints is formulated based on a tree classifier ensemble, where the minimum fpo can guarantee the highest detection speed. Finally, the minimization problem is solved efficiently by using nonlinear fitting based on radial basis function neural networks. In addition, the optimal solution is directly used to instruct classifier training; thus, the training speed could be accelerated greatly. Therefore, a rapid and accurate classification-based detection technique is proposed for the PDS. Experimental results on urban traffic videos show that the proposed method has a high detection speed with an acceptable detection rate and a false-alarm rate for onboard detection; moreover, the training procedure is also very fast.

Real‐time recognition of multiple pedestrians using car‐mounted camera

AbstractIn ITS, there has recently been emphasis on the importance of developing a driving assistance system which can detect pedestrians with a car‐mounted camera and warn of their presence in order to assure pedestrian safety. In particular, the ability to detect multiple pedestrians in real time (at a video rate of 30 fps) is indispensable from the practical point of view. Consequently, this study proposes a method of recognizing multiple moving objects (pedestrians) in real time using dynamic scene input from a car‐mounted camera, as a basic technology for the driving assistance system. The presence of a pedestrian is evaluated on the basis of hue information in the HSI color representation system and model‐based matching using a circular approximation model. Thus, in terms of the evaluation function, the problem of recognition of multiple pedestrians is replaced by the problem of search for multiple local maxima. However, it is difficult to realize the process in real time if the evaluation value is calculated for the whole image. This paper considers the “attention,” which is a function that reduces the burden of human visual information processing, and proposes a method in which a pseudo‐attention function is realized by using a hybrid GA, allowing multiple local maxima to be retrieved efficiently in real time.A real‐time experiment on the recognition of multiple pedestrians in dynamic images obtained from a car‐mounted camera was used to verify the effectiveness of the proposed method. In addition, the practical performance of the system was investigated by an experiment on the recognition of suddenly appearing pedestrians. © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 89(4): 21–33, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20181