Pedestrian Information Research Articles

Image–Text Person Re-Identification with Transformer-Based Modal Fusion

Existing person re-identification methods utilizing CLIP (Contrastive Language-Image Pre-training) mostly suffer from coarse-grained alignment issues. This is primarily due to the original design intention of the CLIP model, which aims at broad and global alignment between images and texts to support a wide range of image–text matching tasks. However, in the specific domain of person re-identification, local features and fine-grained information are equally important in addition to global features. This paper proposes an innovative modal fusion approach, aiming to precisely locate the most prominent pedestrian information in images by combining visual features extracted by the ResNet-50 model with text representations generated by a text encoder. This method leverages the cross-attention mechanism of the Transformer Decoder to enable text features to dynamically guide visual features, enhancing the ability to identify and locate the target pedestrian. Experiments conducted on four public datasets, namely MSMT17, Market1501, DukeMTMC, and Occluded-Duke, demonstrate that our method outperforms the baseline network by 5.4%, 2.7%, 2.6%, and 9.2% in mAP, and by 4.3%, 1.7%, 2.7%, and 11.8% in Rank-1, respectively. This method exhibits excellent performance and provides new research insights for the task of person re-identification.

Novel Surveillance View: A Novel Benchmark and View-Optimized Framework for Pedestrian Detection from UAV Perspectives

To address the issues of insufficient samples, limited scene diversity, missing perspectives, and low resolution in existing UAV-based pedestrian detection datasets, this paper proposes a novel UAV-based pedestrian detection benchmark dataset named the Novel Surveillance View (NSV). This dataset encompasses diverse scenes and pedestrian information captured from multiple perspectives, and introduces an innovative data mining approach that leverages tracking and optical flow information. This approach significantly improves data acquisition efficiency while ensuring annotation quality. Furthermore, an improved pedestrian detection method is proposed to overcome the performance degradation caused by significant perspective changes in top-down UAV views. Firstly, the View-Agnostic Decomposition (VAD) module decouples features into perspective-dependent and perspective-independent branches to enhance the model’s generalization ability to perspective variations. Secondly, the Deformable Conv-BN-SiLU (DCBS) module dynamically adjusts the receptive field shape to better adapt to the geometric deformations of pedestrians. Finally, the Context-Aware Pyramid Spatial Attention (CPSA) module integrates multi-scale features with attention mechanisms to address the challenge of drastic target scale variations. The experimental results demonstrate that the proposed method improves the mean Average Precision (mAP) by 9% on the NSV dataset, thereby validating that the approach effectively enhances pedestrian detection accuracy from UAV perspectives by optimizing perspective features.

Unsupervised person re‐identification based on adaptive information supplementation and foreground enhancement

AbstractUnsupervised person re‐identification has attracted vital interest because of its ability to protect privacy, significantly lower the expense of manual annotation, and eliminate the need for data labels. General unsupervised methods train the network only through global features, which causes the fine‐grained information contained in local features to be ignored in the recognition process, resulting in large amounts of label noise and affecting the recognition accuracy. Moreover, more robust pedestrian features can also improve the accuracy of clustering and enable unsupervised person re‐identification to obtain better results. To address these issues, first, a dual‐branch structure was proposed, which separately obtains the global features of the pedestrian and the local features by dividing the global features into a few equal sections. Then, an adaptive information supplementation (AIS) method based on the k‐nearest neighbor algorithm is designed to ascertain each local feature's relevance to the global features, calculating adaptive weight scores for information supplementation. Finally, these weight scores are used to reallocate the weights of the global features in each part, acquiring features that contain more pedestrian information during the representation learning process. These better features are used to reduce label noise to obtain more accurate pseudo‐labels. Second, an adaptive foreground enhancement module (AFEM) was proposed and inserted before clustering to increase the robustness of pedestrian features, which increases the precision of the pseudo‐labels that are produced after clustering. Experiments on Market‐1501, DukeMTMC‐reID, and MSMT17 demonstrate that the proposed method achieves better results than state‐of‐the‐art methods in fully unsupervised person re‐identification tasks.

A Study on Bilingual Deep Learning PIS Neural Network Model Based on Graph-Text Modal Fusion

This research investigates a multilingual cross-modal pedestrian information search (PIS) technique based on graph-text modal fusion. Initially, we used a combination of replacement neural networks to improve the English Language-Based Pedestrian Information Search model with Graph-Text Modal Fusion (GTMFLPIS) performance. In addition, existing research lacks GTMFLPIS models for other languages. Therefore, we propose to train GTMFLPIS models for Chinese. The Chinese GTMFLPIS model was trained using our previously constructed Chinese CUHK-PEDES dataset. The Rank1 of the Chinese RN50_PMML12V2 model reached 0.5989. In addition, we found that a single model could not adapt to the limitations of multiple languages. Therefore, we propose a novel architecture to implement a single-model multilingual cross-modal GTMFLPIS model in this research. We propose RN50_DBMCV2 and ENB7_DBM-CV2, both of which have improved performance over the existing ones. We constructed a bilingual dataset using our Chinese CUHK-PEDES dataset and existing English CUHK-PEDES dataset to test our novel multilingual cross-modal GTMFLPIS model. In addition, we found that the loss function significantly impacts the model during our experiments. Therefore, we optimized the performance of the existing loss functions for cross-modal GTMFLPIS models. Our proposed CCMPM loss function improves the performance of the model by 2%. The experimental results of this research show that our proposed model has advantages in improving the accuracy of PIS.

Improved Pedestrian Vehicle Detection for Small Objects Based on Attention Mechanism

Abstract This study aims to solve the low detection accuracy and susceptibility to false detection and omission in pedestrian and vehicle detection by proposing an improved YOLOv5s algorithm. Firstly, a small target detection module is added to better acquire and determine the information of pedestrians from long-range vehicles. Secondly, the multi-scale channel attention CBAM attention module is added, and the dual attention mechanism is not only flexible and convenient, but also improves the computational efficiency. Finally, the MPDIoU loss function based on minimum point distance is introduced to replace the original GIoU loss function, and this change not only enhances the regression accuracy of the model. At the same time, the convergence speed of the model is accelerated. KITTI data set was used for experiments, and the experimental results showed that the average accuracy of the model trained by the improved YOLOv5s algorithm on the data set reached 84.9%, which was 3.7% higher than that of the original YOLOv5s algorithm. It is verified that the model is suitable for high accuracy of pedestrian and vehicle recognition in complex environments, and has high value for promotion.

Robust pedestrian multi-object tracking in the intelligent bus environment

Abstract Pedestrian multi-object tracking algorithms aim to maintain identity information of pedestrians by comparing the similarity between trajectories and detections, predicting pedestrian motion trajectories. However, within the context of intelligent bus, challenges arise due to factors such as passenger growth and vehicle vibrations, rendering existing pedestrian multi-object tracking algorithms less accurate. Therefore, this paper proposes an intelligent bus terminal robust pedestrian multi-object tracking algorithm named pure motion (PM), which can consistently and stably track pedestrians. The proposed algorithm employs several key strategies. Firstly, it optimizes trajectory prediction by adapting the aspect ratio of the prediction box based on pedestrian movement, automatically adjusting its shape, and selecting velocity weight coefficients according to different tracking targets. Secondly, it decomposes the homography matrix to acquire motion components and correct predicted results under motion conditions. Subsequently, the algorithm leverages the similarity between detection results and trajectories to retain high-confidence detections, eliminating low-confidence ones associated with background, thereby reducing false negatives and enhancing trajectory coherence. Futhermore, the introduction of detection confidence into trajectory updates to enhances the precision of measurement noise. The proposed algorithm underwent testing in intelligent bus driving scenarios, including turns, waiting for traffic lights, emergency braking, and approaching bus stops. The tracking accuracy on the MOT17-13-val dataset reaches 81.8. The results demonstrate that PM significantly improves the robustness of pedestrian multi-object tracking algorithms in the environment of intelligent bus.

Model-aware privacy-preserving with start trigger method for person re-identification

Person Re-identification (ReID) could search for the same pedestrian from non-overlapping cameras, which completes the pedestrian location and search purpose. However, the process contains much sensitive pedestrian information and raises serious privacy problems. Conventional methods mainly remove identity-related features from pedestrian images to alleviate the privacy issue. Unfortunately, these strategies cause pedestrian information loss and poor data utility. In the paper, we propose a novel Model-Aware Privacy-Preserving with Start Trigger (MPST) method, which not only prevents personal identity for third parties but also achieves accurate pedestrian location for authorized organizations. The core idea is that authorized organizations obtain the start trigger to activate the ReID model that has the ability to search for target pedestrians, while third parties (i.e., hackers) cannot employ the ReID model to complete the pedestrian matching task without the start trigger. To this end, we develop a universal adversarial algorithm to learn an ingenious start trigger for the person ReID system. Afterwards, we further design a model-aware training strategy to facilitate our deep model to perceive issued instructions by synthetically utilizing the start trigger and original pedestrian samples. As a result, we successfully install an activate button to change the ReID model state for deciding whether the deep model has the ability to search for pedestrians or not. Abundant experiments demonstrate that the proposed MPST is effective for pedestrian identity anonymization. Our study achieves superior performance for authorized organizations and completes the privacy protection goal.

Crowdedness information and travel decisions of pedestrians and public transport users in the COVID-19 era: A stated preference analysis

The COVID-19 pandemic has impacted people's everyday lives, as avoiding being in crowded places became the number one societal rule. Crowdedness has therefore increasingly affected decisions such as a place visit via a specific path, the selection of a public transport stop, itinerary, etc., thereby making related information increasingly relevant. The objective of this study is to examine the route and travel choices of pedestrians and public transport users, with the provisioning of travel information related to crowdedness levels. To that end, a choice experiment was designed to elicit travelers' preferences. Discrete choice models were estimated based on data collected from 465 individuals in Greece. Results showed that crowd avoidance plays a significant role in shaping mobility decisions for both pedestrians and public transport users. Factors such as place of residence, age, the importance of COVID-19 measures and arrival time are found to affect the likelihood of switch routes in response to information about high levels of crowdedness.

A Unified Environmental Network for Pedestrian Trajectory Prediction

Accurately predicting pedestrian movements in complex environments is challenging due to social interactions, scene constraints, and pedestrians' multimodal behaviors. Sequential models like long short-term memory fail to effectively integrate scene features to make predicted trajectories comply with scene constraints due to disparate feature modalities of scene and trajectory. Though existing convolution neural network (CNN) models can extract scene features, they are ineffective in mapping these features into scene constraints for pedestrians and struggle to model pedestrian interactions due to the loss of target pedestrian information. To address these issues, we propose a unified environmental network based on CNN for pedestrian trajectory prediction. We introduce a polar-based method to reflect the distance and direction relationship between any position in the environment and the target pedestrian. This enables us to simultaneously model scene constraints and pedestrian social interactions in the form of feature maps. Additionally, we capture essential local features in the feature map, characterizing potential multimodal movements of pedestrians at each time step to prevent redundant predicted trajectories. We verify the performance of our proposed model on four trajectory prediction datasets, encompassing both short-term and long-term predictions. The experimental results demonstrate the superiority of our approach over existing methods.

Elderly drivers’ behaviour due to indication of pedestrian information by HMI

Japan is becoming a super-aging society, where there are concerns regarding traffic accidents caused by elderly drivers. This study aimed to investigate the effectiveness of a human–machine interface (HMI) for assessing elderly drivers’ behaviour. The drivers’ behaviour was investigated when a pedestrian crossed while the drivers turned right at intersections and drove on straight roads, with vehicles travelling in the left lane. We investigated 16 elderly participants by using a driving simulator. The results indicated that the display of pedestrian information increased the closest distance of the vehicle to the crossing pedestrian. Moreover, time to collision increased under conditions where there were three preceding vehicles turning right at an intersection or driving straight on a straight road. In the questionnaire, most participants answered that the HMI was helpful. These findings can serve as the basis while designing an HMI that encourages elderly drivers to notice pedestrians.

Measuring pedestrians' movement and building a visual-based attractiveness map of public spaces using smartphones

Accurately measuring the distribution of vitality in urban public spaces and evaluating the attractiveness index of landscape elements can enhance the precision of public space design and the rational planning of urban microupdate strategies. This study introduces a method based on the SLAM algorithm to create a 3D visual exposure analysis and applied in two squares in Macau to calculate the relationship between pathfinding decisions and visual exposure. The results demonstrate that both the extent of visual exposure (seen from the observer) and the area of isovist field (seen from the object) can statistically indicate the attractiveness index of a space. Consequently, an attractiveness map of the site can be constructed. This study effectively captures walking trajectories and visual data in real-world settings, integrating site-specific and pedestrian information into a digital twin system. This approach not only advances quantitative methodologies but also facilitates postoccupancy evaluations of public space usage and environmental behavior research, thereby expanding the potential for future investigations in this domain.

Dual-Stage Attribute Embedding and Modality Consistency Learning-Based Visible–Infrared Person Re-Identification

Visible–infrared person re-identification (VI-ReID) is an emerging technology for realizing all-weather smart surveillance systems. To address the problem of pedestrian discriminative information being difficult to obtain and easy to lose, as well as the wide modality difference in the VI-ReID task, in this paper we propose a two-stage attribute embedding and modality consistency learning-based VI-ReID method. First, the attribute information embedding module introduces the fine-grained pedestrian information in the attribute label into the transformer backbone, enabling the backbone to extract identity-discriminative pedestrian features. After obtaining the pedestrian features, the attribute embedding enhancement module is utilized to realize the second-stage attribute information embedding, which reduces the adverse effect of losing the person discriminative information due to the deepening of network. Finally, the modality consistency learning loss is designed for constraining the network to mine the consistency information between two modalities in order to reduce the impact of modality difference on the recognition results. The results show that our method reaches 74.57% mAP on the SYSU-MM01 dataset in All Search mode and 87.02% mAP on the RegDB dataset in IR-to-VIS mode, with a performance improvement of 6.00% and 2.56%, respectively, proving that our proposed method is able to reach optimal performance compared to existing state-of-the-art methods.

Attribute-Guided Collaborative Learning for Partial Person Re-Identification.

Partial person re-identification (ReID) aims to solve the problem of image spatial misalignment due to occlusions or out-of-views. Despite significant progress through the introduction of additional information, such as human pose landmarks, mask maps, and spatial information, partial person ReID remains challenging due to noisy keypoints and impressionable pedestrian representations. To address these issues, we propose a unified attribute-guided collaborative learning scheme for partial person ReID. Specifically, we introduce an adaptive threshold-guided masked graph convolutional network that can dynamically remove untrustworthy edges to suppress the diffusion of noisy keypoints. Furthermore, we incorporate human attributes and devise a cyclic heterogeneous graph convolutional network to effectively fuse cross-modal pedestrian information through intra- and inter-graph interaction, resulting in robust pedestrian representations. Finally, to enhance keypoint representation learning, we design a novel part-based similarity constraint based on the axisymmetric characteristic of the human body. Extensive experiments on multiple public datasets have shown that our model achieves superior performance compared to other state-of-the-art baselines.

Obstacle Avoidance Learning for Robot Motion Planning in Human–Robot Integration Environments

In the human-robot integration environment, it is essential for mobile robots to pass through the crowd and complete the navigation task smoothly. However, the current mainstream robot navigation algorithms only treat pedestrians as dynamic obstacles and passively avoid pedestrians in local planning. When encountering fast-moving pedestrians, local path planning often fails, causing the robot to stagnate, spin or shake in place, which in turn reduces the navigation efficiency and results in unnatural navigation trajectories. To address this problem, it is desirable for the robot to find a safe and convenient temporary target to avoid the collision with fast-moving pedestrians. In this paper, we propose an obstacle avoidance learning method with the temporary target for the robot motion planning in the human-robot integration environment. The temporary target distribution is learned from imitations by using a Conditional Variational Autoencoder (CVAE) framework, whereby the dynamic scenario information including pedestrian information, the environmental information, and the robot information are considered as the generation conditions. With the proposed method, the mobile robot first navigates to the temporary target area, and then plans the path toward the final target point. Experimental studies reveal that the proposed method can achieve satisfactory performance with respect to different scenario conditions.

Dynamic coupling model of FDS and cellular automata considering trampling behavior

Fire accidents usually cause large casualties. In this paper, a fire evacuation model based on the dynamic coupling of FDS and CA, which dynamically couples disaster data and evacuation behavior, is designed to reflect the impact of disaster factors on the pedestrian evacuation process in real-time. Furthermore, pedestrians are divided into two categories, one type of pedestrians will choose to move against the wall to reduce the risk of stampede accidents, and the other type of pedestrian will evacuate to the exit according to the normal movement method. And considering the role of information transmission between pedestrians, the two types of pedestrians will transform into each other. In addition, the concept of pedestrian crowding, support, and friction is introduced and viewed as an influencing factor of normal pedestrian falls. Set the panic value as the influencing factor of the efficiency of pedestrian information transmission and the death probability of downed pedestrians. The impact of pedestrian density, pedestrian walking behavior towards walls, evacuation response time, information propagation radius, heat release rate of the fire source, and the interference of smoke on the pedestrians' visibility range on evacuation outcomes has been discussed through numerical simulations. The analysis shows that although the presence of pedestrians walking against the wall will slightly increase the evacuation time, when the available evacuation time allows, the risk of a stampede will be greatly reduced; The increase in evacuation response time will aggravate the panic psychology of pedestrians and increase the risk of stampede accidents; The increase of the pedestrian information transmission radius will reduce the risk of a stampede, and the effect is more obvious when the pedestrian density is smaller; Selecting building materials with lower heat release rates can effectively reduce the risk of fire incidents; The restriction of visibility range by smoke seriously endangers pedestrian safety and reduces the efficiency of evacuation.Our research results can play a reference and guide for future building disaster prevention design and public safety research.

Multi-attention network for pedestrian intention prediction based on spatio-temporal feature fusion

An essential prerequisite for autonomous vehicles deploying in urban scenarios is the ability to accurately recognize the behavioral intentions of pedestrians and other vulnerable road users and take measures to ensure their safety. In this paper, a spatial-temporal feature fusion-based multi-attention network (STFF-MANet) is designed to predict pedestrian crossing intention. Pedestrian information, vehicle information, scene context, and optical flow are extracted from continuous image sequences as feature sources. A lightweight 3D convolutional network is designed to extract temporal features from optical flow. Construct a spatial encoding module to extract the spatial features from the context. Pedestrian motion information are re-encoded using a collection of gated recurrent units. The final network structure is created through ablation research, which introduces attention mechanisms into the network to merge pedestrian motion features and spatio-temporal features. The efficiency of the suggested strategy is demonstrated by comparison experiments on the datasets JAAD and PIE. On the JAAD dataset, the intent recognition accuracy is 9% more accurate than the existing techniques.

Robust feature mining transformer for occluded person re-identification

The person Re-identification (ReID) technology in real scenes is still a challenging task, especially when the target pedestrian is occluded. Existing occluded person ReID methods distinguish visible regions through pose information or semantic information. These methods focus on the body information in the visible region, which can perform well for ReID. However, since these approaches require the help of auxiliary networks, they are not easy to obtain efficient models. In addition, under the pose guidance, these networks focus only on the visible body parts and ignore other regional features. Therefore, we propose a Robust Feature Mining Transformer (RFMT) method to tackle the above problems of occluded person re-identification. Specifically, when dealing with occlusion, the proposed Residual Transformer (ResT) layer allows for a more efficient spread of image features through cross-layer connections, thereby reducing the loss of pedestrian information. Furthermore, obtaining complete pedestrian features becomes a big challenge in occluded scenes. Our proposed Global Attention (GAtt) enhances feature extraction by accurately focusing on the most representative part of images. And then, we design a hyperparametric centroid loss (Lctr) to catch stable features. Meanwhile, this loss can reduce the image search time during training and retrieval. Many experiments have demonstrated that our RFMT performs better than the state-of-the-art ReID methods on public occluded and holistic person re-identification datasets.

Predicting pedestrian tracks around moving vehicles based on conditional variational transformer

Fast and accurate prediction of pedestrian trajectory around vehicles can reduce or even prevent most traffic accidents and improve the safety of traffic participants. This involves the real-time interaction and fusion of various information, such as the vehicle’s motion characteristics, the pedestrian’s historical motion trajectory, and the motion relationship between people and cars. However, most of the existing algorithms use RNN as the skeleton to process the information prediction trajectory, which is weak in extracting the internal relationship between different information, and the running time of the algorithm is long. To solve these problems, we propose a Transformer based deep learning algorithm (CVTF) to complete the first-person perspective pedestrian trajectory prediction task. We have the following innovations about this model: 1: We use the stamp coding method for vehicle speed and pedestrian information to ensure we can learn the characteristics of different information sources. 2: Transformer structure is used, and its attention mechanism is improved (Maybe-self attention), which improves the running speed of the model and reduces the memory consumption. 3: Combined with Conditional Variational Autoencoder (CVAE), hidden variables are introduced to improve the prediction accuracy. Experiments on three pedestrian trajectory prediction benchmarks show that our model achieves the most advanced performance.

Information complementary attention-based multidimension feature learning for person re-identification

With the need for criminal investigation technology and the development of deep learning, the task of person re-identification has gradually become a research hotspot. Recently, various neural network-based person re-identification technologies designed by researchers have shown excellent results. However, most of the frameworks focus on complex structural design or redundant networks to guide model construction, which hugely increases the cost of train and application cost. In addition, the correlation between the channel information and spatial information on the pedestrian feature map is also relatively lacking. Therefore, we design a lightweight attention module to address the lack of correlation question response. The proposed module sequentially extracts person images’ channel and spatial features and effectively associates the two kinds of information through sequential connections. The proposed attention module has a simple structure, and the parameter increase in the backbone network is tiny. We place the fuse module in each feature extraction layer to focus on the pedestrian information extracted by each layer. To solve the problem of complex model structure, we choose the residual network as the backbone network and the attention mechanism to extract person features without using pose point estimation or additional network assistance to reduce model complexity. We adjust the drop rate of the person classification layer to improve the model’s generalization ability. We estimate the performance of our method on three public datasets: Market-1501, DukeMTMC-reID, and CUHK03 (both detected and labeled) demonstrate the proposed method’s effectiveness and obtain highly competitive performance on the three datasets.

Dual-Branch Person Re-Identification Algorithm Based on Multi-Feature Representation

Person re-identification can identify specific pedestrians across cameras and solve the visual limitations of a single fixed camera scene. It achieves trajectory analysis of target pedestrians, facilitating case analysis by public security personnel. Person re-identification has become a challenging problem due to occlusion, blur, posture change, etc. The key to affecting the accuracy is whether sufficient information of pedestrians can be obtained. Most existing deep learning methods only consider a single category of characteristics, resulting in problems of information loss and single feature expression. This paper proposed a dual-branch person re-identification algorithm based on multi-scale and multi-granularity branches to obtain complete pedestrian information, called MSMG-Net. The multi-scale branch extracted features from different layers and fused them through the bidirectional cross-pyramid structure. It made up for the loss of information caused by using only high-level semantic features. The multi-granularity branch combined the convolutional neural network (CNN) and Transformer module based on improved self-attention mechanisms to obtain global information, and it performed horizontal segmentation of feature maps to obtain local information. It combined global and local features and solved the problem of single feature expression. The experimental results show that mAP/Rank-1 of MSMG-Net reached 88.6%/96.3% on the Market-1501 dataset and 80.1%/89.9% on the DukeMTMC-ReID dataset. Compared with many state-of-the-art methods, the performance of the proposed algorithm was improved significantly.