Pedestrian Features Research Articles

Information complementary attention-based multidimension feature learning for person re-identification

With the need for criminal investigation technology and the development of deep learning, the task of person re-identification has gradually become a research hotspot. Recently, various neural network-based person re-identification technologies designed by researchers have shown excellent results. However, most of the frameworks focus on complex structural design or redundant networks to guide model construction, which hugely increases the cost of train and application cost. In addition, the correlation between the channel information and spatial information on the pedestrian feature map is also relatively lacking. Therefore, we design a lightweight attention module to address the lack of correlation question response. The proposed module sequentially extracts person images’ channel and spatial features and effectively associates the two kinds of information through sequential connections. The proposed attention module has a simple structure, and the parameter increase in the backbone network is tiny. We place the fuse module in each feature extraction layer to focus on the pedestrian information extracted by each layer. To solve the problem of complex model structure, we choose the residual network as the backbone network and the attention mechanism to extract person features without using pose point estimation or additional network assistance to reduce model complexity. We adjust the drop rate of the person classification layer to improve the model’s generalization ability. We estimate the performance of our method on three public datasets: Market-1501, DukeMTMC-reID, and CUHK03 (both detected and labeled) demonstrate the proposed method’s effectiveness and obtain highly competitive performance on the three datasets.

Multi-Attention-Guided Cascading Network for End-to-End Person Search

The key procedure is to accurately identify pedestrians in complex scenes and effectively embed features from multiple vision cues. However, it is still a limitation to coordinate two tasks in the unified framework, thus leading to high computational overhead and unsatisfactory search performance. Furthermore, most methods do not take significant clues and key features of pedestrians into consideration. To remedy these issues, we introduce a novel method named Multi-Attention-Guided Cascading Network (MGCN) in this paper. Specifically, we obtain the trusted bounding box through the detection header as the label information for post-process. Based on the end-to-end network, we demonstrate the advantages of jointly learning to construct the bounding box and attention module by maximizing the complementary information from different attention modules, which can achieve optimized person search performance. Meanwhile, by imposing an aligning module on re-id feature extracted network to locate visual clues with semantic information, which can restrain redundant background information. Extensive experimental results for the two benchmark person search datasets are provided to demonstrate that the proposed MGCN markedly outperforms the state-of-the-art baselines.

CAPNet: Context and Attribute Perception for Pedestrian Detection

With a focus on practical applications in the real world, a number of challenges impede the progress of pedestrian detection. Scale variance, cluttered backgrounds and ambiguous pedestrian features are the main culprits of detection failures. According to existing studies, consistent feature fusion, semantic context mining and inherent pedestrian attributes seem to be feasible solutions. In this paper, to tackle the prevalent problems of pedestrian detection, we propose an anchor-free pedestrian detector, named context and attribute perception (CAPNet). In particular, we first generate features with consistent well-defined semantics and local details by introducing a feature extraction module with a multi-stage and parallel-stream structure. Then, a global feature mining and aggregation (GFMA) network is proposed to implicitly reconfigure, reassign and aggregate features so as to suppress irrelevant features in the background. At last, in order to bring more heuristic rules to the network, we improve the detection head with an attribute-guided multiple receptive field (AMRF) module, leveraging the pedestrian shape as an attribute to guide learning. Experimental results demonstrate that introducing the context and attribute perception greatly facilitates detection. As a result, CAPNet achieves new state-of-the-art performance on Caltech and CityPersons datasets.

Effect of Spring-Mass-Damper Pedestrian Models on the Performance of Low-Frequency or Lightweight Glazed Floors

For structural design purposes, human-induced loads on pedestrian systems can be described by several simplified (i.e., deterministic equivalent-force models) or more complex computational approaches. Among others, the Spring-Mass-Damper (SMD), Single Degree of Freedom (SDOF) model has been elaborated by several researchers to describe single pedestrians (or groups) in the form of equivalent body mass m, spring stiffness k and damping coefficient c. For all these literature SMD formulations, it is proved that the biodynamic features of walking pedestrians can be realistically reproduced, with high computational efficiency for vibration serviceability assessment of those pedestrian systems mostly sensitive to human-induced loads (i.e., with vibration frequency f1 < 8 Hz). Besides, the same SMD proposals are characterized by mostly different theoretical and experimental assumptions for calibration. On the practical side, strongly different SMD input parameters can thus be obtained for a given pedestrian. This paper focuses on a selection of literature on SMD models, especially on their dynamic effects on different structural floor systems. Four different floors are explored (F#1 and F#2 made of concrete, F#3 and F#4 of glass), with high- or low-frequency, and/or high- (>1/130th) or low- (1/4th) mass ratio, compared to the occupant. Normal walking scenarios with frequency in the range fp = 1.5–2 Hz are taken into account for a total of 100 dynamic simulations. The quantitative comparison of typical structural performance indicators for vibration serviceability assessment (i.e., acceleration peak, RMS, CREST) shows significant sensitivity to input SMD assumptions. Most importantly, the sensitivity of structural behaviours is observed for low-frequency systems, as expected, but also for low-mass structures, which (as in the case of glazed floor solutions) can be characterized by the use of lightweight modular units with relatively high vibration frequency. As such, major attention can be required for their vibrational analysis and assessment.

Anchor-free infrared pedestrian detection based on cross-scale feature fusion and hierarchical attention mechanism

An anchor-free infrared pedestrian detection algorithm is proposed to address the problems of multiscale, partial occlusion and environmental interference in infrared images, which make it difficult for traditional algorithms to perform accurate detection. First, a cross-scale feature fusion module is devised to improve the detection performance of multiscale and partially occluded objects. This module fuses the global features of different residual layers and multiscale local region features, thereby improving the multiscale feature fusion capability of the model while expanding the feature receptive field of the backbone network. Second, the object texture features of infrared images are weak owing to the imaging mechanism, which affects the accuracy of detection model. Thus, we construct a hierarchical attention mapping module to enhance the significance of pedestrian features in complex environments while suppressing the background information. Moreover, to improve the detection performance further, a novel dual-branch head detector is designed to decouple the input features from the channel dimension, which can be used for classification and location. Finally, by introducing the anchor-free concept into the network prediction mechanism, the prediction box can learn ground truth regression parameters by itself, thereby simplifying the network structure and improving the model generalization ability. The experimental results demonstrate that the proposed algorithm can accurately detect multiscale infrared pedestrian objects in complex environments and exhibits better objective evaluation indices compared to other pedestrian detection algorithms. The average precision and recall reached 98.78% and 98.67%, respectively.

Pedestrian multiple-object tracking based on FairMOT and circle loss

Multi-object Tracking is an important issue that has been widely investigated in computer vision. However, in practical applications, moving targets are often occluded due to complex changes in the background, which leads to frequent pedestrian ID switches in multi-object tracking. To solve the problem, we present a multi-object tracking algorithm based on FairMOT and Circle Loss. In this paper, HRNet is adopted as the baseline. Then, Polarized Self-Attention is added to HRNet-w32 to obtain weights of helpful information based on its modeling advantages. Moreover, the re-identification branch is optimized, and the Circle Loss is selected as the loss function to acquire more discriminative pedestrian features and to distinguish different pedestrians. The method proposed is assessed on the public MOT17 datasets. The experimental results show that the MOTA score achieves 69.5%, IDF1 reaches 70.0%, and the number of ID switches (IDs) decreases 636 times compared to the TraDes algorithm.

Correlation-and-Correction Fusion Attention Network for Occluded Pedestrian Detection

As a significant issue in computer vision, pedestrian detection has achieved certain achievements with the support of deep learning. However, pedestrian detection in congested scenes still encounters the challenging problem of feature loss and obfuscation. To address the issue, we propose a pedestrian detection network based on a correlation-and-correction fusion attention mechanism. First, a multimask correction attention module is proposed, which generates visible part masks of pedestrians, enhancing the visible region’s features and correcting the false one. Besides, the module preserves the features of multiclass pedestrians by generating multiple masks. Then, we fuse a correlation channel attention module to enhance the correlation of various pedestrians’ body features. Next, we studied three fusion methods of correlation and correction attention mechanisms and found that the serial connection of “correlation first and correction behind” works best. Finally, we extend our method to multiclass pedestrian detection in congested scenes. Experimental results on the CityPersons, Caltech, and CrowdHuman datasets demonstrate the effectiveness of our method. On the CityPersons dataset where more than 70% of pedestrians are occluded, our method outperforms the baseline method by 1.12% on the heavy occlusion subset and surpasses many outstanding methods.

A differentiated decision‐making algorithm for automated vehicles based on pedestrian feature estimation

AbstractOne critical difficulty to high‐level automated driving is the decision‐making process of automated vehicles in complicated traffic environments, especially in situations mixed of pedestrians and vehicles. This paper proposes a differentiated decision‐making algorithm to promote passing capability and efficiency in mixed traffic conditions. First, the behavioural characteristic of pedestrians, denoted as the pedestrian feature index, is estimated by a multi‐layer perception module input with quantitative analysis of pedestrian action. Based on estimation results, the decision algorithm merges pedestrian feature index into intelligent driver model and adjusts corresponding parameters, which used to be unchangeable so that the ego‐vehicle can make differential decisions according to various pedestrian features. Validation on the PIE dataset shows that the accuracy of pedestrian feature estimation is ensured. A simulation scenario is established utilizing cellular automata, and the results indicate that the proposed decision‐making algorithm can greatly improve passing efficiency under safety and manoeuvrability prerequisite.

A Small Target Pedestrian Detection Model Based on Autonomous Driving

Since small-target pedestrians account for a small proportion of pixels in images and lack texture features, the feature information of small-target pedestrians is often ignored in the feature extraction process, leading to reduced accuracy and poor robustness. To improve the accuracy of small-target pedestrian detection and the anti-interference ability of the model, a small-target pedestrian detection model that fuses residual networks and feature pyramids is proposed. First, a residual block with a discard layer is constructed to replace the standard residual block in the residual network structure to reduce the complexity of the model computation process and solve the problems of gradient disappearance and explosion in the deep network. Then, feature selection and feature alignment modules are added to the lateral connection part of the feature pyramid to enhance important pedestrian features in the input image, and the multiscale feature fusion capability of the model is enhanced for small-target pedestrians, thereby improving the detection accuracy of small-target pedestrians and solving the problems of feature misalignment and ignored multiscale features in the feature pyramid network. Finally, a cascaded autofocus query module is proposed to increase the inference speed of the feature pyramid network through focusing and querying, thus improving the performance and efficiency of small-target pedestrian detection. The experimental results show that the proposed model achieves better detection results than previous models.

Research on Pedestrian Detection Based on the Multi-Scale and Feature-Enhancement Model

Pedestrian detection represents one of the critical tasks of computer vision; however, detecting pedestrians can be compromised by problems such as the various scale of pedestrian features and cluttered background, which can easily cause a loss of accuracy. Therefore, we propose a pedestrian detection method based on the FCOS network. Firstly, we designed a feature enhancement module to ensure that effective high-level semantics are obtained while preserving the detailed features of pedestrians. Secondly, we defined a key-center region judgment to reduce the interference of background information on pedestrian feature extraction. By testing on the Caltech pedestrian dataset, the AP value is improved from 87.36% to 94.16%. The results of the comparison experiment illustrate that the model proposed in this paper can significantly increase the accuracy.

Lightweight Pedestrian Detection Based on Feature Multiplexed Residual Network

As an important part of autonomous driving intelligence perception, pedestrian detection has high requirements for parameter size, real-time, and model performance. Firstly, a novel multiplexed connection residual block is proposed to construct the lightweight network for improving the ability to extract pedestrian features. Secondly, the lightweight scalable attention module is investigated to expand the local perceptual field of the model based on dilated convolution that can maintain the most important feature channels. Finally, we verify the proposed model on the Caltech pedestrian dataset and BDD 100 K datasets. The results show that the proposed method is superior to existing lightweight pedestrian detection methods in terms of model size and detection performance.

IVP-YOLOv5: an intelligent vehicle-pedestrian detection method based on YOLOv5s

Computer vision is now vital in intelligent vehicle environment perception systems. However, real-time small-scale pedestrian detection in intelligent vehicle environment perception systems is still needs to be improved. This paper proposes an intelligent vehicle-pedestrian detection method based on YOLOv5s, named IVP-YOLOv5, to use in vehicle environment perception systems. Based on the network structure of YOLOv5s, we replaced BottleNeck CSP with Ghost-Bottleneck to reduce the complexity of processing feature maps while maintaining good detection performance. To reduce the error between the ground truth box and the predicted box, we apply Alpha-IoU as the bounding box loss function, improving pedestrian detection accuracy and robustness. We introduce the slicing-aided hyper inference (SAHI) strategy, which enables the lightweight backbone network to capture more detailed features of pedestrians by enlarging image pixels. Experiments on the BDD100 K dataset show that the proposed IVP-YOLOv5 achieves 67.1% AP and 18.5% APs of pedestrian detection, and the GFLOPs and the number of parameters are only 10.5 and 4.9M.

Pedestrian Re-Identification Algorithm Based on Multivariate Manifold Metric-Anti-Noise Manifold Space Learning

Deep learning is widely used in pedestrian re-identification, but pedestrian images are susceptible to changes in illumination, environmental noise, and other factors. These factors have affected the in-depth promotion of this type of method. First, the existing models cannot effectively extract the structural information of low-light pedestrian samples. Second, environmental noise information will cause the model to fail to extract stable and reliable feature information. In this paper, to solve the problems as mentioned above, we propose a multi-group manifold metric learning model. Our method can obtain manifold conditions under different lighting conditions. It can retain the inherent structure of low-dimensional spatial data in high-dimensional space, and realize the extraction of features of pedestrians in low-light conditions. In addition, we proposed an anti-noise manifold space learning method and designed a manifold module network structure. The manifold modules are stacked layer by layer to construct a manifold convolutional neural network further. Our method can extract robust pedestrian image features and reduce noise disturbance to the model. We use noise and low-light images as the main experimental objects. The experimental results show that our proposed method can achieve the most advanced performance in multiple ReID data sets.

Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection

Multispectral pedestrian detection is a technology designed to detect and locate pedestrians in Color and Thermal images, which has been widely used in automatic driving, video surveillance, etc. So far most available multispectral pedestrian detection algorithms only achieved limited success in pedestrian detection because of the lacking take into account the confusion of pedestrian information and background noise in Color and Thermal images. Here we propose a multispectral pedestrian detection algorithm, which mainly consists of a cascaded information enhancement module and a cross-modal attention feature fusion module. On the one hand, the cascaded information enhancement module adopts the channel and spatial attention mechanism to perform attention weighting on the features fused by the cascaded feature fusion block. Moreover, it multiplies the single-modal features with the attention weight element by element to enhance the pedestrian features in the single-modal and thus suppress the interference from the background. On the other hand, the cross-modal attention feature fusion module mines the features of both Color and Thermal modalities to complement each other, then the global features are constructed by adding the cross-modal complemented features element by element, which are attentionally weighted to achieve the effective fusion of the two modal features. Finally, the fused features are input into the detection head to detect and locate pedestrians. Extensive experiments have been performed on two improved versions of annotations (sanitized annotations and paired annotations) of the public dataset KAIST. The experimental results show that our method demonstrates a lower pedestrian miss rate and more accurate pedestrian detection boxes compared to the comparison method. Additionally, the ablation experiment also proved the effectiveness of each module designed in this paper.

A Double Stream Person Re-Identification Method Based on Attention Mechanism and Multi-Scale Feature Fusion

Person re-identification is the key technology for designing a intelligent video surveillance system to retrieve and recongise a specific person in a series of videos. It mainly plays a role in multiple non-overlapping camera monitoring environments to determine whether the target person of interest that has appeared under a camera appears again under others. However, the image data, in the real scene, taken by the surveillance camera may be occluded or blurred, which increased the difficulty of identifying the pedestrian posturn and then lead to dramatically decrease the accuracy of recognition. To solve the above problems, we propose a dualbranch multi-scale feature fusion network, which improves the expression ability of pedestrian features under partial occlusion by learning discriminative pedestrian features. The feature erasure mechanism is introduced into the branch network to effectively enhance the fine-grained feature learning ability of the network. By embedding the lightweight attention module into Resnet-50, the image sequence features of the channel dimension will be extracted, and the interference caused by the cluttered back ground information will be suppressed. In the training phase, the average pooling layer and the maximum pooling layer of different kernels and strides are utilized for different residual stages, and the mixed pooling strategy and mixed loss function of different kernels are designed. Finally, a novel hybrid loss function which combines triplet loss and ID loss is presented to train the network model. By comparison with the existing representative methods on Market1501, DukeMTMC reID and MTMS17 datasets, the experimental results show that the features extracted by the proposed method are more discriminative and with high recognition accuracy.

SKGACN: Social Knowledge-Guided Graph Attention Convolutional Network for Human Trajectory Prediction

Pedestrian trajectory prediction is crucial in driverless applications. To accurately predict the high-quality trajectory of pedestrians, it is necessary to consider the reasonable social interaction and the spatiotemporal relationships between pedestrians. Previous methods couldn’t accurately capture the social features of pedestrians in realistic congested situations and extract spatiotemporal interaction features with high computation. Therefore, in this paper, a novel prediction model is proposed, called the Social Knowledge-guided Graph Attention Convolutional Network (SKGACN), which aims to address the social interactions and the spatiotemporal relationships between pedestrians with low computational requirements. Specifically, the social knowledge-guided graph attention mechanism fully considers multiple information relative to pedestrians to capture their social interaction. For spatiotemporal interactions, an improved Temporal Convolution Network (TCN) model is use as it can parallelize the processing times to get as higher efficiency compared to traditional models. Compared to the state-of-the-art methods, we evaluate our proposed method after applying it on two public datasets (ETH and UCY). The experimental results show that our method performs better in terms of Average Displacement Error (ADE) and Final Displacement Error (FDE) metrics.

Occluded Person Re-Identification by Multi-Granularity Generation Adversarial Network

In order to address the problem that the detailed features of pedestrians are not prominent and the pedestrian pictures are obscured in unique environments in the process of person re-recognition, we propose a person re-recognition method with a multi-grain size generative adversarial network. Firstly, we use the generative adversarial network to recover the occluded pedestrian pictures; secondly, we improve the traditional multi-granularity network by adding an Efficient Channel Attention for Deep Convolutional Neural Networks (ECA-Net) on the coarse-grained branch to focus on the feature information in the pedestrian pictures and use the High-Resolution Net (HRNet) for pose estimation on the fine-grained branch to divide the pedestrian pictures into nine parts, to enhance the network’s learning of more detailed features of pedestrians, and thus improve the accuracy of pedestrian re-recognition learning, which in turn improves the accuracy of person re-identification.

Hybrid Contrastive Learning for Unsupervised Person Re-Identification

Unsupervised person re-identification (Re-ID) aims to learn discriminative features without human-annotated labels. Recently, contrastive learning provides a new prospect for unsupervised person Re-ID, and existing methods mainly constrain the feature similarity among easy sample pairs. However, the feature similarity among hard sample pairs is neglected, which causes suboptimal performance in unsupervised person Re-ID. In the paper, we propose a novel Hybrid Contrastive Model (HCM) to perform the identity-level contrastive learning and the image-level contrastive learning for unsupervised person Re-ID, which adequately explores the feature similarity among hard sample pairs. Specifically, for the identity-level contrastive learning, an identity-based memory is constructed to store pedestrian features. Accordingly, we define the dynamic contrast loss to identify identity information with dynamic factor for distinguishing hard/easy samples. As for the image-level contrastive learning, an image-based memory is established to store each image feature. We design the sample constraint loss to explore the similarity relationship between hard positive and negative sample pairs. Furthermore, we optimize the two contrastive learning processes in one unified framework to make use of their own advantages as so to constrain the feature distribution for extracting potential information. Extensive experiments prove that the proposed HCM distinctly outperforms the state-of-the-art methods.

Efficient Deep Learning Approach to Recognize Person Attributes by Using Hybrid Transformers for Surveillance Scenarios

Numerous deep perception technologies and methods are built on the foundation of pedestrian feature identification. It covers various fields, including autonomous driving, spying, and object tracking. A recent study area is the identification of personality traits that has attracted much interest in video surveillance. Identifying a person’s distinct areas is complex and plays an incredibly significant role. This paper presents a current method applied to networks of primary convolutional neurons to locate the area connected to the Person attribute. Using Individual Feature Identification, the features of a person, such as gender, age, fashion sense, and equipment, have received much attention in video surveillance analytics. This Article adopted a Conv-Attentional image transformer that broke down the most discriminating Attribute and region into multiple grades. The feed-forward system and conv-attention are the components of serial blocks, and parallel blocks have two attention-focused tactics: direct cross-layer attention and feature interpolation. It also provides a flexible Attribute Localization Module (ALM) to learn the regional aspects of each Attribute are considered at several levels, and the most discriminating areas are selected adaptively. We draw the conclusion that hybrid transformers outperform pure transformers in this instance. The extensive experimental results indicate that the proposed hybrid technique achieves higher results than the current strategies on four unique private characteristic datasets, i.e., RapV2, RapV1, PETA, and PA100K.

Occluded Person Re-Identification With Pose Estimation Correction and Feature Reconstruction

In the real-world surveillance system application, the accuracy of person re-identification (Re-ID) still suffers from occlusion. Occluded Re-ID aims to retrieve occluded person images from multiple cameras that do not overlap. To address this issue, we present a multi-branch feature enhancement network in this paper. It has the pose correction module (PCM), the feature reconstruction module (FRM), and the part align module (PAM). PCM builds the key-points confidence corrected mechanism of the self-adaptive weights learning before the part matching stage to strengthen part-level features in the non-occluded region and weaken ones in the occluded region. FRM reconstructs the feature distribution of inter-domain and class-domain using the maximum mean discrepancy to generalize the global features. The proposed feature separation mechanism enables the two branches to attend to distinct pedestrian feature. To increase the power of representation, PAM combines part-level and global features in an outer product. The experimental results on three holistic datasets, two partial datasets, and two occluded datasets demonstrate that our method is superior. It significantly outperforms state-of-the-art by 5.7% and 4.4% rank-1 scores on Occluded-Duke and Occluded-REID, respectively.