High-quality Density Maps Research Articles

Tides in clouds: control of star formation by long-range gravitational force

ABSTRACT Gravity drives the collapse of molecular clouds through which stars form, yet the exact role of gravity in cloud collapse remains a complex issue. Studies point to a picture where star formation occurs in clusters. In a typical, pc-sized cluster-forming region, the collapse is hierarchical, and the stars should be born from regions of even smaller sizes (${\approx} 0.1\,\rm pc$). The origin of this spatial arrangement remains under investigation. Based on a high-quality surface density map towards the Perseus region, we construct a 3D density structure, compute the gravitational potential, and derive eigenvalues of the tidal tensor (λmin, λmid, λmax, λmin < λmid < λmax), analyse the behaviour of gravity at every location, and reveal its multiple roles in cloud evolution. We find that fragmentation is limited to several isolated, high-density ‘islands’. Surrounding them, is a vast amount of the gas ($75~{{ \rm per\ cent}}$ of the mass, $95~{{ \rm per\ cent}}$ of the volume) that stays under the influence of extensive tides where fragmentation is suppressed. This gas will be transported towards these regions to fuel star formation. The spatial arrangement of regions under different tides explains the hierarchical and localized pattern of star formation inferred from the observations. Tides were first recognized by Newton, yet this is the first time its dominance in cloud evolution has been revealed. We expect this link between cloud density structure and role gravity to be strengthened by future studies, resulting in a clear view of the star formation process.

Counting in congested crowd scenes with hierarchical scale-aware encoder–decoder network

As an indispensable component of intelligent monitoring systems, crowd counting plays a crucial role in many fields, particularly crowd management and control during the COVID-19 pandemic. Despite the promising achievements of many methods, crowd scale variations and noise interference in congested crowd scenes remain urgent problems to be solved. In this paper, we propose a novel Hierarchical Scale-aware Encoder–Decoder Network (HSED-Net) for single-image crowd counting to handle scale variations and noise interference, thereby generating high-quality density maps. The HSED-Net is designed as an encoder–decoder architecture, which contains two core networks: Scale-Aware Encoding network (SAEnet) and Multi-path Aggregation Decoding network (MADnet). The SAEnet focuses on extracting rich multi-scale crowd features, which employs cascaded scale-aware encoding branches to collaboratively obtain high-resolution feature representations. During the encoding phase, two adaptive weight generators are proposed to filter the crowd features from different dimensions to resist the interference of noise. Instead of fusing the multi-scale and multi-level features indiscriminately, the MADnet adopts a multi-path adaptive fusion strategy and selectively emphasizes more appropriate features through the spatial and channel guidance modules, further improving the quality of density maps and the robustness of network. Extensive experiments on four challenging datasets have strongly demonstrated the superiority of our HSED-Net.

AMSA-CAFF Net: Counting and high-quality density map estimation from X-ray images of electronic components

Accurately estimating object numbers plays a crucial role in industrial applications. However, object counting in computer vision poses a formidable challenge, particularly when dealing with images containing tiny, similar, and stacked objects. Therefore, this paper proposes an encoder–decoder-based convolutional neural network named AMSA-CAFF Net to accurately count and regress high-quality density maps from X-ray images of highly dense microscopic components. AMSA-CAFF Net has three major components: an adaptive multi-scale feature aggregation (AMSA) module for encoding adaptive multi-scale features in the encoder, a channel-wise adaptive feature fusion (CAFF) module to ensure smooth feature fusion, and a continuous feature enhancement mechanism to promote information integration in the decoder. In addition, there is still a lack of datasets dedicated to industrial object counting tasks. To address the limitations associated with the existing datasets, we construct a novel and large-scale object counting dataset named X-ray-based industrial electronic component counting dataset (XRAY-IECCD), consisting of 1,460 X-ray images of 10 types of components, resulting in a total of 2,915,126 objects annotated with points. To the best of our knowledge, the XRAY-IECCD is the first dataset of tiny, dense, multi-class object counts with a substantial number of annotations. We evaluate the performance of AMSA-CAFF Net on three datasets: the XRAY-IECCD, crowd counting benchmark ShanghaiTech dataset, and object counting benchmark CARPK dataset. The proposed framework outperforms the state-of-the-art approaches over the proposed XRAY-IECCD and CARPK datasets, highlighting its effectiveness. Furthermore, we achieve competitive results on the ShanghaiTech dataset, which demonstrates the generalization capability of our method.

Domain-Adaptive Crowd Counting via High-Quality Image Translation and Density Reconstruction.

Recently, crowd counting using supervised learning achieves a remarkable improvement. Nevertheless, most counters rely on a large amount of manually labeled data. With the release of synthetic crowd data, a potential alternative is transferring knowledge from them to real data without any manual label. However, there is no method to effectively suppress domain gaps and output elaborate density maps during the transferring. To remedy the above problems, this article proposes a domain-adaptive crowd counting (DACC) framework, which consists of a high-quality image translation and density map reconstruction. To be specific, the former focuses on translating synthetic data to realistic images, which prompts the translation quality by segregating domain-shared/independent features and designing content-aware consistency loss. The latter aims at generating pseudo labels on real scenes to improve the prediction quality. Next, we retrain a final counter using these pseudo labels. Adaptation experiments on six real-world datasets demonstrate that the proposed method outperforms the state-of-the-art methods.

Counting with Self-Weighted Multi-Scale Fusion Networks

Because of the large-scale variation, counting in scenes of different densities is an extremely difficult task. In this paper, based on the attention mechanism, we propose a new self-weighted multi-scale fusion network structure named SMFNet to solve the problem of multi-scale changes and can significantly improve the effect of crowd counting in monitoring scene. The proposed SMFNet uses VGG as the backbone network to extract multi-scale features, uses a SMFNet as the neck to fuse multiple-scale features, and uses the atrous spatial pyramid pooling (ASPP) network and ordinary convolution as the head to generate both the attention map and the density map. The attention map highlighting crowd regions in the image contributes to a high-quality density map, and the density map records the crowd distribution. The number of crowd in the image can be obtained by summing the pixel values of the density map. We conduct experiments on three crowd counting datasets and one vehicle counting dataset to show that our proposed SMFNet can improve the state-of-the-art counting methods.

Dilated high-resolution network driven RGB-T multi-modal crowd counting

Crowd counting aims to estimate the number of pedestrians in a scene. However, the problems of insufficient illumination and large-scale variation affect the accuracy of crowd counting. In this paper, a dilated high-resolution network (DHRNet) driven RGB-T multi-modal crowd counting model is proposed to address the above problems. In terms of the importance of RGB and thermal modalities, a thermal-main and RGB-auxiliary strategy is chosen instead of treating them equally as in previous works. In terms of the fusion of RGB and thermal modalities, a cross-modal fusion module is designed and embedded in the input feature level of DHRNet. In terms of DHRNet output feature utilization, a multilayer perceptron regression head is proposed to predict high-quality density maps. The experimental results on public datasets show that our proposed network significantly outperforms state-of-the-art methods and reaches a new level of performance. The ablation studies verify the effectiveness of the thermal-main and RGB-auxiliary strategy and the proposed modules.

Crowd counting via Localization Guided Transformer

The rapidly growing demands on real-world crowd security and commercial applications have drawn widespread attentions to crowd counting, a computer vision task that aims to count all persons that appear in a given image. Recent state-of-the-art crowd counting methods commonly follow the density map regression paradigm, where a density map is estimated from the given image and summed up as the total count. Despite achieving impressive progress, these methods are still significantly challenged by complicated scenarios with severe scale variations of persons and cluttered backgrounds. Considering that localization-based counting methods, though less accurate, are able to learn more discriminative representation of persons through locating their positions, we propose a novel Localization Guided Transformer (LGT) framework in this work. The LGT aims to use the knowledge learned from a leading localization-based method to more accurately guide the estimation on density maps for crowd counting. Specifically, our framework first exploits a point-based model with two output heads, i.e., regression head and classification head, to simultaneously predict the head point proposals and point confidence respectively. Then, an intermediate multi-scale feature map is extracted from the shared backbone network and actively fused with the point location information. Afterwards, the fused features are fed into a Transformer module to explore patch-wise interactions via the self-attention mechanism, yielding a more discriminative representation for high-quality density map estimation. Extensive experiments and comparisons with state-of-the-art methods show the effectiveness of our proposed framework.

Hierarchical feature aggregation network with semantic attention for counting large‐scale crowd

The purpose of crowd counting is to estimate the number of people in an image. Due to the unconstrained imaging conditions, the scale variation and background occlusion in the images make it still challenging to achieve counting accurately. To tackle the two key challenges, we design a Hierarchical Feature Aggregation Network (HFANet) for accurate crowd counting in complex scenarios. The proposed method can extract multiple features at different levels and then aggregate them hierarchically to generate a high-quality density map. To highlight the crowd regions effectively from the cluttered background, we propose the Semantic Attention module to preserve the useful feature information through the attention mechanism for the low-level features extracted by VGG-16. Meanwhile, we employ convolutional kernels of different sizes to extract multi-scale features and global average pooling operations to preserve contextual information. Furthermore, we design the Feature Aggregation module to integrate the extracted multiple features through a progressive approach, which aims to take full advantage of the complementary properties between low-level and high-level features for efficient features aggregation. Finally, we evaluate the performance of the proposed HFANet on four challenging datasets. Extensive experimental results demonstrate that our proposed approach has better performance compared with most state-of-the-art approaches.

CACrowdGAN: Cascaded Attentional Generative Adversarial Network for Crowd Counting

Crowd counting is a valuable technology for extremely dense scenes in the transportation. Existing methods generally have higher-order inconsistencies between ground truth density maps and generated density maps. To address this issue, we incorporate an attentional discriminator to take charge of checking the density map between the generator and the ground truth. Thus, a Cascaded Attentional Generative Adversarial Network (CACrowdGAN) is proposed that enables the attentional-driven discriminator to distinguish implausible density maps and simultaneously to guide the generator to deliver fine-grained high quality density maps. The proposed CACrowdGAN consists of two components: an attentional generator and a cascaded attentional discriminator. The attentional generator has an attention module and a density module. The attention module is developed for the generator to focus on the crowd regions of the input images, while the density module is used to provide the attentional input of the discriminator. In addition, a cascaded attentional discriminator is proposed to synthesize attentional-driven fine-grained details at different crowd regions of the input image and compute a per-pixel fine-grained loss for training generator. The proposed CACrowdGAN achieves the state-of-the-art performance on five popular crowd counting datasets (ShanghaiTech, WorldEXPO’10, UCSD, UCF_CC_50 and UCF_QNRF), which demonstrates the effectiveness and robustness of the proposed approach in the complex scenes.

Semantic-refined spatial pyramid network for crowd counting

• We design a novel semantic-refined spatial pyramid network (SSPNet). • The SSPNet can benefit the fusing of multi-scale information. • The SSPNet can enhance the representation ability of deep semantic features. • The multi-scale feature will be adaptively integrated by our method. • The texture features are obtained to enrich the multi-scale features. In this paper, we propose a novel encoder-decoder model called Semantic-refined Spatial Pyramid Network (SSPNet) for generating high-quality density maps, which aims to build a scale-aware counting network to estimate the number of crowds accurately. The SSPNet consists of the front-end based on VGG-16, spatial pyramid multi-scale module (SPMM), and semantic enhancement module (SEM). First, a series of convolutional neural layers are utilized as the front-end to get deeper features without the extra computational cost. Moreover, the SPMM, which has a spatial pyramid structure with multiple receptive fields, is employed to capture multi-scale features. Furthermore, the SEM is designed to refine the features captured by SPMM, which uses deep semantic information to better integrate multi-scale features. Finally, the shallow texture information is adopted to compensate for the detail of the feature map to enhance the quality of the density map. Extensive experiments and comparisons on three challenge datasets, including ShanghaiTech Part_A & Part_B, UCF_CC_50, and UCF-QNRF, illustrate the superiority of our method.

Cascaded parallel crowd counting network with multi-resolution collaborative representation.

Accurately estimating the size and density distribution of a crowd from images is of great importance to public safety and crowd management during the COVID-19 pandemic, but it is very challenging as it is affected by many complex factors, including perspective distortion and background noise information. In this paper, we propose a novel multi-resolution collaborative representation framework called the cascaded parallel network (CP-Net), consisting of three parallel scale-specific branches connected in a cascading mode. In the framework, the three cascaded multi-resolution branches efficiently capture multi-scale features through their specific receptive fields. Additionally, multi-level feature fusion and information filtering are performed continuously on each branch to resist noise interference and perspective distortion. Moreover, we design an information exchange module across independent branches to refine the features extracted by each specific branch and deal with perspective distortion by using complementary information of multiple resolutions. To further improve the robustness of the network to scale variance and generate high-quality density maps, we construct a multi-receptive field fusion module to aggregate multi-scale features more comprehensively. The performance of our proposed CP-Net is verified on the challenging counting datasets (UCF_CC_50, UCF-QNRF, Shanghai Tech A&B, and WorldExpo’10), and the experimental results demonstrate the superiority of the proposed method.

Multi-scale features fused network with multi-level supervised path for crowd counting

Many CNN-based methods which utilize the density map to regress the count number of crowd are introduced to solve the crowd counting problem lately. Due to the head scale variations caused by the perspective change and background noise, these methods cannot address these two problems well in highly crowded scenario. In order to solve these two problems, we introduce a multi-scale features fused network with multi-level supervised path to produce the high-quality density map in this paper. Our model utilizes the first 13 layers of VGG16 model as the backbone, the multi-level supervised path in our model employs the multi-level dilated convolution module (MLD) to supervise the whole network at multi-level, and generate the attention map for the density map, which is used to handle the scale variations. The other path is used to fuse multi-scale features to generate the density map with soft spatial-channel attention module (SSCA) which aims to produce a saliency weight map of same size. In the end, the final density map is captured by the feature map multiply the attention map. In addition, a new objective function is proposed to train our network. A large number of experimental results show that compared with other networks, our method achieves better experimental results on four challenging datasets (UCF_CC_50, ShanghaiTech, UCF-QRNF and WorldExpo’10 dataset).

DMPNet: densely connected multi-scale pyramid networks for crowd counting.

Crowd counting has been widely studied by deep learning in recent years. However, due to scale variation caused by perspective distortion, crowd counting is still a challenging task. In this paper, we propose a Densely Connected Multi-scale Pyramid Network (DMPNet) for count estimation and the generation of high-quality density maps. The key component of our network is the Multi-scale Pyramid Network (MPN), which can extract multi-scale features of the crowd effectively while keeping the resolution of the input feature map and the number of channels unchanged. To increase the information transfer between the network layer, we used dense connections to connect multiple MPNs. In addition, we also designed a novel loss function, which can help our model achieve better convergence. To evaluate our method, we conducted extensive experiments on three challenging benchmark crowd counting datasets. Experimental results show that compared with the state-of-the-art algorithms, DMPNet performs well in both parameters and results. The code is available at: https://github.com/lpfworld/DMPNet.

A Location-Enhanced and Multiscale-Friendly Crowd Detecting Approach for Tram

The automatic statistics of passengers distribution and movement in some public transportation scenes from online cameras video is significant to public security, passenger flow control, and guidance. However, it is hard to get the crowd counting results accurately in carriages or platforms due to the narrow space, complex background, severe perspective distortion, and scale variation. A new crowd count method called coordinate deformable net (CDNet) is proposed in this article. This method is comprised of four sections: the front end, coordinate attention module (CAM), deformable scale-aware module (DSM), and back end. The front end is a preliminary feature extractor with the structure of VGG-16 but does not contain the full connection layer. The back end is used for density map prediction with the construction of six stacked dilated convolution layers. The new add-in CAM module ensures that the model has more capabilities of resisting complex background noise and focusing on areas of interest that are crucial for generating high-quality density maps. In the new DSM module, multicolumn deformable convolution can extract multiscale features effectively and fuse them according to attention weights. The DSM module can improve the scale variation problem significantly to adapt different scale passengers’ detection and enhance the accuracy of head positioning. The experimental results demonstrate that the accuracy of CDNet is superior to other classical models. In addition, to protect personal privacy, the faces in our Tram dataset are blurred.

Multi-Scale Guided Attention Network for Crowd Counting

The CNN-based crowd counting method uses image pyramid and dense connection to fuse features to solve the problems of multiscale and information loss. However, these operations lead to information redundancy and confusion between crowd and background information. In this paper, we propose a multi-scale guided attention network (MGANet) to solve the above problems. Specifically, the multilayer features of the network are fused by a top-down approach to obtain multiscale information and context information. The attention mechanism is used to guide the acquired features of each layer in space and channel so that the network pays more attention to the crowd in the image, ignores irrelevant information, and further integrates to obtain the final high-quality density map. Besides, we propose a counting loss function combining SSIM Loss, MAE Loss, and MSE Loss to achieve effective network convergence. We experiment on four major datasets and obtain good results. The effectiveness of the network modules is proved by the corresponding ablation experiments. The source code is available at https://github.com/lpfworld/MGANet.

EDENet: Elaborate density estimation network for crowd counting

For the CNN-based density estimation approaches in the field of crowd counting, how to generate a high-quality density map with accurate counting performance and detailed spatial description is still an open question. In this paper, to tackle the aforementioned contradiction, we propose an end-to-end trainable architecture called Elaborate Density Estimation Network for Crowd Counting (EDENet), which can gradually generate high-quality density estimation maps based on distributed supervision. Specifically, EDENet is composed of Feature Extraction Network (FEN), Feature Fusion Network (FFN), Double-Head Network (DHN) and Adaptive Density Fusion Network (ADFN). The FEN adopts VGG as the backbone network and employs Spatial Adaptive Pooling (SAP) to extract coarse-grained features. The FFN can effectively fuse contextual information and localization information for enhancing the spatial description ability of fine-grained features. In the DHN, the Density Attention Module (DAM) can provide attention masks of foreground-background, thereby urging the Density Regression Module (DRM) to focus on the pixels around the head annotations to regress density maps with different resolutions. The ADFN constructed on the basis of the adaptive weighting mechanism can directly introduce coarse-grained density representation into high-resolution density maps to strengthen the commonality and dependency among density maps. Extensive experiments on four benchmark crowd datasets (the ShanghaiTech, the UCF-QNRF, the JHU-CRWORD++ and the NWPU-Crowd) indicate that EDENet can achieve state-of-the-art recognition performance and high robustness. Not only that, the density map with the highest Peak Signal to Noise Ratio (PSNR) can be considered to be of high quality.

Crowd counting method based on cross column fusion attention mechanism

Deep learning has made substantial progress in crowd density estimation, but there are still some problems in existing methods, such as large population density, background interference, and scale change, which makes it difficult to count people. To solve the above problems, we proposed a crowd counting method based on a cross column fusion attention mechanism. First, the first ten layers of VGG16 with good migration ability and feature extraction ability are used as the front-end network to preliminarily extract human head features. Then, a cross column fusion attention module is designed. In this module, feature maps are fused across columns to make the network contain richer deep and shallow features. At the same time, to alleviate the background interference, the attention mechanism is used to guide the network to focus on the head position in the picture, and different weights are assigned to different positions according to the attention score map, so as to highlight the crowd and weaken the background, and finally get a high-quality density map. In addition, a shallow convolution module is designed as another branch. The output feature map of the shallow convolution module and the output feature map of the attention module of cross column fusion are fused to solve the problem of scale change effectively. Finally, in the last layer of the network, the convolution layer of 1 × 1 is used to replace the full connection layer, and fewer network parameters are used to reduce the calculation and the population density map is regressed. The experimental results show that the mean absolute error and mean square error of the proposed algorithm are significantly reduced compared with the comparison algorithm.

Average up-sample network for crowd counting

The task of crowd counting is receiving increased attention recently, but it still faces many challenges, such as extremely dense scene, scale variation and background clutter. The quality of generated density map plays an important role in counting performance. In this paper, we propose an encoder-decoder structure network called Average Up-sample Convolution Neural Network (AU-CNN), for high-quality density map and accurate counting estimation. The encoder extracts the features of input image while the decoder gradually recovers the size of feature map to the original size of input image by developing a simple but effective average up-sample module. The average up-sample module takes the average of interpolation results from three different up-sample methods, without adding any other redundant parameters. Moreover, compared with most existing counting algorithm using only Euclidean loss, we use a combined loss function of Euclidean loss and count loss to optimize the network, which is demonstrated effective in performance improving. Experiments on the ShanghaiTech, UCF_CC_50, and UCF_QNRF demonstrate the great counting performance and robustness of our proposed method.

Metro passengers counting and density estimation via dilated-transposed fully convolutional neural network

Metro passenger counting and density estimation are crucial for traffic scheduling and risk prevention. Although deep learning has achieved great success in passenger counting, most existing methods ignore fundamental appearance information, leading to density maps of low quality. To address this problem, we propose a novel counting method called “dilated-transposed fully convolution neural network” (DT-CNN), which combines a feature extraction module (FEM) and a feature recovery module (FRM) to generate high-quality density maps and accurately estimate passenger counts in highly congested metro scenes. Specifically, the FEM is composed of a CNN, and a set of dilated convolutional layers extract 2D features relevant to scenes containing crowded human objects. Then, the resulting density map produced by the FEM is processed by the FRM to learn potential features, which is used to restore feature map pixels. The DT-CNN is end-to-end trainable and independent of the backbone fully convolutional network architecture. In addition, we introduce a new metro passenger counting dataset (Zhengzhou_MT++) that contains 396 images with 3,978 annotations. Extensive experiments conducted on self-built datasets and three representative crowd-counting datasets show the proposed method achieves superior performance relative to other state-of-the-art methods in terms of counting accuracy and density map quality. The Zhengzhou MT++ dataset is available at https://github.com/YellowChampagne/Zhengzhou_MT .

Convolutional Neural Network for Crowd Counting on Metro Platforms

Owing to the increased use of urban rail transit, the flow of passengers on metro platforms tends to increase sharply during peak periods. Monitoring passenger flow in such areas is important for security-related reasons. In this paper, in order to solve the problem of metro platform passenger flow detection, we propose a CNN (convolutional neural network)-based network called the MP (metro platform)-CNN to accurately count people on metro platforms. The proposed method is composed of three major components: a group of convolutional neural networks is used on the front end to extract image features, a multiscale feature extraction module is used to enhance multiscale features, and transposed convolution is used for upsampling to generate a high-quality density map. Currently, existing crowd-counting datasets do not adequately cover all of the challenging situations considered in this study. Therefore, we collected images from surveillance videos of a metro platform to form a dataset containing 627 images, with 9243 annotated heads. The results of the extensive experiments showed that our method performed well on the self-built dataset and the estimation error was minimum. Moreover, the proposed method could compete with other methods on four standard crowd-counting datasets.