Multiple Views Research Articles

TS-BEV: BEV object detection algorithm based on temporal-spatial feature fusion

In order to accurately identify occluding targets and infer the motion state of objects, we propose a Bird’s-Eye View Object Detection Network based on Temporal-Spatial feature fusion (TS-BEV), which replaces the previous multi-frame sampling method by using the cyclic propagation mode of historical frame instance information. We design a new Temporal-Spatial feature fusion attention module, which fully integrates temporal information and spatial features, and improves the inference and training speed. In response to realize multi-frame feature fusion across multiple scales and views, we propose an efficient Temporal-Spatial deformable aggregation module, which performs feature sampling and weighted summation from multiple feature maps of historical frames and current frames, and makes full use of the parallel computing capabilities of GPUs and AI chips to further improve efficiency. Furthermore, in order to solve the lack of global inference in the context of temporal-spatial fusion BEV features and the inability of instance features distributed in different locations to fully interact, we further design the BEV self-attention mechanism module to perform global operation of features, enhance global inference ability and fully interact with instance features. We have carried out extensive experimental experiments on the challenging BEV object detection nuScenes dataset, quantitative results show that our method achieves excellent performance of 61.5% mAP and 68.5% NDS in camera-only 3D object detection tasks, and qualitative results show that TS-BEV can effectively solve the problem of 3D object detection in complex traffic background with lack of light at night, with good robustness and scalability.

Three-view cotton flower counting through multi-object tracking and RGB-D imagery

Monitoring the number of cotton flowers can provide important information for breeders to assess the flowering time and the productivity of genotypes because flowering marks the transition from vegetative growth to reproductive growth and impacts the final yield. Traditional manual counting methods are time-consuming and impractical for large-scale fields. To count cotton flowers efficiently and accurately, a multi-view multi-object tracking approach was proposed by using both RGB and depth images collected by three RGB-D cameras fixed on a ground robotic platform. The tracking-by-detection algorithm was employed to track flowers from three views simultaneously and remove duplicated counting from single views. Specifically, an object detection model (YOLOv8) was trained to detect flowers in RGB images and a deep learning-based optical flow model Recurrent All-pairs Field Transforms (RAFT) was used to estimate motion between two adjacent frames. The intersection over union and distance costs were employed to associate flowers in the tracking algorithm. Additionally, tracked flowers were segmented in RGB images and the depth of each flower was obtained from the corresponding depth image. Those flowers tracked with known depth from two side views were then projected onto the middle image coordinate using camera calibration parameters. Finally, a constrained hierarchy clustering algorithm clustered all flowers in the middle image coordinate to remove duplicated counting from three views. The results showed that the mean average precision of trained YOLOv8x was 96.4%. The counting results of the developed method were highly correlated with those counted manually with a coefficient of determination of 0.92. Besides, the mean absolute percentage error of all 25 testing videos was 6.22%. The predicted cumulative flower number of Pima cotton flowers is higher than that of Acala Maxxa, which is consistent with what breeders have observed. Furthermore, the developed method can also obtain the flower number distributions of different genotypes without laborious manual counting in the field. Overall, the three-view approach provides an efficient and effective approach to count cotton flowers from multiple views. By collecting the video data continuously, this method is beneficial for breeders to dissect genetic mechanisms of flowering time with unprecedented spatial and temporal resolution, also providing a means to discern genetic differences in fecundity, the number of flowers that result in harvestable bolls. The code and datasets used in this paper can be accessed on GitHub: https://github.com/UGA-BSAIL/Multi-view_flower_counting.

Co-regularized optimal high-order graph embedding for multi-view clustering

Real-world applications frequently involve multiple data modalities in the same samples, which are regarded as multi-view data. Multi-view clustering has been studied extensively in recent years to demonstrate embedded heterogeneity. However, most existing methods emphasize low-order correlation in multiple views, whereas approaches that incorporate high-order correlation are limited by the equal view-specific significance problem or a trade-off between global and local consistency. In this paper, we propose a co-regularized optimal graph-based clustering method known as Co-MSE, which integrates the correlation of different orders. By integrating the first-order and second-order similarities, the local structure is preserved, while an optimized embedding representation for multi-view data is obtained simultaneously through co-regularization. We demonstrate that Co-MSE can aid in providing a more suitable embedding representation and further enable satisfactory clustering performance. Extensive experiments on real-world datasets confirm the effectiveness and advantages of the proposed method.

Up is down: some hypotheses on how to interpret perspective in MS I.33’s illustrations

The oldest known European fight book, Royal Armouries MS I.33 has puzzled scholars and sword fighting enthusiasts alike with its occasionally strange looking illustrations. This paper argues that some of them contain multiple view planes or perspectives ‘folded’ into one, as it also occurs elsewhere in contemporary Gothic manuscript painting. To the modern observer who has no first-hand experience of the type of sword fighting depicted, these specialised drawing conventions are difficult to identify and, consequently, to decode. By comparing ostensibly similar situations as they are illustrated in later fight books that employ different artistic means, this paper suggests a new way of looking at I.33’s illustrations that could be relevant beyond the examples discussed here.

Artificial Intelligence Measurement of Preoperative Radiographs in Adolescent Idiopathic Scoliosis Based on Multiple-View Semantic Segmentation.

Cross-sectional study. Imaging classification of adolescent idiopathic scoliosis (AIS) is directly related to the surgical strategy, but the artificial classification is complex and depends on doctors' experience. This study investigated deep learning-based automated classification methods (DL group) for AIS and validated the consistency of machine classification and manual classification (M group). A total of 506 cases (81 males and 425 females) and 1812 AIS full spine images in the anteroposterior (AP), lateral (LAT), left bending (LB) and right bending (RB) positions were retrospectively used for training. The mean age was 13.6 ± 1.8. The mean maximum Cobb angle was 46.8 ± 12.0. U-Net semantic segmentation neural network technology and deep learning methods were used to automatically segment and establish the alignment relationship between multiple views of the spine, and to extract spinal features such as the Cobb angle. The type of each test case was automatically calculated according to Lenke's rule. An additional 107 cases of adolescent idiopathic scoliosis imaging were prospectively used for testing. The consistency of the DL group and M group was compared. Automatic vertebral body segmentation and recognition, multi-view alignment of the spine and automatic Cobb angle measurement were implemented. Compare to the M group, the consistency of the DL group was significantly higher in 3 aspects: type of lateral convexity (0.989 vs 0.566), lumbar curvature modifier (0.932 vs 0.738), and sagittal plane modifier (0.987 vs 0.522). Deep learning enables automated Cobb angle measurement and automated Lenke classification of idiopathic scoliosis whole spine radiographs with higher consistency than manual measurement classification.

Quantitative Measurements in the Exhaust Flow of a Rotating Detonation Combustor Using Rainbow Schlieren Deflectometry

Abstract This study employs Rainbow Schlieren Deflectometry (RSD) to characterize the unsteady, supersonic/subsonic exhaust plume of a Rotating Detonation Combustor (RDC). Firstly, RSD images are analyzed to quantify the frequency and strength of flow oscillations and their relationship to the detonation wave. Secondly, a 3D tomographic algorithm is used to obtain the local 3D density field across the whole region of interest (ROI). The tomographic analysis relies upon wave rotation to infer projection data of the 3D exhaust plume at multiple view angles using a single RSD/camera system and was previously validated using phantom data from computational fluid dynamics analysis of an RDC. The annular RDC operated on methane and 2/3 O2 - 1/3 N2 oxidizer mixture is equipped with a converging nozzle to pressurize the combustion chamber. The product flow exiting the nozzle throat expands across an unoptimized conical aerospike attached to the center body of the RDC. RSD images provide a temporal resolution of 369 ns and spatial resolution of 100 ?m in a 6.4 high and 25.6 mm wide ROI of the exhaust plume. A rainbow filter is calibrated to convert hue in color schlieren images into deflection angle data. This data is used to characterize the unsteady flow oscillations that show excellent agreement with PCB pressure probe measurements acquired inside the combustion chamber. Tomographic analysis yields a 3D local density field that shows distinct features, consistent with published numerical simulations of the RDC exhaust plume.

Consistent and specific multi-view multi-label learning with correlation information

In multi-view multi-label (MVML) learning, each sample is represented by several heterogeneous distinct feature representations while associated with a set of class labels simultaneously. To achieve MVML learning, most of the existing methods contribute to the recovery of a consistent subspace, i.e., a shared feature representation, among multiple views. Nevertheless, each view has its inherent specific properties used in the discrimination process of labels. These methods lose sight in the specific information exploitation, and therefore are easily trapped in a sub-optimal result. In this study, we present an optimization framework CSVL to solve the learning problem. The main technical contribution in CSVL is a formulation for MVML learning while consistent subspace across views, specific subspace for each view, and the correlations among labels are taken into account. Specifically, consistent subspace is recovered by imposing a low-rank constraint among multiple views, and specific subspace of each view is extra generated with Frobenius norm. To further improve model generalization capability, we preserve both feature manifolds from multiple views and label correlations from multiple labels. Extensive experiments on 7 benchmark datasets show that our proposal CSVL has the advantages in MVML learning.

Efficient Multi-View Clustering via Unified and Discrete Bipartite Graph Learning.

Although previous graph-based multi-view clustering (MVC) algorithms have gained significant progress, most of them are still faced with three limitations. First, they often suffer from high computational complexity, which restricts their applications in large-scale scenarios. Second, they usually perform graph learning either at the single-view level or at the view-consensus level, but often neglect the possibility of the joint learning of single-view and consensus graphs. Third, many of them rely on the k -means for discretization of the spectral embeddings, which lack the ability to directly learn the graph with discrete cluster structure. In light of this, this article presents an efficient MVC approach via u nified and d iscrete b ipartite g raph l earning (UDBGL). Specifically, the anchor-based subspace learning is incorporated to learn the view-specific bipartite graphs from multiple views, upon which the bipartite graph fusion is leveraged to learn a view-consensus bipartite graph with adaptive weight learning. Furthermore, the Laplacian rank constraint is imposed to ensure that the fused bipartite graph has discrete cluster structures (with a specific number of connected components). By simultaneously formulating the view-specific bipartite graph learning, the view-consensus bipartite graph learning, and the discrete cluster structure learning into a unified objective function, an efficient minimization algorithm is then designed to tackle this optimization problem and directly achieve a discrete clustering solution without requiring additional partitioning, which notably has linear time complexity in data size. Experiments on a variety of multi-view datasets demonstrate the robustness and efficiency of our UDBGL approach. The code is available at https://github.com/huangdonghere/UDBGL.

Percutaneous coronary intervention for bifurcation coronary lesions using optimised angiographic guidance: the 18th consensus document from the European Bifurcation Club.

The 2023 European Bifurcation Club (EBC) meeting took place in Warsaw in October, and the latest evidence for the use of intravascular ultrasound (IVUS) and optical coherence tomography (OCT) to optimise percutaneous coronary interventions (PCI) on coronary bifurcation lesions (CBLs) was a major focus. The topic generated deep discussions and general appraisal on the potential benefits of IVUS and OCT in PCI procedures. Nevertheless, despite an increasing recognition of IVUS and OCT capabilities and their recognised central role for guidance in complex CBL and left main PCI, it is expected that angiography will continue to be the primary guidance modality for CBL PCI, principally due to educational and economic barriers. Mindful of the restricted access/adoption of intracoronary imaging for CBL PCI, the EBC board decided to review and describe a series of tips and tricks which can help to optimise angiography-guided PCI for CBLs. The identified key points for achieving an optimal angiography-guided PCI include a thorough analysis of pre-PCI images (computed tomography angiography, multiple angiographic views, quantitative coronary angiography vessel estimation), a systematic application of the technical steps suggested for a given selected technique, an intraprocedural or post-PCI use of stent enhancement and a low threshold for bailout use of intravascular imaging.

Projective Incomplete Multi-View Clustering.

Due to the rapid development of multimedia technology and sensor technology, multi-view clustering (MVC) has become a research hotspot in machine learning, data mining, and other fields and has been developed significantly in the past decades. Compared with single-view clustering, MVC improves clustering performance by exploiting complementary and consistent information among different views. Such methods are all based on the assumption of complete views, which means that all the views of all the samples exist. It limits the application of MVC, because there are always missing views in practical situations. In recent years, many methods have been proposed to solve the incomplete MVC (IMVC) problem and a kind of popular method is based on matrix factorization (MF). However, such methods generally cannot deal with new samples and do not take into account the imbalance of information between different views. To address these two issues, we propose a new IMVC method, in which a novel and simple graph regularized projective consensus representation learning model is formulated for incomplete multi-view data clustering task. Compared with the existing methods, our method not only can obtain a set of projections to handle new samples but also can explore information of multiple views in a balanced way by learning the consensus representation in a unified low-dimensional subspace. In addition, a graph constraint is imposed on the consensus representation to mine the structural information inside the data. Experimental results on four datasets show that our method successfully accomplishes the IMVC task and obtain the best clustering performance most of the time. Our implementation is available at https://github.com/Dshijie/PIMVC.

Multi-stakeholder recommendation system through deep learning-based preference evaluation and aggregation model with multi-view information embedding

Learning the preferences of consumers, providers, and system stakeholders is a challenging problem in the Multi-Stakeholder Recommendation System (MSRS). Existing MSRS methods lack the ability to generate equitable recommendations and investigate implicit relationships between stakeholders and items. Hence, this study addresses this issue by proposing a multi-stakeholder preference learning-based recommendation model that exploits information from multiple views to evaluate stakeholders' preferences. The proposed model learns consumer preferences using users' ratings and reviews of an item, provider preferences with provider utility, and provider-item interaction. Furthermore, the proposed model learns the system-level preference of promoting long-tail items through the probabilistic evaluation of stakeholders' interest in popular and unpopular items. Finally, this study develops a multi-stakeholder, multi-view deep neural network model to aggregate stakeholders' preferences and deliver equitable recommendations. This work utilizes benchmark Movie Lens (ML) 25M, ML-100K, ML-1M, and TripAdvisor datasets to validate and compare the proposed model's performance with other baseline methods using standard evaluation metrics for each stakeholder. Examining the precision metrics, the proposed model attains the minimum enhancement of 7.91%, 18.24%, 10.72%, and 20.12% across the ML-25M, ML-100K, ML-1M, and TripAdvisor datasets. Further concerning the exposure, hit, and reach metrics, the model exhibits a substantial minimum improvement of 19.12%, 14.73%, 5.37%, and 28.46% over the ML-25M, ML-100K, ML-1M, and TripAdvisor datasets. Finally, the proposed model excels in promoting long-tail items and enhancing the cumulative utility gain of the stakeholders, surpassing the baseline methods.

Clean affinity matrix induced hyper-Laplacian regularization for unsupervised multi-view feature selection

Most previous unsupervised multi-view feature selection (UMFS) methods have achieved appealing performance by exploring the consistency among multiple views. However, they have the following shortcomings: (1) They often fail to consider the potential inconsistency that might be caused by view-specific characteristics from the perspective of sparsity. (2) The previously learned hyper-graph might be affected by noise, thereby reducing the quality of the generated graph. To tackle these issues, this paper proposes a clean affinity matrix induced hyper-Laplacian regularization (CAHR) method for UMFS. Firstly, the initial affinity matrix is decomposed into the consistent and inconsistent parts, then a novel diversity penalty term is introduced to enforce the sparsity of the inconsistent part across views, thereby making the consistent part be cleaner. Secondly, a unified affinity matrix is generated by fusing the consistent factors of the initial affinity matrix in a self-weighted manner, thereby considering the consistency of multi-view data. Based on the unified affinity matrix, a hyper-Laplacian matrix is further constructed, which can maintain high-order manifold structure of data. Finally, a loss function is designed to find the best mapping for feature selection. Comprehensive experiments demonstrate that the proposed method significantly outperforms several state-of-the-art UMFS methods.

RLclean: An unsupervised integrated data cleaning framework based on deep reinforcement learning

Data cleaning, a prerequisite to subsequent data analysis, has always been the focus of data science research. Datasets with errors can severely detract from the quality of downstream analytical results. Unfortunately, despite the proliferation of various data cleaning methods, it remains a time-consuming problem and frequently entails considerable labor expenses. In reality, errors are often heterogeneous and require different solutions. As a result, stand-alone methods often inadequate for addressing dirty data with multiple types of errors, while studies have demonstrated that combining such methods always require human intervention and the result remains unsatisfactory.In this paper, we propose an unsupervised integrated data cleaning framework, namely RLclean. Based on deep reinforcement learning, RLclean takes advantages of multiple data cleaning techniques, enabling it to effectively clean multiple types of errors and achieve satisfactory results. Additionally, it eliminates the need for costly human involvement, as the cleaning strategy is learned by data-driven, which further allows the framework to self-adapt to diverse domains. RLclean mainly consists of two parts: (i) an integrated error detection model that unites multiple techniques to detect different types of errors from multiple views; and (ii) an integrated data repair model that learns the optimal repair operations and repairs dirty data in an unsupervised manner. Extensive experiments on benchmark datasets have demonstrated the superiority of RLclean over state-of-the-art methods.

Large-scale multi-view subspace clustering via embedding space and partition matrix

Multi-view subspace clustering (MVSC) has attracted increasing attention because it can extract information from multiple views and explore the underlying structure. In general, most of the existing anchor strategies solve the problem of excessive complexity, but there is a loss of information in the process of affinity graph passing to spectral clustering. In addition, the noise in the original data leads to the learned anchor graph not representing the data features adequately. To solve the above problems, this paper proposes a Large-Scale Multi-View Subspace Clustering via Embedding Space and Partition Matrix(LMVSC-EPM) algorithm that preserves the distribution of the original data by embedding matrix mapping. The algorithm utilizes the centroid matrix and the clustering assignment matrix to derive the clustering results directly. Specifically, LMVSC-EPM employs embedding matrices to map the raw data into the embedding space and adaptively learns anchors using a view-sharing anchor strategy. Moreover, the non-negative orthogonal matrix is adopted to assign the results to the clustering assignment matrix, which avoids the loss of the affinity matrix in the passing process. Furthermore, an alternating minimization optimization method is designed in this paper to solve the optimization problem. Experimental results on seven underlying datasets demonstrate the efficiency and superiority of the proposed method.

Design of an iterative method for environmental-sustainable development: Integrating bioinspired computing techniques

The need for sustainable development has grown in response to global environmental, social, and economic challenges. Conventional computational methods frequently struggle to address the complex nature of the Sustainable Development Goals (SDGs), lacking the ability to balance global search with local optimization and failing to prioritize goals related to sustainability. To address these restrictions, this work introduces the Integrated Bioinspired Computing Model for Sustainable Development (IBCMSD). By combining Genetic Algorithms (GAs), Artificial Neural Networks (ANNs), and Ant Colony Optimization (ACO), a cohesive hybrid model is developed that improves exploration and exploitation, balance for increased efficiency, and solution quality. It is implemented on High-Performance Computing (HPC) clusters to ensure scalability and resilience when dealing with complicated optimization challenges. Furthermore, using a multidisciplinary co-design method completes the model with multiple views, increasing its relevance and applicability in real-world circumstances. IBCMSD makes a significant contribution to computational sustainability by leveraging bioinspired computing, potentially enabling informed decision-making and SDG accomplishment across multiple domains.

Multiview representation learning for identification of novel cancer genes and their causative biological mechanisms.

Tumorigenesis arises from the dysfunction of cancer genes, leading to uncontrolled cell proliferation through various mechanisms. Establishing a complete cancer gene catalogue will make precision oncology possible. Although existing methods based on graph neural networks (GNN) are effective in identifying cancer genes, they fall short in effectively integrating data from multiple views and interpreting predictive outcomes. To address these shortcomings, an interpretable representation learning framework IMVRL-GCN is proposed to capture both shared and specific representations from multiview data, offering significant insights into the identification of cancer genes. Experimental results demonstrate that IMVRL-GCN outperforms state-of-the-art cancer gene identification methods and several baselines. Furthermore, IMVRL-GCN is employed to identify a total of 74 high-confidence novel cancer genes, and multiview data analysis highlights the pivotal roles of shared, mutation-specific, and structure-specific representations in discriminating distinctive cancer genes. Exploration of the mechanisms behind their discriminative capabilities suggests that shared representations are strongly associated with gene functions, while mutation-specific and structure-specific representations are linked to mutagenic propensity and functional synergy, respectively. Finally, our in-depth analyses of these candidates suggest potential insights for individualized treatments: afatinib could counteract many mutation-driven risks, and targeting interactions with cancer gene SRC is a reasonable strategy to mitigate interaction-induced risks for NR3C1, RXRA, HNF4A, and SP1.

Diverse representation-guided graph learning for multi-view metric clustering

Multi-view graph clustering has garnered tremendous interest for its capability to effectively segregate data by harnessing information from multiple graphs representing distinct views. Despite the advances, conventional methods commonly construct similarity graphs straightway from raw features, leading to suboptimal outcomes due to noise or outliers. To address this, latent representation-based graph clustering has emerged. However, it often hypothesizes that multiple views share a fixed-dimensional coefficient matrix, potentially resulting in useful information loss and limited representation capabilities. Additionally, many methods exploit Euclidean distance as a similarity metric, which may inaccurately measure linear relationships between samples. To tackle these challenges, we develop a novel diverse representation-guided graph learning for multi-view metric clustering (DRGMMC). Concretely, raw sample matrix from each view is first projected into diverse latent spaces to capture comprehensive knowledge. Subsequently, a popular metric is leveraged to adaptively learn similarity graphs with linearity-aware based on attained coefficient matrices. Furthermore, a self-weighted fusion strategy and Laplacian rank constraint are introduced to output clustering results directly. Consequently, our model merges diverse representation learning, metric learning, consensus graph learning, and data clustering into a joint model, reinforcing each other for holistic optimization. Substantial experimental findings substantiate that DRGMMC outperforms most advanced graph clustering techniques.

Integrated sources model: A new space-learning model for heterogeneous multi-view data reduction, visualization, and clustering

In machine learning, multi-view data involve multiple distinct sets of attributes (“views”) for a common set of observations; when each view has the same attributes considered in different contexts, the data are said to contain multiple views of homogeneous format, which can be conceptualized as a tensor. In this article, we describe a novel approach for integrating multiple views of heterogeneous format into a common latent space using a workflow that involves non-negative matrix and tensor factorization (NMF/NTF). This approach, which we refer to as the integrated sources model (ISM), consists of two main steps: Embedding and analysis. In the embedding step, the views are transformed into matrices with common non-negative components. In the analysis step, the transformed views are combined into a tensor and decomposed using NTF. We also present a variant of ISM; the integrated latent sources model (ILSM), which offers significant advantages over ISM in terms of computational power and in cases where the views are highly unbalanced with regard to the number of attributes per view. Noteworthy, ISM can be extended to process multi-omic and multi-view datasets even in the presence of missing views. We provide a proof-of-concept analysis using five examples, including the UCI Digits (the University of California Irvine Pen-Based Recognition of Handwritten Digits) dataset, a public cell-type gene signatures dataset, and a multi-omic single-cell dataset. These examples demonstrate that, in most cases, multi-view clustering is better achieved with ISM or its variant ILSM than with other latent space approaches. We also show how the non-negativity and sparsity of the ISM model components enable straightforward interpretations, in contrast to other approaches that involve latent factors of mixed signs. Finally, we present potential applications to single-cell multi-omics and spatial mapping, including spatial imaging, spatial transcriptomics, and computational biology, which are currently under evaluation. ISM relies on state-of-the-art algorithms invoked through a simple workflow implemented in Python.

Dynamic Graph Guided Progressive Partial View-Aligned Clustering.

In recent years, there has been a growing focus on multiview data, driven by its rich complementary and consistent information, which has the potential to significantly enhance the performance of downstream tasks. Although many multiview clustering (MVC) methods have achieved promising results by integrating the information of multiple views to learn the consistent representation or consistent graph, these methods typically require complete and entirely accurate correspondences between multiview data, which is challenging to fulfill in practice leading to the problem of partially view-aligned clustering (PVC). To tackle it, we propose a novel method, called dynamic graph guided progressive partial view-aligned clustering (DGPPVC) in this article. To the best of our knowledge, this could be the first work to employ graph convolutional network (GCN) to address the problem of PVC, which explores GCN with dynamic adjacency matrix to reduce unreliable alignments and locate the feature representation with consistent graph structure. In particular, DGPPVC develops an end-to-end framework that encompasses graph construction, feature representation learning, and alignment relationships learning, in which the three parts mutually influence and benefit each other. Moreover, DGPPVC adopts a novel alignment learning strategy that progresses from simplicity to complexity, enabling the step-by-step acquisition of unknown correspondences between different modalities. By giving priority to simple instance pairs, a variant of Jaccard similarities is designed to identify more reliable and complex alignments progressively. During the gradual learning process of alignment relationships, the graph structure matrix is continually and dynamically optimized, thus acquiring a greater variety of graph information between different views. Experiments on several real-world datasets show our promising performance compared with the state-of-the-art methods in partially view-aligned clustering.

PPM: A boolean optimizer for data association in multi-view pedestrian detection

To accurately localize occluded people in a crowd is a challenging problem in video surveillance. Existing end-to-end deep multi-camera detectors rely heavily on pre-training with the same multiview datasets used for testing, which compromises their real-world applications. An alternative approach presented here is to project the torso lines of the instance segmentation masks from multiple views to the ground plane and propose pedestrian candidates at the intersection points. The candidate selection process is, for the first time, formulated as a logic minimization problem in Boolean algebra. A probabilistic Petrick’s method (PPM) is proposed to seek the minimum number of candidates to account for all the foreground masks while maximizing the joint occupancy likelihoods in multiple views. Experiments on benchmark video datasets have demonstrated the much improved performance of this approach in comparison with the benchmark deep or non-deep algorithms for multiview pedestrian detection.