Feature Constraints Research Articles

Non-probability sampling network based on anomaly pedestrian trajectory discrimination for pedestrian trajectory prediction

Pedestrian trajectory prediction in first-person view is an important support for achieving fully automated driving in cities. However, existing pedestrian trajectory prediction methods still have significant shortcomings in terms of pedestrian trajectory diversity, dynamic scene constraints, and dependence on long-term trajectory prediction. We proposes a non-probability sampling network based on pedestrian trajectory anomaly recognition (ADsampler) to predict multiple possible future pedestrian trajectories. First, by incorporating pose and optical flow information, ADsampler models the multi-dimensional motion characteristics of pedestrians based on observed trajectory information and discriminates trajectory states. The sampling range in the Gaussian latent space is determined based on the recognition results. Next, velocity and yaw information of the car are introduced to model the car's motion state. A subtraction fusion network is employed to remove redundant image feature constraints in highly dynamic scenes. Finally, ADsampler utilizes a novel trajectory decoding network that combines the position encoding capability of GRU with the long-term dependency capturing ability of Transformer to decode and predict the fused features. we evaluate our model on crowded videos in the public datasets JAAD, PIE, ETH and UCY. Experiments demonstrate that the proposed method outperforms state-of-the-art approaches in prediction accuracy.

Feature-constrained automatic geometric deformation analysis method of bridge models toward digital twin

ABSTRACT It is very important to construct digital twin scenes, which can accurately describe the dynamically changing geographical environment and improve the level of refined management in bridge construction. This article proposes a feature constrained automatic diagnostic analysis method for geometric deformation of bridge digital twins. The geometric deformation feature library of bridge twins was first created to accurately describe structural relationships and behavior characteristics. Secondly, line surface feature constraints were used to extract geometric deformation information from bridge digital twins. Then, a geometric deformation diagnosis algorithm was designed based on an improved Hausdorff method. Finally, a case study was conducted to implement experimental analysis. The experimental results show that the method proposed in this paper can automatically extract the geometric morphology and rapidly calculate line and surface deformations for point cloud bridge digital twins. It achieves an efficiency improvement above 90% and with millimeter-level accuracy, which effectively enhances the diagnostic analysis capabilities for geographical digital twin models.

Sketch-guided spatial adaptive normalization and high-level feature constraints based GAN image synthesis for steel strip defect detection data augmentation

Deep learning methods have made remarkable strides in surface defect detection. But, they heavily rely on large amount of training data, which can be a costly endeavor, especially for specific applications like steel strip surface defect detection, where acquiring and labeling large-scale data is impractical due to the rarity of certain defective categories in production environment. Hence, realistic defect image synthesis can greatly alleviate this issue. However, training image generation networks also demand substantial data, making image data augmentation merely an auxiliary effort. In this work, we propose a Generative Adversarial Network (GAN)-based image synthesis framework. We selectively extract the defect edges of the original image as well as the background texture information, and use them as network input through the spatially-adaptive (de)normalization (SPADE) module. This enriches the input information, thus significantly reducing the amount of training data for GAN network in image generation, and enhancing the background details as well as the defect boundaries in the generated images. Additionally, we introduce a novel generator loss term that balances the similarity and perceptual fidelity between synthetic and real images by constraining high-level features at different feature levels. This provides more valuable information for data augmentation in training object detection models using synthetic images. Our experimental results demonstrate the sophistication of the proposed image synthesis method and its effectiveness in data augmentation for steel strip surface defect detection tasks.

Research on Image Stitching Algorithm Based on Point-Line Consistency and Local Edge Feature Constraints.

Image stitching aims to synthesize a wider and more informative whole image, which has been widely used in various fields. This study focuses on improving the accuracy of image mosaic and proposes an image mosaic method based on local edge contour matching constraints. Because the accuracy and quantity of feature matching have a direct influence on the stitching result, it often leads to wrong image warpage model estimation when feature points are difficult to detect and match errors are easy to occur. To address this issue, the geometric invariance is used to expand the number of feature matching points, thus enriching the matching information. Based on Canny edge detection, significant local edge contour features are constructed through operations such as structure separation and edge contour merging to improve the image registration effect. The method also introduces the spatial variation warping method to ensure the local alignment of the overlapping area, maintains the line structure in the image without bending by the constraints of short and long lines, and eliminates the distortion of the non-overlapping area by the global line-guided warping method. The method proposed in this paper is compared with other research through experimental comparisons on multiple datasets, and excellent stitching results are obtained.

FCLGYOLO: Feature Constraint and Local Guided Global Feature for Fire Detection in Unmanned Aerial Vehicle Imagery

FCLGYOLO: Feature Constraint and Local Guided Global Feature for Fire Detection in Unmanned Aerial Vehicle Imagery

Automatic Extraction of High-Voltage Transmission Pylons with Multi Feature Constraints

Automatic Extraction of High-Voltage Transmission Pylons with Multi Feature Constraints

Semantic segmentation of remote sensing imagery for road extraction via joint angle prediction: comparisons to deep learning

Accurate road network information is required to study and analyze the relationship between land usage type and land subsidence, and road extraction from remote sensing images is an important data source for updating road networks. This task has been considered a significant semantic segmentation problem, given the many road extraction methods developed for remote sensing images in recent years. Although impressive results have been achieved by classifying each pixel in the remote sensing image using a semantic segmentation network, traditional semantic segmentation methods often lack clear constraints of road features. Consequently, the geometric features of the results might deviate from actual roads, leading to issues like road fractures, rough edges, inconsistent road widths, and more, which hinder their effectiveness in road updates. This paper proposes a novel road semantic segmentation algorithm for remote sensing images based on the joint road angle prediction. By incorporating the angle prediction module and the angle feature fusion module, constraints are added to the angle features of the road. Through the angle prediction and angle feature fusion, the information contained in the remote sensing images can be better utilized. The experimental results show that the proposed method outperforms existing semantic segmentation methods in both quantitative evaluation and visual effects. Furthermore, the extracted roads were consecutive with distinct edges, making them more suitable for mapping road updates.

Mutually Beneficial Transformer for Multimodal Data Fusion

Multimodal feature fusion representation, e.g., hyperspectral image and light detection and ranging (HSI-LiDAR) fusion, is an essential topic for fusion perception. However, existing networks tend to employ mandatory feature stacking or local context fusion strategies between multiple modalities, ignoring the power of globally mutual-guided feature transmission. Therefore, this paper develops a mutually beneficial transformer method for multimodal data fusion (MBFormer), which contains the following steps. First, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">spatial constraint-based self-attention (SCS) module</i> . In this module, spectralwise attention and a spatialwise convolution are applied to HSI and LiDAR data individually, and then a spatial guide mask generated from LiDAR elevation information is used as an agent to bridge with HSI for spatial feature constraints. Second, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">channel diversity-based transformer (CDT) module</i> . On the basis of local spectral embedding explorations, an adaptive token-mixer mechanism is conducted on the groupwise classification token of HSI and individual LiDAR data for global information connectivity and transitivity. At last, the selected features are embedded into a classification layer for the final result calculation. Experimental results show that the proposed MBFormer can obtain 97.76% and 98.62% classification accuracies on Houston and Trento datasets, respectively, indicating the advantages and competitiveness of the MBFormer over the compared state-of-the-art methods.

Learning Unified Hyper-Network for Multi-Modal MR Image Synthesis and Tumor Segmentation With Missing Modalities.

Accurate segmentation of brain tumors is of critical importance in clinical assessment and treatment planning, which requires multiple MR modalities providing complementary information. However, due to practical limits, one or more modalities may be missing in real scenarios. To tackle this problem, existing methods need to train multiple networks or a unified but fixed network for various possible missing modality cases, which leads to high computational burdens or sub-optimal performance. In this paper, we propose a unified and adaptive multi-modal MR image synthesis method, and further apply it to tumor segmentation with missing modalities. Based on the decomposition of multi-modal MR images into common and modality-specific features, we design a shared hyper-encoder for embedding each available modality into the feature space, a graph-attention-based fusion block to aggregate the features of available modalities to the fused features, and a shared hyper-decoder for image reconstruction. We also propose an adversarial common feature constraint to enforce the fused features to be in a common space. As for missing modality segmentation, we first conduct the feature-level and image-level completion using our synthesis method and then segment the tumors based on the completed MR images together with the extracted common features. Moreover, we design a hypernet-based modulation module to adaptively utilize the real and synthetic modalities. Experimental results suggest that our method can not only synthesize reasonable multi-modal MR images, but also achieve state-of-the-art performance on brain tumor segmentation with missing modalities.

Finding Near-optimal Configurations in Colossal Spaces with Statistical Guarantees

A Software Product Line ( SPL ) is a family of similar programs. Each program is defined by a unique set of features, called a configuration , that satisfies all feature constraints. “What configuration achieves the best performance for a given workload?” is the SPL Optimization ( SPLO ) challenge. SPLO is daunting: just 80 unconstrained features yield 10 24 unique configurations, which equals the estimated number of stars in the universe. We explain (a) how uniform random sampling and random search algorithms solve SPLO more efficiently and accurately than current machine-learned performance models and (b) how to compute statistical guarantees on the quality of a returned configuration; i.e., it is within x% of optimal with y% confidence.

A method for stitching remote sensing images with Delaunay triangle feature constraints

The process of synthesizing multiple images into a seamless panoramic image is referred to as remote sensing image stitching. Existing studies focus less on the influence of topography on the appearance and texture of images and the perturbation of image spectra by topographic changes. This paper presents a remote sensing image stitching method that considers the impact of topography and geomorphology. First, the feature matching was optimized using the Euclidean distance similarity of texture features and the nearest neighbor distance ratio of feature points in remote sensing images as constraints. Then, the Delaunay triangle mesh of feature points in the image overlapping region was constructed, the geometric features of Delaunay triangles were used to optimize the triangle matching and reduce the matching redundancy, and the affine transformation matrix was solved based on the comprehensive consideration of the geometric features of Delaunay triangles and the texture features of the remote sensing images. Finally, the weighted fusion algorithm was applied to stitch and fuse the images. Three image datasets were selected for the experiments, one in which there were large terrain undulations in the imaging regions, one in which the main body of the imaging regions was water, and one in which the overall terrain of the imaging regions had relatively gentle slopes but obscuring features were also present. The results showed that the average correct rates of method feature matching were 89.78%, 94.99%, and 96.17%, which were the best for each algorithm, and the average feature matching times were 4.13, 8.27 and 7.19 s. These times are much lower than those obtained with the APAP and AANAP algorithms and basically the same as those achieved with the SPHP and SURF algorithms. In terms of visual effect, the AG, and indices of the proposed method were all significantly improved compared to the APAP, SPHP, AANAP, and SURF algorithms, the advantages were most obvious when dealing with datasets with large topographic relief in the imaging regions, with the maximum improvement of the AG, and indices were 85.61%, 95.68%, and 93.12%, respectively. Therefore, it is possible to conclude that the proposed method is more suitable for remote sensing image stitching and fusion of different topographic and geomorphological conditions.

Role-oriented representation learning via fusioning local and higher-order feature

Role-oriented representation learning extracts structural features by capturing the structural similarities of nodes to generate representations that preserve the structural equivalence, which is crucial for addressing the role discovery task of real networks. However, most studies directly combine different types of structural features and ignore the correlations among features, which is not enough to accurately portray node roles. Therefore, we propose a novel role-oriented network representation learning method RFLH that mines the correlations among features by normalizing flows to improve the performance on the role discovery tasks. We utilize the Variational Auto-Encoder (VAE) to generate the original representation distribution and learn the structural features of nodes by constraint of local and higher-order features, where local features are learned by the traditional variational module, while higher-order features are learned by the flow-based variational module that introduces normalizing flows to map the original representation distribution to a complex distribution. Moreover, we deduce the new loss by introducing the Jacobian determinant to Evidence Lower Bound (ELBO). Finally, we fuse local and higher-order features into the node representations, which are concatenated by the representations sampled from the original distribution and the new distribution, to generate the final node representations. Extensive experimental results on several real network datasets demonstrate that our model achieves superior performance compared to typical baseline methods.

Atmospheric correction algorithm based on deep learning with spatial-spectral feature constraints for broadband optical satellites: Examples from the HY-1C Coastal Zone Imager

Broadband optical satellites have been widely used for fine monitoring of coastal waters and inland lakes for their high spatial resolution. Atmospheric correction (AC) is one of the essential data processing steps for remote sensing of water environments with high spatial resolution satellites. Most broadband optical satellites, such as HY-1C Coastal Zone Imager (CZI) with a spatial resolution of 50 m, lack of near-infrared and shortwave infrared bands needed for atmospheric correction, making accurate AC difficult. Auxiliary aerosol data is often needed for accurate AC of broadband optical satellite data, which suffers from the differences in spatial resolution and imaging time between auxiliary data and satellite data. Furthermore, existing AC algorithms always perform AC pixel by pixel, which ignores the spatial relationship between adjacent pixels. Taking HY-1C CZI as an example, this paper proposes a novel atmospheric correction algorithm based on deep learning (SSACNet). Considering the inherent spatial-spectral features of satellite images, the SSACNet combines 2D and 3D convolution. The 3D convolution was used to mine the spatial and spectral features of the image and 2D convolution was explored to perform spatial information compensation. The SSACNet was trained and evaluated using the spatio-temporally synchronized dataset of HY-1C CZI and Landsat8 Operational Land Imager (OLI). The evaluation results by in-situ data show that the SSACNet has good performance, with the average correlation coefficient of 0.89, and the absolute percentage deviation (APD) of four bands ranging from 21.53 % to 35.41 %. Compared with the quasi-synchronous Landsat8 OLI remote sensing reflectance (Rrs), the APD is less than 10 %. Compared with traditional atmospheric correction algorithms, SSACNet has significantly improved the spatial and spectral information fidelity. In addition, SSACNet also shows good applicability, as it can be used in both clear ocean waters and turbid coastal waters. This study lays a foundation for the quantitative remote sensing of water environment by broadband optical satellites.

Mutual information-based feature disentangled network for anomaly detection under variable working conditions

Anomaly detection is of great significance to ensure operational safety of advanced equipment. However, existing anomaly detection algorithms presume that the working conditions are invariable, which cannot adapt to actual industrial scenarios and may cause false or missing alarms when the working conditions change. To address this issue, we expect to learn a condition-independent feature space, from which anomaly indicators can be derived to enable condition-agnostic anomaly detection. The objective to find such space is then deduced intuitively and justified by information theory. Correspondingly, a Mutual Information-based Feature Disentangled Network (MIFD-Net) is proposed for anomaly detection under variable working conditions. Specifically, to ensure feature independence from working conditions, the mutual information between latent representations and conditions is minimized by reducing its variational upper bound. Meanwhile, a self-supervised contrastive loss is deduced to maximize the mutual information between latent features and observed signals, which improves the fault discriminability of latent features. Finally, both feature constraints are integrated via dynamic weights in model training. Reconstruction errors derived from latent features are defined as anomaly scores for condition-agnostic anomaly detection. Experiments on part-level and component-level fault datasets demonstrate the superiority of the proposed method under both multiple and varying working condition scenarios.

The Method of Moving Frames for Surface Global Parametrization

This article introduces a new representation of surface global parametrization based on Cartan’s method of moving frames . We show that a system of structure equations , characterizing the local coordinates changes with respect to a local frame system, completely characterizes the set of possible cone parametrizations. The discretization of this system provably provides necessary and sufficient conditions for the existence of a valid mapping. We are able to derive a versatile algorithm for surface parametrization, allowing feature constraints and singularities. As the first structure equation is independent of the global coordinate system, we do not require prior knowledge of cuts or cone positions. So, a single non-linear least-square problem is enough to place quantized cones while minimizing a given distortion energy. We are therefore able to take full advantage of the link between the parametrization geometry and the topology of its cone metric to solve challenging constrained parametrization problems.

Three-Dimensional Reconstruction and Geometric Morphology Analysis of Lunar Small Craters within the Patrol Range of the Yutu-2 Rover

Craters on the lunar surface are the most direct method for the study of geological processes and are of great significance to the study of lunar evolution. In order to fill the research gap on small craters (diameter less than 3 m), we focus on the small craters around the moving path of the Yutu-2 lunar rover and carry out a 3D reconstruction and geometrical morphology analysis on them. First, a self-calibration model with multiple feature constraints is used to calibrate the navigation camera and obtain the internal and external parameters. Then, the sequence images with overlapping regions from neighboring stations are used to obtain the precise position of the rover through the bundle adjustment (BA) method. After that, a cross-scale cost aggregation for a stereo matching network is proposed to obtain a parallax map, which can further obtain 3D point clouds of the lunar surface. Finally, the indexes of the craters are extracted (diameter D, depth d, and depth–diameter ratio dr), and the different indicators are fitted and analyzed. The results suggest that CscaNet has an anomaly percentage value of 1.73% in the KITTI2015 dataset, and an EPE of 0.74 px in the SceneFlow dataset, both of which are superior to GC-Net, DispNet, and PSMnet, and have higher reconstruction accuracy. The correlation between D and d is high and exhibits a positive correlation, while the correlation between D and dr is low. The geometric morphology expressions of small craters fitted by using D and d are significantly different from the expressions proposed by other scholars for large craters. This study provides a priori knowledge for the subsequent Von Karmen crater survey mission in the SPA Basin.

On exploring pose estimation as an auxiliary learning task for Visible–Infrared Person Re-identification

Visible–infrared person re-identification (VI-ReID) has been challenging due to the existence of large discrepancies between visible and infrared modalities. Most pioneering approaches reduce intra-modality variations and inter-modality discrepancies by learning modality-shared features. However, an explicit modality-shared cue, i.e., body keypoints, has not been fully exploited in VI-ReID. Additionally, existing feature learning paradigms imposed constraints on either global features or partitioned feature stripes, which neglect the prediction consistency of global and part features. To address the above problems, we exploit Pose Estimation as an auxiliary learning task to assist VI-ReID in an end-to-end framework. By jointly training these two tasks in a mutually beneficial manner, our model learns higher quality ID-related features. On top of it, the learnings of global features and local features are seamlessly synchronized by Hierarchical Feature Constraint (HFC), where the former supervises the latter using the knowledge distillation strategy. Experimental results on two benchmark VI-ReID datasets show that the proposed method consistently improves state-of-the-art methods by significant margins. Specifically, our method achieves nearly 20% mAP improvements against the state-of-the-art method on the RegDB dataset. Our intriguing findings highlight the usage of auxiliary task learning in VI-ReID. Our source code is available at https://github.com/yoqim/Pose_VIReID.

Primordial feature constraints from BOSS + eBOSS

Understanding the universe in its pristine epoch is crucial in order to obtain a concise comprehension of the late-time universe. Although current data in cosmology are compatible with Gaussian primordial perturbations whose power spectrum follows a nearly scale-invariant power law, this need not be the case when a fundamental theoretical construction is assumed. These extended models lead to sharp features in the primordial power spectrum, breaking its scale invariance. In this work, we obtain combined constraints on four primordial feature models by using the final data release of the BOSS galaxies and eBOSS quasars. By pushing towards the fundamental mode of these surveys and using the larger eBOSS volume, we were able to extend the feature parameter space (i.e. the feature frequency ω) by a factor of four compared to previous analyses using BOSS. While we did not detect any significant features, previous work showed that next-generation galaxy surveys such as DESI will improve the sensitivity to features by a factor of 7, and will also extend the parameter space by a factor of 2.5.

Feature-constrained real-time simultaneous monitoring of monocular vision odometry for bridge bearing displacement and rotation

The displacement and rotation are critical indicators of bridge's health, especially for the key components of the bridge such as bridge bearings. This study presents a real-time simultaneous measurement method of bridge bearing displacement and rotation to realize round-the-clock structure monitoring. The main contributions include: (1) A displacement and rotation simultaneous measurement algorithm and system is developed based on feature-constrained monocular visual odometry (VO), which uses image sequences of infrared array lamp targets with multiple feature constraints as visual input to construct the mathematical relationship between displacement and rotation and coordinate system transformation matrix. (2) A two-step parameter optimization strategy is proposed based on adaptive multi-window centroid array tracking and transformation matrix purification and refinement, effectively improving displacement and rotation measurement accuracy and robustness. The effectiveness of the proposed method is verified in the laboratory experiment and the actual bridge bearing monitoring of the Huangpu Pearl River Bridge.

Information-diffused graph tracking with linear complexity

Mainstream tracking approaches have achieved remarkable performance by adopting transformer structures. However, transformer structures’ inherent design of dot-product with softmax normalization incurs quadratic computation complexity regarding sequence length. This issue is further complicated when vision tasks employ softmax attention, as sequence length scales with the square of images’ sizes. Even though sparse attention and low-rank decomposition can alleviate over-inflated computation, it is still laborious to balance trackers’ accuracy, computation cost, and inference speed. To tackle the above problems, we propose an Information-Diffused Graph tracking pipeline with linear complexity (IDGtrack). As the feature constraint relationship in the physical world is an important cue for vision tasks, graph modules are constructed with information-diffused adjacency matrices to substitute softmax attention, which is not only efficient for linear computations but also maintains the non-negativity and global distribution of the attention matrix. Distinct from traditional linear attention methods exclusive to self-attention, a self-integrated and cross-context graph module with linear complexity is explored where a complete bipartite graph is established between the target and search region, facilitating a comprehensive perception of local and background information. Extensive experiments are conducted on public tracking benchmarks, demonstrating that our method achieves state-of-the-art (SOTA) performance with 111 FPS on GPU RTX3090.