Multiscale Matching Research Articles

Multilevel identity fine authentication method based on time-frequency domain feature extraction in industrial internet of things system

The cyber physical system is widely applied to industrial Internet of Things system and its security determines the safety of the industrial IoT system. This paper proposes a multilevel fine fingerprint authentication method for the security protection of key operating equipment. Firstly, Because of the acquired signals, the mathematical representation models are established separately, including the self-encoding model on the basis of the multiorder moment features and the multi-scale representation model based on the wavelet. Secondly, the multigranularity features of the time domain and frequency domain are extracted respectively based on the real-time data's self-encoding representation and multiscale representation. Then, the extracted features of different granularities can be built into the image representation based on the time-frequency fusion portrait. Based on the images of different granularities and the legitimate image library, a multiscale matching authentication method is established. Finally, simulation experiments show that the proposed method is effective.

Multi-scale matching and simulating flows of ecosystem service supply and demand in the Wuhan metropolitan area, China

Metropolitan areas face the challenge of balancing increasing social needs against limited natural resources. The multi-scale evaluation of ecosystem services (ES) supply, demand and flows is crucial but under explored. Four prominent ES, food provision (FP), carbon sequestration (CS), soil conservation (SC), and outdoor recreation (OR) in Wuhan metropolitan area (WMA) were selected as cases. ES supply and demand were assessed by InVEST model, geographic information system (GIS) spatial analysis, and statistical methods. A linear programming model was employed to reveal the characteristics of ES flows. Results showed the following: (1) the total ES supply capacity can meet the demand of the study area. At different scales, the supply patterns were fundamentally consistent, except for FP, which was significantly higher in Wuhan's suburban areas. In contrast, the other three ES exhibited high supply in the northeast and south WMA, encompassing the Dabie and Mufu Mountains. The high demand for all ES radiated from the city center, except for SC. (2) A significant scale effect of ES mismatch was found, and ecological deficit patches decreased with increasing research scale. The Mismatch was evident on finer scales but lessened at coarser ones. (3) ES flows, consisting of FP, CS, and OR, formed a flow pattern from the suburbs to the city center. Coarse-scale management integrating ecological flows may better maintain ES balance than fine-scale measures. These findings provide insights for multi-scale management strategies that not only contribute to localized ecological restoration, but also optimize strategies for ecological corridors and compensation mechanisms.

Control and prediction of bedding-parallel fractures in fine-grained sedimentary rocks: A case from the Permian Lucaogou Formation in Jimusar Sag, Junggar Basin, Western China

The fine-grained sedimentary rocks have numerous bedding-parallel fractures that are essential for the migration, enrichment, and efficient development of oil and gas. However, because of their variety and the complexity of the factors that affect them, their spatial prediction by the industrial community becomes challenging. Based on sample cores, thin sections, and well-logging and seismic data, this study employed a multi-scale data matching approach to quantitatively predict the development of bedding-parallel fractures and investigate their spatial distribution. Bedding-parallel fractures in the Lucaogou Formation in Jimusar Sag frequently occur along preexisting bedding planes and lithological interfaces. Unfilled bedding-parallel fractures inside or near source-rocks exhibit enhanced oil-bearing capacity. They were identified on micro-resistivity scanning images by the presence of regularly continuous black or nearly black sinusoidal curves. Overall, the developmental degree of bedding-parallel fractures was positively related to the brittle mineral and total organic carbon contents and negatively related to single reservoir interval thickness. The maintained porosity of the reservoir matrix contributed to a thorough response to factors affecting the development of bedding-parallel fractures. Here, an effective and objective method was proposed for predicting the development and distribution of bedding-parallel fractures in the fine-grained sedimentary rocks. The method was based on the matched reservoir interval density, reservoir interval density, and matched sweet spot density of bedding-parallel fractures. The prediction method integrated the significant advantages of high vertical resolution from logging curves and strong lateral continuity from seismic data. The average relative prediction error was 8% in the upper sweet spot in the Lucaogou Formation, indicating that the evaluation parameters for bedding-parallel fractures in fine-grained sedimentary rocks were reasonable and reliable and that the proposed prediction method has a stronger adaptability than the previously reported methods. The workflow based on multi-scale matching and stepwise progression can be applied in similar fine-grained sedimentary rocks, providing reliable technological support for the exploration and development of hydrocarbons.

Enhanced large-scale building extraction evaluation: developing a two-level framework using proxy data and building matching

ABSTRACT Deep learning-based building extraction methods have widespread applications in diverse fields. However, the evaluation of large-scale extraction results remains challenging, due to traditional evaluation metrics rely on manually created ground-truth samples and the lack of comprehensive reference-building data for developing countries. To address these problems, we proposed a two-level framework for evaluating large-scale footprint extraction. First, we utilised global open-source population and land use data as the proxy data, to assess grid-level completeness for the areas with insufficient reference data. Second, we introduced an improved two-way area-overlapping method to match the extracted footprints with the reference buildings, thereby enabling a comprehensive evaluation of the study region. Tested in Hyogo Prefecture and Numazu City, Japan, the results demonstrated a 2.6-% improvement in grid classification accuracy and an increase of 0.53 in the completeness correlation, compared with the results obtained using a single proxy indicator. Moreover, the optimised matching method achieved an outstanding semantic matching accuracy of 99%, with high efficiency and robustness in multi-scale matching. Therefore, the proposed approach can effectively evaluate large-scale footprint extraction results and interpret their semantic relationship with actual buildings, applicable globally regardless of the availability of reference building datasets.

A two-stage optimization configuration model for multi-type flexible resource considering spatiotemporal response characteristics

As China gradually transitions towards a low-carbon energy structure, the proportion of grid-connected new energy sources like wind and solar power continues to increase. To ensure the safe and reliable operation of the power system while meeting the capacity planning for future new energy installations, there is a need for flexible resources with corresponding adjustment capabilities in the power system. In response to this situation, this paper presents an optimization model for the allocation of multiple types of flexible resources that takes into account spatiotemporal response characteristics. Firstly, a flexibility evaluation model is developed based on spatial and temporal response characteristics. Flexibility evaluation indices, such as flexibility average deficit and flexibility coverage index, are constructed. These indices are used for screening nodes with inadequate flexibility in the power system and analyzing the flexibility adequacy at various nodes. Next, the adjustment characteristics of multiple types of flexible resources are analyzed, and a model for their adjustment capabilities is established. Finally, by considering constraints based on time flexibility evaluation indices, a two-stage optimization model for flexible resource allocation is constructed. This model leverages the multiscale matching characteristics between flexibility resources and the fluctuation patterns of new energy sources to guide the allocation of flexible resources at nodes with insufficient flexibility. The effectiveness and applicability of the proposed flexible resource allocation method are validated using the IEEE 9-node system.

One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching.

One-shot skeleton action recognition, which aims to learn a skeleton action recognition model with a single training sample, has attracted increasing interest due to the challenge of collecting and annotating large-scale skeleton action data. However, most existing studies match skeleton sequences by comparing their feature vectors directly which neglects spatial structures and temporal orders of skeleton data. This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching. We represent skeleton data at multiple spatial and temporal scales and achieve optimal feature matching from two perspectives. The first is multi-scale matching which captures the scale-wise semantic relevance of skeleton data at multiple spatial and temporal scales simultaneously. The second is cross-scale matching which handles different motion magnitudes and speeds by capturing sample-wise relevance across multiple scales. Extensive experiments over three large-scale datasets (NTU RGB+D, NTU RGB+D 120, and PKU-MMD) show that our method achieves superior one-shot skeleton action recognition, and outperforms SOTA consistently by large margins.

DIML: Deep Interpretable Metric Learning via Structural Matching.

In this paper, we present a new framework named DIML to achieve more interpretable deep metric learning. Unlike traditional deep metric learning method that simply produces a global similarity given two images, DIML computes the overall similarity through the weighted sum of multiple local part-wise similarities, making it easier for human to understand the mechanism of how the model distinguish two images. Specifically, we propose a structural matching strategy that explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. We also devise a multi-scale matching strategy, which considers both global and local similarities and can significantly reduce the computational costs in the application of image retrieval. To handle the view variance in some complicated scenarios, we propose to use cross-correlation as the marginal distribution of the optimal transport to leverage semantic information to locate the important region in the images. Our framework is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods. To extend our DIML to more advanced architectures like vision Transformers (ViTs), we further propose truncated attention rollout and partial similarity to overcome the lack of locality in ViTs. We evaluate our method on three major benchmarks of deep metric learning including CUB200-2011, Cars196, and Stanford Online Products, and achieve substantial improvements over popular metric learning methods with better interpretability.

End-to-End Edge-Guided Multi-Scale Matching Network for Optical Satellite Stereo Image Pairs

Acquiring disparity maps by dense stereo matching is one of the most important methods for producing digital surface models. However, the characteristics of optical satellite imagery, including significant occlusions and long baselines, increase the challenges of dense matching. In this study, we propose an end-to-end edge-guided multi-scale matching network (EGMS-Net) tailored for optical satellite stereo image pairs. Using small convolutional filters and residual blocks, the EGMS-Net captures rich high-frequency signals during the initial feature extraction phase. Subsequently, pyramid features are derived through efficient down-sampling and consolidated into cost volumes. To regularize these cost volumes, we design a top–down multi-scale fusion network that integrates an attention mechanism. Finally, we innovate the use of trainable guided filter layers in disparity refinement to improve edge detail recovery. The network is trained and evaluated using the Urban Semantic 3D and WHU-Stereo datasets, with subsequent analysis of the disparity maps. The results show that the EGMS-Net provides superior results, achieving endpoint errors of 1.515 and 2.459 pixels, respectively. In challenging scenarios, particularly in regions with textureless surfaces and dense buildings, our network consistently delivers satisfactory matching performance. In addition, EGMS-Net reduces training time and increases network efficiency, improving overall results.

Aeroengine Remaining Life Prediction Based on Advanced Health Index Construction and Grey Similarity Multi-Scale Matching

Aeroengine Remaining Life Prediction Based on Advanced Health Index Construction and Grey Similarity Multi-Scale Matching

Flexible Iris Matching Based on Spatial Feature Reconstruction

In an iris recognition-at-a-distance (IAAD) system used in surveillance scenarios, the camera usually captures a large number of low-quality images. These images exhibit partial occlusions due to eyelids and eyelashes, specular reflections, and severe deformations caused by pupil dilations and contractions. Recognizing these low-quality images is a challenging yet dominant problem in IAAD. To mitigate this issue, current iris recognition systems mostly filter out low-quality images by using strict criteria based on image quality evaluation. This strategy, however, wastes device capabilities and produces low throughput of subjects. Other systems require highly cooperative users. In this work, we propose a novel occlusion-robust, deformation-robust, and alignment-free framework for low-quality iris matching, which integrates the merits of deep features and sparse representation in an end-to-end learning process known as iris spatial feature reconstruction (ISFR). Here each probe image can be sparsely reconstructed on the basis of appropriate feature maps from gallery high-quality images. ISFR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two iris images, which naturally avoids the time-consuming alignment step. In summary, the distinctiveness of deep features, the robustness of sparse reconstruction, and the flexibility of multiscale matching strategy are unified in a general framework to attain more accurate and reasonable iris matching. Extensive experimental results on four public iris image databases demonstrate that the proposed method significantly outperforms both traditional and deep learning-based iris recognition methods.

Cross-Viewpoint Template Matching Based on Heterogeneous Feature Alignment and Pixel-Wise Consensus for Air- and Space-Based Platforms

Template matching is the fundamental task in remote sensing image processing of air- and space-based platforms. Due to the heterogeneous image sources, different scales and different viewpoints, the realization of a general end-to-end matching model is still a challenging task. Considering the abovementioned problems, we propose a cross-view remote sensing image matching method. Firstly, a spatial attention map was proposed to solve the problem of the domain gap. It is produced by two-dimensional Gaussian distribution and eliminates the distance between the distributed heterogeneous features. Secondly, in order to perform matching at different flight altitudes, a multi-scale matching method was proposed to perform matching on three down-sampling scales in turn and confirm the optimal result. Thirdly, to improve the adaptability of the viewpoint changes, a pixel-wise consensus method based on a correlation layer was applied. Finally, we trained the proposed model based on weakly supervised learning, which does not require extensive annotation but only labels one pair of feature points of the template image and search image. The robustness and effectiveness of the proposed methods were demonstrated by evaluation on various datasets. Our method accommodates three types of template matching with different viewpoints, including SAR to RGB, infrared to RGB, and RGB to RGB.

Image registration based fault localization in panoramas of mountain-mounted PV plants

Large-scale photovoltaic (PV) plants have been widely distributed in the mountains. However, the complex geographical environment and the irregular arrangement of PV strings bring great challenges to maintenance. In this paper, a novel scheme combining an improved YOLOv5s model and image registration algorithms is proposed to detect and locate module faults in mountain-mounted PV plants (MPVs) via aerial dual-light videos and panoramas. The improved YOLOv5s used to detect faults in IR images becomes more lightweight owing to the integration of MobileNetV3 and ECA-Net. To realize fault localization in panoramas of MPVs, image registration algorithms are adopted. IR and visual images are simultaneously extracted frame by frame from dual-light videos. Faults detected in the IR image are first mapped to the corresponding visual image by the homography (H1) matrix. To map faults from the visual image to the panorama, another registration between the visual image and panorama is achieved. Multi-scale matching between the visual image and the panorama is first performed to reduce the area of registration. The conversion of grayscale probability density function (GPDF) of the visual image and the adoption of AKAZE feature detection algorithm make the homography (H2) matrix used for registration more accurate. A MPV with a power generation of 212.85 kW is used for testing. The precision and recall of fault detection are 88.31% and 91.67%, respectively. The accuracy of locating faults to the corresponding modules is 85.3%, and the recall of locating is 94.12%. Experimental results show that the proposed scheme can effectively identify faults and accurately locate faults in MPVs.

A comprehensive artificial intelligence framework for dental diagnosis and charting

BackgroundThe aim of this study was to develop artificial intelligence (AI) guided framework to recognize tooth numbers in panoramic and intraoral radiographs (periapical and bitewing) without prior domain knowledge and arrange the intraoral radiographs into a full mouth series (FMS) arrangement template. This model can be integrated with different diseases diagnosis models, such as periodontitis or caries, to facilitate clinical examinations and diagnoses.MethodsThe framework utilized image segmentation models to generate the masks of bone area, tooth, and cementoenamel junction (CEJ) lines from intraoral radiographs. These masks were used to detect and extract teeth bounding boxes utilizing several image analysis methods. Then, individual teeth were matched with a patient’s panoramic images (if available) or tooth repositories for assigning tooth numbers using the multi-scale matching strategy. This framework was tested on 1240 intraoral radiographs different from the training and internal validation cohort to avoid data snooping. Besides, a web interface was designed to generate a report for different dental abnormalities with tooth numbers to evaluate this framework’s practicality in clinical settings.ResultsThe proposed method achieved the following precision and recall via panoramic view: 0.96 and 0.96 (via panoramic view) and 0.87 and 0.87 (via repository match) by handling tooth shape variation and outperforming other state-of-the-art methods. Additionally, the proposed framework could accurately arrange a set of intraoral radiographs into an FMS arrangement template based on positions and tooth numbers with an accuracy of 95% for periapical images and 90% for bitewing images. The accuracy of this framework was also 94% in the images with missing teeth and 89% with restorations.ConclusionsThe proposed tooth numbering model is robust and self-contained and can also be integrated with other dental diagnosis modules, such as alveolar bone assessment and caries detection. This artificial intelligence-based tooth detection and tooth number assignment in dental radiographs will help dentists with enhanced communication, documentation, and treatment planning accurately. In addition, the proposed framework can correctly specify detailed diagnostic information associated with a single tooth without human intervention.

Automatic matching of multi-scale road network under the constraint of small-scale road mesh

Aiming at the problem of road network multi-scale matching, a multi-scale road matching method under the constraint of road mesh of small-scale data has been proposed. First, two road meshes with different scale data are constructed; Secondly, under the constraint of the small-scale road mesh, the composite mesh composed of several road meshes in the large-scale road is extracted, and the mesh matching with the small-scale road mesh is completed; Then, many-to-many matching of road meshes with different scales is realized; finally, the matching relationship between composite mesh and small-scale road mesh is transformed into the matching between multi-scale road mesh boundary roads and internal roads, and the matching of the whole road network is completed. The experimental results show that this method can better realize the matching of multi-scale road network.

HMSM-Net: Hierarchical multi-scale matching network for disparity estimation of high-resolution satellite stereo images

Disparity estimation of satellite stereo images is an essential and challenging task in photogrammetry and remote sensing. Recent researches have greatly promoted the development of disparity estimation algorithms by using CNN (Convolutional Neural Networks) based deep learning techniques. However, it is still difficult to handle intractable regions that are mainly caused by occlusions, disparity discontinuities, texture-less areas, and repetitive patterns. Besides, the lack of training datasets for satellite stereo images remains another major issue that blocks the usage of CNN techniques due to the difficulty of obtaining ground-truth disparities. In this paper, we propose an end-to-end disparity learning model, termed hierarchical multi-scale matching network (HMSM-Net), for the disparity estimation of high-resolution satellite stereo images. First, multi-scale cost volumes are constructed by using pyramidal features that capture spatial information of multiple levels, which learn correspondences at multiple scales and enable HMSM-Net to be more robust in intractable regions. Second, stereo matching is executed in a hierarchical coarse-to-fine manner by applying supervision to each scale, which allows a lower scale to act as prior knowledge and guides a higher scale to attain finer matching results. Third, a refinement module that incorporates the intensity and gradient information of the input left image is designed to regress a detailed full-resolution disparity map for local structure preservation. For network training and testing, a dense stereo matching dataset is created and published by using GaoFen-7 satellite stereo images. Finally, the proposed network is evaluated on the Urban Semantic 3D and GaoFen-7 datasets. Experimental results demonstrate that HMSM-Net achieves superior accuracy compared with state-of-the-art methods, and the improvement on intractable regions is noteworthy. Additionally, results and comparisons of different methods on the GaoFen-7 dataset show that it can severs as a challenging benchmark for performance assessment of methods applied to disparity estimation of satellite stereo images. The source codes and evaluation dataset are made publicly available at https://github.com/Sheng029/HMSM-Net.

Target tracking algorithm based on multiscale analysis and combinatorial matching

Target tracking algorithm based on multiscale analysis and combinatorial matching

MS-HLMO: Multiscale Histogram of Local Main Orientation for Remote Sensing Image Registration

Multi-source image registration is challenging due to intensity, rotation, and scale differences among the images. Considering the characteristics and differences of multi-source remote sensing images, a feature-based registration algorithm named Multi-scale Histogram of Local Main Orientation (MS-HLMO) is proposed. Harris corner detection is first adopted to generate feature points. The HLMO feature of each Harris feature point is extracted on a Partial Main Orientation Map (PMOM) with a Generalized Gradient Location and Orientation Histogram-like (GGLOH) feature descriptor, which provides high intensity, rotation, and scale invariance. The feature points are matched through a multi-scale matching strategy. Comprehensive experiments on 17 multi-source remote sensing scenes demonstrate that the proposed MS-HLMO and its simplified version MS-HLMO$^+$ outperform other competitive registration algorithms in terms of effectiveness and generalization.

Method of multiscale coincidence of pulses for measuring temporary loads from moving vehicles in bridges and overpasses

Modern development of road transport is characterized by constant updating of the nomenclature of cars (including heavy-duty vehicles), by increasing of intensity and speed of their movement, especially in large metropolises and on roads of national and local significance. This complicates the operating conditions of spans of highway bridges to some extent, leads to their damage and early wear. The deterioration of the technical condition of the structures of the spans is largely due to the increased dynamic impact of the cars on the bridges. The purpose of this article is to measure the amplitude of oscillations of a bridge structure from temporary loads of rolling stock with non-periodic long-term loading by optical methods. In accordance with this goal, it is necessary to develop a technique and device for measuring the amplitude of oscillations of the bridge structure by converting the amplitude of oscillations in time intervals and their subsequent measurement by multi-scale pulse coincidence. In a number of works, digital deployment systems that use number systems with different bases, represented by a family of deployment functions, are considered in detail.
 Such digital systems in general, are described by the corresponding operating scheme. According to this operating scheme, a method of multiscale matching for measuring time intervals was developed, which formed the basis for measuring the vibration amplitude of the bridge structure. The given mathematical model of calculation of the number of measurement channels was used in the synthesis of velocity transducers and linear displacements based on optical (laser) deployment systems, information conversion devices of radar velocity sensors based on the Doppler effect. Similarly, a frequency comparison device, a digital frequency meter, a digital phase meter, and a number of other frequency-time group devices were synthesized.

Multiscale Spatial Polygonal Object Granularity Factor Matching Method Based on BPNN

Spatial object matching is one of the fundamental technologies used for updating and merging spatial data. This study focused mainly on the matching optimization of multiscale spatial polygonal objects. We proposed a granularity factor evaluation index that was developed to promote the recognition ability of complex matches in multiscale spatial polygonal object matching. Moreover, we designed the granularity factor matching model based on a backpropagation neural network (BPNN) and designed a multistage matching workflow. Our approach was validated experimentally using two topographical datasets at two different scales: 1:2000 and 1:10,000. Our results indicate that the granularity factor is effective both in improving the matching score of complex matching and reducing the occurrence of missing matching, and our matching model is suitable for multiscale spatial polygonal object matching, with a high precision and recall reach of 97.2% and 90.6%.

Retinal Vessel Segmentation Based on Multiscale Matched Filtering

Retinal vascular function is complex, morphological structure varies from person to person, and is susceptible to vascular diseases and systemic vascular diseases. Its accurate segmentation is of great significance for disease diagnosis and identification. In this paper, a multi-scale matching filtering algorithm is proposed for the uneven size of retinal blood vessels. On the basis of the traditional singlescale Gaussian matching filter, multiscale Gaussian matched filters with two sizes are used to enhance grayscale images. Enhancement is performed, and the superimposed image is binarized using a twodimensional maximum entropy threshold segmentation algorithm. The algorithm is tested in the DRIVE database with sensitivity, specificity and accuracy of 0.803, 0.959, 0.981, respectively. Comparing with the traditional algorithm, the algorithm has high sensitivity, fast running speed and rich details of segmentation results.