Template Matching Research Articles

TomoCPT: a generalizable model for 3D particle detection and localization in cryo-electron tomograms.

Cryo-electron tomography is a rapidly developing field for studying macromolecular complexes in their native environments and has the potential to revolutionize our understanding of protein function. However, fast and accurate identification of particles in cryo-tomograms is challenging and represents a significant bottleneck in downstream processes such as subtomogram averaging. Here, we present tomoCPT (Tomogram Centroid Prediction Tool), a transformer-based solution that reformulates particle detection as a centroid-prediction task using Gaussian labels. Our approach, which is built upon the SwinUNETR architecture, demonstrates superior performance compared with both conventional binary labelling strategies and template matching. We show that tomoCPT effectively generalizes to novel particle types through zero-shot inference and can be significantly enhanced through fine-tuning with limited data. The efficacy of tomoCPT is validated using three case studies: apoferritin, achieving a resolution of 3.0 Å compared with 3.3 Å using template matching, SARS-CoV-2 spike proteins on cell surfaces, yielding an 18.3 Å resolution map where template matching proved unsuccessful, and rubisco molecules within carboxysomes, reaching 8.0 Å resolution. These results demonstrate the ability of tomoCPT to handle varied scenarios, including densely packed environments and membrane-bound proteins. The implementation of the tool as a command-line program, coupled with its minimal data requirements for fine-tuning, makes it a practical solution for high-throughput cryo-ET data-processing workflows.

Knowledge-driven semantic converting method of multimodal models toward a geospatial perspective

ABSTRACT In the virtual geographic environment, conducting status analysis on urban structures and similar objects is crucial for enhancing their detailed management level. However, it is challenging to directly convert the same object across various software systems with different modalities (such as spatial analysis, BIM design, numerical simulation etc.). Therefore, the effective conversion of multi-modal models becomes pivotal. Due to the characteristics of inconsistent spatial description and complex association relationship among multimodal models, resulting in low knowledge reuse rate, poor accuracy of unit mapping, and low efficiency of state sharing in the process of model conversion. Aiming at these problems, this article delves into the knowledge-driven semantic conversion techniques for multi-modal models from a geospatial viewpoint. The mapping relationships between multimodal models in terms of spatial, geometric, and semantic information were first clarified. Subsequently, a structural matching template based on knowledge reuse was established, and a knowledge-guided algorithm for multimodal model transformation was designed. Finally, using a suspension bridge as a case study, a prototype system was developed and experimental analysis was conducted. The experimental results show that the method proposed in this article can accurately convert between BIM models, numerical analysis models, and GIS scene models, with spatial coordinate accuracy controlled within 1 mm and a conversion efficiency increase of more than 10 times. This can effectively enhance the integrated performance of models in applications such as digital geospatial twin scenarios.

Assessment of Pine Tree Crown Delineation Algorithms on UAV Data: From K-Means Clustering to CNN Segmentation

The crown area is a critical metric for evaluating tree growth and supporting various ecological and forestry analyses. This study compares three approaches, i.e., unsupervised clustering, region-based, and deep learning, to estimate the crown area of Pinus eldarica Medw. using UAV-acquired RGB imagery (2 cm ground sampling distance) and high-density point clouds (1.27 points/cm3). The first approach applied unsupervised clustering techniques, such as Mean-shift and K-means, to directly estimate crown areas, bypassing tree top detection. The second employed a region-based approach, using Template Matching and Local Maxima (LM) for tree top identification, followed by Marker-Controlled Watershed (MCW) and Seeded Region Growing for crown delineation. The third approach utilized a Convolutional Neural Network (CNN) that integrated Digital Surface Model layers with the Visible Atmospheric Resistance Index for enhanced segmentation. The results were compared against field measurements and manual digitization. The findings reveal that CNN and MCW with LM were the most effective, particularly for small and large trees, though performance decreased for medium-sized crowns. CNN provided the most accurate results overall, with a relative root mean square error (RRMSE) of 8.85%, a Nash–Sutcliffe Efficiency (NSE) of 0.97, and a bias score (BS) of 1.00. The CNN crown area estimates showed strong correlations (R2 = 0.83, 0.62, and 0.94 for small, medium, and large trees, respectively) with manually digitized references. This study underscores the value of advanced CNN techniques for precise crown area and shape estimation, highlighting the need for future research to refine algorithms for improved handling of crown size variability.

Aluminum Reservoir Welding Surface Defect Detection Method Based on Three-Dimensional Vision

Welding is an important process in the production of aluminum reservoirs for motor vehicles. The welding quality affects product performance. However, rapid and accurate detection of weld surface defects remains a huge challenge in the field of industrial automation. To address this problem, we proposed a 3D vision-based aluminum reservoir welding surface defect detection method. First of all, a scanning system based on laser line scanning camera was constructed to acquire the point cloud data of weld seams on the aluminum reservoir surface. Next, a planar correction algorithm was used to adjust the slope of the contour line according to the slope of the contour line in order to minimize the effect of systematic disturbances when acquiring weld data. Then, the surface features of the weld, including curvature and normal vector direction, were extracted to extract holes, craters, and undercut defects. For better extraction of the defect, a double-aligned template matching method was used to ensure comprehensive extraction and measurement of defect areas. Finally, the detected defects were categorized according to their morphology. Experimental results show that the proposed method using 3D laser scanning data can detect and classify typical welding defects with an accuracy of more than 97.1%. Furthermore, different types of defects, including holes, undercuts, and craters, can also be accurately detected with precision 98.9%.

Automated karyogram analysis for early detection of genetic and neurodegenerative disorders: a hybrid machine learning approach

Anomalous chromosomes are the cause of genetic diseases such as cancer, Alzheimer's, Parkinson's, epilepsy, and autism. Karyotype analysis is the standard procedure for diagnosing genetic disorders. Identifying anomalies is often costly, time-consuming, heavily reliant on expert interpretation, and requires considerable manual effort. Efforts are being made to automate karyogram analysis. However, the unavailability of large datasets, particularly those including samples with chromosomal abnormalities, presents a significant challenge. The development of automated models requires extensive labeled and incredibly abnormal data to accurately identify and analyze abnormalities, which are difficult to obtain in sufficient quantities. Although the deep learning-based architecture has yielded state-of-the-art performance in medical image anomaly detection, it cannot be generalized well because of the lack of anomalous datasets. This study introduces a novel hybrid approach that combines unsupervised and supervised learning techniques to overcome the challenges of limited labeled data and scalability in chromosomal analysis. An Autoencoder-based system is initially trained with unlabeled data to identify chromosome patterns. It is fine-tuned on labeled data, followed by a classification step using a Convolutional Neural Network (CNN). A unique dataset of 234,259 chromosome images, including the training, validation, and test sets, was used. Marking a significant achievement in the scale of chromosomal analysis. The proposed hybrid system accurately detects structural anomalies in individual chromosome images, achieving 99.3% accuracy in classifying normal and abnormal chromosomes. We also used a structural similarity index measure and template matching to identify the part of the abnormal chromosome that differed from the normal one. This automated model has the potential to significantly contribute to the early detection and diagnosis of chromosome-related disorders that affect both genetic health and neurological behavior.

Research on Concrete Beam Damage Detection Using Convolutional Neural Networks and Vibrations from ABAQUS Models and Computer Vision

Researchers have already used vibration data and deep learning methods, such as Convolutional Neural Networks (CNNs), to detect structural damage. Moreover, some researchers have employed image-based displacement sensors (such as the template matching and edge detection methods) to obtain structural vibration information. It is necessary to verify whether deep learning methods can detect minor damage inside beams, for example, small hollowing in concrete. In addition, there is an urgent need to develop an effective image-based displacement sensor that can simultaneously detect a large number of reliable vibration data from different measurement points. In this study, the vibration data of two beam-ABAQUS models were used as the input data for a newly designed deep learning-based structural health monitoring method. There were 500 vibration samples for each case, and the peak of vibrations was several millimeters. The proposed CNN model can locate damage positions in beams with high accuracy (close to 100%), and the damage sizes are 3 cm and 6 cm. Laboratory experiments were carried out on four beams with different damage. The optimized displacement sensor developed based on the edge detection method was used to detect the displacement of the beams. Each beam had 200 vibration data, and there were 800 vibration data in total. These vibration data were used as input data to train the proposed deep learning architecture, and satisfactory accuracy was achieved in detecting the damage of the beams with an accuracy of 97%. The training process is satisfactory in that the training loss and validation loss dropped very quickly.

A fast monocular 6D pose estimation method for textureless objects based on perceptual hashing and template matching

A fast monocular 6D pose estimation method for textureless objects based on perceptual hashing and template matching

Multisensory naturalistic decoding with high-density diffuse optical tomography.

Decoding naturalistic content from brain activity has important neuroscience and clinical implications. Information about visual scenes and intelligible speech has been decoded from cortical activity using functional magnetic resonance imaging (fMRI) and electrocorticography, but widespread applications are limited by the logistics of these technologies. High-density diffuse optical tomography (HD-DOT) offers image quality approaching that of fMRI but with the silent, open scanning environment afforded by optical methods, thus opening the door to more naturalistic research and applications. Although early visual decoding studies with HD-DOT have been promising, decoding of naturalistic auditory and multisensory stimulus information from HD-DOT data has not been established. Audiovisual decoding was investigated using HD-DOT data collected from participants who viewed a library of movie clips. A template-matching strategy was used to decode which movie clip a participant viewed based on their HD-DOT data. Factors affecting decoding performance-including trial duration and number of decoding choices-were systematically evaluated. Decoding accuracy was 94.2% for four-way decoding utilizing 4min of data per trial as a starting point. As parameters were made more stringent, decoding performance remained significantly above chance with strong effect sizes down to 15-s trials and up to 32 choices. Comparable decoding accuracies were obtained when cortical sampling was confined to visual and auditory regions and when participants were presented with purely auditory or visual clips. HD-DOT data sample cortical hemodynamics with sufficient resolution and fidelity to support decoding complex, naturalistic, multisensory stimuli via template matching. These results provide a foundation for future studies on more intricate decoding algorithms to reconstruct diverse features of novel naturalistic stimuli from HD-DOT data.

Cosine Similarity Template Matching Networks for Optical and SAR Image Registration

Cosine Similarity Template Matching Networks for Optical and SAR Image Registration

Precision in planetary exploration: Crater detection with residual U-Net34/50 and matching template algorithm

Precision in planetary exploration: Crater detection with residual U-Net34/50 and matching template algorithm

Accelerating the Discovery of Abiotic Vesicles with AI-Guided Automated Experimentation.

The first protocells are speculated to have arisen from the self-assembly of simple abiotic carboxylic acids, alcohols, and other amphiphiles into vesicles. To study the complex process of vesicle formation, we combined laboratory automation with AI-guided experimentation to accelerate the discovery of specific compositions and underlying principles governing vesicle formation. Using a low-cost commercial liquid handling robot, we automated experimental procedures, enabling high-throughput testing of various reaction conditions for mixtures of seven (7) amphiphiles. Multitemplate multiscale template matching (MMTM) was used to automate confocal microscopy image analysis, enabling us to quantify vesicle formation without tedious manual counting. The results were used to create a Gaussian process surrogate model, and then active learning was used to iteratively direct the laboratory experiments to reduce model uncertainty. Mixtures containing primarily trimethyl decylammonium and decylsulfate in equal amounts formed vesicles at submillimolar critical vesicle concentrations, and more than 20% glycerol monodecanoate prevented vesicles from forming even at high total amphiphile concentrations.

Research on wave measurement and simulation experiments of binocular stereo vision based on intelligent feature matching

Waves are crucial in ocean observation and research. Stereo vision-based wave measurement, offering non-contact, low-cost, and intelligent processing, is an emerging method. However, improving accuracy remains a challenge due to wave complexity. This paper presents a novel approach to measure wave height, period, and direction by combining deep learning-based stereo matching with feature matching techniques. To improve the discontinuity and low accuracy in disparity maps from traditional wave image matching algorithms, this paper proposes the use of a high-precision stereo matching method based on Pyramid Stereo Matching Network (PSM-Net).A 3D reconstruction method integrating Scale-Invariant Feature Transform (SIFT) with stereo matching was also introduced to overcome the limitations of template matching and interleaved spectrum methods, which only provide 2D data and fail to capture the full 3D motion of waves. This approach enables accurate wave direction measurement. Additionally, a six-degree-of-freedom platform was proposed to simulate waves, addressing the high costs and attenuation issues of traditional wave tank simulations. Experimental results show the prototype system achieves a wave height accuracy within 5%, period accuracy within 4%, and direction accuracy of ±2°, proving the method’s effectiveness and offering a new approach to stereo vision-based wave measurement.

Two-stage unmanned aerial vehicle localization method based on multi-category semantic segmentation and template matching

Two-stage unmanned aerial vehicle localization method based on multi-category semantic segmentation and template matching

Railway reconstruction from 3D point cloud using Deep Learning and Parametric Modeling

Abstract. Railway infrastructure is crucial for transporting goods and passengers, making its maintenance and reconstruction vital for safety and reliability. Traditional methods reliant on manual surveys are time-consuming and prone to inaccuracies. Although 3D point cloud data provides detailed representations of railway environments, its unstructured nature complicates processing and modeling. This paper presents a methodology that combines deep learning with parametric modeling to reconstruct railway environments from 3D point cloud data, focusing on key components such as rails, catenary wires and poles. The results are represented in a standardized CityJSON format, in compliance with the Transportation module of CityGML 3.0, and textured to create photo-realistic 3D railway models. The proposed approach uses the KPConv (Kernel Point Convolution) architecture for semantic segmentation to classify railway components. The model is trained on Rail3D dataset and achieved a mean Intersection-over-Union (mIoU) of 84%. Instance segmentation of catenary poles is performed using Label Connected Components (LCC) algorithm, followed by a second-level classification through template matching using Fast Global Registration (FGR) and Iterative Closest Point (ICP). Rail reconstruction combines Region Growing and H-DBSCAN algorithms for clustering, vectorization for linear geometry extraction, and extension to ensure continuity despite gaps or noise in the data. Catenary poles are reconstructed using parametric models, taking as input a scale factor and a rotation matrix calculated from the extracted height and azimuth. Wires are added accordingly to connect the reconstructed poles. The methodology was validated on Belgian railway data, producing accurate, interoperable and photo-realistic 3D models suitable for digital twin integration, infrastructure monitoring and urban simulations.

An Algorithm for Automatic Low-contrast Detection System and Its Evaluation for Images with Various Phantom Rotations

The purpose of this study is to develop an algorithm for automatic low-contrast object detection on the ACR 464 CT phantom and to investigate its evaluation for images with various phantom rotations. A software for automatic low-contrast detection was implemented with MATLAB R2013a. An algorithm was based on a template matching method. The reference points for the template matching method was centers of phantom and largest low-contrast object. The centers of the phantom and the largest low-contrast object were calculated using centroid formulae from the segmented phantom and objects with specific threshold values. Region of interests (ROIs) were located at each low-contrast object and background. The mean CT number and noise were calculated from pixel values within each ROI. The contrast-to-noise ratio (CNR) was then calculated based on the contrast between low-contrast object and background. The CNR cut-off is one. Object that has a CNR value more than the cut-off is considered resolved object and object that has a CNR value less than the cut-off is considered unresolved object. The algorithm was evaluated on images of the ACR CT phantom with various rotations of 0°, 22.5°, 45°, and 60°. Statistic evaluation was performed by ANOVA one-way to compare results of mean CT number, noise, and CNR for various rotations. The p-value more than 0.05 indicated that there is no significant difference. The proposed method was successful in placing ROIs at all low-contrast objects and at background for various phantom rotations. The CNR increases with the increase of size of the low-contrast object. The p-values for contrast, noise and CNR were 0.99, indicating that there are no statistically significant differences for various rotations. The minimum resolved object detectability in various rotations was 4 mm. An automatic technique for detecting low-contrast objects is accurate for various rotations.

Research Advanced in Human Pose Estimation based on Deep Learning

Human pose estimation constitutes a crucial task within the realm of computer vision. Its objective is to precisely identify and locate the main features of the human body as shown in images or videos, thereby enabling the understanding and analysis of human behavior. In the early stages, pose estimation efforts predominantly focused on single-person scenes, relying mainly on template matching, direct regression, and detection methods. Thanks to the rapid progress of deep learning technologies, research on human pose estimation has evolved from single-person scenarios to complex multi-person ones. Multi-person pose estimation can be separated into top-down and bottom-up approaches. The top-down method initially detects the human body and then conducts pose estimation. Although relatively straightforward to implement, it has limitations when dealing with multi-person scenes and occlusion situations. In contrast, the bottom-up method first detects main features and then assigns them to different individuals, proving more effective in handling occlusion and multi-person scenes. Considering the above factors, this paper undertakes a comprehensive and in-depth study on human pose estimation based on deep learning, covering multiple aspects such as single-person pose estimation, multi-person pose estimation, datasets, evaluation metrics, algorithms, and experimental methods. Additionally, it discusses the existing problems and future development directions from perspectives including improving the model architecture, optimizing data processing, and integrating multi-modal information. Moreover, recent advancements in deep learning architectures, including convolutional neural networks and recurrent neural networks, have further enhanced the performance and accuracy of human pose estimation. The combination of different modalities, such as depth information and infrared imaging, also holds promise for addressing some of the challenges in complex scenes.

A clot waveform analysis-based system for differential diagnosis of prolonged activated partial thromboplastin time in plasma samples.

Prolonged activated partial thromboplastin time (aPTT) in plasma samples requires quick and accurate differential diagnosis. We developed two methods using clot waveform analysis (CWA) for plasma samples with prolonged aPTT, particularly for factor (F)VIII deficiency. One method estimates FVIII activity (FVIII:C) using CWA without measuring FVIII:C by template matching. The other utilizes CWA in the mixing test to quickly differentiate FVIII deficiency (FD), FVIII inhibitor (Inh), and lupus anticoagulant (LA). To establish a more accurate system for differential diagnosis of aPTT prolongation, including FIX deficiency, by combining a CWA-based mixing test and template matching. Samples with FD (n = 96), LA (n = 19), and Inh (n = 28) were incubated with normal plasma. FD, LA, and Inh were differentiated using a mixing test, followed by CWA-based template matching. In the mixing test, sensitivity for FD, Inh, and LA was 100%, 93%, and 100%, and specificity was 96%, 100%, and 100%. Samples with FIX inhibitor (> 0.6 BU/mL) were differentiated as the inhibitor sample. In template matching, almost all severe hemophilia A and B were judged as the respective severe types. This novel CWA-based measurement system could aid in differential diagnosis of prolonged aPTT.

Research on a Panoramic Image Stitching Method for Images of Corn Ears, Based on Video Streaming

Background: Corn is the main grain crop grown in China, and the ear shape index of corn is an important parameter for breeding new varieties, including ear length, diameter, row number of ears, row number of grains per ear, and so on. Objective: In order to solve the problem of limited field of view associated with computer detection of the corn ear shape index against a complex background, this paper proposes a panoramic splicing method for corn ears against a complex background, which can splice 10 corn ear panoramic images at the same time, to improve information collection efficiency, display comprehensive information, and support data analysis, so as to realize automatic corn seed examination. Methods: A summary of corn ear panoramic stitching methods under complex backgrounds is presented as follows: 1. a perceptual hash algorithm and histogram equalization were used to extract video frames; 2. the U-Net image segmentation model based on transfer learning was used to predict corn labels; 3. a mask preprocessing algorithm was designed; 4. a corn ear splicing positioning algorithm was designed; 5. an algorithm for irregular surface expansion was designed; 6. an image stitching method based on template matching was adopted to assemble the video frames. Results: The experimental results showed that the proposed corn ear panoramic stitching method could effectively solve the problems of virtual stitching, obvious stitching seams, and too-high similarity between multiple images. The success rate of stitching was as high as 100%, and the speed of single-corn-ear panoramic stitching was about 9.4 s, indicating important reference value for corn breeding and disease and insect detection. Discussions: Although the experimental results demonstrated the significant advantages of the panoramic splicing method for corn ear images proposed in this paper in terms of improving information collection efficiency and automating corn assessment, the method still faces certain challenges. Future research will focus on the following points: 1. addressing the issue of environmental interference caused by diseases, pests, and plant nutritional status on the measurement of corn ear parameters in order to enhance the stability and accuracy of the algorithm; 2. expanding the dataset for the U-Net model to include a wider range of corn ears with complex backgrounds, different growth stages, and various environmental conditions to improve the model’s segmentation recognition rate and precision. Recently, our panoramic splicing algorithm has been deployed in practical applications with satisfactory results. We plan to continue optimizing the algorithm and more broadly promote its use in fields such as corn breeding and pest and disease detection in an effort to advance the development of agricultural automation technology.

HSF-IBI: A Universal Framework for Extracting Inter-Beat Interval from Heterogeneous Unobtrusive Sensors.

Heartbeat inter-beat interval (IBI) extraction is a crucial technology for unobtrusive vital sign monitoring, yet its precision and robustness remain challenging. A promising approach is fusing heartbeat signals from different types of unobtrusive sensors. This paper introduces HSF-IBI, a novel and universal framework for unobtrusive IBI extraction using heterogeneous sensor fusion. Specifically, harmonic summation (HarSum) is employed for calculating the average heart rate, which in turn guides the selection of the optimal band selection (OBS), the basic sequential algorithmic scheme (BSAS)-based template group extraction, and the template matching (TM) procedure. The optimal IBIs are determined by evaluating the signal quality index (SQI) for each heartbeat. The algorithm is morphology-independent and can be adapted to different sensors. The proposed algorithm framework is evaluated on a self-collected dataset including 19 healthy participants and an open-source dataset including 34 healthy participants, both containing heterogeneous sensors. The experimental results demonstrate that (1) the proposed framework successfully integrates data from heterogeneous sensors, leading to detection rate enhancements of 6.25 % and 5.21 % on two datasets, and (2) the proposed framework achieves superior accuracy over existing IBI extraction methods, with mean absolute errors (MAEs) of 5.25 ms and 4.56 ms on two datasets.

Quantitative Spatial Analysis of Chromatin Biomolecular Condensates using Cryo-Electron Tomography.

Phase separation is an important mechanism to generate certain biomolecular condensates and organize the cell interior. Condensate formation and function remain incompletely understood due to difficulties in visualizing the condensate interior at high resolution. Here we analyzed the structure of biochemically reconstituted chromatin condensates through cryo-electron tomography. We found that traditional blotting methods of sample preparation were inadequate, and high-pressure freezing plus focused ion beam milling was essential to maintain condensate integrity. To identify densely packed molecules within the condensate, we integrated deep learning-based segmentation with novel context-aware template matching. Our approaches were developed on chromatin condensates, and were also effective on condensed regions of in situ native chromatin. Using these methods, we determined the average structure of nucleosomes to 6.1 and 12 Å resolution in reconstituted and native systems, respectively, and found that nucleosomes have a nearly random orientation distribution in both cases. Our methods should be applicable to diverse biochemically reconstituted biomolecular condensates and to some condensates in cells.