Template Matching Research Articles

Bounding Box and Thresholding in Optical Character Recognition for Car License Plate Recognition

License plate recognition plays a central role in a variety of application contexts, including traffic management, automated parking, and law enforcement. Among the various approaches available, the Optical Character Recognition (OCR) technique has proven its effectiveness in recognizing characters in license plate images. This study describes an approach for detecting and recognizing vehicle license plates by utilizing the OCR method with Bounding Box, Thresholding, and template matching. In addition, this study uses MATLAB R2022a software as the main tool in developing and implementing the method. The goal is to recognize vehicle license plates from images, describe their characteristics, and generate relevant information. This approach involves a series of image processing steps starting with the pre-processing stage, followed by the process of binarization and license plate segmentation. After successfully isolating the license plate area, isolating the character using a bounding box is performed using image separation techniques. The OCR method is used to recognize license plate characters through comparison using the correlation method. Through a series of experiments on several image datasets, this approach succeeded in showing that out of 20 sampled license plate images, the results obtained were a reading accuracy of 93.55% of 100%, recognizing 13 out of 20 license plate images accurately when tested. Thus, the findings of this research are expected to contribute to the recognition of vehicle license plates that are accurate and efficient, by utilizing image processing techniques and OCR methods implemented using MATLAB R2022a software.

Improving pulsar polarization and timing measurements with the Nançay Radio Telescope

Context. Accurate polarimetric calibration of the radio pulse profiles from pulsars is crucial for studying their radiation properties at these wavelengths. Additionally, inaccurate calibration can distort recorded pulse profiles, introducing noise in time of arrival (TOA) data and thus degrading pulsar timing analyses. One method for determining the full polarimetric response of a given telescope is to conduct observations of bright polarized pulsars over wide ranges of parallactic angles, to sample different orientations of their polarization angle and in turn determine the cross-couplings between polarization feeds. Aims. The Nançay decimetric Radio Telescope (NRT) is a 94 m equivalent meridian telescope, capable of tracking a given pulsar for approximately 1 h around transit. The NRT therefore cannot sample wide ranges of parallactic angles when observing a given pulsar, so until late 2019 the polarimetric calibration of 1.4 GHz pulsar observations with the NRT was rudimentary. We therefore aimed to develop a method for improving the calibration of NRT observations, overcoming the above-mentioned limitation. Ultimately, our goal was to improve the quality of NRT pulsar timing, with better calibrated pulsar pulse profiles. Methods. In November 2019, we began conducting regular observations of the bright and highly linearly polarized pulsar PSR J0742−2822, in a special observing mode in which the feed horn rotates by ~180° over the course of the 1 h observation, mimicking wide parallactic angle variations and in principle enabling us to determine the polarimetric response of the NRT at 1.4 GHz. In addition, we assessed the quality of the NRT timing of a selection of millisecond pulsars (MSPs), namely, J1730−2304, J1744−1134, and J1857+0953, with conventional TOAs extracted from total intensity pulse profiles, and TOAs extracted with the Matrix Template Matching (MTM) technique, designed to compensate for putative polarimetric calibration errors. Results. From the analysis of the rotating horn observations of PSR J0742−2822 we could determine the cross-couplings between the polarization feeds and also constrain the Stokes parameters of the noise diode signal, which prior to this work was erroneously assumed to be ideal and was used as the only reference source for the calibration of pulsar observations. The improved polarimetric response of the NRT as determined from these observations was applied to observations of a selection of MSPs with published polarimetric properties. We find that the new polarimetric profiles and polarization position angles are consistent with previous findings, unlike NRT polarimetric results obtained with the previously used method of calibration. The analysis of the timing data shows that the new calibration method improves the quality of the timing, and the MTM method proves very effective at reducing noise from imperfect calibration. For pulsars with sufficient degrees of polarization, the MTM method appears to be the preferred method of extracting TOAs from NRT observations.

Robust crystal structure identification at extreme conditions using a density-independent spectral descriptor and supervised learning

The increased time- and length-scale of classical molecular dynamics simulations have led to raw data flows surpassing storage capacities, necessitating on-the-fly integration of structural analysis algorithms. As a result, algorithms must be computationally efficient, accurate, and stable at finite temperature to reliably extract the relevant features of the data at simulation time. In this work, we leverage spectral descriptors to encode local atomic environments and build crystal structure classification models. In addition to the classical way spectral descriptors are computed, i.e. over a fixed radius neighborhood sphere around a central atom, we propose an extension to make them independent from the material’s density. Models are trained on defect-free crystal structures with moderate thermal noise and elastic deformation, using the linear discriminant analysis (LDA) method for dimensionality reduction and logistic regression (LR) for subsequent classification. The proposed classification model is intentionally designed to be simple, incorporating only a limited number of parameters. This deliberate simplicity enables the model to be trained effectively even when working with small databases. Despite the limited training data, the model still demonstrates inherent transferability, making it applicable to a broader range of scenarios and datasets. The accuracy of our models in extreme conditions (high temperature, high density, large deformation) is compared to traditional algorithms from the literature, namely adaptive common neighbor analysis (a-CNA), polyhedral template matching (PTM) and diamond structure identification (IDS). Finally, we showcase two applications of our method: tracking a solid–solid BCC-to-HCP phase transformation in Zirconium at high pressure up to high temperature, and visualizing stress-induced dislocation loop expansion in single crystal FCC Aluminum containing a Frank–Read source, at high temperature.

A study on the Detection and Visualization of Buried Objects Using 2.5D Template Matching Techniques

A study on the Detection and Visualization of Buried Objects Using 2.5D Template Matching Techniques

A Review of Face Detection Methods Based on Feature Approach

Face detection technology is the first and essential step for facial-based analysis algorithms such as face recognition, face feature extraction, face alignment, face enhancement, and face parsing. That is besides serving other applications related to human intention and act analysis such as facial expression recognition, gender recognition, and age classification. Face detection is used to detect faces in digital images. It is a special case of object detection and can be used in many areas such as biometrics, security, law enforcement, entertainment, and personal safety. There are various methods proposed in the field of face detection, and they all compete to make it more advanced and accurate. These methods belong to two main approaches; feature-based approach and image-based approach. This paper reviews the face detection methods that belong to the feature-based approach. Moreover, their work concepts, strengths, and limitations are mentioned. This paper concentrates on the feature-based approach due to its simplicity and high applicability in real-time detection compared to the image-based approach. Index Terms— Face Detection Methods, Feature-Based Approach, Viola-Jones Algorithm, Deformable Template Matching.

ECG-Free Heartbeat Detection in Seismocardiography and Gyrocardiography Signals Provides Acceptable Heart Rate Variability Indices in Healthy and Pathological Subjects.

Cardio-mechanical monitoring techniques, such as Seismocardiography (SCG) and Gyrocardiography (GCG), have received an ever-growing interest in recent years as potential alternatives to Electrocardiography (ECG) for heart rate monitoring. Wearable SCG and GCG devices based on lightweight accelerometers and gyroscopes are particularly appealing for continuous, long-term monitoring of heart rate and its variability (HRV). Heartbeat detection in cardio-mechanical signals is usually performed with the support of a concurrent ECG lead, which, however, limits their applicability in standalone cardio-mechanical monitoring applications. The complex and variable morphology of SCG and GCG signals makes the ECG-free heartbeat detection task quite challenging; therefore, only a few methods have been proposed. Very recently, a template matching method based on normalized cross-correlation (NCC) has been demonstrated to provide very accurate detection of heartbeats and estimation of inter-beat intervals in SCG and GCG signals of pathological subjects. In this study, the accuracy of HRV indices obtained with this template matching method is evaluated by comparison with ECG. Tests were performed on two public datasets of SCG and GCG signals from healthy and pathological subjects. Linear regression, correlation, and Bland-Altman analyses were carried out to evaluate the agreement of 24 HRV indices obtained from SCG and GCG signals with those obtained from ECG signals, simultaneously acquired from the same subjects. The results of this study show that the NCC-based template matching method allowed estimating HRV indices from SCG and GCG signals of healthy subjects with acceptable accuracy. On healthy subjects, the relative errors on time-domain indices ranged from 0.25% to 15%, on frequency-domain indices ranged from 10% to 20%, and on non-linear indices were within 8%. The estimates obtained on signals from pathological subjects were affected by larger errors. Overall, GCG provided slightly better performances as compared to SCG, both on healthy and pathological subjects. These findings provide, for the first time, clear evidence that monitoring HRV via SCG and GCG sensors without concurrent ECG is feasible with the NCC-based template matching method for heartbeat detection.

Automating Assessment and Providing Personalized Feedback in E-Learning: The Power of Template Matching

This research addressed the need to enhance template-matching performance in e-learning and automated assessments within Egypt’s evolving educational landscape, marked by the importance of e-learning during the COVID-19 pandemic. Despite the widespread adoption of e-learning, robust template-matching feedback mechanisms should still be developed for personalization, engagement, and learning outcomes. This study augmented the conventional best-buddies similarity (BBS) approach with four feature descriptors, Harris, scale-invariant feature transform (SIFT), speeded-up robust features (SURF), and maximally stable extremal regions (MSER), to enhance template-matching performance in e-learning. We systematically selected algorithms, integrated them into enhanced BBS schemes, and assessed their effectiveness against a baseline BBS approach using challenging data samples. A systematic algorithm selection process involving multiple reviewers was employed. Chosen algorithms were integrated into enhanced BBS schemes and rigorously evaluated. The results showed that the proposed schemes exhibited enhanced template-matching performance, suggesting potential improvements in personalization, engagement, and learning outcomes. Further, the study highlights the importance of robust template-matching feedback in e-learning, offering insights into improving educational quality. The findings enrich e-learning experiences, suggesting avenues for refining e-learning platforms and positively impacting the Egyptian education sector.

Near-Infrared Laser Methane Remote Monitoring Based on Template Matching Algorithm of Harmonic Signals

Trustworthy technology for the monitoring of fugitive methane emissions is an indispensable component for low−carbon emission reduction and safe production in oil and gas infrastructure. A miniaturization and low-cost methane remote monitoring prototype based on near-infrared laser absorption spectroscopy is developed to retrieve the path−integral concentration by employing the harmonic conjoint analysis method for the backscattered echo signals from a distant non-cooperative target. A distorted harmonic template matching (DHTM) algorithm is proposed based on wavelength modulation spectroscopy with second harmonic normalized via the first harmonic (WMS−2f/1f) method, which suppresses the output concentration fluctuation caused by open path monitoring with non-cooperative target and avoids the issue of false alarms during the detection process without target gas. The reliability of the WMS−2f/1f−DHTM algorithm is verified by calibration and dynamic experiments. The results showed that this algorithm avoids the dilemma of false alarms in the absence of gas compared to the conventional WMS−2f/1f algorithm, while the root mean square error (RMSE) of the concentration inversion with a detection distance of 20 m is reduced by 57.6% compared to direct absorption spectroscopy (DAS) algorithm. And the minimum detection limit of system is 3.79 ppm·m. The methane telemetry sensor with the WMS−2f/1f−DHTM algorithm exhibits substantial application potential in carbon monitoring of oil and gas industry.

Design and Verification of a Novel Structural Strain Measuring Method Based on Template Matching and Microscopic Vision

Strain measurements have a significant role in evaluating the condition of various structural types and have become an essential component in the area of structural health monitoring. However, there are some limitations in the current means of strain measurement, and this study aims to improve these methods. We have designed a novel strain measurement method based on template matching algorithms and microscopic vision techniques, developed a new sliding strain sensor, and paired it with a new microscope to realize strain measurement. The method has the function of remote wireless acquisition with a cell phone, which is more widely applicable. In the laboratory performance testing, the zero drift of the sensor is mainly concentrated in the fluctuation range of ±2 με, and the effective range reaches nearly 40,000 με. In the comparison experiments with the linear variable differential transformer, the maximum error of the static loading is only 5 με, and the maximum error rate of the dynamic loading is less than 1%, which proves that it has a relatively high accuracy. Finally, the short-term real-time monitoring of the local structure of the footbridge was accomplished, and the strain changes on the surface of the structure were captured instantly, stably, and efficiently in the actual measurements. The proposed strain measurement system has the advantages of high accuracy, a low cost, convenient measurement, and wide applicability, and it provides a novel alternative means for strain measurement in the field of structural health monitoring.

Robust shape-based template matching algorithm for target localization under SEM.

A novel acceleration algorithm for geometric template matching is proposed based on the Cauchy-Schwartz inequality (C-S inequality) in this paper. The proposed approach is validated under a scanning electron microscope (SEM), and its effectiveness is demonstrated. In this approach, the object shape features are represented as column vectors with complex elements. Then, a threshold is determined to exclude the sliding windows without any matched objects. Finally, surface fitting is employed to obtain the subpixel positions of the targets, and least squares adjustment is utilized to fine-tune the obtained results. The experimental results demonstrate that the proposed method can significantly reduce the matching time by 59% - 96% compared with the traditional shape-based method. Furthermore, the strong robustness and high accuracy are verified under different disturbances. Additionally, the approach is shown to be robust and accurate under different types of disturbances, thus confirming its suitability for real-time targets tracking on SEM with high accuracy.

A Generic, Multimodal Geospatial Data Alignment System for Aerial Navigation

We present a template matching algorithm based on local descriptors for aligning two geospatial products of different modalities with a large area asymmetry. Our system is generic with regards to the modalities of the geospatial products and is applicable to the self-localization of aerial devices such as drones and missiles. This algorithm consists in finding a superposition such that the average dissimilarity of the superposed points is minimal. The dissimilarity of two points belonging to two different geospatial products is the distance between their respective local descriptors. These local descriptors are learned. We performed experiments consisting in estimating a translation between optical (Pléiades) and SAR (Miranda) images onto vector data (OpenStreetMap), onto optical images (DOP) and onto SAR images (KOMPSAT-5). Each remote sensing image to be aligned covered 0.64 km2, and each reference geospatial product spanned over 225 km2. We conducted a total of 381 alignment experiments, with six unique modality combinations. In aggregate, the precision reached was finer than 10 m with 72% probability and finer than 20 m with 96% probability. This is considerably more than with traditional methods such as normalized cross-correlation and mutual information.

A Real-Time Eye Tracking System for the Detection of Eye Blinks.

This paper proposes an eye blink detection system that automatically detects eye blinks, which can be an indicator of fatigue or cognitive load, among others. As a key feature, the real-time capability of the system is being required to use it, for example, as a monitoring system for people in potentially critical situations (e.g., drivers or operators of heavy machinery). The system uses the Viola-Jones algorithm for face detection and the median flow tracker to track the face in video sequences. Eye detection is implemented using face proportions, and template matching is used for blink detection. The resulting system processes 40-47 frames per second on default consumer hardware and achieves an accuracy of 80.33% and a precision of 85.22% in the evaluation. The proposed system shows promising results under ideal viewing conditions but has difficulty maintaining high precision during head movements. The proposed system could be integrated with various health-related assistance systems to monitor the individual's well-being in real time, as long as their head is observed from the front if possible.

Self-supervised Vision Transformers for 3D pose estimation of novel objects

Object pose estimation is important for object manipulation and scene understanding. In order to improve the general applicability of pose estimators, recent research focuses on providing estimates for novel objects, that is, objects unseen during training. Such works use deep template matching strategies to retrieve the closest template connected to a query image, which implicitly provides object class and pose. Despite the recent success and improvements of Vision Transformers over CNNs for many vision tasks, the state of the art uses CNN-based approaches for novel object pose estimation. This work evaluates and demonstrates the differences between self-supervised CNNs and Vision Transformers for deep template matching. In detail, both types of approaches are trained using contrastive learning to match training images against rendered templates of isolated objects. At test time such templates are matched against query images of known and novel objects under challenging settings, such as clutter, occlusion and object symmetries, using masked cosine similarity. The presented results not only demonstrate that Vision Transformers improve matching accuracy over CNNs but also that for some cases pre-trained Vision Transformers do not need fine-tuning to achieve the improvement. Furthermore, we highlight the differences in optimization and network architecture when comparing these two types of networks for deep template matching.

RadarSense: Accurate Recognition of Mid-air Hand Gestures with Radar Sensing and Few Training Examples

Microwave radars bring many benefits to mid-air gesture sensing due to their large field of view and independence from environmental conditions, such as ambient light and occlusion. However, radar signals are highly dimensional and usually require complex deep learning approaches. To understand this landscape, we report results from a systematic literature review of ( N =118) scientific papers on radar sensing, unveiling a large variety of radar technology of different operating frequencies and bandwidths and antenna configurations but also various gesture recognition techniques. Although highly accurate, these techniques require a large amount of training data that depend on the type of radar. Therefore, the training results cannot be easily transferred to other radars. To address this aspect, we introduce a new gesture recognition pipeline that implements advanced full-wave electromagnetic modeling and inversion to retrieve physical characteristics of gestures that are radar independent, i.e., independent of the source, antennas, and radar-hand interactions. Inversion of radar signals further reduces the size of the dataset by several orders of magnitude, while preserving the essential information. This approach is compatible with conventional gesture recognizers, such as those based on template matching, which only need a few training examples to deliver high recognition accuracy rates. To evaluate our gesture recognition pipeline, we conducted user-dependent and user-independent evaluations on a dataset of 16 gesture types collected with the Walabot, a low-cost off-the-shelf array radar. We contrast these results with those obtained for the same gesture types collected with an ultra-wideband radar made of a vector network analyzer with a single horn antenna and with a computer vision sensor, respectively. Based on our findings, we suggest some design implications to support future development in radar-based gesture recognition.

Registration of Large Optical and SAR Images with Non-Flat Terrain by Investigating Reliable Sparse Correspondences

Optical and SAR image registration is the primary procedure to exploit the complementary information from the two different image modal types. Although extensive research has been conducted to narrow down the vast radiometric and geometric gaps so as to extract homogeneous characters for feature point matching, few works have considered the registration issue for non-flat terrains, which will bring in more difficulties for not only sparse feature point matching but also outlier removal and geometric relationship estimation. This article addresses these issues with a novel and effective optical-SAR image registration framework. Firstly, sparse feature points are detected based on the phase congruency moment map of the textureless SAR image (SAR-PC-Moment), which helps to identify salient local regions. Then a template matching process using very large local image patches is conducted, which increases the matching accuracy by a significant margin. Secondly, a mutual verification-based initial outlier removal method is proposed, which takes advantage of the different mechanisms of sparse and dense matching and requires no geometric consistency assumption within the inliers. These two procedures will produce a putative correspondence feature point (CP) set with a low outlier ratio and high reliability. In the third step, the putative CPs are used to segment the large input image of non-flat terrain into dozens of locally flat areas using a recursive random sample consensus (RANSAC) method, with each locally flat area co-registered using an affine transformation. As for the mountainous areas with sharp elevation variations, anchor CPs are first identified, and then optical flow-based pixelwise dense matching is conducted. In the experimental section, ablation studies using four precisely co-registered optical-SAR image pairs of flat terrain quantitatively verify the effectiveness of the proposed SAR-PC-Moment-based feature point detector, big template matching strategy, and mutual verification-based outlier removal method. Registration results on four 1 m-resolution non-flat image pairs prove that the proposed framework is able to produce robust and quite accurate registration results.

How to identify pollen like a palynologist: A prior knowledge-guided deep feature learning for real-world pollen classification

Airborne pollen identification is crucial to help patients prevent pollinosis symptoms. Existing data-driven methods rely on large-scale pollen images with simple backgrounds. In real scenarios, the background is complex and the data scale is small. Therefore, these methods suffer from two challenges: (1) Irrelevant information interference; (2) Incomplete feature attention. To overcome these challenges, we propose a prior knowledge-guided deep feature learning (PK-DFL) for real-world optical microscope image classification. Its main steps are as follows: Pollen location is designed to locate pollen grains based on color features, aiming to boost the accuracy of shape and texture prior feature extraction. Shape-texture awareness helps to extract the shape and texture of pollen grains via predefined feature extractors (i.e., a set of shape descriptors and an improved SFTA). These features are used to construct two types of prior knowledge, namely shape-texture attention maps (STA maps) and shape-texture feature vectors (STF vectors). Pollen classification uses a deep network (CNN) to classify pollen via imitating the pollen identification procedure of palynologists. It uses STA maps to weight pollen images and convolutional feature maps for instructing the CNN to focus on critical areas of pollen images (for the first challenge). STF vectors are employed to obtain the inter-class similarity of pollen via template matching. This information is further converted to soft targets that are used to supervise the CNN attending to comprehensive key features (for the second challenge). Extensive experiments on real-world datasets demonstrate the effectiveness of our PK-DFL (with accuracy and F1-score over 88%).

Construction of an innovative model of English online teaching based on a multimedia English learning system model

Abstract This paper explores a multimedia learning system suitable for students’ independent English learning through the current development of assisted English learning systems, university-level English teaching requirements, and learner characteristics. Based on the model of the proposed multimedia English learning system, in the endpoint detection stage of speech, short-time excess rate and short-time energy are combined to process the endpoint detection of speech. In the final speech comparison stage, the dynamic time planning algorithm DTW is analyzed and improved before the similarity recognition of speech. We also complete the requirement analysis of the English-speaking learning system by combining the relevant theories and applying the speech signal processing technology and template matching technology to realize a multimedia-assisted learning system suitable for college students to conduct independent English learning. Through the English-speaking test of two classes, the control class using traditional online teaching had many failures reaching 25%, while the experimental class using the English-assisted learning platform passed all of them. 80 points or more were achieved by only 5% of the control class, while 20% of the experimental class. The average score of the control class was 63.4, while the experimental class was 72.1, and the experimental class was 8.7 points higher than the control class. Therefore, teaching English-assisted learning platforms can improve students’ English-speaking abilities.

Brain-like position measurement method based on improved optical flow algorithm

In this paper, a brain-like navigation scheme based on fuzzy kernel C-means (FKCM) clustering assisted pyramid Lucas Kanade (LK) optical flow algorithm is developed to measure the position of vehicle. The Speed Cell and Place Cell in animals’ brain are introduced to construct the brain-like navigation mechanism which involves the optical flow method and image template matching to imitate the cells above-mentioned separately. To eliminate the singular values during optical flow calculation, the output of pyramid LK algorithm is clustered by FKCM algorithm firstly. Then, the velocity is calculated and integrated to get the position of the vehicle, and the brain-like navigation scheme is introduced to correct the position measurement errors by eliminating the accumulated errors resulting from velocity integration. The prominent advantages of the presented method are: (i) a pure visual brain-like position measurement method based on the concept of speed cells and place cells is proposed, making visual navigation more accurate and intelligent; (ii) the FKCM algorithm is used to eliminate the singular value of the pyramid LK algorithm, which improves the calculated velocity accuracy. Also, experimental comparison with classical pyramid LK algorithm is given to illustrate the superiority of the proposed method in position measurement.

Simultaneous object detection and segmentation for patient-specific markerless lung tumor tracking in simulated radiographs with deep learning.

Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimatortracking. The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individualcases. We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing forsignificance. A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p < 0.001) over the 37 PCT-DRR IoU > 0.2 cases, but not over the undifferentiated 84 testingcases. The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lungcases.

Structural displacement estimation by a hybrid computer vision approach

Structural displacement is an important indicator of structural safety in structural health monitoring (SHM). As one of the vision-based methods, optical flow can provide displacement measurement of pixels in images. However, estimating structural displacement by predicting the optical flow between the current frame and the initial frame may be subject to limited accuracy due to environmental variations. Alternatively, calculating structural velocity by predicting the optical flow of adjacent frames and integrating to obtain displacement may introduce displacement drift attributable to cumulative errors. This study proposes a hybrid structural displacement estimation method to eliminate the effects of environmental variations and cumulative errors. In comparison to the existing methods, the novelty of the proposed method is to effectively integrate deep learning-based dense optical flow and correlation template matching (CM), for achieving both high accuracy and improved robustness. Deep learning-based dense optical flow was used for optical flow prediction between adjacent frames through correlation calculations and iterative updating to obtain the structural velocity. Pyramid-accelerated CM was employed to locate the regions of interest (ROI) of the structure in each frame, and the structural displacement was then estimated by counting temporal changes in these locations. By fusing estimated structural velocity and displacement using the Kalman filter, optimized structural displacement results were obtained, and temporal cumulative errors using dense optical flow could be eliminated. The proposed method was validated in an indoor shaking table test of a three-story reinforced concrete structure, and an outdoor shaking table test of a cold-formed steel wall system. The results indicated that the proposed method reduced the root mean square error of estimated displacement by over 89% compared with dense optical flow and by over 36% compared with CM. Furthermore, the proposed method was able to process 1080p high-definition images at a rate of 5.43 frames per second, indicating its high efficiency for applications.