Low Frame Research Articles

Infrared and Visible Camera Integration for Detection and Tracking of Small UAVs: Systematic Evaluation

Given the recent proliferation of Unmanned Aerial Systems (UASs) and the consequent importance of counter-UASs, this project aims to perform the detection and tracking of small non-cooperative UASs using Electro-optical (EO) and Infrared (IR) sensors. Two data integration techniques, at the decision and pixel levels, are compared with the use of each sensor independently to evaluate the system robustness in different operational conditions. The data are submitted to a YOLOv7 detector merged with a ByteTrack tracker. For training and validation, additional efforts are made towards creating datasets of spatially and temporally aligned EO and IR annotated Unmanned Aerial Vehicle (UAV) frames and videos. These consist of the acquisition of real data captured from a workstation on the ground, followed by image calibration, image alignment, the application of bias-removal techniques, and data augmentation methods to artificially create images. The performance of the detector across datasets shows an average precision of 88.4%, recall of 85.4%, and mAP@0.5 of 88.5%. Tests conducted on the decision-level fusion architecture demonstrate notable gains in recall and precision, although at the expense of lower frame rates. Precision, recall, and frame rate are not improved by the pixel-level fusion design.

Collaborative Low Frame Rate UAV Tracking by Proposals

Collaborative Low Frame Rate UAV Tracking by Proposals

A vibration transducer framework for the measurement and reconstruction of vibration signals via low-rate sampling

This study presents a novel, cost-effective, and highly accurate computer vision-based vibration transducer framework, enabling the measurement and reconstruction of vibration signals using video frames captured at a low frame per second (fps) relative to the Nyquist rate of the measured signal. In the proposed framework, an assembly consisting of several spring-mass systems serves as the primary sensor, while a camera module tracks the motion of the seismic masses as the secondary sensor. The setup, along with the modified compressive sensing equation (CS) equation, results in a transducer scheme that can recover the base motion signals from the displacement history of seismic masses sampled at rates much lower than the maximum predominant frequency of the original signal. The performance of the proposed transducer framework has been successfully verified by numerical simulations. In addition, an experimental study has been conducted in order to validate the concept under modelling and measurement error.

Nuclear instance segmentation and tracking for preimplantation mouse embryos.

For investigations into fate specification and morphogenesis in time-lapse images of preimplantation embryos, automated 3D instance segmentation and tracking of nuclei are invaluable. Low signal-to-noise ratio, high voxel anisotropy, high nuclear density, and variable nuclear shapes can limit the performance of segmentation methods, while tracking is complicated by cell divisions, low frame rates, and sample movements. Supervised machine learning approaches can radically improve segmentation accuracy and enable easier tracking, but they often require large amounts of annotated 3D data. Here we first report a novel mouse line expressing near-infrared nuclear reporter H2B-miRFP720. We then generate a dataset (termed BlastoSPIM) of 3D images of H2B-miRFP720-expressing embryos with ground truth for nuclear instances. Using BlastoSPIM, we benchmark seven convolutional neural networks and identify Stardist-3D as the most accurate instance segmentation method. With our BlastoSPIM-trained Stardist-3D models, we construct a complete pipeline for nuclear instance segmentation and lineage tracking from the 8-cell stage to the end of preimplantation development (>100 nuclei). Finally, we demonstrate BlastoSPIM's usefulness as pre-train data for related problems, both for a different imaging modality and for different model systems.

Flexible Soft X-Ray Image Sensors based on Metal Halide Perovskites With High Quantum Efficiency.

Soft X-ray imaging is a powerful tool to explore the structure of cells, probe material with nanometer resolution, and investigate the energetic phenomena in the universe. Conventional soft X-ray image sensors are by and large Si-based charge coupled devices that suffer from low frame rates, complex fabrication processes, mechanical inflexibility, and required cooling below -60°C. Here, a soft X-ray photodiode is reported based on low-cost metal halide perovskite with comparable performance to commercial Si-based device. Nanothrough network electrodeminimized the optical loss due to the shadowing of insensitive layers, while a multidimensional perovskite heterojunctionis generated to reduce the photo-generated carrier loss. This strategy promoted a record quantum efficiency of 8×103% without cooling, several orders of magnitude greater than the previously achieved. Flexible and curved soft X-ray imaging arrays are fabricated based on this high-performance device structure, demonstrating stable soft X-ray response and sharp imaging capabilities. This work highlights the low-cost and efficient perovskite photodiode as a strong candidate for the next-generation soft X-ray image sensors.

Microbubble tracking based on partial smoothing-based adaptive generalized labelled Multi-Bernoulli filter for super-resolution imaging

Super-resolution ultrasound (SRUS) can image the vasculature at microscopic resolution according to microbubble (MB) localization, with velocity vector maps obtained based on MB tracking information. High MB concentrations can reduce the acquisition time of SRUS imaging, however adjacent and intersecting vessels are difficult to distinguish, thus decreasing resolution. Low acquisition frame rates affect the precision of flow velocity estimation. This study proposes a partial smoothing-based adaptive generalized labeled multi-Bernoulli filter (SAGLMB) to precisely track the MB motion at different flow velocities. SAGLMB employs a generalized labelled multi-Bernoulli filter (GLMB) for MB trajectory allocation to separate adjacent and intersecting vessels. Furthermore, the nonlinear motion of MB was predicted by an unscented Kalman filter, and a cardinalized probability hypothesis density filter was applied to suppress clutter interference. Finally, the trajectories were smoothed by unscented Rauch-Tung-Striebel to improve the resolution of the SRUS image. The simulation results demonstrate that SAGLMB outperforms the conventional bipartite graph-based tracking at high MB concentrations, achieving at least an 8.55 % improvement in the correctly paired precision, with 3 times increase in the structural similarity index measure. Moreover, SAGLMB can obtain more precise flow velocity estimations with a 4 times improvement than the conventional method. The SRUS results of rabbit kidney show that the proposed method significantly improves resolution of adjacent and intersecting vessels at higher MB concentrations and maintains this performance as the acquisition frame rate decreases. Furthermore, the rat brain microvascular network was reconstructed with 9.21 μm (λ/11.1) resolution. Therefore, SAGLMB can achieve robust SRUS imaging at high concentrations and low acquisition frame rates.

Detection and Tracking of Low-Frame-Rate Water Surface Dynamic Multi-Target Based on the YOLOv7-DeepSORT Fusion Algorithm

This study aims to address the problem in tracking technology in which targeted cruising ships or submarines sailing near the water surface are tracked at low frame rates or with some frames missing in the video image, so that the tracked targets have a large gap between frames, leading to a decrease in tracking accuracy and inefficiency. Thus, in this study, we proposed a water surface dynamic multi-target tracking algorithm based on the fusion of YOLOv7 and DeepSORT. The algorithm first introduces the super-resolution reconstruction network. The network can eliminate the interference of clouds and waves in images to improve the quality of tracking target images and clarify the target characteristics in the image. Then, the shuffle attention module is introduced into YOLOv7 to enhance the feature extraction ability of the target features in the recognition network. Finally, Euclidean distance matching is introduced into the cascade matching of the DeepSORT algorithm to replace the distance matching of IOU to improve the target tracking accuracy. Simulation results showed that the algorithm proposed in this study has a good tracking effect, with an improvement of 9.4% in the improved YOLOv7 model relative to the mAP50-95 value and an improvement of 13.1% in the tracking accuracy in the DeepSORT tracking network compared with the SORT tracking accuracy.

Combining optical flow and Swin Transformer for Space-Time video super-resolution

Space–time video super-resolution is a task that aims to interpolate low frame rate, low resolution videos to high frame rate, high resolution ones. While existing Transformer-based methods have achieved results comparable to convolutional neural networks-based methods, the computational cost of Transformer limits its performance with constrained computational resources. Moreover, Swin Transformer may fail to fully exploit the spatio-temporal information of video frames due to the limitation of window size, impeding its effectiveness in handling large motions. To address these limitations, we propose an end-to-end space–time video super-resolution architecture based on optical flow alignment and Swin Transformer. The alignment module is introduced to extract spatio-temporal information from adjacent frames without significantly increasing the computational burden. Additionally, we design a residual convolution layer to enhance the translational invariance of the features extracted by Swin Transformer and introduces additional nonlinear transformations. Experimental results demonstrate that our proposed method achieves superior performance on various benchmark datasets compared to state-of-the-art methods. In terms of Peak Signal-to-Noise Ratio, our method outperforms the state-of-the-art methods by at least 0.15 dB on Vimeo-Medium dataset.

Application of motion prediction based on a long short-term memory network for imaging dose reduction in real-time tumor-tracking radiation therapy

PurposeTo demonstrate the possibility of using a lower imaging rate while maintaining acceptable accuracy by applying motion prediction to minimize the imaging dose in real-time image-guided radiation therapy. MethodsTime-series of three-dimensional internal marker positions obtained from 98 patients in liver stereotactic body radiation therapy were used to train and test the long-short-term memory (LSTM) network. For real-time imaging, the root mean squared error (RMSE) of the prediction on three-dimensional marker position made by LSTM, the residual motion of the target under respiratory-gated irradiation, and irradiation efficiency were evaluated. In the evaluation of the residual motion, the system-specific latency was assumed to be 100 ms. ResultsExcept for outliers in the superior–inferior (SI) direction, the median/maximum values of the RMSE for imaging rates of 7.5, 5.0, and 2.5 frames per second (fps) were 0.8/1.3, 0.9/1.6, and 1.2/2.4 mm, respectively. The median/maximum residual motion in the SI direction at an imaging rate of 15.0 fps without prediction of the marker position, which is a typical clinical setting, was 2.3/3.6 mm. For rates of 7.5, 5.0, and 2.5 fps with prediction, the corresponding values were 2.0/2.6, 2.2/3.3, and 2.4/3.9 mm, respectively. There was no significant difference between the irradiation efficiency with and that without prediction of the marker position. The geometrical accuracy at lower frame rates with prediction applied was superior or comparable to that at 15 fps without prediction. In comparison with the current clinical setting for real-time image-guided radiation therapy, which uses an imaging rate of 15.0 fps without prediction, it may be possible to reduce the imaging dose by half or more. ConclusionsMotion prediction can effectively lower the frame rate and minimize the imaging dose in real-time image-guided radiation therapy.

High-frequency nonlinear vibration analysis through low-frequency stereo-camera systems

This paper describes a new methodology that expands the capabilities of low-frame, high-resolution stereo-camera systems in studying the dynamic behavior of components in the presence of nonlinear phenomena. A new downsampling technique called the Smoothed Harmonics Analysis (SHA) is proposed. This technique addresses the limitations due to the low-frame rate cameras for the study of high-frequency periodic steady-state nonlinear oscillations. SHA enables accurate reconstruction of downsampled signals, thus opening up numerous potential applications. The feasibility of this technique is demonstrated by analyzing the motion of a beam with nonlinear behavior. The nonlinearity is caused by intermittent contact while the beam is subjected to harmonic excitation.

Towards clinical application of freehand optical ultrasound imaging

Freehand optical ultrasound (OpUS) imaging is an emerging ultrasound imaging paradigm that uses an array of fibre-optic, photoacoustic ultrasound sources and a single fibre-optic ultrasound detector to perform ultrasound imaging without the need for electrical components in the probe head. Previous freehand OpUS devices have demonstrated capability for real-time, video-rate imaging of clinically relevant targets, but have been hampered by poor ultrasound penetration, significant imaging artefacts and low frame rates, and their designs limited their clinical applicability. In this work we present a novel freehand OpUS imaging platform, including a fully mobile and compact acquisition console and an improved probe design. The novel freehand OpUS probe presented utilises optical waveguides to shape the generated ultrasound fields for improved ultrasound penetration depths, an extended fibre-optic bundle to improve system versatility and an overall ruggedised design with protective elements to improve probe handling and protect the internal optical components. This probe is demonstrated with phantoms and the first multi-participant in vivo imaging study conducted with freehand OpUS imaging probes, this represents several significant steps towards the clinical translation of freehand OpUS imaging.

STDAN: Deformable Attention Network for Space-Time Video Super-Resolution.

The target of space-time video super-resolution (STVSR) is to increase the spatial-temporal resolution of low-resolution (LR) and low-frame-rate (LFR) videos. Recent approaches based on deep learning have made significant improvements, but most of them only use two adjacent frames, that is, short-term features, to synthesize the missing frame embedding, which cannot fully explore the information flow of consecutive input LR frames. In addition, existing STVSR models hardly exploit the temporal contexts explicitly to assist high-resolution (HR) frame reconstruction. To address these issues, in this article, we propose a deformable attention network called STDAN for STVSR. First, we devise a long short-term feature interpolation (LSTFI) module that is capable of excavating abundant content from more neighboring input frames for the interpolation process through a bidirectional recurrent neural network (RNN) structure. Second, we put forward a spatial-temporal deformable feature aggregation (STDFA) module, in which spatial and temporal contexts in dynamic video frames are adaptively captured and aggregated to enhance SR reconstruction. Experimental results on several datasets demonstrate that our approach outperforms state-of-the-art STVSR methods. The code is available at https://github.com/littlewhitesea/STDAN.

An efficient ultrasonic wavenumber-domain plane wave imaging method towards the inspection of curved structures

Ultrasonic phased array testing is commonly employed for inspecting curved structures. Conventional plane wave imaging techniques, based on delay-and-sum in the time-domain, offer high image quality and inspection accuracy but suffer from low frame rates due to their high computational complexity. In this work, an efficient wavenumber-domain imaging method that combines non-stationary wavefield extrapolation and f-k migration is proposed for curved structure inspection. Special emission focal laws are designed to generate a sequence of steered plane waves through the curved interface. The raw data is then extrapolated to the top boundary of the region of interest, followed by f-k migration to reconstruct images with high time efficiency. Simulation and experimental evaluations demonstrate a time reduction by a factor of up to 32.24 compared to conventional time-domain plane wave image reconstruction with equivalent image quality, highlighting its potential for monitoring flaws in real-time.

Monitoring of motor vehicle exhaust emissions using Gaussian process regression frame interpolation optical flow algorithm

In fluid pollutant monitoring, the spatial continuity of pixel motion is disrupted by infrared cameras, primarily due to factors like low frame rate. This disruption impedes the accurate capture of pollutant distribution and evolution, resulting in substantial errors in monitoring outcomes. To address this challenge, we introduce the Gaussian Process Regression Frame Interpolation Optical Flow (GPR-FIOF), aimed at restoring the spatial continuity of pixel motion. Consequently, this facilitates a more precise estimation of fluid pollutant motion. Experimental results from fluid simulations demonstrate that, when compared to conventional algorithms, GPR-FIOF significantly enhances accuracy and stability, improving by 80.30% and 66.39%, respectively. Field experiments employing infrared gas correlation spectroscopy methods revealed improvements in accuracy and stability of emission rate inversion results, with enhancements of 18.24% and 61.77%, respectively. GPR-FIOF effectively mitigates the disruption in spatial continuity, enhancing the accuracy of pollutant gas emission monitoring and bolstering its feasibility for environmental monitoring applications.

Running sprint force-velocity-power profile obtained with a low-cost and low frame rate acquisition video technique: reliability and concurrent validity

ABSTRACT The Force-velocity (F-v) and Power-velocity (P-v) relationships quantify athlete’s horizontal force production capacities during sprinting. Efforts are underway to enhance ecological validity for practitioners and sports coaches. This study provides detailed data comparison of a low frames per second setup (30 Hz; FPSlow) with splits from a high FPS camera to derive F-v and P-v relationships. Sixty-six sprints performed by 11 university track and field athletes (6 male, 5 female) were evaluated. Data were recorded using FPSlow, photocells, and a high-speed camera (240 Hz; MySprint). In the FPSlow setup, bias was 0.17s, and Limits of agreement was 0.09s compared to photocells. ICC was 1.00, and the coefficient of variation (CV) was 1.0% [0.8–1.1%]. Time acquisition comparison between MySprint and FPSlow setups revealed high consistency (ICC = 0.99) and low CV (2.9% [2.8–3.1%]). F-v profile variables exhibited biases from trivial to small, with ICC ranging from moderate to nearly perfect. CV ranged from 2.7% to 11.8%, and improved using the average of three sprints (CV between 1.8% and 8.6%). The ‘simple method’ applied to data from the low FPS video setup yielded kinetic and kinematic parameters comparable to those obtained by the validated previous method and photocells.

Method for noninvasive HV/MV switchgear motion analysis using kernel-based algorithm with adaptive feature extraction

When designing, developing, and testing medium-voltage (MV) and high-voltage (HV) switchgears, it is of utmost importance to analyze the movements of their mechanical parts, such as drive trains, contact nozzles, etc. This ensures the safety of switchgear operations and enables detecting and predicting issues that could lead to component damage, power interruptions, reduced efficiency, or even switch failure. Conventional invasive measuring setups, including encoders and laser distance measurements, are often difficult or expensive to use, due to undesirable environmental properties such as high temperatures and/or high voltages, or dimensional constraints often encountered in testing laboratories, compact substations or confined equipment rooms. Video object tracking can be a viable solution in such conditions, allowing for the extraction of the trajectory of mechanical parts of an object under analysis. This paper proposes a novel kernel-based algorithm with adaptive feature extraction for precise colour-based object tracking through video processing. The implemented method is based on the principles of the CamShift algorithm, augmented with fusion with the Kalman filter for continuous estimation and prediction of the object’s position based on the available measurements while reducing sensitivity to noise. Beyond precise tracking of the object of interest, the algorithm automatically adapts the mask used in the standard CamShift algorithm by extracting and processing the features of the selected object. This approach exhibits flexibility and robustness in adverse industrial environments. It supports modifications according to user preferences and represents a cost-effective alternative to conventional methods, while performing real-time processing. Experimental results underscore this noninvasive approach is flexible, highly robust, and enables tracking of coloured markers even in challenging conditions like poor lighting, significant blur, and low frame rates.

Performance of cone-beam computed tomography imaging during megavoltage beam irradiation under phase-gated conditions

PurposeTarget positions should be acquired during beam delivery for accurate lung stereotactic body radiotherapy. We aimed to perform kilovoltage (kV) imaging during beam irradiation (intra-irradiation imaging) under phase-gated conditions and evaluate its performance. MethodsCatphan 504 and QUASAR respiratory motion phantoms were used to evaluate image quality and target detectability, respectively. TrueBeam STx linac and the Developer Mode was used. The imaging parameters were 125 kVp and 1.2 mAs/projection. Flattened megavoltage (MV) X-ray beam energies 6, 10 and 15 MV and un-flattened beam energies 6 and 10 MV were used with field sizes of 5 × 5 and 15 × 15 cm2 and various frame rates for intra-irradiation imaging. In addition, using a QUASAR phantom, intra-irradiation imaging was performed during intensity-modulated plan delivery. The root-mean-square error (RMSE) of the CT-number for the inserted rods, image noise, visual assessment, and contrast-to-noise ratio (CNR) were evaluated. ResultsThe RMSEs of intra-irradiation cone-beam computed tomography (CBCT) images under gated conditions were 50–230 Hounsfield Unit (HU) (static < 30 HU). The noise of the intra-irradiation CBCT images under gated conditions was 15–35 HU, whereas that of the standard CBCT images was 8.8–27.2 HU. Lower frame rates exhibited large RMSEs and noise; however, the iterative reconstruction algorithm (IR) was effective at improving these values. Approximately 7 fps with the IR showed an equivalent CNR of 15 fps without the IR. The target was visible on all the gated intra-irradiation CBCT images. ConclusionSeveral image quality improvements are required; however, intra-irradiated CBCT images showed good visual target detection.

P‐16: A Novel Driving Scheme to Achieve Low Frame Rate and Low Power Consumption with Narrow Border

To improve the display effect at low illumination of low frame rate display, two sets of scan driving circuits must be adopted. Therefore, the border must be widened to place two sets of scan driving circuits and one set of emit circuits, which conflicts with the need of narrow border for AMOLED display. In this paper, a novel driving architecture that placed two set of circuits alternatively by one side is proposed to meet the low frame rate and narrow border at the same time.

15‐4: WUXGA LTPS Pad with Only 1‐IC‐Chip and 8‐Photo‐Mask Processes

In order to enhance the competitiveness with a‐Si and IGZO products, LTPS tablet product with comprehensive advantages in performance and cost have been developed. In this article, innovative technologies including 8‐Photo‐Mask process and only 1‐IC‐Chip solution are applied, and the performances such as low frame rate, active pens and narrow borders can meet the request of customer.

Near-zero fluoroscopy technique for cardiac electronic device implantation: long-term outcomes

Abstract Introduction Radiation hazards are a major concern for both patients and staff. We have previously described a near-zero fluoroscopy technique for cardiac electronic device implantation. In this study, we aimed to investigate the long-term safety of this technique and compare outcomes and procedural metrics with a conventional approach in a larger population. Methods Low-dose fluoroscopy technique. Radiation exposure during fluoroscopy is directly proportional to the time the unit is activated. Typically, the pedal switch is depressed for at least a few seconds per imaging sequence in order to visualize "live" movements of either the leads during their positioning or the needle during a fluoroscopy-guided puncture of the subclavian vein. Our technique consists of setting a low frame rate per second (0.5-3.75fps) and momentarily triggering the foot pedal switch only to obtain screenshots of the position of the leads/needle. Use of live electrograms to assist spatial awareness of lead position (e.g. Atrial, Coronary Sinus, Ventricular signals) acts as an adjunct to guidance. Study design We included consecutive patients undergoing permanent pacemaker (PPM) or cardiac defibrillator (ICD) implant at our center from July 2018 to December 2022 using the near-zero fluoroscopy technique. Procedures were performed by three operators (AC, MD, MF). Cumulative radiation dose was measured using dose-area product (DAP). We assessed procedure success and complication rate during long-term follow-up. We compared the procedural metrics and outcomes with a cohort of patients with PPM or ICD implanted in our centre by senior operators, using a traditional fluoroscopic approach. Propensity score was adopted to match baseline characteristics between the study and control group. Results The total population consisted of 412 patients (74.2±13.9 years, 60.9% male). We included 206 who underwent PPM or ICD implantation using the low-dose fluoroscopy technique, and 206 matched-control. A PPM or ICD was successfully implanted in all patients. Fluoroscopy time and DAP were significantly lower in the low-dose fluoroscopy group, mean 2.3±2.0 seconds versus 258.4±252.1 seconds (P&amp;lt;0.001) and 3.4±3.5 µGym2 versus 29.8±55.4 µGym2 (P&amp;lt;0.001), respectively. Mean procedure time was 49.0±20.8 minutes in the low fluoroscopy group and 65.7±26.8 minutes in the control group (P&amp;lt;0.001). Median frame per second setting was 0.5 in the low-dose cohort. After a mean follow-up of 441±470 days, the rate of complications was identical across groups (2.4%, p = 1.0). Conclusion This near-zero dose fluoroscopy technique is effective and safe, enabling a 10x reduction in radiation exposure during device implant. Modification of traditional implant and imaging techniques can provide material reductions in fluoroscopy dose to even below that for a standard PA chest radiograph.