Low Frame Research Articles

Elevated Frame Rates During Exercise Echocardiography Improve Speckle Tracking Success Rate and Augment Deformation Values.

Background: Although 2D speckle-tracking echocardiography (STE) is important for the clinical quantification of myocardial function it remains unknown whether increased frame rates during exercise STE augment tracking success and absolute deformation values. Method: 19 participants (15 male and 4 female; age 26.7±4.8) underwent step-wise exercise testing on a recumbent bicycle. Exercise started at 50 W, increasing by 30 W every 3 min until a target heart rate of 130-140 bpm was reached. During last 90 seconds of each exercise stage, echocardiographic sequences for offline quantification of longitudinal strain (LS), peak twist, untwisting velocity, basal rotation and apical rotation were acquired with high (HFPS), medium (MFPS), and low frame rates (LFPS). Differences in tracking success were determined using chi-square test and the impact of different frames rates on absolute deformation values were compared using mixed-model-analysis. Results: Utilization of HFPS significantly improved tracking success for parasternal short-axis images. LS acquired at HFPS was highest at baseline and across all exercise stages. Similar trends were observed for twist, peak untwisting velocity and apical rotation, while basal rotation showed no differences. Mixed model analysis revealed a significant effect of frame rate setting on LS (p<0.05) and untwisting velocity (p<0.05). Conclusion: In contrast to recommendations by leading organizations advocating for frame rates between 40 and 80 fps during resting conditions, with a proportional increase as heart rate rises, our findings suggest that consistently maintaining the frame rate at the highest feasible level is preferable for achieving optimal tracking success and accuracy in STE.

Efficient Hyperspectral Video Reconstruction via Dual-Channel DMD Encoding

Hyperspectral video acquisition requires a precise balance between spectral and temporal resolution, often achieved through compressive sampling using two-dimensional detectors and spectral reconstruction algorithms. However, the reliance on spatial light modulators for coding reduces optical efficiency, while complex recovery algorithms hinder real-time reconstruction. To address these challenges, we propose a digital-micromirror-device-based complementary dual-channel hyperspectral (DMD-CDH) video imaging system. This system employs a DMD for simultaneous light splitting and spatial encoding, enabling one channel to perform non-aliasing spectral sampling at lower frame rates while the other provides complementary high-rate sampling for panchromatic video. Featuring high optical throughput and efficient complementary sampling, the system ensures reliable hyperspectral video reconstruction and serves as a robust ground-based validation platform for remote sensing applications. Additionally, we introduce tailored optical error calibration and fixation techniques alongside a lightweight hyperspectral fusion network for reconstruction, achieving hyperspectral frame rates exceeding 30 fps. Compared to the existing models, this system simplifies the calibration process and provides a practical high-performance solution for real-time hyperspectral video imaging.

Quantifying Similarities Between MediaPipe and a Known Standard to Address Issues in Tracking 2D Upper Limb Trajectories: Proof of Concept Study.

Markerless motion tracking methods have promise for use in a range of domains, including clinical settings where traditional marker-based systems for human pose estimation are not feasible. Artificial intelligence (AI)-based systems can offer a markerless, lightweight approach to motion capture. However, the accuracy of such systems, such as MediaPipe, for tracking fine upper limb movements involving the hand has not been explored. The aim of this study is to evaluate the 2D accuracy of MediaPipe against a known standard. Participants (N=10) performed a touchscreen-based shape-tracing task requiring them to trace the trajectory of a moving cursor using their index finger. Cursor trajectories created a reoccurring or random shape at 5 different speeds (500-2500 ms, in increments of 500 ms). Movement trajectories on each trial were simultaneously captured by the touchscreen and a separate video camera. Movement coordinates for each trial were extracted from the touchscreen and compared to those predicted by MediaPipe. Specifically, following resampling, normalization, and Procrustes transformations, root-mean-squared error (RMSE; primary outcome measure) was calculated between predicted coordinates and those generated by the touchscreen computer. Although there was some size distortion in the frame-by-frame estimates predicted by MediaPipe, shapes were similar between the 2 methods and transformations improved the general overlap and similarity of the shapes. The resultant mean RMSE between predicted coordinates and those generated by the touchscreen was 0.28 (SD 0.06) normalized px. Equivalence testing revealed that accuracy differed between MediaPipe and the touchscreen, but that the true difference was between 0 and 0.30 normalized px (t114=-3.02; P=.002). Additional analyses revealed no differences in resultant RMSE between methods when comparing across lower frame rates (30 and 60 frames per second [FPS]), although there was greater RMSE for 120 FPS than for 60 FPS (t35.43=-2.51; P=.03). Overall, we quantified similarities between one AI-based approach to motion capture and a known standard for tracking fine upper limb movements, informing applications of such systems in domains such as clinical and research settings. Future work should address accuracy in 3 dimensions to further validate the use of AI-based systems, including MediaPipe, in such domains.

Single-pixel orbital angular momentum detection in the temporal domain

Abstract Orbital angular momentum (OAM) beam’s spatial information captured using a camera, employs millions of single-pixel detectors arranged in a two-dimensional matrix. Poses challenges in OAM detection due to significant storage requirements, reduced speed due to low frame rate, and increased post-processing time. To address this bottleneck, a novel OAM detection technique utilizing only a diffuser and a single-pixel fast photodiode (FPD) is reported. The sixteen OAM beams, l = [ − 8 + 8 ] interact with the rotating diffuser resulting in a temporally varying speckle field. The 1D temporal speckle information (TSI) has been segregated from the temporally varying 2D speckle pattern images for each beam. The segregated 1D TSI has been classified by employing a machine learning model and achieved a classification accuracy of 97.1% and 97.3% on simulated and experimental data respectively. To further validate the proposed single-pixel OAM detection method, the 1D TSI is captured directly using a FPD. The experimentally captured 1D TSI captured via photodiode has been classified by a lightweight custom-designed 1D convolutional neural network and achieved an increased classification accuracy of 96%. This approach redefines OAM detection, operating in the temporal domain with a single-pixel FPD, thereby significantly reducing storage and computational costs while promising high-speed OAM detection.

Dynamic light field reconstruction via densely connected deep equilibrium model

High-resolution consumer plenoptic cameras usually feature low frame rates, making them not well-suited for capturing high-speed motion scenes. To compensate for this limitation, we extend the original snapshot compressive imaging system to plenoptic cameras and propose a densely connected deep equilibrium (DEQ) model for high-quality dynamic light field (LF) reconstruction, abbreviated as DLFDEQ. Specifically, we perform temporal compression encoding on a dynamic LF and model the reconstruction process as an inverse problem with an implicit regularization term. To solve this inverse problem, we present a densely connected DEQ model based on gradient descent. Our approach demonstrates stronger robustness and better detail retention than existing methods. We can practically quadruple the original camera’s frame rate by continually capturing and retrieving these measurement frames with high reconstruction accuracy.

Feasibility of fetal cardiac function and anatomy assessment by real-time spiral bSSFP MRI at 0.55T

BackgroundContemporary 0.55T MRI is promising for fetal MRI, due to the larger bore, reduced safety concerns, lower acoustic noise, and improved fast imaging capability. In this work, we explore improved fetal cardiac MRI (CMR) without relying on any synchronizing devices, prospective, or retrospective gating. PurposeTo determine the feasibility of real-time MRI evaluation of fetal cardiac function as well as cardiac and great vessel anatomies by using spiral balanced steady-state free precession (bSSFP) at 0.55T. MethodsA real-time spiral bSSFP pulse sequence for fetal CMR was implemented and optimized on a 0.55T whole-body MRI. Fetal CMR was prospectively performed between May 2022 and August 2023. The protocol included: 1) real-time images at standard cardiac views, for 10-20seconds/view and 40 to 43.6 ms/frame and 2) 4-9 stacks of slices at standard cardiac views that each cover the whole heart, with 15-30 slices/stack, and 2-5seconds/slice, at 320 to 349 ms/frame. Images were evaluated by a fetal cardiologist. Quantitative measurements of cardiothoracic area ratio and cardiac axis were compared with previous reports. Diagnostic accuracy was compared against postnatal echocardiographic findings. ResultsTwenty-nine participants were enrolled for 32 CMR exams, with mean maternal age 33.6±5.8 year (range 22-44 year) and mean gestational age 32.8±3.9 weeks (range 23-38 weeks). The proposed sequence enabled evaluation of the fetal heart in <30minutes in all cases (average 22minutes). Real-time MRI allowed easy adjustment of scan plan, automatic whole-heart volumetric sweeping, and flexible choice of reconstruction temporal resolution. For key cardiac anatomic features 60% were delineated well. Mean cardiothoracic area ratio and cardiac axis were 0.27±0.04 and 45.8±7.8. Diagnostic agreement with postnatal echocardiographic findings was 84%. ConclusionA spiral real-time bSSFP pulse sequence at 0.55T can provide both low framerate and high framerate fetal heart images without relying on maternal breath-hold, specialized gating devices, or cardiac gating. The low framerate images offer high diagnostic quality structural evaluations of the fetal heart, while the high framerate images capture fetal heart motion and may enable functional assessments.

Comparison of Coronary Interventions with Low fluoroscopic frame rate and normal fluoroscopic frame rate - Prospective, Randomized study

Comparison of Coronary Interventions with Low fluoroscopic frame rate and normal fluoroscopic frame rate - Prospective, Randomized study

Mitigating high frame rate demands in shear wave elastography using radial basis function-based reconstruction: An experimental phantom study.

Shear wave elastography (SWE) is a technique that quantifies tissue stiffness by assessing the speed of shear waves propagating after being excited by acoustic radiation force. SWE allows the quantification of elastic tissue properties and serves as an adjunct to conventional ultrasound techniques, aiding in tissue characterization. To capture this transient propagation of the shear wave, the ultrasound device must be able to reach very high frame rates. In this paper, our aim is to relax the high frame rate requirement for SWE imaging. To this end, we propose lower frame rate SWE imaging followed by employing a 2-dimensional (2D) radial basis functions (RBF)-based interpolation. More specifically, the process involves obtaining low frame rate data and then temporal upsampling to reach a synthetic high frame rate data by inserting the 'UpS-1' image frames with missing values between two successive image frames (UpS: Upsampling rate). Finally, we apply the proposed interpolation technique to reconstruct the missing values within the incomplete high frame rate data. The results obtained from employing the proposed model on two experimental datasets indicate that we can relax the frame rate requirement of SWE imaging by a factor of 4 while maintaining shear wave speed (SWS), group velocity, and phase velocity estimates closely align with the high frame rate SWE model so that the error is less than 3%. Furthermore, analysis of the structural similarity index (SSIM) and root mean squared error (RMSE) on the 2D-SWS maps highlights the efficacy of the suggested technique in enhancing local SWS estimates, even at a downsampling (DS) factor of 4. For DS≤4, the SSIM values between the 2D-SWS maps produced by the proposed technique and those generated by the original high frame rate data consistently remain above 0.94. Additionally, the RMSE values is below 0.37 m/s, indicating promising performance of the proposed technique in reconstruction of SWS values.

Infrared and Visible Camera Integration for Detection and Tracking of Small UAVs: Systematic Evaluation

Given the recent proliferation of Unmanned Aerial Systems (UASs) and the consequent importance of counter-UASs, this project aims to perform the detection and tracking of small non-cooperative UASs using Electro-optical (EO) and Infrared (IR) sensors. Two data integration techniques, at the decision and pixel levels, are compared with the use of each sensor independently to evaluate the system robustness in different operational conditions. The data are submitted to a YOLOv7 detector merged with a ByteTrack tracker. For training and validation, additional efforts are made towards creating datasets of spatially and temporally aligned EO and IR annotated Unmanned Aerial Vehicle (UAV) frames and videos. These consist of the acquisition of real data captured from a workstation on the ground, followed by image calibration, image alignment, the application of bias-removal techniques, and data augmentation methods to artificially create images. The performance of the detector across datasets shows an average precision of 88.4%, recall of 85.4%, and mAP@0.5 of 88.5%. Tests conducted on the decision-level fusion architecture demonstrate notable gains in recall and precision, although at the expense of lower frame rates. Precision, recall, and frame rate are not improved by the pixel-level fusion design.

Flexible Soft X-Ray Image Sensors based on Metal Halide Perovskites With High Quantum Efficiency.

Soft X-ray imaging is a powerful tool to explore the structure of cells, probe material with nanometer resolution, and investigate the energetic phenomena in the universe. Conventional soft X-ray image sensors are by and large Si-based charge coupled devices that suffer from low frame rates, complex fabrication processes, mechanical inflexibility, and required cooling below -60°C. Here, a soft X-ray photodiode is reported based on low-cost metal halide perovskite with comparable performance to commercial Si-based device. Nanothrough network electrodeminimized the optical loss due to the shadowing of insensitive layers, while a multidimensional perovskite heterojunctionis generated to reduce the photo-generated carrier loss. This strategy promoted a record quantum efficiency of 8×103% without cooling, several orders of magnitude greater than the previously achieved. Flexible and curved soft X-ray imaging arrays are fabricated based on this high-performance device structure, demonstrating stable soft X-ray response and sharp imaging capabilities. This work highlights the low-cost and efficient perovskite photodiode as a strong candidate for the next-generation soft X-ray image sensors.

Nuclear instance segmentation and tracking for preimplantation mouse embryos.

For investigations into fate specification and morphogenesis in time-lapse images of preimplantation embryos, automated 3D instance segmentation and tracking of nuclei are invaluable. Low signal-to-noise ratio, high voxel anisotropy, high nuclear density, and variable nuclear shapes can limit the performance of segmentation methods, while tracking is complicated by cell divisions, low frame rates, and sample movements. Supervised machine learning approaches can radically improve segmentation accuracy and enable easier tracking, but they often require large amounts of annotated 3D data. Here, we first report a previously unreported mouse line expressing near-infrared nuclear reporter H2B-miRFP720. We then generate a dataset (termed BlastoSPIM) of 3D images of H2B-miRFP720-expressing embryos with ground truth for nuclear instances. Using BlastoSPIM, we benchmark seven convolutional neural networks and identify Stardist-3D as the most accurate instance segmentation method. With our BlastoSPIM-trained Stardist-3D models, we construct a complete pipeline for nuclear instance segmentation and lineage tracking from the eight-cell stage to the end of preimplantation development (>100 nuclei). Finally, we demonstrate the usefulness of BlastoSPIM as pre-train data for related problems, both for a different imaging modality and for different model systems.

Collaborative Low Frame Rate UAV Tracking by Proposals

Collaborative Low Frame Rate UAV Tracking by Proposals

A vibration transducer framework for the measurement and reconstruction of vibration signals via low-rate sampling

This study presents a novel, cost-effective, and highly accurate computer vision-based vibration transducer framework, enabling the measurement and reconstruction of vibration signals using video frames captured at a low frame per second (fps) relative to the Nyquist rate of the measured signal. In the proposed framework, an assembly consisting of several spring-mass systems serves as the primary sensor, while a camera module tracks the motion of the seismic masses as the secondary sensor. The setup, along with the modified compressive sensing equation (CS) equation, results in a transducer scheme that can recover the base motion signals from the displacement history of seismic masses sampled at rates much lower than the maximum predominant frequency of the original signal. The performance of the proposed transducer framework has been successfully verified by numerical simulations. In addition, an experimental study has been conducted in order to validate the concept under modelling and measurement error.

Microbubble tracking based on partial smoothing-based adaptive generalized labelled Multi-Bernoulli filter for super-resolution imaging

Super-resolution ultrasound (SRUS) can image the vasculature at microscopic resolution according to microbubble (MB) localization, with velocity vector maps obtained based on MB tracking information. High MB concentrations can reduce the acquisition time of SRUS imaging, however adjacent and intersecting vessels are difficult to distinguish, thus decreasing resolution. Low acquisition frame rates affect the precision of flow velocity estimation. This study proposes a partial smoothing-based adaptive generalized labeled multi-Bernoulli filter (SAGLMB) to precisely track the MB motion at different flow velocities. SAGLMB employs a generalized labelled multi-Bernoulli filter (GLMB) for MB trajectory allocation to separate adjacent and intersecting vessels. Furthermore, the nonlinear motion of MB was predicted by an unscented Kalman filter, and a cardinalized probability hypothesis density filter was applied to suppress clutter interference. Finally, the trajectories were smoothed by unscented Rauch-Tung-Striebel to improve the resolution of the SRUS image. The simulation results demonstrate that SAGLMB outperforms the conventional bipartite graph-based tracking at high MB concentrations, achieving at least an 8.55 % improvement in the correctly paired precision, with 3 times increase in the structural similarity index measure. Moreover, SAGLMB can obtain more precise flow velocity estimations with a 4 times improvement than the conventional method. The SRUS results of rabbit kidney show that the proposed method significantly improves resolution of adjacent and intersecting vessels at higher MB concentrations and maintains this performance as the acquisition frame rate decreases. Furthermore, the rat brain microvascular network was reconstructed with 9.21 μm (λ/11.1) resolution. Therefore, SAGLMB can achieve robust SRUS imaging at high concentrations and low acquisition frame rates.

Detection and Tracking of Low-Frame-Rate Water Surface Dynamic Multi-Target Based on the YOLOv7-DeepSORT Fusion Algorithm

This study aims to address the problem in tracking technology in which targeted cruising ships or submarines sailing near the water surface are tracked at low frame rates or with some frames missing in the video image, so that the tracked targets have a large gap between frames, leading to a decrease in tracking accuracy and inefficiency. Thus, in this study, we proposed a water surface dynamic multi-target tracking algorithm based on the fusion of YOLOv7 and DeepSORT. The algorithm first introduces the super-resolution reconstruction network. The network can eliminate the interference of clouds and waves in images to improve the quality of tracking target images and clarify the target characteristics in the image. Then, the shuffle attention module is introduced into YOLOv7 to enhance the feature extraction ability of the target features in the recognition network. Finally, Euclidean distance matching is introduced into the cascade matching of the DeepSORT algorithm to replace the distance matching of IOU to improve the target tracking accuracy. Simulation results showed that the algorithm proposed in this study has a good tracking effect, with an improvement of 9.4% in the improved YOLOv7 model relative to the mAP50-95 value and an improvement of 13.1% in the tracking accuracy in the DeepSORT tracking network compared with the SORT tracking accuracy.

Combining optical flow and Swin Transformer for Space-Time video super-resolution

Space–time video super-resolution is a task that aims to interpolate low frame rate, low resolution videos to high frame rate, high resolution ones. While existing Transformer-based methods have achieved results comparable to convolutional neural networks-based methods, the computational cost of Transformer limits its performance with constrained computational resources. Moreover, Swin Transformer may fail to fully exploit the spatio-temporal information of video frames due to the limitation of window size, impeding its effectiveness in handling large motions. To address these limitations, we propose an end-to-end space–time video super-resolution architecture based on optical flow alignment and Swin Transformer. The alignment module is introduced to extract spatio-temporal information from adjacent frames without significantly increasing the computational burden. Additionally, we design a residual convolution layer to enhance the translational invariance of the features extracted by Swin Transformer and introduces additional nonlinear transformations. Experimental results demonstrate that our proposed method achieves superior performance on various benchmark datasets compared to state-of-the-art methods. In terms of Peak Signal-to-Noise Ratio, our method outperforms the state-of-the-art methods by at least 0.15 dB on Vimeo-Medium dataset.

Application of motion prediction based on a long short-term memory network for imaging dose reduction in real-time tumor-tracking radiation therapy

PurposeTo demonstrate the possibility of using a lower imaging rate while maintaining acceptable accuracy by applying motion prediction to minimize the imaging dose in real-time image-guided radiation therapy. MethodsTime-series of three-dimensional internal marker positions obtained from 98 patients in liver stereotactic body radiation therapy were used to train and test the long-short-term memory (LSTM) network. For real-time imaging, the root mean squared error (RMSE) of the prediction on three-dimensional marker position made by LSTM, the residual motion of the target under respiratory-gated irradiation, and irradiation efficiency were evaluated. In the evaluation of the residual motion, the system-specific latency was assumed to be 100 ms. ResultsExcept for outliers in the superior–inferior (SI) direction, the median/maximum values of the RMSE for imaging rates of 7.5, 5.0, and 2.5 frames per second (fps) were 0.8/1.3, 0.9/1.6, and 1.2/2.4 mm, respectively. The median/maximum residual motion in the SI direction at an imaging rate of 15.0 fps without prediction of the marker position, which is a typical clinical setting, was 2.3/3.6 mm. For rates of 7.5, 5.0, and 2.5 fps with prediction, the corresponding values were 2.0/2.6, 2.2/3.3, and 2.4/3.9 mm, respectively. There was no significant difference between the irradiation efficiency with and that without prediction of the marker position. The geometrical accuracy at lower frame rates with prediction applied was superior or comparable to that at 15 fps without prediction. In comparison with the current clinical setting for real-time image-guided radiation therapy, which uses an imaging rate of 15.0 fps without prediction, it may be possible to reduce the imaging dose by half or more. ConclusionsMotion prediction can effectively lower the frame rate and minimize the imaging dose in real-time image-guided radiation therapy.

High-frequency nonlinear vibration analysis through low-frequency stereo-camera systems

This paper describes a new methodology that expands the capabilities of low-frame, high-resolution stereo-camera systems in studying the dynamic behavior of components in the presence of nonlinear phenomena. A new downsampling technique called the Smoothed Harmonics Analysis (SHA) is proposed. This technique addresses the limitations due to the low-frame rate cameras for the study of high-frequency periodic steady-state nonlinear oscillations. SHA enables accurate reconstruction of downsampled signals, thus opening up numerous potential applications. The feasibility of this technique is demonstrated by analyzing the motion of a beam with nonlinear behavior. The nonlinearity is caused by intermittent contact while the beam is subjected to harmonic excitation.

Towards clinical application of freehand optical ultrasound imaging

Freehand optical ultrasound (OpUS) imaging is an emerging ultrasound imaging paradigm that uses an array of fibre-optic, photoacoustic ultrasound sources and a single fibre-optic ultrasound detector to perform ultrasound imaging without the need for electrical components in the probe head. Previous freehand OpUS devices have demonstrated capability for real-time, video-rate imaging of clinically relevant targets, but have been hampered by poor ultrasound penetration, significant imaging artefacts and low frame rates, and their designs limited their clinical applicability. In this work we present a novel freehand OpUS imaging platform, including a fully mobile and compact acquisition console and an improved probe design. The novel freehand OpUS probe presented utilises optical waveguides to shape the generated ultrasound fields for improved ultrasound penetration depths, an extended fibre-optic bundle to improve system versatility and an overall ruggedised design with protective elements to improve probe handling and protect the internal optical components. This probe is demonstrated with phantoms and the first multi-participant in vivo imaging study conducted with freehand OpUS imaging probes, this represents several significant steps towards the clinical translation of freehand OpUS imaging.

STDAN: Deformable Attention Network for Space-Time Video Super-Resolution.

The target of space-time video super-resolution (STVSR) is to increase the spatial-temporal resolution of low-resolution (LR) and low-frame-rate (LFR) videos. Recent approaches based on deep learning have made significant improvements, but most of them only use two adjacent frames, that is, short-term features, to synthesize the missing frame embedding, which cannot fully explore the information flow of consecutive input LR frames. In addition, existing STVSR models hardly exploit the temporal contexts explicitly to assist high-resolution (HR) frame reconstruction. To address these issues, in this article, we propose a deformable attention network called STDAN for STVSR. First, we devise a long short-term feature interpolation (LSTFI) module that is capable of excavating abundant content from more neighboring input frames for the interpolation process through a bidirectional recurrent neural network (RNN) structure. Second, we put forward a spatial-temporal deformable feature aggregation (STDFA) module, in which spatial and temporal contexts in dynamic video frames are adaptively captured and aggregated to enhance SR reconstruction. Experimental results on several datasets demonstrate that our approach outperforms state-of-the-art STVSR methods. The code is available at https://github.com/littlewhitesea/STDAN.