Frame Interval Research Articles

Toward Super Temporal Resolution by Suppression of Mixing Effects of Electrons

The theoretical temporal resolution limit of silicon (Si) image sensors is 11.1 ps. The super temporal resolution (STR) is defined as a frame interval less than this limit. The temporal resolution of burst image sensors is significantly affected by “mixing” of signal electrons traveling from different starting positions. For example, the penetration depths of photons incident to a surface of a photodiode (PD) disperse exponentially, resulting in mixing of the photoelectrons from different depths at the other end, which causes a distribution of the arrival times. The temporal resolution is proportional to the standard deviation of the arrival time. Toward STR imaging, this article proposes measures to suppress various mixing phenomena during the travel of the signal electrons: 1) a germanium (Ge) PD for visible light to practically eliminate the effect of vertical mixing caused by the distribution of the penetration depth; 2) an inverted pyramid PD to efficiently suppress the horizontal mixing, keeping a 100% fill factor; 3) a standard column PD with negative potential on the vertical walls to squeeze the trajectory bundle of electrons falling on the guide gate on the front side; and 4) a resistive guide gate to linearize the potential profile to maximize the horizontal drift velocity to minimize the mixing together with the squeezed electron bundle.

Communication‐awareness adaptive resource scheduling strategy for multiple target tracking in a multiple radar system

In this study, a communication-awareness adaptive resource scheduling (CARS) strategy for multiple target tracking in a multiple radar system (MRS) is proposed. The CARS strategy aims to maximise the tracking performance of MRS, whilst minimising the interference from MRS to communication systems (CSs), whose mechanism is to simultaneously control the revisit frame interval of each target and determine the activation of radar nodes, the radar-target assignment and the allocation of transmitted power. Mathematically, the CARS strategy is formulated as an optimization problem, which contains both the continuous variable and the discrete (integer) variable. To tackle the resultant mixed-integer, non-convex, and non-linear problem efficiently, incorporating with the proposed hybrid particle swarm optimization algorithm based on the Kullback–Leibler divergence, a two-stage solution technique is developed to obtain the near-optimal solution. Numerical simulation results are provided to validate the proposed CARS strategy and demonstrate its superiority over the traditional scheduling strategies.

An application of Chasles' theorem to describe the rotation axis of golf swing

The purpose of this study was to propose a method for describing the rotational motion of the entire golf swing by two lines about which the trunk and the lower limbs rotate respectively, and to visualize individual differences in the rotation axes. Three-dimensional whole-body motion data of three male golfers were collected using an IMU motion capture system and a helical axis was computed for each body segment at each interval of frames. The weighted average of the helical axes of the head, upper trunk, and lower trunk segments was computed for each interval to represent the rotation axis of the trunk. Similarly, the weighted average was calculated for femur, leg, and foot segments to represent the rotation axis of the lower limbs. The mean values and standard deviations of the position and orientation of each rotation axis were computed over each of the two swing phases (backswing phase & downswing phase) and animated as cones in computer graphics. Comparisons of the rotation axes among the three golfers revealed that the position and the direction of the axes are unique to each player.

Inferring CT perfusion parameters and uncertainties using a Bayesian approach

Computed tomography perfusion imaging is commonly used for the rapid assessment of patients presenting with symptoms of acute stroke. Maps of perfusion parameters, such as cerebral blood volume (CBV), cerebral blood flow (CBF), and mean transit time (MTT) derived from the perfusion scan data, provide crucial information for stroke diagnosis and treatment decisions. Most CT scanners use singular value decomposition (SVD)-based methods to calculate these parameters. However, some known problems are associated with conventional methods. In this work, we propose a Bayesian inference algorithm, which can derive both the perfusion parameters and their uncertainties. We apply the variational technique to the inference, which then becomes an expectation-maximization problem. The probability distribution (with Gaussian mean and variance) of each estimated parameter can be obtained, and the coefficient of variation is used to indicate the uncertainty. We perform evaluations using both simulations and patient studies. In a simulation, we show that the proposed method has much less bias than conventional methods. Then, in separate simulations, we apply the proposed method to evaluate the impacts of various scan conditions, i.e., with different frame intervals, truncated measurement, or motion, on the parameter estimate. In one patient study, the method produced CBF and MTT maps indicating an ischemic lesion consistent with the radiologist's report. In a second patient study affected by patient movement, we showed the feasibility of applying the proposed method to motion corrected data. The proposed method can be used to evaluate confidence in parameter estimation and the scan protocol design. More clinical evaluation is required to fully test the proposed method.

Improved localization precision via restricting confined biomolecule stochastic motion in single-molecule localization microscopy

Abstract Single-molecule localization microscopy (SMLM) plays an irreplaceable role in biological studies, in which nanometer-sized biomolecules are hardly to be resolved due to diffraction limit unless being stochastically activated and accurately located by SMLM. For biological samples preimmobilized for SMLM, most biomolecules are cross-linked and constrained at their immobilizing sites but still expected to undergo confined stochastic motion in regard to their nanometer sizes. However, few lines of direct evidence have been reported about the detectability and influence of confined biomolecule stochastic motion on localization precision in SMLM. Here, we access the potential stochastic motion for each immobilized single biomolecule by calculating the displacements between any two of its localizations at different frames during sequential imaging of Alexa Fluor-647-conjugated oligonucleotides. For most molecules, localization displacements are remarkably larger at random frame intervals than at shortest intervals even after sample drift correction, increase with interval times and then saturate, showing that biomolecule stochastic motion is detected and confined around the immobilizing sizes in SMLM. Moreover, localization precision is inversely proportional to confined biomolecule stochastic motion, whereas it can be deteriorated or improved by enlarging the biomolecules or adding a post-crosslinking step, respectively. Consistently, post-crosslinking of cell samples sparsely stained for tubulin proteins results in a better localization precision. Overall, this study reveals that confined stochastic motion of immobilized biomolecules worsens localization precision in SMLM, and improved localization precision can be achieved via restricting such a motion.

Evolution of nanosecond laser-induced phase explosion based on a high-speed continuous imaging system

A high-speed continuous imaging system is established based on the principle of multiwavelength shadow imaging with a 15-ns exposure time and 1-ns frame interval. Furthermore, the spatiotemporal evolution of phase explosion products induced by Nd: YAG nanosecond laser ablation of aluminum was observed. The results show that, at the laser flux of 1.34 J/cm2, only shockwave is caused by strong evaporation; above the laser flux of 2.05 J/cm2, the shadow image shows significant explosive particles jet. When the injected laser flux reaches 3.32 J/cm2, the plasma explosion is accompanied by the phase explosion. Moreover, at a laser flux between 1.34 and 2.05 J/cm2 the velocity of shockwave fronts surges owing to the phase explosion. At a laser flux of 2.86–3.32 J/cm2, the wavefront velocity resurges under the effects of phase explosion and plasma explosion.

End-diastolic segmentation of intravascular ultrasound images enables more reproducible volumetric analysis of atheroma burden.

Volumetric intravascular ultrasound (IVUS) analysis is currently performed at a fixed frame interval, neglecting the cyclic changes in vessel dimensions occurring during the cardiac cycle that can affect the reproducibility of the results. Analysis of end-diastolic (ED) IVUS frames has been proposed to overcome this limitation. However, at present, there is lack of data to support its superiority over conventional IVUS. The present study aims to compare the reproducibility of IVUS volumetric analysis performed at a fixed frame interval and at the ED frames, identified retrospectively using a novel deep-learning methodology. IVUS data acquired from 97 vessels were included in the present study; each vessel was segmented at 1 mm interval (conventional approach) and at ED frame twice by an expert analyst. Reproducibility was tested for the following metrics; normalized lumen, vessel and total atheroma volume (TAV), and percent atheroma volume (PAV). The mean length of the analyzed segments was 50.0 ± 24.1 mm. ED analysis was more reproducible than the conventional analysis for the normalized lumen (mean difference: 0.76 ± 4.03 mm3 vs. 1.72 ± 11.37 mm3 ; p for the variance of differences ratio < 0.001), vessel (0.30 ± 1.79 mm3 vs. -0.47 ± 10.26 mm3 ; p < 0.001), TAV (-0.46 ± 4.03 mm3 vs. -2.19 ± 14.39 mm3 ; p < 0.001) and PAV (-0.12 ± 0.59% vs. -0.34 ± 1.34%; p < 0.001). Results were similar when the analysis focused on the 10 mm most diseased segment. The superiority of the ED approach was due to a more reproducible detection of the segment of interest and to the fact that it was not susceptible to the longitudinal motion of the IVUS probe and the cyclic changes in vessel dimensions during the cardiac cycle. ED IVUS segmentation enables more reproducible volumetric analysis and quantification of TAV and PAV that are established end points in longitudinal studies assessing the efficacy of novel pharmacotherapies. Therefore, it should be preferred over conventional IVUS analysis as its higher reproducibility is expected to have an impact on the sample size calculation for the primary end point.

Technical note: Calibration of frame intervals of video recorders using Global Positioning System (GPS) signal as time reference

A method has been developed for calibrating the time intervals of video recorders using a LED panel known as “SEXTA”, which utilizes Global Positioning System (GPS) signal as time reference. This method directly measures the frame intervals frame by frame down to a time resolution of 2 ms of the recordings by video recorders equipped with CCD or CMOS sensors in global shutter or rolling shutter modes respectively. The method has been validated by using a Digital Single Lens Reflex (DSLR) camera and a previous studied dashboard camera. Frame by frame analysis of the calibrated video recording of SEXTA can reveal characteristics of sensors, such as type of sensors (CMOS vs CCD), exposure time, the direction and speed of rolling shutter for CMOS sensors as well as the common characteristics of dashboard cameras such as frame skipping.

Laryngeal Adductor Reflex Movement Latency Following Tactile Stimulation.

To measure the latency of laryngeal adductor reflex (LAR) motion onset at 2 laryngopharyngeal subsites using calibrated aesthesiometers. Cross-sectional. Academic institution. Twenty-one asymptomatic, healthy subjects (11 male, 10 female) underwent laryngopharyngeal sensory testing with tactile stimuli delivered to the aryepiglottic fold and medial pyriform sinus using 30-mm Cheung-Bearelly monofilaments (4-0 and 5-0 nylon sutures) via channeled flexible laryngoscope. The LAR onset latency, defined as the first visual detection of ipsilateral vocal fold adduction following tactile stimulation, was measured with frame-by-frame analysis of video recordings. The overall mean LAR latency across both subsites and stimulation forces was 176.6 (95% CI, 170.3-183.0) ms, without significant difference between subsites or forces. The critical value for LAR response latency prolongation at the .01 significance level was 244 ms. At 30 frames/s video capture resolution, LAR response latency ≥8 frame intervals would indicate abnormal prolongation. Aesthesiometer-triggered LAR latency appears to be invariant over an 8.7-dB force range and between the aryepiglottic fold and medial pyriform sinus subsites in controls. Laryngeal adductor reflex latency incongruences between stimulation forces or laryngopharyngeal subsites may serve as pathophysiological features to dissect mechanisms of upper aerodigestive tract disorders. Level 3B.

Comparative Analysis of PI controller, Fuzzy controller and Interval Type-2 Fuzzy based Performance of Dynamic Voltage Restorer for Compensation of Voltage Sag and Swell

In today’s scenario, the power quality issues like voltage sag and swell are major concerns in power systems. On the distribution, side to overcome this issue a custom power device DVR(Dynamic Voltage Restorer) is proven to be a more effective solution that is lower in cost, smaller in size, and offers a faster dynamic response for the protection of sensitive load. DVR is a series compensating device which operates as a voltage booster. The application of DVR is entitled to be a flawless compensating device for the compensation of voltage sag and swell. This paper presents the modeling and simulation using various controller topologies like PI controller based on synchronous reference frame algorithm, fuzzy controller, and interval type 2 fuzzy controller. The results are then verified using a Matlab-Simulink environment.

Single-shot spectral-volumetric compressed ultrafast photography

In ultrafast optical imaging, it is critical to obtain the spatial structure, temporal evolution, and spectral composition of the object with snapshots in order to better observe and understand unrepeatable or irreversible dynamic scenes. However, so far, there are no ultrafast optical imaging techniques that can simultaneously capture the spatial–temporal–spectral five-dimensional (5D) information of dynamic scenes. To break the limitation of the existing techniques in imaging dimensions, we develop a spectral-volumetric compressed ultrafast photography (SV-CUP) technique. In our SV-CUP, the spatial resolutions in the x, y and z directions are, respectively, 0.39, 0.35, and 3 mm with an 8.8 mm × 6.3 mm field of view, the temporal frame interval is 2 ps, and the spectral frame interval is 1.72 nm. To demonstrate the excellent performance of our SV-CUP in spatial–temporal–spectral 5D imaging, we successfully measure the spectrally resolved photoluminescent dynamics of a 3D mannequin coated with CdSe quantum dots. Our SV-CUP brings unprecedented detection capabilities to dynamic scenes, which has important application prospects in fundamental research and applied science.

Deep Learning-Driven Gaussian Modeling and Improved Motion Detection Algorithm of the Three-Frame Difference Method

To enhance the effect of motion detection, a Gaussian modeling algorithm is proposed to fix holes and breaks caused by the conventional frame difference method. The proposed algorithm uses an improved three-frame difference method. A three-frame image sequence with one frame interval is selected for pairwise difference calculation. The logical “OR” operation is used to achieve fast motion detection and to reduce voids and fractures. The Gaussian algorithm establishes an adaptive learning model to make the size and contour of the motion detection more accurate. The motion extracted by the improved three-frame difference method and Gaussian model is logically summed to obtain the final motion foreground picture. Moreover, a moving target detection method, based on the U-Net deep learning network, is proposed to reduce the dependency of deep learning on the number of training datasets. It helps the algorithm to train models on small datasets. Next, it calculates the ratio of the number of positive and negative samples in the dataset and uses the reciprocal of the ratio as the sample weight to deal with the imbalance of positive and negative samples. Finally, a threshold is set to predict the results for obtaining the moving object detection accuracy. Experimental results show that the algorithm can suppress the generation and rupture of holes and reduce the noise. Also, it can quickly and accurately detect movement to meet the design requirements.

Ultra-high speed video observations of intracloud lightning flash initiation

This study describes results from video observations of five intracloud flashes located ≤ 20 km from the camera and recorded with 6.1 µs exposure time and 6.66 µs frame intervals. Video data are supported with electric field change (E-change) and VHF measurements, with emphasis on the flash initiating event (IE) and initial breakdown (IB) stage. In four of the five flashes, the IE is accompanied by weak luminosity, ≤ 5% above background, lasting for 300–500 µs. Two of these four IEs were positive Narrow Bipolar Events (NBEs) with VHF powers of 43 and 990 W; these are the first (known) data showing visible light detected with a positive NBE. Two other IEs with weak luminosity had powers of 0.5 and 1 W, and the IE with no detected luminosity had a VHF power of 3 W. A typical IB cluster consists of several narrow pulses and one classic pulse in E-change data (along with many VHF pulses), and each example flash has 2–10 IB clusters in the first 5–50 ms. The luminosity of IB clusters was substantially greater than IE luminosity, ranging from 10 to 40% above background in four examples, while for one flash with 10 IB clusters, the luminosity range was 35–360% above background (average 190%). Luminosity durations of IB clusters were 520–1750 µs with average 1210 µs. For both IEs and IB clusters, increases in the detected luminosity were closely timed with substantial VHF emissions and decreased when VHF emissions weakened.

Single-Shot Real-Time Ultrafast Imaging of Femtosecond Laser Fabrication

Femtosecond laser fabrication outperforms the traditional fabrication techniques with high precision, high efficiency, low collateral damage and wide applicability, which has shown to be a powerful tool in precision machining. Imaging the ultrafast dynamics of femtosecond laser fabrication is necessary for understanding the processing mechanism and for establishing the corresponding physical models. Up to now, ultrafast measurement techniques based on the pump–probe strategy are the most used methods. However, they are limited by laser energy stability and materials surface uniformity, which have a heavy impact on the dynamic measurement precision of femtosecond laser fabrication. To overcome this limitation of the traditional pump–probe techniques, we developed chirped spectral mapping ultrafast photography (CSMUP), which can achieve single-shot real-time ultrafast imaging with a frame rate of about 250 billion frames per second (temporal frame interval of 4 ps) and a spatial resolution of less than 833 nm. We experimentally imaged the dynamics of femtosecond laser ablation in silicon under a 400 nm femtosecond laser exposure with CSMUP, and the experimental result agreed well with previous theoretical models. CSMUP provides a new strategy to improve the efficiency and accuracy of femtosecond laser fabrication by a single-shot dynamic measurement of the interaction between the femtosecond laser and materials, and it is expected to work as a real-time detection method for various ultrafast phenomena.

Experiment Analysis of CAN-FD Factors

With the growing number of onboard ECUs, the load of classical CAN (Controller Area Network) is aggravated to produce transmission delay issue. A new protocol CAN-FD (Controller Area Network with Flexible Data-Rate) was presented by the BOSCH company to meet the high speed and real-time data exchange needs of the intelligent vehicles and ITS (Intelligent Transport System). For the vehicle electronics, the CAN-FD data network is still growing up. In this paper, the classical CAN and CAN-FD in protocol are compared through experiments. Results show that: (1) the terminal resistors have more effects on the CAN-FD than the CAN; (2) the bit rate of the CAN-FD can be ten times of that of the CAN; and (3) the data transmission efficiency of the CAN-FD network is affected by both the bit rate and the data length. Additionally, the physical network configuration takes decisive effect on the minimum frame interval of the CAN-FD frame.

High-spatial-resolution ultrafast framing imaging at 15 trillion frames per second by optical parametric amplification

We report a framing imaging based on noncollinear optical parametric amplification (NCOPA), named FINCOPA, which applies NCOPA for the first time to single-shot ultrafast optical imaging. In an experiment targeting a laser-induced air plasma grating, FINCOPA achieved 50 fs-resolved optical imaging with a spatial resolution of ∼83 lp / mm and an effective frame rate of 10 trillion frames per second (Tfps). It has also successfully visualized an ultrafast rotating optical field with an effective frame rate of 15 Tfps. FINCOPA has simultaneously a femtosecond-level temporal resolution and frame interval and a micrometer-level spatial resolution. Combining outstanding spatial and temporal resolutions with an ultrahigh frame rate, FINCOPA will contribute to high-spatiotemporal resolution observations of ultrafast transient events, such as atomic or molecular dynamics in photonic materials, plasma physics, and laser inertial-confinement fusion.

Single-shot ultrafast burst imaging using an integral field spectroscope with a microlens array.

To fully utilize the functions of a center-wavelength-sweeping pulse train generated by a free-space angular-chirp-enhanced delay optical layout for a probe laser pulse in sequentially timed all-optical mapping photography (STAMP), we introduced an integral field spectroscopy (IFS) method using a microlens array (MLA) to produce hyperspectral images, referring to the technique as lens array (LA)-STAMP. Compared with the previous STAMP utilizing spectral filtering where a bandpass filter generated hyperspectral images, LA-STAMP achieved much higher optical throughput. In a prototype setup, we used a 60×60 MLA and demonstrated single-shot burst imaging of a femtosecond laser-induced ablation process on a glass surface with 300 ps frame intervals in a 1.8 ns time window. Each frame image was constructed by assembling spectrally dispersed 36×36 monochromatic segments distributed by each lenslet on 5×5 pixels of a CCD camera. The spatial resolution was ∼4.4µm, which was determined by the MLA's pitch and the magnification of the microscope lens. We limited the number of frames to seven in this prototype setup, although it can be scaled to ∼24 with a spatial resolution of ∼1µm by designing IFS with a fine pitch MLA.

Cameras replace human observers in multi‐species aerial counts in Murchison Falls, Uganda

AbstractWildlife counts in Africa and elsewhere are often implemented using light aircraft with ‘rear‐seat‐observer’ (RSO) counting crews. Previous research has indicated thatRSOs often fail to detect animals, and that population estimates are therefore biased. We conducted aerial wildlife surveys in Murchison Falls Protected Area, Uganda, in which we replacedRSOs with high‐definition ‘oblique camera count’ (OCC) systems. The survey area comprises forests, woodlands and grasslands. Four counts were conducted in 2015–2016 using a systematic‐reconnaissance‐flight (SRF) strip‐transect design. Camera inclination angles, focal lengths, altitude and frame interval were calibrated to provide imaged strips of known sample size on the left and right sides of the aircraft. Using digital cameras, 24 000 high‐definition images were acquired for each count, which were visually interpreted by four airphoto interpreters. We used the standard JollyII SRFanalysis to derive population estimates. OurOCCestimates of the antelopes – hartebeest, Uganda kob, waterbuck and oribi – were, respectively, 25%, 103%, 97% and 2100% higher than in the most recentRSOcount conducted in 2014. TheOCCsurveys doubled the 2014RSOestimate of 58 000 Uganda kob to over 118 000. Population size estimates of elephants and giraffes did not differ significantly. Although all fourOCCbuffalo estimates were higher than theRSOestimates – in one count by 60% – these differences were not significant due to the clumped distribution and high variation in herd sizes, resulting in imprecise estimation by sampling. We conclude thatRSOwildlife counts in Murchison have been effective in enumerating elephants and giraffe, but that many smaller species have not been well detected. We emphasize the importance of 60 years ofRSO‐based surveys across Africa, but suggest that new imaging technologies are embraced to improve accuracy.

1000-fps consecutive ultrafast 2D-burst imaging with a sub-nanosecond temporal resolution by a frequency-time encoding of SF-STAMP

We demonstrated real-time, in-situ operation of a single-shot, frequency-time-encoding burst imaging method of sequentially timed all-optical mapping photography utilizing spectral filtering (SF-STAMP) by employing a high-speed camera with the frame rate of a 1000 fps as a detector. We obtained single-shot burst images of a pulse-by-pulse femtosecond laser ablation of a glass by a chirped probe laser pulse with a 1–2-ps frame interval or by a spectrally sweeping burst laser pulse train with a 300-ps interval. We observed burst images of plasma generation during the early stage of laser ablation in a glass with a short frame interval by a chirped probe laser pulse. We also captured burst images of plume and shock-wave generation in air with a 1.5-ns observation time window by a spectrally sweeping probe laser pulse train.

Hyperspectrally Compressed Ultrafast Photography.

The spatial, temporal, and spectral information in optical imaging play a crucial role in exploring the unknown world and unencrypting natural mysteries. However, the existing optical imaging techniques can only acquire the spatiotemporal or spatiospectral information of the object with the single-shot method. Here, we develop a hyperspectrally compressed ultrafast photography (HCUP) that can simultaneously record the spatial, temporal, and spectral information of the object. In our HCUP, the spatial resolution is 1.26 lp/mm in the horizontal direction and 1.41 lp/mm in the vertical direction, the temporal frame interval is 2ps, and the spectral frame interval is 1.72nm. Moreover, HCUP operates with receive-only and single-shot modes, and therefore it overcomes the technical limitation of active illumination and can measure the nonrepetitive or irreversible transient events. Using our HCUP, we successfully measure the spatiotemporal-spatiospectral intensity evolution of the chirped picosecond laser pulse and the photoluminescence dynamics. This Letter extends the optical imaging from three- to four-dimensional information, which has an important scientific significance in both fundamental research and applied science.