Frame Generation Research Articles

System Science Can Relax the Tension Between Data and Theory

The actual hype around machine learning (ML) methods has pushed the old epistemic struggle between data-driven and theory-driven scientific styles well beyond the academic realm. The potential consequences of the widespread adoption of ML in scientific work have fueled a harsh debate between opponents predicting the decay of basic curiosity-driven science and enthusiasts hoping for the advent of a ‘theory-free’ objective science. In this work, I suggest how the system science style of reasoning could drastically de-potentiate this (sometimes deceptive) opposition through the generation of multi-purpose relational theoretical frames stemming from the network paradigm. The recognition of the virtual non-existence of purely ‘theoryfree’ approaches and the need for a careful balancing of theoretical and empirical contributions is the main claim of the present work.

Enhanced neural video compression for cloud gaming videos with aligned frame generation

The burgeoning popularity of cloud gaming makes it critical for efficient video compression to relieve the growing bandwidth pressure. While existing neural video coding approaches have demonstrated strong compression potential on natural videos, there is an absence of efficient neural codecs dedicated to gaming videos. To bridge this gap, in this paper, we propose an end-to-end neural video compression method designed specifically for cloud gaming videos. By effectively utilizing the unique camera motion information inherent to cloud gaming, the previous reconstructed frame is maximally aligned to the current frame through a learning-based module with multiple losses, which then replaces the previous reconstructed frame for optical flow estimation. By significantly reducing the displacement between two consecutive frames caused by camera motion, the motion estimation accuracy is enhanced, effectively handling the large and abrupt motion scenarios frequently present in gaming videos. Furthermore, the aligned tensor obtained in the previous step is used to enhance the latent prior of the entropy model, providing a superior temporal prior for coding. Extensive experimental results demonstrate the superior performance of our proposed method compared to one of the previous state-of-the-art approaches, DCVC-HEM, providing significant progress in end-to-end neural compression in cloud gaming videos.

ДОСЛІДЖЕННЯ ЧИСЛОВОЇ ВИПАДКОВОЇ ПОСЛІДОВНОСТІ, ЩО ОДЕРЖАНА З ВЕБ КАМЕРИ

The investigation object of this paper is a random numbers sequence (RNS) obtained from a single frame of a webcam, which can be practically used as a source of chaos for a hardware RNS generator. The problem under consideration was to calculate the primary statistical characteristics of the RNS obtained from the webcam frame and compare them with the crypto-resistance requirements. The obtained statistics confirmed the hypothesis of a high level of chaos using a webcam. In successive frames from image capture (40 milliseconds gap), the photodiode matrix records changes that are not visible to the human eye. Even in absolute darkness , 60% of the changes in the brightness values of the matrix pixels are recorded. Which is 10% higher than the software engineering crypto-resistance requirements for the avalanche effect parameter. This is probably explained by the chaos that accompanies the stochastic interaction of photons of light with the atoms of the material of the sensor. Thus, it is possible to speak about a high level of chaos in the generated RNS. The peculiarity of the research is that for the purity of the experiment, frame generation was carried out in total darkness (approx. luminosity 10-4 lux), and a uniformly illuminated (luminosity 200 lux) white surface. Testing the web camera under extreme conditions gives a complete picture of the unpredictability and chaos of RNS generation. The primary investigation of the statistics of the generated RNS showed a good compliance with some crypto-resistance requirements. The method built on this property allows designing an affordable hardware RNS generator in laboratory conditions without the involvement of special equipment.

Gabor frame multipliers and Parseval duals on the half real line

Recently, Gabor analysis on locally compact abelian (LCA) groups has become the focus of an active research. In practice, the time variable cannot be negative. The half real line [Formula: see text] is an LCA group under multiplication and the usual topology, with the Haar measure [Formula: see text]. This paper addresses Gabor frame multipliers and Parseval duals for [Formula: see text]. We introduce and characterize Gabor frame multipliers and Parseval Gabor frame multipliers based on Zak transform matrices. Our Zak transform matrix is essentially different from the conventional Zibulski–Zeevi matrix. It allows us to define Gabor frame generators by designing suitable matrix-valued functions of finite size. We also prove that an arbitrary Gabor frame [Formula: see text] admits a Parseval dual frame/tight dual frame whenever [Formula: see text] are rational numbers not greater than [Formula: see text].

Video Frame Prediction from a Single Image and Events

Recently, the task of Video Frame Prediction (VFP), which predicts future video frames from previous ones through extrapolation, has made remarkable progress. However, the performance of existing VFP methods is still far from satisfactory due to the fixed framerate video used: 1) they have difficulties in handling complex dynamic scenes; 2) they cannot predict future frames with flexible prediction time intervals. The event cameras can record the intensity changes asynchronously with a very high temporal resolution, which provides rich dynamic information about the observed scenes. In this paper, we propose to predict video frames from a single image and the following events, which can not only handle complex dynamic scenes but also predict future frames with flexible prediction time intervals. First, we introduce a symmetrical cross-modal attention augmentation module to enhance the complementary information between images and events. Second, we propose to jointly achieve optical flow estimation and frame generation by combining the motion information of events and the semantic information of the image, then inpainting the holes produced by forward warping to obtain an ideal prediction frame. Based on these, we propose a lightweight pyramidal coarse-to-fine model that can predict a 720P frame within 25 ms. Extensive experiments show that our proposed model significantly outperforms the state-of-the-art frame-based and event-based VFP methods and has the fastest runtime. Code is available at https://npucvr.github.io/VFPSIE/.

Dual generators and dual discriminators generative adversarial network for video anomaly detection

Video anomaly detection (VAD) has garnered substantial attention from researchers due to its broad applications, including fire detection, drop detection, and vibration detection. In the current context of VAD, existing methods prioritize detection efficiency but overlook the impact of motion and appearance information. Additionally, achieving accurate predictions while retaining motion and appearance information poses a significant challenge. This paper proposes a novel semi-supervised method for VAD based on Generative Adversarial Network (GAN) structures with dual generators and dual discriminators, namely Dual-GAN. The future frame generator utilizes an improved encoder-decoder network to preserve more spatial information. Motion information for the future flow generator is obtained by estimating optical flow between reconstruction frames, complementing the optical flow between prediction frames. The introduction of a frame discriminator and a motion discriminator against the frame generator enhances the realism of prediction frames, which facilitates the identification of unexpected abnormal events. This method significantly outperforms comparative approaches in synthesizing video frames and predicting future flows, showcasing its effectiveness in handling diverse video data. Extensive experiments are performed on four publicly available datasets to ensure a comprehensive evaluation of the model performance. Further exploration could include refining the model architecture, exploring additional datasets, and adapting the methodology to specific application domains.

User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networks

The Metaverse emerging as maturing technologies are empowering the different facets. Virtual Reality (VR) technologies serve as the backbone of the virtual universe within the Metaverse to offer a highly immersive user experience. As mobility is emphasized in the Metaverse context, VR devices reduce their weights at the sacrifice of local computation abilities. In this paper, for a system consisting of a Metaverse server and multiple VR users, we consider two cases of (i) the server generating frames and transmitting them to users, and (ii) users generating frames locally and thus consuming device energy. As Metaverse emphasizes on the accessibility for all users anywhere and anytime, the users can have totally different characteristics, devices and demands. In this paper, the channel access arrangement (including the decisions on frame generation location), and transmission powers for the downlink communications from the server to the users are jointly optimized by our proposed user-centric Deep Reinforcement Learning (DRL) algorithm, namely User-centric Critic with Heterogenous Actors (UCHA). Comprehensive experiments demonstrate that our UCHA algorithm leads to remarkable results under various requirements and constraints.

Audio-Driven Talking Video Frame Restoration

Talking video frames occasionally drop while streaming for reasons like network errors, which greatly hurts the online team collaboration and user experiences. Directly generating the dropped frames from the remaining ones is unfavorable, since a person's lip motion is usually non-linear and thus hard to be restored when consecutive frames are missing. Nevertheless, the audio content provides strong signals for lip motion and is less likely to drop during transmitting. Inspired by this, as an initial attempt, we present the task of audio-driven talking video frame restoration in this paper, i.e., restoring dropped video frames by jointly leveraging the audio and remaining video frames. Towards the high-quality frame generation, we devise a cross-modal frame restoration network. This network aligns the complete audio content with video frames, precisely identifies and sequentially generates the dropped frames. To justify our model, we construct a new dataset, Talking Video Frames Drop, TVFD for short, consisting of 2.5K video and 144K frames in total. We conduct extensive experiments over TVFD and another publicly accessible dataset - Voxceleb2. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.

An intelligent system for high-density small target pest identification and infestation level determination based on an improved YOLOv5 model

Purpose:A deep learning-based intelligent system has been developed for the identification and detection of high-density small target pests with the aim of addressing the limitations observed in previous detection systems and manual sorting. The novel system promises to overcome issues of low detection accuracy, low detection efficiency, and serious leakage and error detection. It is expected to significantly improve the efficiency of pest detection, thereby offering a potential solution to the challenges posed by the presence of these pests in agricultural settings. Methods:In this paper, we propose an enhanced YOLOv5s model to tackle challenges in pest detection, such as small targets, high densities, and species diversity. Our method embeds Channel Attention (CA) modules into the algorithm and broadens the shallow feature detection scale of the original FPNet, bolstering small-target detection. Network convergence is accelerated using the Kmeans++ algorithm for prior frame generation and YOLOv5s’ intrinsic weights for transfer learning. The Efficient Intersection over Union (EIOU) loss function is adopted to address unbalanced data labels. Furthermore, we develop an algorithm to determine pest infestation levels, transforming detection results into pest grade evaluations. These advancements form the core of our proposed intelligent pest identification and detection system. Results:In our enhanced YOLOv5s model, we assessed the system’s performance using a dataset comprising six prevalent pest species. This dataset was specifically curated to emulate real-life detection scenarios, featuring small pest targets with similar appearances and imbalanced data labels. We compared our model with a common one-stage model and noted average improvements of 5.5%, 2%, and 3.95% in accuracy, recall, and detection precision respectively across both sparse and dense scenarios. Importantly, the enhanced model maintained a detection time of roughly 10ms per image and was only 1MB larger than the original model, illustrating a balance between accuracy and efficiency. Furthermore, our algorithm enables the assessment of pest grades based on the input detection results, illustrating the degree of pest infestation. Conclusion:The present paper describes the development of an intelligent system for the automatic identification and detection of pests. Experimental results demonstrate the high efficiency and efficacy of the proposed system in identifying and detecting common pests, even in complex scenarios characterized by high pest density and data label imbalance. Notably, the proposed system has the potential to assist practitioners in the fields of agriculture and forestry to make informed decisions on pest control, thereby improving work efficiency and productivity.

Numerical model generation of test frames for pre-launch studies of EarthCARE's retrieval algorithms and data management system

Abstract. The Earth Cloud, Aerosol and Radiation Explorer (EarthCARE) satellite consists of active and passive sensors whose observations will be acted on by an array of retrieval algorithms. EarthCARE's retrieval algorithms have undergone pre-launch verifications within a virtual observing system that consists of 3D atmosphere–surface data produced by the Global Environmental Multiscale (GEM) numerical weather prediction (NWP) model, as well as instrument simulators that when applied to NWP data yield synthetic observations for EarthCARE's four sensors. Retrieval algorithms operate on the synthetic observations, and their estimates go into radiative transfer models that produce top-of-atmosphere solar and thermal broadband radiative quantities, which are compared to synthetic broadband measurements, thus mimicking EarthCARE's radiative closure assessment. Three high-resolution test frames were simulated; each measures ∼6200 km along-track by 200 km across-track. Horizontal grid spacing is 250 m, and there are 57 atmospheric layers up to 10 mbar. The frames span wide ranges of conditions and extend over (i) Greenland to the Caribbean, crossing a cold front off Nova Scotia; (ii) Nunavut to Baja California, crossing over Colorado's Rocky Mountains; and (iii) the central equatorial Pacific Ocean, which includes a mesoscale convective system. This report discusses how the test frames were produced and presents their key geophysical features. All data are publicly available and, owing to their high-resolution, could be used to simulate observations for other measurement systems.

An 82-nW 0.53-pJ/SOP Clock-Free Spiking Neural Network With 40-μs Latency for AIoT Wake-Up Functions Using a Multilevel-Event-Driven Bionic Architecture and Computing-in-Memory Technique

This article presents a clock-free spiking neural network (SNN) intelligent inference engine (IIE) for artificial intelligence of things (AIoT) sensor nodes, which often operate in random-sparse-event (RSE) scenarios. The IIE drastically reduces the system’s long-term average (LTA) power consumption, improves energy efficiency, and achieves microsecond level inference latency. Three techniques are proposed: 1) A clock-free SNN architecture without clock tree, frame generator, and arbiter, is driven by the output spikes, which are encoded with level-crossing (LC) sampling method; the circuit activity is completely related to event activity and spike rates, dramatically reducing the overall power consumption and latency. 2) The bioinspired leaky-integrate-fire (LIF) neurons directly extract the time-domain information from asynchronous spikes, reducing the network size and number of operations. 3) The computing-in-memory (CIM) and mixed-signal synapse-neuron circuits are employed to increase the SNN parallelism and avoid weight movements, thus improving the energy efficiency and response speed. The measured LTA power is bounded at 82 nW while the event-driven chip is on call and waiting for events; the energy efficiency is 0.53 pJ per synapse operation (SOP), only 1/3 that of state-of-the-art methods at 4bit weights even with 180 nm technology. We demonstrate electrocardiogram (ECG) recognition as a typical AIoT application, and the power consumption is less than 350 nW. The measured accuracy of abnormal ECG detection is 90.5%. Moreover, the latency is only 40 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu $</tex-math> </inline-formula> s to realize real-time NN inference. This work provides an effective solution for AIoT nodes that require both ultralow power and fast response.

Operation Control Method for High-Speed Maglev Based on Fractional-Order Sliding Mode Adaptive and Diagonal Recurrent Neural Network

The speed profile tracking calculation of high-speed maglev trains is mainly affected by running resistance. In order to reduce the adverse effects and improve tracking accuracy, this paper presents a maglev train operation control method based on a fractional-order sliding mode adaptive and diagonal recurrent neural network (FSMA-DRNN). First, the kinematic resistance equation is established due to the three types of resistance that occur during the actual operation of a train: air resistance, guide eddy current resistance, and suspension frame generator coil resistance. Then, the FSMA-DRNN control law and parameter update law are designed, and a FSMA-DRNN operation controller is composed of three parts: speed feed forward, fractional-order sliding mode adaptive equivalent control, and diagonal recurrent neural network resistance compensation. Furthermore, by using the designed operation controller, it is proven effective by the Lyapunov theory for the stability of the closed-loop control system. Apart from the proposed theoretical analysis, the proposed approaches are verified by experiments on the high-speed maglev hardware-in-the-loop simulation platform Rt-Lab, in line with the 29.86 km test line and a five-car train from the Shanghai maglev, showing the effectiveness and superiority for operation optimization.

Non‐linear Rough 2D Animation using Transient Embeddings

AbstractTraditional 2D animation requires time and dedication since tens of thousands of frames need to be drawn by hand for a typical production. Many computer‐assisted methods have been proposed to automatize the generation of inbetween frames from a set of clean line drawings, but they are all limited by a rigid workflow and a lack of artistic controls, which is in the most part due to the one‐to‐one stroke matching and interpolation problems they attempt to solve. In this work, we take a novel view on those problems by focusing on an earlier phase of the animation process that uses rough drawings (i.e., sketches). Our key idea is to recast the matching and interpolation problems so that they apply to transient embeddings, which are groups of strokes that only exist for a few keyframes. A transient embedding carries strokes between keyframes both forward and backward in time through a sequence of transformed lattices. Forward and backward strokes are then cross‐faded using their thickness to yield rough inbetweens. With our approach, complex topological changes may be introduced while preserving visual motion continuity. As demonstrated on state‐of‐the‐art 2D animation exercises, our system provides unprecedented artistic control through the non‐linear exploration of movements and dynamics in real‐time.

A 915–1220 TOPS/W, 976–1301 GOPS Hybrid In-Memory Computing Based Always-On Image Processing for Neuromorphic Vision Sensors

Neuromorphic vision sensors (NVSs) save energy and reduce data at the source by asynchronously recording changes in temporal contrast. Thus, NVS provides an opportunity to exploit temporal and spatial redundancy in video streams by enabling the following deep neural network (DNN) processor for object recognition to process only foreground object regions in valid frames. However, the NVS data inevitably contains noise leading to false frame generation. Moreover, objects may be fragmented due to a lack of events leading to wrong object region proposals (RPs). Hence, it is important to have an always ON image processor to perform image restoration (IR) and RP operations for NVS data. In this article, we propose a hybrid memory bitcell with collocated static random access memory (SRAM) and dynamic random access memory (DRAM) consisting of 11 transistors [11T-collocated SRAM and DRAM (CRAM)] to perform in-memory computing (IMC)-based IR and RP for event-based binary image (EBBI) frame from a stationary NVS. We propose IMC-based charge diffusion for IR (denoise and region filling) by enabling a 2-D interconnection of bitcells across the whole array for globally parallel computing. The proposed CRAM supports projection mode for IMC-based RP, which enables 1-D projection of objects on the horizontal and vertical axes and finds regions through a recently proposed iterative algorithm. We also proposed an RP update (RPU) algorithm and hardware to improve RP accuracy by 1.6 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> over the prior art. Implemented in 65 nm CMOS, the 320 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 240 quarter video graphics array (QVGA) macro achieves a maximal energy efficiency (throughput) of 1220 TOPS/W (1301 GOPS) without RPU and 915 TOPS/W (976 GOPS) with RPU, both of which are superior to the prior art. We also show that the accuracy of IR and RP obtained by the proposed architecture is better than earlier methods.

Female Genius: Eliza Harriot and George Washington at the Dawn of the Constitution

Female Genius: Eliza Harriot and George Washington at the Dawn of the Constitution

Perceptions into causes and consequences of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) variants

AbstractSevere acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) variants are emerging worldwide and pathogenicity varies widely from no symptoms to death. The SARS‐CoV‐2 is evolving as lineages like Alpha, Beta, Gamma, Epsilon, Iota, Delta, and Omicron in the course of time. The main reasons for such viral evolution are (a) the imperfect nature of SARS‐CoV‐2 RNA polymerase, and viral exonuclease mediated proofreading functions resulting in the generation of mutations in viral genomes; (b) fusions of the 5′ leader sequence to unexpected 3′ sites, and transcription regulatory sequences (TRSs) in subgenomic RNAs (sgRNAs), which result in the generation of structural variants and novel open reading frames; (c) these viruses are combated by the host type I interferons (IFNs). In such a process IFNs upregulate viral RNA editing APOBEC3G/F and ADAR1 genes, which induce mutations in viral genomes. These factors play important roles in causing viral evolution and the emergence of more efficient SARS‐CoV‐2 genomes, which escape the host immune defense system, and vaccine‐elicited antibodies and impede therapeutic strategies. The main challenges we now face are how to control future SARS‐CoV‐2 evolution, the elimination of their deleterious side effects, and the onset of new diseases as aftermaths of SARS‐CoV‐2 infections. Preventive measures like (a) the development of broadly neutralizing antibodies and novel vaccines, therapies based on genomics and proteomics data will help in avoiding, and/or minimizing SARS‐CoV‐2 infections; (b) targeted therapies, application of patient‐based precision medicine methodology can help in achieving the goal and avoiding unwanted deleterious side effects and the onset of SARS‐CoV‐2 infections mediated several diseases in future.

MAC-Based Compression Ratio Improvement for CAN Security

Information security in a controller area network (CAN) is becoming more important as the connections between a vehicle’s internal and external networks increase. Encryption and authentication techniques can be applied to CAN data frames to enhance security. To authenticate a data frame, a message authentication code (MAC) needs to be transmitted with the CAN data frame. Therefore, space for transmitting the MAC is required within the CAN frame. Recently, the Triple ID algorithm has been proposed to create additional space in the data field of the CAN frame. The Triple ID algorithm ensures every CAN frame is authenticated by at least 4 bytes of MAC without changing the original CAN protocol. However, since the Triple ID algorithm uses six header bits, there is a problem associated with low data compression efficiency. In this paper, we propose an algorithm that can remove up to 15 bits from frames compressed with the Triple ID algorithm. Through simulation using CAN signals of a Kia Sorento vehicle and an LS Mtron tractor, we show that the generation of frames containing compressed messages of 4 bytes or more is reduced by up to 99.57% compared to the Triple ID method.

Demonstration of PS-QAM Based Flexible Coherent PON in Burst-Mode With 300G Peak Rate and Ultra-Wide Dynamic Range

Coherent passive optical network (PON) has been proposed to support the next generation high-speed optical access network owing to the superior receiver sensitivity and thereby the extended power budget for 100 G-and-beyond. In addition, PON's flexibility and adaptability are gaining increasing interest. Flexible PON is believed to be a powerful solution in optimizing the overall capacity based on the available channel conditions, i.e., the optical path loss (OPL) and the associated signal-to-noise ratio and nonlinearity impairments of the different groups of optical network units. In this study, we demonstrate the rate-flexible coherent TDM-PON in burst-mode based on probabilistic-shaping (PS) QAM modulations and local oscillator (LO) power adjustment technology, achieving a 300G peak rate and ultra-wide dynamic range. The principle, the burst-mode frame generation as well as the burst-mode signal processing are discussed in detail. To indicate the overall performance of achievable capacity and flexibility, a new metric, namely, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dynamic-range and net-rate product</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DRNRP</i> ) is introduced as the indicator to show the optimized throughput. Finally, we experimentally demonstrated a flexible coherent TDM-PON in burst-mode based on 25-GBaud PS-QAMs with the dynamic range of >35-dB in back-to-back. A record <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DRNRP</i> up to 7,104 dB·Gbps was achieved in the current system with net data rate varying from 255-Gbits/s to 85-Gbits/s as the OPL varies over 20-km fiber transmission.

High-Performance Plant Pest and Disease Detection Based on Model Ensemble with Inception Module and Cluster Algorithm.

Protecting crop yields is the most important aspect of agricultural production, and one of the important measures in preserving yields is the control of crop pests and diseases; therefore, the identification of crop pests and diseases is of irreplaceable importance. In recent years, with the maturity of computer vision technology, more possibilities have been provided for implementing plant disease detection. However, although deep learning methods are widely used in various computer vision tasks, there are still limitations and obstacles in practical applications. Traditional deep learning-based algorithms have some drawbacks in this research area: (1) Recognition accuracy and computational speed cannot be combined. (2) Different pest and disease features interfere with each other and reduce the accuracy of pest and disease diagnosis. (3) Most of the existing researches focus on the recognition efficiency and ignore the inference efficiency, which limits the practical production application. In this study, an integrated model integrating single-stage and two-stage target detection networks is proposed. The single-stage network is based on the YOLO network, and its internal structure is optimized; the two-stage network is based on the Faster-RCNN, and the target frame size is first clustered using a clustering algorithm in the candidate frame generation stage to improve the detection of small targets. Afterwards, the two models are integrated to perform the inference task. For training, we use transfer learning to improve the model training speed. Finally, among the 37 pests and 8 diseases detected, this model achieves 85.2% mAP, which is much higher than other comparative models. After that, we optimize the model for the poor detection categories and verify the generalization performance on open source datasets. In addition, in order to quickly apply this method to real-world scenarios, we developed an application embedded in this model for the mobile platform and put the model into practical agricultural use.

Generation of Multiple Frames for High Resolution Video SAR Based on Time Frequency Sub-Aperture Technique

Video Synthetic Aperture Radar (ViSAR) operating in spotlight mode has received widespread attention in recent years because of its ability to form a sequence of SAR images for a region of interest (ROI). However, due to the heavy computational burden of data processing, the application of ViSAR is limited in practice. Although back projection (BP) can avoid unnecessary repetitive processing of overlapping parts between consecutive video frames, it is still time-consuming for high-resolution video-SAR data processing. In this article, in order to achieve the same or a similar effect to BP and reduce the computational burden as much as possible, a novel time-frequency sub-aperture technology (TFST) is proposed. Firstly, based on azimuth resampling and full aperture azimuth scaling, a time domain sub-aperture (TDS) processing algorithm is proposed to process ViSAR data with large coherent integration angles to ensure the continuity of ViSAR monitoring. Furthermore, through frequency domain sub-aperture (FDS) processing, multiple high-resolution video frames can be generated efficiently without sub-aperture reconstruction. In addition, TFST is based on the range migration algorithm (RMA), which can take into account the accuracy while ensuring efficiency. The results of simulation and X-band airborne SAR experimental data verify the effectiveness of the proposed method.