Real-time Digital Video Research Articles

Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework

The rhythmic movement of bioinspired robotic fish brings about undesirable visual jitter. Note that the unstable camera path of this kind of robot is characterized by obvious regularity and predictability, of which traditional stabilization methods have not made full advantage. This article proposes a novel estimation-and-prediction framework for real-time digital video stabilization of bioinspired robotic fish. First, based on the attitude information of an inertial measurement unit (IMU), a camera–IMU model is established, where the homography transformation with eight degrees of freedom (DOFs) is reduced to translation transformation with two DOFs. Second, traditional optical flow and gray projection methods as well as a novel translation estimation network are employed to estimate the translations between consecutive frames. Third, a lightweight long short-term memory (LSTM) network is constructed, allowing remarkable prediction and smoothing of the camera path. Finally, aquatic experiments under various scenarios are conducted on a manta-inspired robot, demonstrating the effectiveness of the proposed method. Specifically, compared with the state-of-the-art commercial offline stabilization software, our online stabilization algorithm achieves approximate visual stability and remarkably faster stabilization speed. The obtained results shed light on visual sensing and control applications of bioinspired underwater vehicles.

Real-Time Optical-Wireless Video Surveillance System for High Visual-Fidelity Underwater Monitoring

Real-time, dynamic, and visual monitoring of underwater scenes would become an important research topic with the development of underwater wireless optical communication (UWOC), autonomous underwater vehicle, and video fusion technologies. To this end, we developed a UWOC-based 2K real-time digital video surveillance prototype named AquaF-seer. Using a light-emitting diode and an avalanche photodetector based diffused line-of-sight UWOC system, real-time video transmission with a high resolution of 1920 × 1080 pixels is achieved over various channels, i.e., 1.5-m free space channel, pure water channel, pure water channel with 1.53-mL/s bubble-induced turbulence, pure water channel with 42.40-mL/s bubble-induced turbulence, simulated pure sea water channel, and coastal ocean water channel. Moreover, 46-m and 5-m video monitoring are implemented in free space and an outdoor diving pool. It indicates the reliability of the prototype, which is the first step to realize underwater visual monitoring in future human-robot interaction applications.

Precautionary measures before tailoring and commencing a tele-supervised home-based exercise oncology program for older patients with cancer and post-treatment cancer survivors in the COVID-19 era

A tele-supervised home-based exercise oncology program can be defined as the use of two-way real-time digital video conferencing technologies for the purpose of clinical supervision of exercise practice, tolerance, adherence, and progression among older patients with cancer and post-treatment cancer survivors to ensure safety, and to confirm that the patients are carrying out the exercise routine in the right manner at home. Precautions should be taken before tailoring and commencing a tele-supervised home-based exercise oncology program for older patients with cancer undergoing treatment and cancer survivors after treatment in the era of coronavirus disease (COVID-19) pandemic [1,2].

UHD 8K energy-quality scalable HEVC intra-prediction SAD unit hardware using optimized and configurable imprecise adders

Real-time digital video coding became a mandatory feature in current consumer electronic devices due to the popularization of video applications. However, efficiently encoding videos is an extremely processing/energy-demanding task, especially at high resolutions and frame rates. Thus, the limited energy resources and the dynamically varying system status (such as workload, battery level, user settings, etc.) require energy-efficient solutions capable to support run-time energy-quality scalability. In this work, we present an energy-quality scalable SAD Unit hardware architecture for the HEVC intra-frame prediction targeting real-time processing of UHD 8K (7680 × 4320) videos at 60 frames per second. Approximate computing is used to provide energy-quality scalability by employing configurable imprecise operators. The proposed Energy-Quality scalable architecture supports four operation points: precise computing, and 3-bit, 5-bit or 7-bit imprecision. When implemented in a 45-nm technology using Nangate standard cells library and running at 269 MHz, the proposed architecture consumes from 8.42 to 7.38 mJ to process each UHD 8K frame, according to the selected imprecision level. As a drawback, the coding efficiency (measured in BD rate) is reduced from 0.28 to 1.72%. Compared to the related works, this is the only intra-frame prediction SAD unit able to provide energy-quality scalability.

Стабилизация видеоизображения в режиме реального времени с использованием MEMS-датчиков

This article describes our ongoing research on real-time digital video stabilization using MEMS-sensors. The authors propose to use the described method for stabilizing the video that is transmitted to the mobile robot operator who controls the vehicle remotely, as well as increasing the precision of video-based navigation for subminiature autonomous models. The article describes the general mathematical models needed to implement the video stabilization module based on the MEMS sensors readings. These models includes the camera motion model, frame transformation model and rolling-shutter model. The existing approaches to stabilization using sensors data were analyzed and considered from the point of view of the application in a real-time mode. This article considers the main problems that came up during the experiments that were not resolved in the previous research papers. Such problems include: calibration of the camera and sensors, synchronization of the camera and sensors, increasing the accuracy of determining the camera position from sensors data. The authors offer possible solutions to these problems that would help improve quality of the work of existing algorithms, such as a system for parallel synchronized recording of video and sensor data based on the Android operating system. As the main result, the authors represent a framework for implementing video stabilization algorithms based on MEMS sensors readings.

Hardware implementation and validation of a traffic road sign detection and identification system

Reconfigurability and parallel computing capability of field programmable gate array (FPGA) devices are highly exploited in real-time digital image and video processing applications. In this field, real-time traffic road signs detection systems present a huge interest since they help to assist drivers and decrease accidents. In this paper, we propose an FPGA-based hardware implementation of road signs detection and identification system. The proposed system can achieve real-time video constraints while assuring a high-level accuracy in terms of detection rate. The performance of the system in terms of processing latency was evaluated relatively to the reaction distance, the braking distance and the vehicle speed. The evaluation results show that our system can support real-time driving conditions until the speed of 110 km/h. To prove the validity of the proposed implementation, a hardware co-simulation strategy was applied. This is based on the use of Matlab/Xilinx system generator. A comparison of the co-simulation results shows the effectiveness of the developed architecture.

Security in big data

Security in big data

A compact hardware architecture for digital image stabilization using integral projections

We present a hardware architecture for real-time digital video stabilization for high-performance embedded systems. The stabilization algorithm analyzes the current and past video frames and obtains a motion estimation vector, which is then filtered to isolate unwanted camera movements from intentional panning. The vector is then used to correct the output video frame. The paper describes our hardware architecture for motion estimation, filtering and correction and its implementation on a Xilinx Spartan-6 LX45 FPGA. We evaluate our results on several benchmark video sequences, both visually and using the Inter-frame Transformation Fidelity index (ITF). Running on the 640×480-pixel video output of an infrared camera, our FPGA implementation successfully compensates involuntary camera motion at a maximum throughput of 104.15 frames per second with a 100MHz clock. The power consumption added to the FPGA by the image stabilization core is only 24.16mW.

The pedagogy of the observed: how does surveillance technology influence dance studio education?

A local trend in commercial dance studio education is the implementation of real-time digital video surveillance. This case study explores how digital video cameras in the dance studio environment affect asymmetrical power relationships already present in the commercial studio setting, as well as how surveillance impacts feminist pedagogical strategies. The panoptic mechanism in the commercial studio discussed in this case study consists of studio video cameras with live feed to monitors in the waiting area, the front desk area, as well as the director’s office. This research seeks to initiate a discussion regarding how such mediation may intensify the already panoptic atmosphere of a dance studio. Survey results indicate some emphasis placed by the parents on their positionality as client-consumers, as well as their desire to be involved in their children’s dance education, which results in their support of the surveillance. Qualitative research reveals how the teachers’ feminist pedagogical strategies are affected by video surveillance. This research adds a new dimension to the discussion around the panopticism of dance studios by focusing on an audience mediated though digital video cameras.

QoS-Optimized Adaptive Multi-layer (OQAM) architecture of wireless network for high quality digital video transmission

Quality of Service (QoS) optimization is an important design goal in wireless video transmission. The application (APP) layer 802.11e medium access control (MAC) layer, and physical (PHY) layer of the wireless protocol stack can be jointly designed for the exchange of information. This will optimize the performance of wireless network for real-time digital video transmission. This paper proposes an innovative ‘QoS-Optimized Adaptive Multi-layer (OQAM)’ architecture. It ensures reliable and high-quality video transmission over communication channels. The channel exhibits wide variability in throughput, delay, and packet loss. The simulation is performed using EvalVid and Network Simulator-2. Enhanced Intra Prediction (EnIP) algorithm with H.264/MPEG4 Advanced Video Coding (AVC) is proposed at the APP layer. This H.264/MPEG4 AVC is a non-scalable video enCOder/DECoder (CODEC). Improved MAC Adaptive Retry Limit (IMALr) is proposed as smart-packet drop mechanism at the 802.11e MAC layer. It uses packet overflow drop (Pov) and expired-time packet discard (Pex) algorithm. Enhanced Adaptive Forward Error Correction (EnAFEC) is proposed at the PHY layer. The aforementioned algorithms are jointly considered in the proposed OQAM architecture, which increases the coding efficiency, reduces end-to-end delay and increases reliability of wireless network for real-time Video over Wireless (VoW) transmission.

System-Level Multicriteria Modelling of Payload Operational Times for Communication Satellite Missions in LEO

The spacecraft payload operational time (SPOT) is amongst the main critical design considerations that must be optimised and validated during the system engineering analysis of a spacecraft mission. This requirement becomes more demanding for a communication satellite mission that relies on the functionality of the payload for its operation and service delivery. The design principles and performance budgets of the payload module are based on the subsubsystems and subsystems that enable the mission to be accomplished. The SPOT constraint is tied to the spacecraft size, weight and power (SWAP) limitations, operational modes of the subsystems and orbital patterns where the system is being deployed. This paper presents a system-level multicriteria optimisation of POTs for communication satellite (ComSat) missions in low-Earth orbit (LEO). The parameter space investigation (PSI) method was utilised to accomplish the multicriteria optimisation of the payload power and spacecraft mass. In the multicriteria optimisation and vector investigation (MOVI) process, 2048 tests were performed and the PSI was conservatively designed to yield 658 pareto optimal solutions vectors; a pareto optimal solution of 37.119 W for the payload module yielded highly adaptive microsatellite (HAM) mass and power margin of 97.021 kg and 23.366 W respectively. The required maximum subsystem power consumption for the power-storing mode is 25.703 W. From the analysis, the solar array capability was calculated to deliver 116.828 W for the mission; this forms the beginning-of-life design point. The prototype design for the ComSat mission yielded a maximum POT of approximately 494 minutes for the onboard payload processing and communication (downlink and uplink). The findings promise to enhance the design of a reliable and capability-based payload module for real-time digital video and broadband media, mobile services, interactive data transfer and voice communication.

The use of real-time digital video in the assessment of post-operative outcomes of breast reconstruction

Outcome measures of breast reconstruction have used panel assessment of photographs. This provides limited information to the assessor as these images are static.. The aim of this study was to assess whether the use of digital video was a valid assessment tool and to compare its use against photography. 35 patients post-reconstruction underwent photography, digital video capture and completed Breast Cancer Treatment Outcomes Scale (BCTOS) questionnaires. The photographs/video clips were randomised and shown to a 21 member panel. Opinions on aesthetic aspects of the reconstruction were assessed using the BCTOS and Harris scale. Panel inter-rater agreement and patient-panel correlation was assessed using Kendall's Coefficient of Concordance and Spearman's rank correlation tests respectively. There was a "moderate" degree of inter-rater agreement amongst panel members in all categories. Greater agreement occurred using video footage to assess overall cosmesis (0.548 vs 0.507) and shape (0.505 vs 0.486). Video showed a greater degree of correlation with patient self-assessment scores in comparison to photography (0.311 vs 0.281). Video footage coupled with panel assessment is a valid method of assessing post-operative outcomes of breast reconstruction and appears superior to still photographs in terms of inter-rater agreement and correlation with patient self-assessment.

Frame layer CBR encoding to reduce large picture quality fluctuation

Constant bit-rate (CBR) encoding methods used for the real-time digital video applications such as terrestrial high definition television (HDTV) broadcasting often produce large picture quality fluctuation due to constraints on buffer size and buffering delay. In this paper, we propose a simple frame-layer CBR encoding method which reduces large picture quality fluctuation of an encoded video under constant bit-rate and limited buffer size. We first define an intermediate encoder buffer level to be lower than the half of the maximum encoder buffer level. Then, the encoder buffer level is kept from being close to the maximum (or minimum) buffer level during encoding by slightly decreasing (or increasing) the number of bits estimated to encode each frame if the resulting buffer level becomes higher (or lower) than the intermediate buffer level. The experimental results show that the proposed CBR encoding method reduces the average local PSNR variation by 20.6% compared with previous CBR encoding method on average under constant bit-rate and limited buffer size.

Wireless multi-gigabit data transmission using active MMIC components at 220 GHz

In this paper we present a wireless data link for multi-gigabit transmission in the mmW frequency range. The link is realized using waveguide modules based on fully integrated 50 nm metamorphic high electron mobility transistor (mHEMT) transmit and receive millimeter wave monolithic integrated circuits (MMICs) and provides transmission of data rates up to 12.5 Gbit/s modulated on a 220 GHz carrier. The receiver input sensitivity is characterized down to values of −38.6 dBm and shows an error rate of 3.0 × 10−9. Measurements using a coherent link setup show bit error rates better than 1.6 × 10−9 for a data rate of 10 Gbit/s and a transmission distance of 2 m. For a distance of 20 m a real-time Digital Video Broadcasting – Satellite (DVB-S) television transmission is realized. The signal quality after the wireless link is sufficient for a commercial DVB-S receiver to display the television programs.

Real-time perceptual watermarking architectures for video broadcasting

Existing secure embedded systems are primarily cryptography based. However, for effective Digital Rights Management (DRM) of multimedia in the framework of embedded systems, both watermarking and cryptography are necessary. In this paper, a watermarking algorithm and corresponding VLSI architectures are presented that will insert a broadcaster's logo into video streams in real-time to facilitate copyrighted video broadcasting and Internet protocol television (IP-TV). The VLSI architecture is prototyped using a hardware description language (HDL) and when realized in silicon can be deployed in any multimedia producing consumer electronics equipment to enable real-time DRM right at the source. The watermark is inserted into the video stream before MPEG-4 compression, resulting in simplified hardware requirements and superior video quality. The watermarking processing is performed in the frequency (DCT) domain. The system is initially simulated and validated in MATLAB/Simulink ® and subsequently prototyped on an Altera ® Cyclone-II FPGA using VHDL. Its maximum throughput is 43 frames/s at a clock speed of 100 MHz which makes it suitable for emerging real-time digital video broadcasting applications such as IP-TV. The watermarked video is of high quality, with an average Peak-Signal-to-Noise Ratio (PSNR) of 21.8 dB and an average Root-Mean-Square Error (RMSE) of 20.6.

QoS guarantee and provisioning for realtime digital video over mobile ad hoc cdma networks with cross-layer design

In this article we investigate the trade-offs and the constraints for multimedia over mobile ad hoc CDMA networks, and propose a cross-layer distributed power control and scheduling protocol to resolve those trade-offs and constraints in order to provide high-quality video over wireless ad hoc CDMA networks. In particular, a distributed power control and scheduling protocol is proposed to control the incurred delay of video streaming over multihop wireless ad hoc networks, as well as the multiple access interference (MAI). We also investigate the impacts of Doppler spread and noisy channel estimates upon the end-to-end video quality, and provide a relatively robust system which employs a combination of power control and coding/interleaving to combat the effects of Doppler spread by exploiting the increased time diversity when the Doppler spread becomes large. Thus, more robust end-to-end video quality can be achieved over a wide range of channel conditions

Neocortical Hyperexcitability in a Genetic Model of Absence Seizures and Its Reduction by Levetiracetam

To study the effect of the antiepileptic drug levetiracetam (LEV) on the patterns of intrinsic optical signals (IOSs) generated by slices of the somatosensory cortex obtained from 3- and 6-month-old WAG/Rij and age-matched, nonepileptic control (NEC) rats. WAG/Rij and NEC animals were anesthetized with enfluorane and decapitated. Brains were quickly removed, and neocortical slices were cut coronally with a vibratome, transferred to a submerged tissue chamber, and superfused with oxygenated artificial cerebrospinal fluid (aCSF). Slices were illuminated with a dark-field condensor and examined with a x2.5 objective; images were processed with a real time digital video image-enhancement system. Images were acquired before (background) and during electrical stimulation with a temporal resolution of 10 images/s and were displayed in pseudocolors. Extracellular stimuli (200 micros; <4 V) were delivered through bipolar stainless steel electrodes placed in the white matter. IOSs recorded in NEC slices bathed in control aCSF became less intense and of reduced size with age (p < 0.05); this trend was not seen in WAG/Rij slices. Age-dependent decreases in IOS intensity and area size were also seen in NEC slices superfused with aCSF containing the convulsant 4-aminopyridine (4-AP, 5 microM); in contrast, significant increases in both parameters occurred with age in 4-AP-treated WAG/Rij slices (p < 0.05). Under any of these conditions, the IOS intensity and area size slices were larger in WAG/Rij than in NEC slices. LEV (50-500 microM) application to WAG/Rij slices caused dose-dependent IOS reductions that were evident both in control and in 4-AP-containing aCSF and were more pronounced in 6-month-old tissue. These data demonstrate age-dependent IOS modifications in NEC and WAG/Rij rat slices and identify a clear pattern of hyperexcitability that occurs in 6-month-old WAG/Rij neocortical tissue, an age when absence seizures occur in all animals. The ability of LEV to reduce these patterns of network hyperexcitability supports the potential use of this new antiepileptic drug in primary generalized epileptic disorders.

普及モデルPC上のソフトウェアによるリアルタイム動画電子透かし埋め込みシステム

普及モデルPC上のソフトウェアによるリアルタイム動画電子透かし埋め込みシステム

A Real-Time Video Decoder for Digital HDTV

In this paper, an architecture for real-time digital HDTV video decoding is presented. Our architecture is based on a dual decoding datapath controlled in a fixed schedule with an efficient write-back scheme for anchor pictures. The decoding datapath is synchronized at the block (8 × 8 pixels) level. Unlike other decoding approaches such as the slice bar decoding method and the cross-divide method, our scheme reduces memory access contention problem to achieve real-time HDTV decoding without a high cost in overall decoder buffers, architecture, and bus. In comparison to data-flow approaches, our method eliminates the complexity associated with tagged data operations. Our anchor picture storage is organized to minimize page-breaks during memory accesses. Simulation shows that with a relatively low rate 81 MHz clock, our decoder can decode MPEG-2 MP@HL HDTV in real-time, based on an ATSC video format of 1,920 × 1,080 pixels/frame at 30 frames/s, at a bit rate of 18 to 20 Mbps.

Real-time video decoding scheme for HDTV set-top boxes

We present a scheme for real-time digital HDTV video decoding suitable for DVB or ATSC set-top boxes. Our technique is based on a dual decoding datapath controlled in two fixed-scheduling combinations with an efficient memory interface scheme for anchor pictures. Unlike other decoding approaches such as the slice bar decoding method and the crossing-divided method, our scheme reduces memory access contention problem to achieve real-time HDTV decoding without a high cost in overall decoder buffers, architecture, and bus. Our simulation shows that with a relatively low rate 81 MHz clock, our decoder can decode MPEG-2 MP@HL HDTV in real-time, based on a video format of 1920 /spl times/ 1080 pixels/frame at 30 frames/s, at a bit rate of 18-22 Mbps.