Increased Frame Rate Research Articles

Grape Target Detection Method in Orchard Environment Based on Improved YOLOv7

In response to the poor detection performance of grapes in orchards caused by issues such as leaf occlusion and fruit overlap, this study proposes an improved grape detection method named YOLOv7-MCSF based on the You Only Look Once v7 (YOLOv7) framework. Firstly, the original backbone network is replaced with MobileOne to achieve a lightweight improvement of the model, thereby reducing the number of parameters. In addition, a Channel Attention (CA) module was added to the neck network to reduce interference from the orchard background and to accelerate the inference speed. Secondly, the SPPFCSPC pyramid pooling is embedded to enhance the speed of image feature fusion while maintaining a consistent receptive field. Finally, the Focal-EIoU loss function is employed to optimize the regression prediction boxes, accelerating their convergence and improving regression accuracy. The experimental results indicate that, compared to the original YOLOv7 model, the YOLOv7-MCSF model achieves a 26.9% reduction in weight, an increase in frame rate of 21.57 f/s, and improvements in precision, recall, and mAP of 2.4%, 1.8%, and 3.5%, respectively. The improved model can efficiently and in real-time identify grape clusters, providing technical support for the deployment of mobile devices and embedded grape detection systems in orchard environments.

VOXEL Video Streaming Over Wireless Networks

High-density point clouds expressing attractive 3D images are attracting attention. These gigantic media require large bandwidth allocations, making them problematic to stream to resource-constrained hand-held devices. This paper proposes a method for point cloud compression and streaming of large point clouds using a web server. Storing large point cloud videos on a web server allows users to publish data sets without using additional applications or sending large amounts of data ahead of time. HTTP/2 improves transfer efficiency by compressing headers into binary format and reduces latency by sending many multiplexed requests using a single TCP session. The large size of the point cloud data and the current limitations of wireless channels demand point cloud compression methods. MATLAB is used for the purpose of analysis, at results show an average compression ratio of 30:1. Our proposed system achieves lower average latency between successive frames resulting in an increased frame rate to 72.46 fps when streaming 4M points, and 82.51 fps in the case of streaming 800K points compared to much lower rates in conventional work.

Accounting for intensity variation within pixels of Shack-Hartmann wavefront sensors

The accuracy of centroiding algorithms in Shack-Hartmann wavefront sensing is limited by the implicit approximation of uniform pixel illumination. Iterative centroiding algorithms are further limited by the consideration of full pixels to define the image domain for centroiding. Here, we demonstrate two practical and complementary approaches to mitigate both these sources of error. First, we consider partial or ‘fractional’ pixels to maintain centroiding area symmetry around the center of mass. Secondly, we propose methods to perform piece-wise polynomial interpolation to calculate intensity distribution within pixels, which is then used to estimate the centroid within each pixel area. This approach that accounts for intensity non-uniformity across pixels notably reduces centroid errors up to a factor of 5 across lenslet image widths ranging from 1.33 to 3.10 pixels full-width-half-maximum (FWHM). Consequently, wavefront sensing errors decrease from 14 % to 4 %, on average, for FWHM = 1.35 pixels, demonstrating a substantial benefit when the number of pixels per lenslet is minimized to enhance the signal-to-noise ratio or increase frame rate.

Decoupled Edge Physics Algorithms for Collaborative XR Simulations

ABSTRACTThis work proposes a novel approach to transform any modern game engine pipeline, for optimized performance and enhanced user experiences in extended reality (XR) environments decoupling the physics engine from the game engine pipeline and using a client‐server architecture creates a scalable solution, efficiently serving multiple graphics clients on head‐mounted displays (HMDs) with a single physics engine on edge‐cloud infrastructure. This approach ensures better synchronization in multiplayer scenarios without introducing overhead in single‐player experiences, maintaining session continuity despite changes in user participation. Relocating the Physics Engine to an edge or cloud node reduces strain on local hardware, dedicating more resources to high‐quality rendering and unlocking the full potential of untethered HMDs. We present four algorithms that decouple the physics engine, increasing frame rates and Quality of Experience (QoE) in VR simulations, supporting advanced interactions, numerous physics objects, and multiuser sessions with over 100 concurrent users. Incorporating a Geometric Algebra interpolator reduces inter‐calls between dissected parts, maintaining QoE and easing network stress. Experimental validation, with more than 100 concurrent users, 10,000 physics objects, and softbody simulations, confirms the technical viability of the proposed architecture, showcasing transformative capabilities for more immersive and collaborative XR applications without compromising performance.

Effect of frame rate on image quality in cardiology evaluated using an indirect conversion dynamic flat-panel detector.

To verify the effect of the frame rate on image quality in cardiology, we used an indirect conversion dynamic flat-panel detector (FPD). We quantified the input-output characteristics, and determined the modulation transfer function (MTF) and normalized noise power spectrum (NNPS) of the equipment used in cardiology at 7.5, 10, 15, and 30 frames per second (fps). We also calculated the noise power spectrum for still images and videos at all frame rates and obtained the image lag correction factor r. The input-output characteristics and the MTF agreed even when the frame rate was varied. The NNPS tended to decrease uniformly as a function of frequency at increasing frame rates. The factor r decreased as a function of the frame rate, and its minimum value was 30 fps. Our results suggest that high-frame-rate imaging in cardiology using indirect conversion dynamic FPDs is affected by image lag.

Research on Automated Fiber Placement Surface Defect Detection Based on Improved YOLOv7

Due to the black and glossy appearance of the carbon fiber prepreg bundle surface, the accurate identification of surface defects in automated fiber placement (AFP) presents a high level of difficulty. Currently, the enhanced YOLOv7 algorithm demonstrates certain performance advantages in this detection task, yet issues with missed detections, false alarms, and low confidence levels persist. Therefore, this study proposes an improved YOLOv7 algorithm to further enhance the performance and generalization of surface defect detection in AFP. Firstly, to enhance the model’s feature extraction capability, the BiFormer attention mechanism is introduced to make the model pay more attention to small target defects, thereby improving feature discriminability. Next, the AFPN structure is used to replace the PAFPN at the neck layer to strengthen feature fusion, preserve semantic information to a greater extent, and finely integrate multi-scale features. Finally, WIoU is adopted to replace CIoU as the bounding box regression loss function, making it more sensitive to small targets, enabling more accurate prediction of object bounding boxes, and enhancing the model’s detection accuracy and generalization capability. Through a series of ablation experiments, the improved YOLOv7 shows a 10.5% increase in mAP and a 14 FPS increase in frame rate, with a maximum detection speed of 35 m/min during the AFP process, meeting the requirements of online detection and thus being able to be applied to surface defect detection in AFP operations.

Content adaptive spatial–temporal rescaling for video coding optimization

With the increase of resolution and frame rate, videos can provide users with a better viewing experience. However, the amount of data is also greatly increased, making it much more difficult for storage and transmission. To address this challenge, we propose to optimize video coding configurations based on video content, aiming to reduce bitrate while preserving perceptual quality. In our method, we extract hand-crafted spatial–temporal features from original videos to delineate their content characteristics. Subsequently, we employ support vector machines (SVM) to perform video coding optimization through spatial–temporal rescaling, thereby improving video coding efficiency. Our approach is compatible with existing encoders and decoders, which is advantageous in practical applications. Experimental results on videos with different contents demonstrate that our method can predict the optimal encoding configuration with high accuracy, achieving considerable bitrate savings for videos of various resolutions and frame rates with low computational costs.

An OLED display driver IC with high‐gain fast‐slew circuit and on‐the‐fly self‐repair technique for high‐resolution display

AbstractThis paper presents an OLED display driver IC with high‐gain fast‐slew (HGFS) and on‐the‐fly self‐repair (OFSR) circuits for a spatial resolution of WQHD and a frame rate of 120 Hz. The proposed HGFS circuit increases the slewing current of the unity‐gain amplifier of the source driver when the input voltage is largely changed, reducing the settling time of the source driver as the resolution and frame rate increase. To enhance image quality by correcting line defects, the OFSR circuit replaces a faulty channel with a spare dummy channel. The OLED display driver IC was manufactured using a 28‐nm high‐voltage process with 8 V/20 V transistors and has 2880 source drivers for WQHD display. The measurement results indicate a horizontal line time of 2.7 μs with a slew rate of 7.675 V/μs, supporting WQHD resolution and 120‐Hz frame rate. The displayed image demonstrates that the vertical line defect has disappeared with the use of OFSR.

NEURAL NETWORK ARCHITECTURE FOR TEXT DECODING BASED ON SPEAKER'S LIP MOVEMENTS

In this paper, we tested a command recognition system using the SSI approach and conducted a series of experiments on modern solutions based on ALR interfaces. The main goal was to improve the accuracy of speech recognition in cases where it is not possible to use the speaker's non-noisy audio sequence, for example, at a great distance from the speaker or in a noisy environment. The obtained results showed that training the neural network on a GPU accelerator allowed to reduce the training time by 26.2 times using a high-resolution training sample with a size of the selected mouth area of 150 × 100 pixels. The results of the analysis of the selected speech recognition quality assessment metrics (word recognition rate (WRR), word error rate (WER), and character error rate (CER)) showed that the maximum word recognition rate of the speaker's speech is 96.71% and is achieved after 18 epochs of training. If we evaluate the character regonition rate of viseme recognition, the highest rate can be obtained after 13 epochs of training. Future research will focus on the use of depth cameras and stereo vision methods with increased frame rates to further improve the accuracy of voice command decoding in conditions of high background noise.

Compressive video via IR-pulsed illumination.

We propose and demonstrate a compressive temporal imaging system based on pulsed illumination to encode temporal dynamics into the signal received by the imaging sensor during exposure time. Our approach enables >10x increase in effective frame rate without increasing camera complexity. To mitigate the complexity of the inverse problem during reconstruction, we introduce two keyframes: one before and one after the coded frame. We also craft what we believe to be a novel deep learning architecture for improved reconstruction of the high-speed scenes, combining specialized convolutional and transformer architectures. Simulation and experimental results clearly demonstrate the reconstruction of high-quality, high-speed videos from the compressed data.

Interlacing in Atomic Resolution Scanning Transmission Electron Microscopy.

Fast frame rates are desirable in scanning transmission electron microscopy for a number of reasons: controlling electron beam dose, capturing in situ events, or reducing the appearance of scan distortions. While several strategies exist for increasing frame rates, many impact image quality or require investment in advanced scan hardware. Here, we present an interlaced imaging approach to achieve minimal loss of image quality with faster frame rates that can be implemented on many existing scan controllers. We further demonstrate that our interlacing approach provides the best possible strain precision for a given electron dose compared with other contemporary approaches.

A birefringent spectral demultiplexer enables fast hyper-spectral imaging of protoporphyrin IX during neurosurgery

Hyperspectral imaging and spectral analysis quantifies fluorophore concentration during fluorescence-guided surgery1–6. However, acquisition of the multiple wavelengths required to implement these methods can be time-consuming and hinder surgical workflow. To this end, a snapshot hyperspectral imaging system capable of acquiring 64 channels of spectral data simultaneously was developed for rapid hyperspectral imaging during neurosurgery. The system uses a birefringent spectral demultiplexer to split incoming light and redirect wavelengths to different sections of a large format microscope sensor. Its configuration achieves high optical throughput, accepts unpolarized input light and exceeds channel count of prior image-replicating imaging spectrometers by 4-fold. Tissue-simulating phantoms consisting of serial dilutions of the fluorescent agent characterize system linearity and sensitivity, and comparisons to performance of a liquid crystal tunable filter based hyperspectral imaging device are favorable. The new instrument showed comparable, if not improved, sensitivity at low fluorophore concentrations; yet, acquired wide-field images at more than 70-fold increase in frame rate. Image data acquired in the operating room during human brain tumor resection confirm these findings. The new device is an important advance in achieving real-time quantitative imaging of fluorophore concentration for guiding surgery.

PAM, not spam: Towards quantitative, reproducible, and energy-preserving cavitation imaging

Since the development of the original passive acoustic mapping (PAM) algorithm for cavitation imaging in 2008, which employed a conventional beamformer adapted from seismology (time exposure acoustics), sequential algorithmic improvements have sought to enhance spatial resolution, reduce computation time and enable increasingly setup-independent reporting of cavitation activity. These enhancements have included the use of frequency-domain rather than time-domain implementations to improve computational efficiency, the introduction of the Robust Capon Beamformer to reduce artefacts, sub-aperture processing to further increase frame rate and the implementation of corrections for frequency-dependent attenuation and for array sensitivity and diffraction to account for probe- and depth-related variability. Once these corrections are taken into account, current PAM algorithms produce accurate estimates of cavitation source strength at a single location, but not in the surrounding region, leading to inaccurate estimates of the total cavitation energy over the treatment volume. We present an energy-preserving and computationally efficient PAM algorithm based on Lucy-Richardson deconvolution, that enables the calculation of a set-up independent Cavitational Radiated Energy Density (CRED) that ensures cavitation images accurately reflect the total energy of radiated acoustic emissions within the imaging domain.

Increasing frame rate of echocardiography based on a novel 2D spatio-temporal meshless interpolation

Background:Increasing temporal resolution through numerical methods aids clinicians to evaluate fast moving structures of the heart with more confidence. Methodology:In this study, a spatio-temporal numerical method is proposed to increase the frame rate based on two-dimensional (2D) interpolation. More specifically, we propose a novel intensity variation time surface (IVTS) strategy to incorporate both temporal and spatial information in the reconstruction. In this regard, we exploit radial basis functions (RBFs) for 2D interpolation. The reason for choosing RBFs for this task is manifold. First, RBFs are able to interpolate on large-scale datasets. Moreover, their mathematical implementation is simple. Another important property of this interpolation technique, which is addressed in this study, is its meshless nature. The meshless property enables higher up-sampling (UpS) rates for echocardiography to improve temporal resolution without noticeably degrading image quality. To evaluate the proposed approach, we tested the RBF interpolation on 2D/3D echocardiography datasets. The reconstructed frames were analyzed using different image quality metrics, and the results were compared with two popular techniques from the literature. Results:The findings demonstrated that, with a down-sampling rate of 3, the proposed technique outperformed the best existing method by 42%, 87%, 8%, and 11%, respectively, in terms of mean square error (MSE), contrast to noise ratio (CNR), peak signal-to-noise ratio (PSNR), and figure of merit (FOM). It should be noted that the proposed method is comparable to the best available method in terms of structural similarity (SSIM) index. Furthermore, when compared to the original images, the results of employing our technique on radio-frequency (RF) level analysis demonstrated that the reconstruction accuracy is satisfactory in terms of image quality criterion. Conclusion:Finally, it is worthwhile noting that the proposed method is better than (or comparable to) the other methods in terms of reconstruction performance and processing time. Therefore, the RBF interpolation can be a promising alternative to the existing methods.

Special Vehicle Detection from UAV Perspective via YOLO-GNS Based Deep Learning Network

At this moment, many special vehicles are engaged in illegal activities such as illegal mining, oil and gas theft, the destruction of green spaces, and illegal construction, which have serious negative impacts on the environment and the economy. The illegal activities of these special vehicles are becoming more and more rampant because of the limited number of inspectors and the high cost required for surveillance. The development of drone remote sensing is playing an important role in allowing efficient and intelligent monitoring of special vehicles. Due to limited onboard computing resources, special vehicle object detection still faces challenges in practical applications. In order to achieve the balance between detection accuracy and computational cost, we propose a novel algorithm named YOLO-GNS for special vehicle detection from the UAV perspective. Firstly, the Single Stage Headless (SSH) context structure is introduced to improve the feature extraction and facilitate the detection of small or obscured objects. Meanwhile, the computational cost of the algorithm is reduced in view of GhostNet by replacing the complex convolution with a linear transform by simple operation. To illustrate the performance of the algorithm, thousands of images are dedicated to sculpting in a variety of scenes and weather, each with a UAV view of special vehicles. Quantitative and comparative experiments have also been performed. Compared to other derivatives, the algorithm shows a 4.4% increase in average detection accuracy and a 1.6 increase in detection frame rate. These improvements are considered to be useful for UAV applications, especially for special vehicle detection in a variety of scenarios.

Real-Time Multi-Object Detection Using Enhanced Yolov5-7S on Multi-GPU for High-Resolution Video

Multiple objects tracking in a video sequence can be performed by detecting and distinguishing the objects that appear in the sequence. In the context of computer vision, the robust multi-object tracking problem is a difficult problem to solve. Visual tracking of multiple objects is a vital part of an autonomous driving vehicle’s vision technology. Wide-area video surveillance is increasingly using advanced imaging devices with increased megapixel resolution and increased frame rates. As a result, there is a huge increase in demand for high-performance computation system of video surveillance systems for real-time processing of high-resolution videos. As a result, in this paper, we used a single stage framework to solve the MOT problem. We proposed a novel architecture in this paper that allows for the efficient use of one and multiple GPUs are used to process Full High Definition video in real time. For high-resolution video and images, the suggested approach is real-time multi-object detection based on Enhanced Yolov5-7S on Multi-GPU Vertex. We added one more layer at the top in backbone to increase the resolution of feature extracted image to detect small object and increase the accuracy of model. In terms of speed and accuracy, our proposed approach outperforms the state-of-the-art techniques.

FEATURES OF MODERN TV PRODUCTION TECHNOLOGY

Modern television production is characterized by the use of information technology and the transition to high-definition formats Ultra HD 4K and 8K. This entails high speeds of digital information transmission and large volumes of media information storage. In television production and broadcasting, two main trends can be identified – improving image quality, its resolution and increasing frame rates and the transition to IP and open communication standards, which indicates the merging of television and information technologies. After the advent of digital TV, it became even more obvious how much the impact of television on the viewer depends on the method of presenting the material, and therefore on the choice of technical means for producing TV programs. The improvement of TV production indicators was facilitated by the widespread introduction of non-linear editing technologies, which initially involve working with video and audio materials in digital format. An important step in the development of television technologies is the use of modern data storage systems based on information and network methods. The article discusses the technologies and features of TV production and the use of network systems, including Network Attached Storage and Storage Area Network.

Spatiotemporal Registration of 3-D Multi-perspective Ultrasound Images of Abdominal Aortic Aneurysms.

Methods for patient-specific abdominal aortic aneurysm (AAA) progression monitoring and rupture risk assessment are widely investigated. Three-dimensional ultrasound can visualize the AAA's complex geometry and displacement fields. However, ultrasound has a limited field of view and low frame rate (i.e., 3-8 Hz). This article describes an approach to enhance the temporal resolution and the field of view. First, the frame rate was increased for each data set by sequencing multiple blood pulse cycles into one cycle. The sequencing method uses the original frame rate and the estimated pulse wave rate obtained from AAA distension curves. Second, the temporal registration was applied to multi-perspective acquisitions of the same AAA. Third, the field of view was increased through spatial registration and fusion using an image feature-based phase-only correlation method and a wavelet transform, respectively. Temporal sequencing was fully correct in aortic phantoms and was successful in 51 of 62 AAA patients, yielding a factor 5 frame rate increase. Spatial registration of proximal and distal ultrasound acquisitions was successful in 32 of 37 different AAA patients, based on the comparison between the fused ultrasound and computed tomography segmentation (95th percentile Haussdorf distances and similarity indices of 4.2 ± 1.7 mm and 0.92 ± 0.02 mm, respectively). Furthermore, the field of view was enlarged by 9%-49%.

Dynamic Frame Interpolation in Wavelet Domain.

Video frame interpolation is an important low-level vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. However, the spatial redundancy when synthesizing the target frame has not been fully explored, that can result in lots of inefficient computation. On the other hand, the computation compression degree in frame interpolation is highly dependent on both texture distribution and scene motion, which demands to understand the spatial-temporal information of each input frame pair for a better compression degree selection. In this work, we propose a novel two-stage frame interpolation framework termed WaveletVFI to address above problems. It first estimates intermediate optical flow with a lightweight motion perception network, and then a wavelet synthesis network uses flow aligned context features to predict multi-scale wavelet coefficients with sparse convolution for efficient target frame reconstruction, where the sparse valid masks that control computation in each scale are determined by a crucial threshold ratio. Instead of setting a fixed value like previous methods, we find that embedding a classifier in the motion perception network to learn a dynamic threshold for each sample can achieve more computation reduction with almost no loss of accuracy. On the common high resolution and animation frame interpolation benchmarks, proposed WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.

Efficient Hardware Accelerator Design of Non-Linear Optimization Correlative Scan Matching Algorithm in 2D LiDAR SLAM for Mobile Robots.

Simultaneous localization and mapping (SLAM) is the major solution for constructing or updating a map of an unknown environment while simultaneously keeping track of a mobile robot's location. Correlative Scan Matching (CSM) is a scan matching algorithm for obtaining the posterior distribution probability for the robot's pose in SLAM. This paper combines the non-linear optimization algorithm and CSM algorithm into an NLO-CSM (Non-linear Optimization CSM) algorithm for reducing the computation resources and the amount of computation while ensuring high calculation accuracy, and it presents an efficient hardware accelerator design of the NLO-CSM algorithm for the scan matching in 2D LiDAR SLAM. The proposed NLO-CSM hardware accelerator utilizes pipeline processing and module reusing techniques to achieve low hardware overhead, fast matching, and high energy efficiency. FPGA implementation results show that, at 100 MHz clock, the power consumption of the proposed hardware accelerator is as low as 0.79 W, while it performs a scan match at 8.98 ms and 7.15 mJ per frame. The proposed design outperforms the ARM-A9 dual-core CPU implementation with a 92.74% increase and 90.71% saving in computing speed and energy consumption, respectively. It has also achieved 80.3% LUTs, 84.13% FFs, and 20.83% DSPs saving, as well as an 8.17× increase in frame rate and 96.22% improvement in energy efficiency over a state-of-the-art hardware accelerator design in the literature. ASIC implementation in 65 nm can further reduce the computing time and energy consumption per scan to 5.94 ms and 0.06 mJ, respectively, which shows that the proposed NLO-CSM hardware accelerator design is suitable for resource-limited and energy-constrained mobile and micro robot applications.