Video Processing Algorithms Research Articles

Intelligent Power Grid Video Surveillance Technology Based on Efficient Compression Algorithm Using Robust Particle Swarm Optimization

Companies that produce energy transmit it to any or all households via a power grid, which is a regulated power transmission hub that acts as a middleman. When a power grid fails, the whole area it serves is blacked out. To ensure smooth and effective functioning, a power grid monitoring system is required. Computer vision is among the most commonly utilized and active research applications in the world of video surveillance. Though a lot has been accomplished in the field of power grid surveillance, a more effective compression method is still required for large quantities of grid surveillance video data to be archived compactly and sent efficiently. Video compression has become increasingly essential with the advent of contemporary video processing algorithms. An algorithm’s efficacy in a power grid monitoring system depends on the rate at which video data is sent. A novel compression technique for video inputs from power grid monitoring equipment is described in this study. Due to a lack of redundancy in visual input, traditional techniques are unable to fulfill the current demand standards for modern technology. As a result, the volume of data that needs to be saved and handled in live time grows. Encoding frames and decreasing duplication in surveillance video using texture information similarity, the proposed technique overcomes the aforementioned problems by Robust Particle Swarm Optimization (RPSO) based run-length coding approach. Our solution surpasses other current and relevant existing algorithms based on experimental findings and assessments of different surveillance video sequences utilizing varied parameters. A massive collection of surveillance films was compressed at a 50% higher rate using the suggested approach than with existing methods.

Approximate arithmetic circuits

Low power consumption is the necessity for the integrated circuit design in CMOS technology of nanometerscale. Recent research proves that to achieve low power dissipation, implementation of approximate designs is the best design when compared to accurate designs. In most of the multimedia ap- plications, DSP blocks has been used as the core blocks. Most of the video and image processing algorithms implemented by these DSP blocks, where result will be in the form of image or video for human observing. As human sense of observation isless, the output of the DSP blocks allows being numerically approx- imate instead of being accurate. The concession on numerical exactness allows proposing approximate analysis. In this project approximate adders, approximate compressors and multipliers are proposed. Two approximate adders namely PA1 and PA2 are proposed which are of type TGA which provides better results like PA1 comprises of 14 transistors and 2 error distance, achieves reduction in delay by 64.9 % and reduction in power by 74.33% whereas the TGA1 had 16 transistors and more power dissipation.PA2 comprises of 20 transistors and 2 error distance. Similarly PA2 achieves delay reduction by 51.43%, power gets reduced by 67.2%. PDP is reduced by 61.97 % whereas TGA2 had 22 transistors. Approximate 4-2 compressor was proposed in this project to reduce number of partial produt. The compressor design in circuit level took 30 transistors with 4 errors out of 16 combinations whereas existing compressor design 1took 38 and design 2 took 36 transistors. By using the proposed adder and compressors, approximate 4x4 multiplier is proposed. The proposed multiplier achieves delay 124.56 (ns) and power 29.332 (uW)which is reduced by 68.01% in terms of delay and 95.97 % in terms of power when compared to accurate multiplier.

WITHDRAWN: Video teaching of computer web design course based on FPGA and sobel algorithm

A general-purpose programmable logic device that allows Field Programmable Gate Arrays (FPGAs) to be manufactured on-chip from simple logic gate operations, or a Sobel algorithm complex system implementation is done by the customer after configuration. All tools are appropriate for the learning process, which can be accessed or downloaded from all over the world with the use of the Internet. Video materials provide computer courses web design concept, based on the potential problems, and also provide a special auxiliary FPGA, the teaching, problem-based approach which is much easier. Therefore, the purpose of the system is to provide intuitive tools, web design courses, understand the Sobel algorithm test before major physical system real-time video processing algorithms, and the creation and practical implementation based on video quality. The purpose is also to prove the potential purpose of developing FPGA-based systems and Sobel algorithms. Video teaching is a procedure of hiding the message into video in such a way that no one can detect its presence. In proposed work, sobel algorithm namely random scan and edge detection is used for hiding the data into video. The development of practical examples of video lessons and reasons to support real-time video processing has been proposed. It re-programmes the FPGA, allowing you not only to verify integrity of the design itself but also the establishment of an early prototype sub-circuits using the same development board.

Hierarchical low-rank and sparse tensor micro defects decomposition by electromagnetic thermography imaging system.

With the advancement of electromagnetic induction thermography and imaging technology in non-destructive testing field, this system has significantly benefitted modern industries in fast and contactless defects detection. However, due to the limitations of front-end hardware experimental equipment and the complicated test pieces, these have brought forth new challenges to the detection process. Making use of the spatio-temporal video data captured by the thermal imaging device and linking it with advanced video processing algorithm to defects detection has become a necessary alternative way to solve these detection challenges. The extremely weak and sparse defect signal is buried in complex background with the presence of strong noise in the real experimental scene has prevented progress to be made in defects detection. In this paper, we propose a novel hierarchical low-rank and sparse tensor decomposition method to mine anomalous patterns in the induction thermography stream for defects detection. The proposed algorithm offers advantages not only in suppressing the interference of strong background and sharpens the visual features of defects, but also overcoming the problems of over- and under-sparseness suffered by similar state-of-the-art algorithms. Real-time natural defect detection experiments have been conducted to verify that the proposed algorithm is more efficient and accurate than existing algorithms in terms of visual presentations and evaluation criteria. This article is part of the theme issue 'Advanced electromagnetic non-destructive evaluation and smart monitoring'.

Show me where the action is!

Reality TV shows have gained popularity, motivating many production houses to bring new variants for us to watch. Compared to traditional TV shows, reality TV shows have spontaneous unscripted footage. Computer vision techniques could partially replace the manual labour needed to record and process this spontaneity. However, automated real-world video recording and editing is a challenging topic. In this paper, we propose a system that utilises state-of-the-art video and audio processing algorithms to, on the one hand, automatically steer cameras, replacing camera operators and on the other hand, detect all audiovisual action cues in the recorded video, to ease the job of the film editor. This publication has hence two main contributions. The first, automating the steering of multiple Pan-Tilt-Zoom PTZ cameras to take aesthetically pleasing medium shots of all the people present. These shots need to comply with the cinematographic rules and are based on the poses acquired by a pose detector. Secondly, when a huge amount of audio-visual data has been collected, it becomes labour intensive for a human editor retrieve the relevant fragments. As a second contribution, we combine state-of-the-art audio and video processing techniques for sound activity detection, action recognition, face recognition, and pose detection to decrease the required manual labour during and after recording. These techniques used during post-processing produce meta-data allowing for footage filtering, decreasing the search space. We extended our system further by producing timelines uniting generated meta-data, allowing the editor to have a quick overview. We evaluated our system on three in-the-wild reality TV recording sessions of 24 hours (× 8 cameras) each taken in real households.

Modeling Lane-Changing Behaviors in Merging Areas of Urban Expressways in Nanjing, China

Merging bottlenecks in urban expressways have attracted much attention in recent years. In this paper, vehicular mandatory lane-changing (MLC) data are collected from Yingtian Avenue in Nanjing, China using cameras. Based on a series of video processing algorithms, 656 MLC behaviors of 1,560 vehicles are extracted from videos. A logistic regression model is proposed to depict MLC at the merging bottleneck and estimate the possibility of accepting gaps for merging, which is validated by precision testing and simulation. During the simulation, a discretionary lane-changing (DLC) model is utilized and calibrated to describe vehicular DLC behaviors for the sake of consistency and completeness. Finally, by simulating different arrival rates of mainline and ramp, a linear regression model is built to predict breakdown at merging bottlenecks. According to data analysis, the MLC model represents high precision during the decision-making process. Besides, the breakdown prediction model implies strong correlation between traffic demand and breakdown occurrence.

Combined spatial temporal based In-loop filter for scalable extension of HEVC

Abstract Deep learning plays a major role in the present video processing tools and algorithms. To​ alleviate the limitations of present in-loop filters in the scalable extension of HEVC (SHVC), a combined residual network (CResNet) in-loop filtering is proposed in this paper. The proposed CResNet in-loop filter exploits layer information available in the spatial temporal domain to restrain the visual artifacts like blocking and ringing. Particularly, the block information related to current and co-located blocks of the spatial and temporal base layer reference frames are considered to optimize the in-loop filtering. The proposed architecture has four convolution layers at the base layer and two convolution layers at the enhancement layer that significantly reduces the coding complexity and memory. Additionally to completely train the input content and also to enhance the in-loop filter performance, the on/off level control flag for coding tree Unit (CTU) is sensed using rate distortion optimization (RDO) approach. The experimental results demonstrate that the proposed architecture provides up to 6.2% to 7.2% reduction in bit rate and 1.01 dB improvement in PSNR compared to the standard SHVC codec.

Multimedia processing using deep learning technologies, high‐performance computing cloud resources, and Big Data volumes

SummaryThe last few years have been marked by the presence of very large sets of images and videos in our everyday lives. These multimedia objects have a very fast frequency of creation and sharing since images and videos can come from different devices such as smartphones, satellites, cameras, or drones. They are generally used to illustrate objects in different situations (public areas, train stations, hospitals, political and sport events and competitions, etc). As consequence, image and video processing algorithms have got increasing importance for several computer vision applications that should be adapted for managing large‐scale volumes and exploiting high performance computing resources (local or cloud). In this work, we propose a cloud‐based toolbox (platform) for computer vision applications. This platform integrates a toolbox of image and video processing algorithms that can (i) exploit high performance computing cloud resources, (ii) execute applications in real time, and (iii) manage large‐scale database using Big Data technologies. The related libraries and hardware drivers are automatically integrated and configured in order to offer to users an access to the different applications without the need to download, install, and configure software or hardware. Experiments were conducted using three kinds of applications: (i) image and video processing applications, (ii) deep learning techniques for images classification and multiobject localization, and (iii) images indexation and retrieval. These experiments demonstrated the interest of our platform for sharing, in an efficient way, our scientific contributions and annotated databases in order to improve the quality and performance of computer vision applications.

Automatic gauze tracking in laparoscopic surgery using image texture analysis

Background and ObjectiveInadvertent retained surgical gauzes are an infrequent medical error but can have devastating consequences in the patient health and in the surgeon professional reputation. This problem seems easily preventable implementing standardized protocols for counting but due to human errors it still persists in surgery. The omnipresence of gauzes, their small size, and their similar appearance with tissues when they are soaked in blood make this error eradication really complex. In order to reduce the risk of accidental retention of surgical sponges in laparoscopy operations, in this paper we present an image processing system that tracks the gauzes on the video captured by the endoscope. MethodsThe proposed image processing application detects the presence of gauzes in the video images using texture analysis techniques. The process starts dividing the video frames into square blocks and each of these blocks is analyzed to determine whether it is similar to the gauze pattern. The video processing algorithm has been tested in a laparoscopic simulator under different conditions: with clean, slightly stained and soaked in blood gauzes as well as against different biological background tissues. Several methods, including different Local Binary Patterns (LBP) techniques and a convolutional neural network (CNN), have been analyzed in order to achieve a reliable detection in real time. ResultsThe proposed LBP algorithm classifies the individual blocks in the image with 98% precision and 94% sensitivity which is sufficient to make a robust detection of any gauze that appears in the endoscopic video even if it is stained or soaked in blood. The results provided by the CNN are superior with 100% precision and 97% sensitivity, but due to the high computational demand, real-time video processing is not attainable in this case with standard hardware. ConclusionsThe algorithm presented in this paper is a valuable tool to avoid the retention of surgical gauzes not only because of its reliability but also because it processes the video transparently and unattended, without the need for additional manipulation of special equipment in the operating room.

An Automated Unified Framework for Video Deraining and Simultaneous Moving Object Detection in Surveillance Environments

In many instances, outdoor surveillance systems suffer from atrocious weather conditions such as rain, since images or videos captured by such vision systems in rainy days may undergo severe visual dilapidations. This can cause a glitch in those algorithms which are further used for object detection and tracking. Therefore, an ancillary video processing algorithm namely, video deraining is necessary prior to the implementation of object detection and tracking. This indicates the requirement of a time-consuming and complicated two-step process for the detection of moving objects in a rainy environment. This paper proposes an automated single-stage formulation for the conventional three-stage procedure such as rain steak removal, original data recovery and moving object detection as simultaneous operation in the tensor framework. The brilliance of this work is confined in the efficient formulation of an operator termed as Slice Rotational Total Variation (SRTV), to unify the different rain patterns into a common pattern so that any form of rain pattern can be effectively removed from the rainy data in a visually appealing manner by preserving the important details of the data. In this paper, SRTV regularization and tensor low-rank minimization are utilized for the effective deraining as well as efficient retrieval of clean background. Besides, $l_{1}$ norm and Tensor Total Variation (TTV) regularizers together with SRTV regularizer are employed for the faithful detection of derained moving objects. The experimental results show that the proposed method outperforms the state of the art methods in terms of deraining capability and accurate moving object detection.

Real-time moving human detection using HOG and Fourier descriptor based on CUDA implementation

Real-time applications of image and video processing algorithms have seen explosive growth in number and complexity over the past decade driven by consumer, scientific and defense applications exploiting inexpensive digital video cameras and networked computing device. This growth has opened up different alternatives to greatly enhance the surveillance capabilities using new architectures and parallelization strategies developed due to the increased accessibility of multicore, multi-threaded processors along with general purpose graphics processing units (GPUs). In this paper, we present a new implementation of a moving human detection algorithm on GPU based on the programming language CUDA. In our approach, the moving object is extracted by background subtraction based on the GMM (Gaussian Mixture Model) on GPU. Then, two complementary features are extracted for moving object classification. They are contour-based description: FD or Fourier Descriptor and region-based description: HOG or Histogram of Oriented Gradient. Both descriptors will then be effectively integrated to SVM (Support Vector Machine), which is able to provide the posterior probability, to achieve better performance. The implementation of such algorithm on a GPU allows a great performance in terms of execution time since it is 19 times faster than that on a CPU. Experimental results show also that the proposed approach outperforms some existing techniques and can detect pedestrians in real-time effectively.

A Python-based laboratory course for image and video signal processing on embedded systems

The usage of embedded systems is omnipresent in our everyday life, e.g., in smartphones, tablets, or automotive devices. These devices are able to deal with challenging image processing tasks like real-time detection of faces or high dynamic range imaging. However, the size and computational power of an embedded system is a limiting demand. To help students understanding these challenges, a new lab course "Image and Video Signal Processing on Embedded Systems" has been developed and is presented in this paper. The Raspberry Pi 3 Model B and the open source programming language Python have been chosen, because of low hardware cost and free availability of the programming language. In this lab course the students learn handling both hard- and software, Python as an alternative to MATLAB, the image signal processing path, and how to develop an embedded image processing system, from the idea to implementation and debugging. At the beginning of the lab course an introduction to Python and the Raspberry Pi is given. After that, various experiments like the implementation of a corner detector and creation of a panorama image are prepared in the lab course. Students participating in the lab course develop a profound understanding of embedded image and video processing algorithms which is verified by comparing questionnaires at the beginning and the end of the lab course. Moreover, compared to a peer group attending an accompanying lecture with exercises, students having participated in this lab course outperform their peer group in the exam for the lecture by 0.5 on a five-point scale.

Multiple-sensor fault detection and isolation using video processing in production lines

ABSTRACT Production-line sensors are essential for synchronisation and execution of the correct workflow in a manufacturing process. Usually, sensor faults lead to serious delays, or even the complete termination of the manufacturing process, to carry out maintenance. Thus, sensor faults compromise the reliability of the manufacturing system and disturb the production schedule. This study proposes a multiple-sensor fault detection and isolation scheme for manufacturing series production lines. The proposed scheme adopts a global-redundancy method, using a digital camera and an ad-hoc video processing algorithm to detect and isolate faulty sensors. The main objective of this research is to preserve continuity of the production workflow and solve the problem of production delays and interruptions. Moreover, the scheme provides the possibility of online and post-process system maintenance. Further, the collected information on sensor false alarm rates can be used for a reliability analysis of the production line. The proposed scheme was tested using a laboratory production line model. The results show that the proposed scheme achieves the established objectives and improves the reliability of the manufacturing process.

Real-time UHD video super-resolution and transcoding on heterogeneous hardware

Videos have become the major type of data produced and consumed every day. With screens grow larger, ultra high definition (UHD) videos are becoming more popular since they provide better visual experience. However, video contents with UHD resolution are still scarce. High-performance video super-resolution (SR) techniques that can obtain high resolution (HR) videos from low resolution (LR) sources are recently used in UHD video production. Deep learning (DL)-based SR methods can provide HR videos with appreciable objective and subjective qualities, while their massive computational complexity makes the processing speed far slower than real-time even on GPU servers when producing UHD videos. Moreover, transcoding and other video processing algorithms executed during the enhancement are also time and resource consuming, which performs relatively slow on ordinary CPU and GPU servers. Nowadays, hardware including GPU, field-programmable gate array (FPGA) and application specific integrated circuit (ASIC) are proved to have outstanding capability on image and video processing tasks in different aspects, and there are also dedicated hardware accelerators meant for specific video processing tasks. In this paper, we focus on accelerating a UHD video enhancement workflow on heterogeneous system with multiple hardware accelerators. First, we optimize the most time consuming task, video SR, with CUDNN and CUDA libraries to achieve real-time processing speed for a single UHD output frame on an ordinary GPU. Second, we design a GPU-friendly multi-thread scheduling algorithm for data and computation to better utilize GPU resources and achieve real-time performance on outputting UHD video clips. Third, targeting on production environment, we build a UHD video enhancement application on selected heterogeneous hardware, with an integrated command line tool of our proposed algorithm, and achieve 60 fps real-time end to end processing speed. Experiments show high efficiency, robustness and compatibility of our approach.

A real-time 3D video analyzer for enhanced 3D audio–visual systems

With the recent advent of three-dimensional (3D) sound home theater systems (HTS), more and more TV viewers are experiencing rich, immersive auditory presence at home. In this paper, visual processing approaches are provided to make 3D audio–visual (AV) systems more realistic to the viewers. In the proposed system, a visual engine processes stereo video streams to extract a disparity map for each pair of left and right video frames. Then, the engine determines the video depth level representing each frame of the disparity map. An audio engine then gives 3D sound depth effects according to the estimated video depth, thereby making viewers’ audio–visual experiences more synchronized. Two video processing algorithms are devised to extract the video depth from each frame: one is based on object segmentation, which turns out to be too complex to be implemented in the field-programmable gate array (FPGA) employed for real-time processing; the other algorithm uses a much simpler histogram-based approach to determine the depth of each video frame, and hence, it is more suitable for FPGA implementations. Subjective listening test results support the effectiveness of the proposed approaches.

Motion Estimation for Fisheye Video With an Application to Temporal Resolution Enhancement

Surveying wide areas with only one camera is a typical scenario in surveillance and automotive applications. Ultra wide-angle fisheye cameras employed to that end produce video data with characteristics that differ significantly from conventional rectilinear imagery as obtained by perspective pinhole cameras. Those characteristics are not considered in typical image and video processing algorithms such as motion estimation, where translation is assumed to be the predominant kind of motion. This contribution introduces an adapted technique for use in block-based motion estimation that takes into the account the projection function of fisheye cameras and thus compensates for the non-perspective properties of fisheye videos. By including suitable projections, the translational motion model that would otherwise only hold for perspective material is exploited, leading to improved motion estimation results without altering the source material. In addition, we discuss extensions that allow for a better prediction of the peripheral image areas, where motion estimation falters due to spatial constraints, and further include calibration information to account for lens properties deviating from the theoretical function. Simulations and experiments are conducted on synthetic as well as real-world fisheye video sequences that are part of a data set created in the context of this paper. Average synthetic and real-world gains of 1.45 and 1.51 dB in luminance PSNR are achieved compared against conventional block matching. Furthermore, the proposed fisheye motion estimation method is successfully applied to motion compensated temporal resolution enhancement, where average gains amount to 0.79 and 0.76 dB.

RETRACTED ARTICLE: An Improvised video coding algorithm for deep learning-based video transmission using HEVC

In the previous decades, a great deal of quick intra Coding Unit (CU) decision algorithms has developed for High Efficiency Video Coding (HEVC/H.265). Regardless of the way that such video processing algorithms accomplish minimum intracoding time with less coding productivity misfortune, these are not appropriate for normally utilized between inter-prediction setups. Here, an algorithm is proposed for HEVC named as speedy (quick) inter-coding unit decision algorithm. In this technique, a movement decent variety of collocated CU was figured to decide for collocated CU partition. Likewise, the previous mode-based detection and termination decision have performed by utilizing a discriminant function limiting expected hazard. However, it requires promoting a decrease in computational complexity. Therefore, an energy minimization function is introduced with motion diversity computation to split the CU as the min cut in the graph for obtaining more efficient gesture-based split-to-split decision of collocated CU in a neighboring frame and also, the estimated motion vectors are clustered to find non-homogeneous regions depend on the deep learning approach such as fuzzy c-means clustering algorithm. Moreover, the thresholds and parameters used in motion diversity and energy function were learned by using deep learning RF classifier to enhance the HEVC video coding performance of an initial CU splitting and termination decision for HEVC inter-prediction.

A Robust and Efficient Video Anti-Shaking Algorithm for Low-End Smartphone Platforms

Feature phones or low-end smart phones are an important part of our daily life. While lacking the powerful embedded camera and processing capability of the high-end smart phone, the feature phones with the low-end camera and processing capability are still widely used owing to its lower cost. Consequently, image and video processing algorithms are needed for many intelligent applications on the low-end smart phones. In this paper, we propose a novel robust and efficient video anti-shaking algorithm for the low-end smartphone platform. It provides an integrated application for the low-end smartphone platform, which includes a novel global motion estimation method, and a rapid and efficient local motion detection method. The overall framework is proposed considering the limited camera perception quality and limited mobile processing capability of the feature phone. The proposed algorithm can not only obtain a good visual effect but also be implemented in real-time for the low-end mobile platforms with very limited computational resource. Experimental results demonstrate that the proposed method can achieve a similar video stabilization effect to the Optical Image Stabilization hardware in high-end smartphones.

Complexity measurement and characterization of 360-degree content

The appropriate characterization of the test material, used for subjective evaluation tests and for benchmarking image and video processing algorithms and quality metrics, can be crucial in order to perform comparative studies that provide useful insights. This paper focuses on the characterisation of 360-degree images. We discuss why it is important to take into account the geometry of the signal and the interactive nature of 360-degree content navigation, for a perceptual characterization of these signals. Particularly, we show that the computation of classical indicators of spatial complexity, commonly used for 2D images, might lead to different conclusions depending on the geometrical domain used to represent the 360-degree signal. Finally, new complexity measures based on the analysis of visual attention and content exploration patterns are proposed.

Real Time Feature Point Detectors for Determining Ego Motions

Abstract Ego motion determination is an important application to recognize the motion of an observer in real time. Here we have implemented feature point detector using FFME for ego-motion determination. The main objective of this work is to detect the motions of any objects such as humans, animals, vehicles, missiles and so on. The camera plays a vital role in surveillance of isolated and or restricted area. To improve the intelligence of camera some efficient video processing algorithms are needed. In this process, the detection of unwanted features were derived and developed for some video processing applications. There are various types of feature detectors like edge, corner and points which can be detected from a video frame. To detect those features, feature point detectors like SIFT, ORB, FAST, BRIEF and FFME are more reliable with good efficiency, less memory size and high processing speed. These feature point detectors are compared and evaluated under parameters of speed and number of feature point matches. Although, FFME algorithm has high speed when compared to other feature point detection algorithms, it is prone to more errors caused by illumination differences, as it detects the feature points using Gradient magnitude of the pixel. To overcome this, we have introduced a novel work that uses FFME with gradient phase technique. The resultant output exhibits high processing speed with less error when compared to other feature point detectors and even FFME with gradient magnitude.