Real-time Image Processing Applications Research Articles

An ultra-fast and energy-efficient CNTFET-based image corner detection hardware for real-time image processing applications

In image processing and machine vision, corner detection is pivotal in diverse applications, including computer vision, 3D reconstruction, face detection, object tracking, and video technologies. Despite the wide usage, the real-time and energy-efficient hardware implementation of corner detection algorithms remains a critical challenge because of the computational resource limitations. On the other hand, owing to the complicated nature of corner detection algorithms, their hardware implementation has been limited to the graphics processing unit (GPU) and field programmable gate arrays (FPGA) platforms. In this regard, this work aims to propose a novel and ultra-efficient carbon nanotube field-effect transistor (CNTFET)-based hardware for image corner detection. Thanks to the proposed corner detection algorithm, the designed hardware has been realized using 2742 transistors with competitive accuracy. The proposed corner detection hardware indicated a remarkable salt-and-pepper noise immunity without using any noise reduction circuit. Our comprehensive simulations demonstrate 78%, 87%, and 94.5% total average improvements in delay, power, and energy compared to the other related corner detection hardware. Moreover, the proposed CNTFET-based corner detection hardware shows a 43 ps propagation delay, demonstrating its real-time operation. The proposed corner detection algorithm at the system level shows suitable accuracy metrics such as Recall, Precision, and error of detection (EoD) compared to the other well-known corner detectors. Our method has established a new pathway for real-time circuit-level hardware design for image processing and machine vision applications.

Neuromorphic Computing Architectures for Real-time Image Processing and Pattern Recognition

Real-time image processing and pattern recognition applications have found a new paradigm in neuromorphic computing systems. In this paper, we quantitatively compare neuromorphic architecture performance to that of conventional computing techniques. We study processing speed, accuracy, and energy usage for diverse image processing jobs using a controlled experimental methodology. The outcomes highlight the advantages of the Neuromorphic architecture, which is distinguished by quicker processing times and greater precision. These results demonstrate the effectiveness of event-driven spiking neural networks and are consistent with earlier studies. Comparisons with hybrid architectures highlight the Neuromorphic architecture's strength as a stand-alone system and point to simpler implementations. However, issues with accuracy fluctuation and the requirement for scalability continue, emphasizing areas for more study. The energy economy of neuromorphic architectures makes them essential parts of real-time image processing and pattern recognition as the field develops.

Object Detection Using Adaptive Block Partition and RCNN Algorithm

Advancements in image and video processing are growing over the years for industrial robots, autonomous vehicles, cryptography, surveillance, medical imaging and computer-human interaction applications. One of the major challenges in real-time image and video processing is the execution of complex functions and high computational tasks. To overcome this issue, a hardware acceleration of different filter algorithms for both image and video processing is implemented on Xilinx Zynq®-7000 System on-Chip (SoC) device consists of Dual-core Cortex™-A9 processors which provides computing ability to perform with the help of software libraries using Vivado® High-Level Synthesis (HLS). The acceleration of object detection algorithms include Sobel-Feldman filter, posterize and threshold filter algorithms implemented with 1920 x 1080 image resolutions for real-time object detection. The implementation results exhibit effective resource utilization such as 45.6% of logic cells, 51% of Look-up tables (LUTs), 29.47% of Flipflops, 15% of Block RAMs and 23.63% of DSP slices under 100 MHz frequency on comparing with previous works. There are a few reasons why tracking is preferable over detecting objects in each frame. Tracking facilitates in identifying the identity of various items across frames when there are several objects. Object detection may fail in some instances, but tracking may still be achievable which takes into account the location and appearance of the object in the previous frame. The key hurdles in real-time image and video processing applications are object tracking and motion detection. Some tracking algorithms are extremely fast because they perform a local search rather than a global search. Tracking algorithms such as meanshift, Regional Neural Network probabilistic data association, particle filter, nearest neighbor, Kalman filter and interactive multiple model (IMM) are available to estimate and predict the state of a system.

Real-time approximate and combined 2D convolvers for FPGA-based image processing

Convolution widely has been used as the main part of the improvement in digital image processing applications. In convolutional computations, a large number of memory accesses and a huge amount of computations challenge its performance. Many of the related proposed convolvers are based on exact computations. Although exact convolvers keep the accuracy of the convolution operation at the top level, sometimes by missing a negligible amount of accuracy, the performance can be improved. Approximate computing is a new technique for solving computation overhead problems. In this paper, approximate 2D convolvers are presented which minimize the memory access rate and computations by a special factor of multiply-and-accumulate (MAC) terms. On the other hand, to preserve the flexibility for supporting different required accuracy, the proposed approximate convolvers are combined with the exact designs with real-time pre-processing stages by exploiting innovative methods which manage the hardware overhead. In comparison with conventional convolvers, the proposed designs improve the number of active resources which causes a significant reduction in power consumption. For 3 × 3 kernel size, the evaluation results on the Xilinx Virtex-7 (XC7V2000t) FPGA device show 34% and 20% power optimization of the proposed approximate and combined convolvers, respectively, in comparison with exact convolver (EC). Also, this improvement grows by increasing the kernel size. Finally, a comparison based on RMSE and PSNR for different sample images and filters reveals that the error rate and image quality reduction are acceptable for many real-time image processing applications.

Hardware Acceleration of Video Edge Detection with Hight Level Synthesis on the Xilinx Zynq Platform

The study conducted in the current paper consists of validating an original design flow for the rapid prototyping of real-time image and video processing applications on FPGAs. A video application for edge detection with Simulink HDL coder and Vivado High-Level Synthesis (HLS) has been designed as if the code was going to be executed on a conventional processor. The developed tools will automatically translate the code into VHDL hardware language using an advanced compilation technique. This amounts to embedding processors on Xilinx Zynq-7000 System on-Chip (SoC) device in an optimal manner. This automated hardware design flow reduces the time to create a prototype since only the high-level description is required. The design of the video edge detection system is implemented on Xilinx Zynq-7000 platform. The result of the implementation gave effective resource utilization and a good frame rate (95 FPS) under 170MHz frequency.

Algorithms to Speed up Contour Tracing in Real Time Image Processing Systems

Contour tracing is an important pre-processing step in many image processing applications such as feature recognition, biomedical imaging, security and surveillance. As single processor architectures reach their performance limits, parallel processing architectures offer energy efficient and high performance solutions for real time applications. Parallel processing architectures, are thus used for several real time image processing applications. Among the several interconnection schemes available, Cayley graph based interconnections offer easy routing and symmetric implementation capabilities. For parallel processing systems with a Cayley graph based interconnection scheme, torus, we developed three accelerated algorithms corresponding to three existing families of contour tracing algorithms. We simulated these algorithms on a parallel processing framework to quantify the normalized speed-up possible in any torus connected parallel processing system. We also compared our best performing algorithm with the existing parallel processing implementations for Nvidia GPUs. We observed a speed-up of up to 468 times using our algorithms on a parallel processing architecture in comparison to the corresponding algorithm on a single processor architecture. We evaluated a speed up of 194 (and 47) compared to the existing parallel processing contour tracing implementation on Tesla K40c (and Quadro RTX 5000 GPU hardware respectively). We observe that for torus connected parallel processing architectures used for image processing, our algorithms can be used to speed up contour tracing, without any hardware modification.

FPGA implementation of multi-dimensional Kalman filter for object tracking and motion detection

Object tracking and motion detection are the major challenges in the real-time image and video processing applications. There are several tracking and prediction algorithms available to estimate and predict the state of a system. Kalman filter is the most widely used prediction algorithm as it is very simple, efficient and easy to implement for linear measurements. However, these types of filter algorithms are customized on hardware platforms such as Field-Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs) to achieve design requirements for embedded applications. In this work, a multi-dimensional Kalman filter (MDKF) algorithm is proposed for object tracking and motion detection. The numerical analysis of proposed tracking algorithm achieves competitive tracking performance in contrast with state-of-the-art tracking algorithms trained on standard benchmarks. Furthermore, MDKF is implemented on Xilinx Zynq™-7000 System-on-a chip (SoC). The implementation of MDKF on SoC performs 2× times tracking speed than that of software approach. The experimental results provide resource utilization of about 61.43% of Block RAMs (BRAMs), 90.09% of DSPs, 83.27% of Look-up tables (LUTs) and 82.35% of logic cells operating at 140 MHz with power consumption of 780 mW which outperforms previous related methods.

A technical review of no-reference image quality assessment algorithms for contrast distorted images

Automatic image quality assessment similar to human vision perception is an essential process for real-time image processing applications to perform perceptual image assessments for effectively achieving their goals. As no-reference image quality assessment (NR-IQA) schemes perform perceptual assessments of images without any information about their original version, these algorithms suit real-time computer vision techniques because of the non-availability of reference images. Contrast and colorfulness play important roles in determining the quality of color images. By combining many IQA metrics, a number of combined metrics had been devised. This study provides an insight into major NR-IQA methods and their effectiveness in assessing contrast, colorfulness, and overall quality of contrast-degraded images with technical analysis. The effectiveness of top-ranking NR-IQA methods is experimentally assessed with benchmark assessment methods on images from benchmarked databases. The study provides insight into open research challenges in the area of NR-IQA for developing new promising methods by clearly demarcating the difficulties of top-ranking NR-IQA methods.

A Low Redundancy Wavelet Entropy Edge Detection Algorithm.

Fast edge detection of images can be useful for many real-world applications. Edge detection is not an end application but often the first step of a computer vision application. Therefore, fast and simple edge detection techniques are important for efficient image processing. In this work, we propose a new edge detection algorithm using a combination of the wavelet transform, Shannon entropy and thresholding. The new algorithm is based on the concept that each Wavelet decomposition level has an assumed level of structure that enables the use of Shannon entropy as a measure of global image structure. The proposed algorithm is developed mathematically and compared to five popular edge detection algorithms. The results show that our solution is low redundancy, noise resilient, and well suited to real-time image processing applications.

Gradient information distillation network for real-time single-image super-resolution

In recent years, deep convolutional neural networks have played an increasingly important role in single-image super-resolution (SR). However, with the increase of the depth and width of networks, the super-resolution methods based on convolution neural networks are facing training difficulties, memory consumption, running slowness and other problems. Furthermore, most of the methods do not make full use of the image gradient information which leads to the loss of geometric structure information of the image. To solve these problems, we propose a gradient information distillation network in this paper. On the one hand, the advantages of fast and lightweight are maintained through information distillation. On the other hand, the SR performance is improved by gradient information. Our network has two branches named gradient information distillation branch (GIDB) and image information distillation branch. To combine features in both branches, we also introduce a residual feature transfer mechanism (RFT). Under the function of GIDB and RFT, our network can retain the rich geometric structure information which can make the edge details of the reconstructed image sharper. The experimental results show that our method is superior to the existing methods while well limits the parameters, computation and running time of the model. It provides the possibility for real-time image processing and mobile applications.

AN EFFICIENT FPGA OVERLAY FOR COLOR TRANSFORMATION FUNCTION USING HIGH LEVEL SYNTHESIS

Image Processing is a significantly desirable in commercial, industrial, and medical applications. Processor based architectures are inappropriate for real time applications as Image processing algorithms are quite intensive in terms of computations. To reduce latency and limitation in performance due to limited amount of memory and fixed clock frequency for synthesis in processor-based architecture, FPGA can be used in smart devices for implementing real time image processing applications. To increase speed of real time image processing custom overlays (Hardware Library of programmable logic circuit) can be designed to run on FPGA fabric. The IP core generated by the HLS (High Level Synthesis) can be implemented on a reconfigurable platform which allows effective utilization of channel bandwidth and storage. In this paper we have presented FPGA overlay design for color transformation function using Xilinx’s python productivity board PYNQ-Z2 to get benefit in performance over a traditional processor. Performance comparison of custom overlay on FPGA and Processor based platform shows FPGA execution yields minimum computation time.

An Efficient Modification of Grey Wolf Optimization Using Cuckoo Search, Levy Fly and Mantegna Algorithm for Real-Time Image Processing Applications

Optimization methods, frequently used in several image and video processing algorithms for the attainment of optimal solutions, pose severe hurdle in case of real-time processing. For catering to the needs of real-time operations in a cost-effective way, dedicated hardware is inevitable. The huge computational load of any optimization method strikes down its feasibility of being realized in terms of dedicated hardware. The computational complexities of meta-heuristic optimization methods are even more than any other conventional optimization methods. So, in spite of having the capability of providing global solution by dodging local optima, meta-heuristic optimizations are avoided in real-time systems. To overcome the bottleneck, in this article, a modified GWOA (modified grey wolf optimization algorithm)is formulated by blending the advantages of CS (cuckoo search), Levy fly (LV), and MA (Mantegna algorithm). This modified GWOA is articulated to be computationally efficient and precise, so that, it can easily be realized in terms of dedicated VLSI architecture while maintaining the accuracy at a high level. The proposed method helps to diminish the cost and power requirement of high end and costly real-time imaging/ video processing systems while upholding its precision. The proposed method is tested by using MATLAB R2018b. The high-level synthesis (HLS) tool of Xilinx Vivado18.2softwareisusedtosynthesizethisMGWOA, thus establishing the viability of the proposed algorithm to be implemented on FPGA/ ASIC.

Multilevel image thresholding with multimodal optimization

Thresholding method is one of the most popular approaches for image segmentation where an objective function is defined in terms of threshold numbers and their locations in a histogram. If only a single threshold is considered, a segmented image with two classes is achieved. On the other hand, multiple classes in the output image are created with multilevel thresholding. Otsu and Kapur’s procedures have been conventional steps for defining objective functions. Nevertheless, the fundamental problem with thresholding techniques is the determination of threshold numbers, which must be selected by the user. In that respect, thresholding methods with both techniques are user-dependent, and may not be practical for real-time image processing applications. In this study, a novel thresholding algorithm without any objective function has been proposed. Histogram curve was considered as an objective function. The peaks and valley in histogram have been detected by means of multimodal particle swarm optimization algorithms. Accordingly, valleys between two peaks have been assigned as thresholds. Consequently, the developed scheme does not need any user intervention and finds the number of thresholds automatically. Furthermore, computation time is independent of the number of thresholds, whereas computation time in Otsu and Kapur procedures depends on the number of thresholds.

CGMBE: a model-based tool for the design and implementation of real-time image processing applications on CPU–GPU platforms

Processing large images in real time requires effective image processing algorithms as well as efficient software design and implementation to take full advantage of all CPU cores and GPU resources on state of the art CPU/GPU platforms. Efficiently coordinating computations on both the host (CPU) and devices (GPUs), along with host–device data transfers is critical to achieving real-time performance. However, such coordination is challenging for system designers given the complexity of modern image processing applications and the targeted processing platforms. In this paper, we present a novel model-based design tool that automates and optimizes these critical design decisions for real-time image processing implementation. The proposed tool consists of a compile-time static analyzer and a run-time dynamic scheduler. The tool automates the process of scheduling dataflow tasks (actors) and coordinating CPU–GPU data transfers in an integrated manner. The approach uses an unfolded dataflow graph representation of the application along with thread-pool-based executors, which are optimized for efficient operation on the targeted CPU–GPU platform. This approach automates the most complicated aspects of the design and implementation process for image processing system designers, while maximizing the utilization of computational power, reducing the memory footprint for both the CPU and GPU, and facilitating experimentation for tuning performance-oriented designs.

A Pre-processing Step for Efficient Edge Extraction

In this paper, we propose a pre-processing step for an efficient edge extraction technique that takes input as an original image to generate an edge map. Generated edge maps could be inputted for state-of-art traditional edge detection algorithms like Canny, Sobel, Prewitt, and recent edge detection algorithms gb-UCM, CED Contours, Structured forest, Sparse Code Gradients and CNN based edge detection Deep Edge, N4 to get better performance. Further, the proposed algorithm has not required any training or learning to improve the edge detection method and is not depending on any parameters. Visual experiments and quantitative evaluation results show that our proposed algorithm greatly improves the modal quality of edge/edge maps. It preserves the original shape, structure of the objects and local features, which presents in an input image. The proposed method takes very less amount of time to execute and making it more suitable for real-time image processing and computer vision applications that depend on edge like classification, object localization, object recognition, image retrieval, segmentation, shape representation.

Detecting Human and Classification of Gender using Facial Images MSIFT Features Based GSVM

Classification of gender using face recognition system is an essential concept for different types of applications in human-computer interaction and computer-aided related applications. It defines a wide range of features from human images to detect male, female and others using real-time data. There are different machine learning approaches were implemented to classify gender and also detects other images during the classification phase, which are not humans based on features extracted from human images datasets. All these existing techniques mostly depend on controlled conditions like features and other representations of the human image. Because of significant and uncertain variations of a particular image, it may be a challenging task in gender classification for real-time image processing application, whether it is male, female and others. So that in this document, we propose a Human detection and Face based gender Recognition System (HDFGR); to investigate male or female classification on real life faces using real world face databases. Our proposed approach consists Multi-Scale Invariant Feature Transform (MSIFT) to describes faces and Gaussian distance-based support vector machine (GSVM) classifier is used to classify gender and objects, i.e. male, female and other from features extracted human image datasets. We obtain an experimental performance of 98.7% by applying DSVM with boosted MSIFT features. Our proposed approach gives better classification accuracy and other performance parameters compared to different existing approaches with benchmark and evaluation of possible databases.

A Hybrid Color Quantization Algorithm That Combines the Greedy Orthogonal Bi-Partitioning Method With Artificial Ants

A color quantization technique that combines the operations of two existing methods is proposed. The first method considered is the Greedy orthogonal bi-partitioning method. This is a very popular technique in the color quantization field that can obtain a solution quickly. The second method, called Ant-tree for color quantization, was recently proposed and can obtain better images than some other color quantization techniques. The solution described in this article combines both methods to obtain images with good quality at a low computational cost. The resulting images are always better than those generated by each method applied separately. In addition, the results also improve those obtained by other well-known color quantization methods, such as Octree, Median-cut, Neuquant, Binary splitting or Variance-based methods. The features of the proposed method make it suitable for real-time image processing applications, which are related to many practical problems in diverse disciplines, such as medicine and engineering.

Investigation on Efficient FPGA Architectures for Image Coding Algorithm

In this research, we provide an effective hardware architecture for various image processing, enhancement, and filtering algorithms that is based on FPGAs. The inherent spatial and temporal parallelism in FPGA architecture makes them a popular choice as implementation platforms for real-time image processing applications. The filters are applied by iteratively cycling over an image's pixels using a windowing operator method. Software becomes less effective and real-time hardware solutions are required as picture sizes and bit depths increase. While the findings shown here are for a picture with a resolution of 585 x 450 pixels, the stated method may be used to photos of any resolution, provided that the FPGA memory can accommodate it. The design was developed using the Nexys3 board and Xilinx Spartan-6 FPGA in mind.

FPGA Implementation of a Novel Gaussian Filter Using Power Optimized Approximate Adders

Smoothing filters are essential for noise removal and image restoration. Gaussian filters are used in many digital image and video processing systems. Hence the hardware implementation of the Gaussian filter becomes a reliable solution for real time image processing applications. This paper discusses the implementation of a novel Gaussian smoothing filter with low power approximate adders in Field Programmable Gate Array (FPGA). The proposed Gaussian filter is applied to restore the noisy images in the proposed system. Original test images with 512x512 pixels were taken and divided in to 4x4 blocks with 256x256 pixels. The proposed technique has been applied and the performance metrics were measured for various simulation criteria. The proposed algorithm is also implemented using approximate adders, since approximate adders had been recognized as a reliable alternate for error tolerant applications in circuit based metrics such as power, area and delay where the accuracy may be considered for trade off.

Implementation of High Speed Vedic Multiplier Using Vertical and Crosswise Algorithm

Multiplication is an important fundamental function in arithmetic operations. Multiplication-based operations such as Multiply and Accumulate (MAC) and inner product are some of the frequently used operations in many Digital Signal Processing (DSP) applications such as convolution, Fast Fourier Transform(FFT), filtering and in microprocessors in its arithmetic and logic unit. Since multiplication dominates the execution time of most DSP algorithms, so there is a need of high speed multiplier. Higher throughput arithmetic operations are important to achieve the desired performance in many real-time signal and image processing applications. In this project, the comparative study of Vedic multiplier and Sequential multiplier is done for low power requirement and high speed. The proposed architecture is based on the Vertical and Crosswise algorithm of ancient Indian Vedic Mathematics, which increases the speed of multiplier by reducing the number of clock cycles thus achieving the greater speed of the processor or system.