Accelerate Image Processing Research Articles

Error Resilient GPU Accelerated Image Processing for Space Applications

Significant advances in spaceborne imaging payloads have resulted in new big data problems in the Earth Observation (EO) field. These challenges are compounded onboard satellites due to a lack of equivalent advancement in onboard data processing and downlink technologies. We have previously proposed a new GPU accelerated onboard data processing architecture and developed parallelised image processing software to demonstrate the achievable data processing throughput and compression performance. However, the environmental characteristics are distinctly different to those on Earth, such as available power and the probability of adverse single event radiation effects. In this paper, we analyse new performance results for a low power embedded GPU platform, investigate the error resilience of our GPU image processing application and offer two new error resilient versions of the application. We utilise software based error injection testing to evaluate data corruption and functional interrupts. These results inform the new error resilient methods that also leverages GPU characteristics to minimise time and memory overheads. The key results show that our targeted redundancy techniques reduce the data corruption from a probability of up to 46 percent to now less than 2 percent for all test cases, with a typical execution time overhead of 130 percent.

GPU Accelerated Image Processing in CCD-Based Neutron Imaging

Image processing is an important step in every imaging path in the scientific community. Especially in neutron imaging, image processing is very important to correct for image artefacts that arise from low light and high noise statistics. Due to the low global availability of neutron sources suitable for imaging, the development of these algorithms is not in the main scope of research work and once established, algorithms are not revisited for a long time and therefore not optimized for high throughput. This work shows the possible speed gain that arises from the usage of heterogeneous computing platforms in image processing along the example of an established adaptive noise reduction algorithm.

Could We Realize the Fully Flexible System by Real-Time Computing with Thin-Film Transistors?

Flexible electronic devices, such as the typical thin-film transistors, are widely adopted in the area of sensors, displayers, wearable equipment, and such large-area applications, for their features of bending and stretching; additionally, in some applications of lower-resolution data converters recently, where a trend appears that implementing more parts of system with flexible devices to realize the fully flexible system. Nevertheless, relatively fewer works on the computation parts with flexible electronic devices are reported, due to their poor carrier mobility, which blocks the way to realize the fully flexible systems with uniform manufacturing process. In this paper, a novel circuit architecture for image processing accelerator using Oxide Thin-film transistor (TFT), which could realize real-time image pre-processing and classification in the analog domain, is proposed, where the performance and fault-tolerance of image signal processing is exploited. All of the computation is done in the analog signal domain and no clock signal is needed. Therefore, certain weaknesses of flexible electronic devices, such as low carrier mobility, could be remedied dramatically. In this paper, Simulations based on Oxide TFT device model have demonstrated that the flexible computing parts could perform 5 × 5 Gaussian convolution operation at a speed of 3.3 MOPS/s with the energy efficiency of 1.83 TOPS/J, and realize image classification at a speed of 10 k fps, with the energy efficiency of 5.25 GOPS/J, which means that the potential applications to realize real-time computing parts of complex algorithms with flexible electronic devices, as well as the future fully flexible systems containing sensors, data converters, energy suppliers, and real-time signal processing modules, all with flexible devices.

Programming Heterogeneous Systems from an Image Processing DSL

Specialized image processing accelerators are necessary to deliver the performance and energy efficiency required by important applications in computer vision, computational photography, and augmented reality. But creating, “programming,” and integrating this hardware into a hardware/software system is difficult. We address this problem by extending the image processing language Halide so users can specify which portions of their applications should become hardware accelerators, and then we provide a compiler that uses this code to automatically create the accelerator along with the “glue” code needed for the user’s application to access this hardware. Starting with Halide not only provides a very high-level functional description of the hardware but also allows our compiler to generate a complete software application, which accesses the hardware for acceleration when appropriate. Our system also provides high-level semantics to explore different mappings of applications to a heterogeneous system, including the flexibility of being able to change the throughput rate of the generated hardware. We demonstrate our approach by mapping applications to a commercial Xilinx Zynq system. Using its FPGA with two low-power ARM cores, our design achieves up to 6× higher performance and 38× lower energy compared to the quad-core ARM CPU on an NVIDIA Tegra K1, and 3.5× higher performance with 12× lower energy compared to the K1’s 192-core GPU.

Thresholding approach to radiography image processing acceleration

An approach for detecting low-value Laplacian pyramid coefficients which can be used to accelerate radiography image processing is presented. The acceleration is achieved through modifying Laplacian pyramid construction algorithm. Detection of low-value coefficients in lower pyramid layers is based on corresponding low-value higher layer coefficients which are lower than the threshold. Detected coefficients are then omitted in pyramid calculation and by the radiography image enhancement algorithm. Clinical radiography images are used to evaluate the proposed approach. Structural similarity index (SSIM) is employed to measure similarity between images processed with and without proposed acceleration. Threshold value analysis shows that high image quality with a mean SSIM of 0.995 can be achieved with mean processing time reduction of 4.62 %.

Підхід до вдосконалення методу Віоли - Джонса при вирішенні задач розпізнавання зображення

In this article, I discuss options for improving a speed of processing and recognition of individual shapes in the image according to the method of Viola - Jones. The main attention is paid to the various methods of boosting, which could replace or enhance the standard solution in the method Viola – Jones. Using new technology, we have accelerated image processing based on Haar basis and avoided excess during calculations. In addition, the article addressed the issue of replacing the method of training the classifier. Improved filtering method will enable us to build a more robust cascade that will produce fewer false-positive results during image processing.

FPGA Implementation of a Geometric Voting Scheme for the Extraction of Geometric Entities from Images

In this work, we present a novel architecture in a field programmable gate array (FPGA) for accelerating image processing algorithms based on conformal geometric algebra (CGA). This implementation specifically accelerates the execution of the Conformal Geometric Algebra Voting Scheme for detection of circles and lines in images. All geometric operations, such as meet, join, and transformation entities, are computed in hardware using FPGA. Other non-geometric operations, such as DBSCAN, are also executed in the FPGA. The full voting scheme consists of two main stages: a local stage computed using neighborhoods in the image and a global stage using the results of the local voting stage. In this implementation, we focused on the most computationally demanding stage: local voting. The top level design consists of five main hardware cores and an ARM processor interconnected using the AXI4 BUS. The cores exchange data using memory sections in a DRAM directly connected to the processing system side in the FPGA, which can be accessed by the hardware cores through the memory controller using a High performance AXI BUS. The design has been validated by comparing the results of the FPGA with the results previously obtained in a software reference application using real image data.

Streaming Elements for FPGA Signal and Image Processing Accelerators

Field-programmable gate array (FPGA) devices boast abundant resources with which custom accelerator components for signal, image, and data processing may be realized; however, realizing high-performance, low-cost accelerators currently demands manual register transfer level design. Software-programmable soft processors have been proposed as a way to reduce this design burden, but they are unable to support performance and cost comparable to custom circuits. This paper proposes a new soft processing approach for FPGA that promises to overcome this barrier. A high-performance, fine-grained streaming processor, known as a streaming accelerator element, is proposed, which realizes accelerators as large-scale custom multicore networks. By adopting a streaming execution approach with advanced program control and memory addressing capabilities, typical program inefficiencies can be almost completely eliminated to enable performance and cost, which are unprecedented among software-programmable solutions. When used to realize accelerators for fast Fourier transform, motion estimation, matrix multiplication, and sobel edge detection, it is shown how the proposed architecture enables real-time performance and with performance and cost comparable with hand-crafted custom circuit accelerators and up to two orders of magnitude beyond existing soft processors.

Accelerating Spaceborne SAR Imaging Using Multiple CPU/GPU Deep Collaborative Computing.

With the development of synthetic aperture radar (SAR) technologies in recent years, the huge amount of remote sensing data brings challenges for real-time imaging processing. Therefore, high performance computing (HPC) methods have been presented to accelerate SAR imaging, especially the GPU based methods. In the classical GPU based imaging algorithm, GPU is employed to accelerate image processing by massive parallel computing, and CPU is only used to perform the auxiliary work such as data input/output (IO). However, the computing capability of CPU is ignored and underestimated. In this work, a new deep collaborative SAR imaging method based on multiple CPU/GPU is proposed to achieve real-time SAR imaging. Through the proposed tasks partitioning and scheduling strategy, the whole image can be generated with deep collaborative multiple CPU/GPU computing. In the part of CPU parallel imaging, the advanced vector extension (AVX) method is firstly introduced into the multi-core CPU parallel method for higher efficiency. As for the GPU parallel imaging, not only the bottlenecks of memory limitation and frequent data transferring are broken, but also kinds of optimized strategies are applied, such as streaming, parallel pipeline and so on. Experimental results demonstrate that the deep CPU/GPU collaborative imaging method enhances the efficiency of SAR imaging on single-core CPU by 270 times and realizes the real-time imaging in that the imaging rate outperforms the raw data generation rate.

A Hardware-Efficient Method for Extracting Statistic Information of Connected Component

The statistic information of connected components are fundamental for image processing, which could be acquired through connected components labeling. This paper proposes a hardware-efficient method for extracting statistic information of connected components in a binary image to accelerate image processing in embedded application. The proposed method scans two adjacent rows with 2?×?2 template simultaneously, meanwhile, statistic information of runs are recorded. After scanning two rows, the equivalent runs are merged, and then statistic information of completed connected region is exported directly. This method scans an image only once, which could reduce off-chip memory access massively. For a determined image resolution, the requirement of on-chip memory resource is also confirmed and not affected by the number of connected components. This algorithm is modeled with Verilog, and the simulation result shows that average processing speed could be real-time for various images with different resolution. Furthermore, the memory cost is little compared to other hardware based algorithms for labeling connected components, and the proposed method is appropriated for hardware implementation.

Quasi real-time analysis of mixed-phase clouds using interferometric out-of-focus imaging: development of an algorithm to assess liquid and ice water content

According to changes in aircraft certifications rules, instrumentation has to be developed to alert the flight crews of potential icing conditions. The technique developed needs to measure in real time the amount of ice and liquid water encountered by the plane. Interferometric imaging offers an interesting solution: It is currently used to measure the size of regular droplets, and it can further measure the size of irregular particles from the analysis of their speckle-like out-of-focus images. However, conventional image processing needs to be speeded up to be compatible with the real-time detection of icing conditions. This article presents the development of an optimised algorithm to accelerate image processing. The algorithm proposed is based on the detection of each interferogram with the use of the gradient pair vector method. This method is shown to be 13 times faster than the conventional Hough transform. The algorithm is validated on synthetic images of mixed phase clouds, and finally tested and validated in laboratory conditions. This algorithm should have important applications in the size measurement of droplets and ice particles for aircraft safety, cloud microphysics investigation, and more generally in the real-time analysis of triphasic flows using interferometric particle imaging.

Synthesis and optimization of image processing accelerators using domain knowledge

In the domain of image processing, often real-time constraints are required. In particular, in safety-critical applications, timing is of utmost importance. A common approach to maintain real-time capabilities is to offload computations to dedicated hardware accelerators, such as Field Programmable Gate Arrays (FPGAs). Designing such architectures is per se already a challenging task, but finding the right design point between achieving as much throughput as necessary while spending as few resources as possible is an even bigger challenge.To address this design challenge in the domain of image processing, several approaches have been presented that introduce an additional layer of abstraction between the developer and the actual target hardware. One approach is to use a Domain-Specific Language (DSL) to generate highly optimized code for synthesis by general purpose High-Level Synthesis (HLS) frameworks. Another approach is to instantiate a generic VHDL IP-Core library for local imaging operators. Elevating the description of image algorithms to such a higher abstraction level can significantly reduce the complexity for designing hardware accelerators targeting FPGAs. We provide a comparison of results for both approaches, a non-expert algorithm developer can achieve. Furthermore, we present an automatic optimization process to give the algorithm developer even more control over trading execution time for resource usage, that could be applied on top of both approaches. To evaluate our optimization procedure, we compare the resulting FPGA accelerators to highly optimized Graphics Processing Unit (GPU) implementations of several image filters relevant for close-to-sensor image and video processing with stringent real-time constraints, such as in the automotive domain.

Early Stage Automatic Strategy for Power-Aware Signal Processing Systems Design

Complexity management, portability and long term adaptivity are common challenges in different fields of embedded systems, normally colliding with the needs of efficient resource utilization and power balance. Image/signal processing systems, though required to offer a large variety of complex functions, have also to deal with battery-life limitations. Wearable signal processing systems, for example, should provide high performance and support new generation standards without compromising their portability and their long-term usability. These constraints challenge hardware designers: early stage trade-off analysis and power management automated techniques are helpful to guarantee a reasonable time-to-market. In the field of video codec specifications, the MPEG standard known as Reconfigurable Video Coding (RVC) framework addresses functional complexity and adaptivity leveraging on the intrinsic modularity of the dataflow model of computation, but it still lacks in offering power management support. The main contribution of this work is providing an automatic early-stage power management methodology to be adopted within the MPEG-RVC context. Starting from different high-level specifications, our mapping methodology identifies directly on the high-level models disjointed homogeneous logic clock regions, where the platform resources can be enabled/disabled together without affecting the overall system performance. To extend its usability to the RVC community, we have integrated this methodology within the Multi-Dataflow Composer (MDC) tool. MDC is a tool for on-the-fly reconfigurable signal processing platforms deployment. In this paper, we extended MDC to address power-aware multi-context systems. To prove the effectiveness of our work, a coprocessor for image and video processing acceleration has been assembled. This latter has been synthesized on a 90 nm ASIC technology, where demonstrated up to 90 % of reduction in the dynamic power consumption on different dataflow-intensive applications. The coprocessor has been implemented also on FPGA, confirming, partially, the benefits of adopting the proposed methodology.

Image processing acceleration for intelligent unmanned aerial vehicle on mobile GPU

In this paper, we present an algorithm for providing visually-guided unmanned aerial vehicle (UAV) control using visual information that is processed on a mobile graphic processing unit (GPU). Most real-time machine vision applications for UAVs exploit low-resolution images because the shortage of computational resources comes from size, weight and power issue. This leads to the limitation that the data are insufficient to provide the UAV with intelligent behavior. However, GPUs have emerged as inexpensive parallel processors that are capable of providing high computational power in mobile environments. We present an approach for detecting and tracking lines that use a mobile GPU. Hough transform and clustering techniques were used for robust and fast tracking. We achieved accurate line detection and faster tracking performance using the mobile GPU as compared with an x86 i5 CPU. Moreover, the average results showed that the GPU provided approximately five times speedup as compared to an ARM quad-core Cortex-A15. We conducted a detailed analysis of the performance of proposed tracking and detection algorithm and obtained meaningful results that could be utilized in real flight.

Embedded platform for local image descriptor based object detection

The article presents novel idea of a hardware accelerated image processing algorithm for embedded systems. The system is based on the well known Fast Retina Keypoint (FREAK) local image description algorithm. The solution utilizes Field Programmable Gate Array (FPGA) as a flexible module that is used to implement hardware acceleration of a given part of the image processing algorithm. The approach presented in this paper is slightly different. Since we are using very fast FREAK descriptor it is not our purpose to implement full feature extraction algorithm in hardware but just its most time-consuming part which is brute force matcher based on the Hamming distance. Moreover our goal was to design very flexible system so that the feature detection and extraction algorithm can be replaced without any interruption in the hardware accelerated part.

Towards a performance-portable description of geometric multigrid algorithms using a domain-specific language

High Performance Computing (HPC) systems are nowadays more and more heterogeneous. Different processor types can be found on a single node including accelerators such as Graphics Processing Units (GPUs). To cope with the challenge of programming such complex systems, this work presents a domain-specific approach to automatically generate code tailored to different processor types. Low-level CUDA and OpenCL code is generated from a high-level description of an algorithm specified in a Domain-Specific Language (DSL) instead of writing hand-tuned code for GPU accelerators. The DSL is part of the Heterogeneous Image Processing Acceleration (HIPAcc) framework and was extended in this work to handle grid hierarchies in order to model different cycle types. Language constructs are introduced to process and represent data at different resolutions. This allows to describe image processing algorithms that work on image pyramids as well as multigrid methods in the stencil domain. By decoupling the algorithm from its schedule, the proposed approach allows to generate efficient stencil code implementations. Our results show that similar performance compared to hand-tuned codes can be achieved.

A 1 GHz Hardware Loop-Accelerator With Razor-Based Dynamic Adaptation for Energy-Efficient Operation

Dynamic adaptation using Razor-based detection and correction of timing errors has demonstrated substantial improvements in performance and energy-efficiency in microprocessors. In this work, we apply Razor to hardware accelerators that find increasing application in System-on-Chip designs with high-performance requirements that must be delivered under stringent power budgets. We describe the implementation and silicon measurement results from a Razor-based hardware loop-accelerator (RZLA), implementing the Sobel edge-detection algorithm. Unlike in microprocessors, the RZLA pipeline is datapath-dominated with statically-scheduled control that has queue-based storage structures which are simply extended to support check-pointing and recovery. We exploit these characteristics typical of DSP and image-processing accelerators to implement Razor recovery in manner that is amenable to RTL validation and verification. We show a low-overhead pulsed-latch based Razor Flip-flop (RFF) architecture that adds only a single extra transistor on clock to minimize clock power overhead. The RFF is deployed in conjunction with a level-sensitive latch-insertion based algorithm to address the minimum-delay constraint present in all Razor systems. This algorithm enables the use of 50% of the clock period for timing speculation leading to robust error detection and correction across a wide dynamic voltage- and frequency-scaling range. Fabricated in 65 nm CMOS, the RZLA reclaims voltage margins to demonstrate 34% energy-efficiency improvements on a per-device basis and 33% overall, for the entire batch of devices at 1 GHz operation.

SU‐E‐I‐37: Low‐Dose Real‐Time Region‐Of‐Interest X‐Ray Fluoroscopic Imaging with a GPU‐Accelerated Spatially Different Bilateral Filtering

Purpose:The purpose of our study is to reduce imaging radiation dose while maintaining image quality of region of interest (ROI) in X‐ray fluoroscopy. A low‐dose real‐time ROI fluoroscopic imaging technique which includes graphics‐processing‐unit‐ (GPU‐) accelerated image processing for brightness compensation and noise filtering was developed in this study.Methods:In our ROI fluoroscopic imaging, a copper filter is placed in front of the X‐ray tube. The filter contains a round aperture to reduce radiation dose to outside of the aperture. To equalize the brightness difference between inner and outer ROI regions, brightness compensation was performed by use of a simple weighting method that applies selectively to the inner ROI, the outer ROI, and the boundary zone. A bilateral filtering was applied to the images to reduce relatively high noise in the outer ROI images. To speed up the calculation of our technique for real‐time application, the GPU‐acceleration was applied to the image processing algorithm. We performed a dosimetric measurement using an ion‐chamber dosimeter to evaluate the amount of radiation dose reduction. The reduction of calculation time compared to a CPU‐only computation was also measured, and the assessment of image quality in terms of image noise and spatial resolution was conducted.Results:More than 80% of dose was reduced by use of the ROI filter. The reduction rate depended on the thickness of the filter and the size of ROI aperture. The image noise outside the ROI was remarkably reduced by the bilateral filtering technique. The computation time for processing each frame image was reduced from 3.43 seconds with single CPU to 9.85 milliseconds with GPU‐acceleration.Conclusion:The proposed technique for X‐ray fluoroscopy can substantially reduce imaging radiation dose to the patient while maintaining image quality particularly in the ROI region in real‐time.

Fast Design Exploration for Performance, Power and Accuracy Tradeoffs in FPGA-Based Accelerators

The ease-of-use and reconfigurability of FPGAs makes them an attractive platform for accelerating algorithms. However, accelerating becomes a challenging task as the large number of possible design parameters lead to different accelerator variants. In this article, we propose techniques for fast design exploration and multi-objective optimization to quickly identify both algorithmic and hardware parameters that optimize these accelerators. This information is used to run regression analysis and train mathematical models within a nonlinear optimization framework to identify the optimal algorithm and design parameters under various objectives and constraints. To automate and improve the model generation process, we propose the use of L 1 -regularized least squares regression techniques.We implement two real-time image processing accelerators as test cases: one for image deblurring and one for block matching. For these designs, we demonstrate that by sampling only a small fraction of the design space (0.42% and 1.1%), our modeling techniques are accurate within 2%--4% for area and throughput, 8%--9% for power, and 5%--6% for arithmetic accuracy. We show speedups of 340× and 90× in time for the test cases compared to brute-force enumeration. We also identify the optimal set of parameters for a number of scenarios (e.g., minimizing power under arithmetic inaccuracy bounds).

Accelerate the Face Detection Optimization with Edge detection and the Discrete Cosine Transform (DCT)

This study presents a new method for accelerating and optimizing face detection, while preserving a high level of accuracy. This method uses local features that are extracted using block-base discrete cosine transform (DCT). This study uses edge detection method for face recognition with ICA algorithms. The technique used for edge detection is Laplacian of Gaussian (LOG). To find objects, face position and local information the discrete cosine transform is used. Here the main idea is edge detection and finding face position in the picture for DCT processing. Edge detection was applied for accelerating image processing. In this paper we use CohnKanade AU-Coded Facial Expression Database.