Abstract
We present a method to automatically synthesize efficient, high-quality demosaicking algorithms, across a range of computational budgets, given a loss function and training data. It performs a multi-objective, discrete-continuous optimization which simultaneously solves for the program structure and parameters that best tradeoff computational cost and image quality. We design the method to exploit domain-specific structure for search efficiency. We apply it to several tasks, including demosaicking both Bayer and Fuji X-Trans color filter patterns, as well as joint demosaicking and super-resolution. In a few days on 8 GPUs, it produces a family of algorithms that significantly improves image quality relative to the prior state-of-the-art across a range of computational budgets from 10 s to 1000 s of operations per pixel (1 dB–3 dB higher quality at the same cost, or 8.5–200× higher throughput at same or better quality). The resulting programs combine features of both classical and deep learning-based demosaicking algorithms into more efficient hybrid combinations, which are bandwidth-efficient and vectorizable by construction. Finally, our method automatically schedules and compiles all generated programs into optimized SIMD code for modern processors.
Highlights
At the same time, demosaicking must often be performed under extreme computational budgets: a single stream of 4K 60 FPS video requires processing 0.5 gigapixels per second
Our programs are Pareto-dominant: they offer both significantly higher quality (1 dB–3 dB) at the same computational cost as any prior algorithm in the same range, and can deliver comparable or better image quality at dramatically lower computational cost (8.5–220× or more). They are designed for efficient streaming SIMD implementation, and automatically compile to highlyoptimized kernels for modern processors. We generate this family of new algorithms automatically by developing a multi-objective, discrete-continuous search which simultaneously solves for the program structure and parameters to find the best tradeoff between computational cost and image quality in a target range of computational budgets
Our search process significantly improves the quality vs. performance tradeoff of existing programs in the real-time performance regime, and spans a frontier of stateof-the-art algorithms covering a throughput range of 10—100 Megapixels per second on a single CPU core
Summary
Demosaicking is among the most ubiquitous and performancecritical image processing tasks. Our programs are Pareto-dominant: they offer both significantly higher quality (1 dB–3 dB) at the same computational cost as any prior algorithm in the same range, and can deliver comparable or better image quality at dramatically lower computational cost (8.5–220× or more) They are designed for efficient streaming SIMD implementation, and automatically compile to highlyoptimized kernels for modern processors. We generate this family of new algorithms automatically by developing a multi-objective, discrete-continuous search which simultaneously solves for the program structure and parameters to find the best tradeoff between computational cost and image quality in a target range of computational budgets. — We define a search space that generates SIMD and localityfriendly algorithms by construction, and a compiler that exploits this structure to automatically generate highlyoptimized streaming implementations
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
