End-to-End Industrial IoT: Software Optimization and Acceleration

Abstract

Application of artificial intelligence and machine learning is transforming Industrial Internet of Things (IIoT) segments by enabling higher productivity, better insights, less downtime, and superior product quality. Through AI-inspired innovations, businesses are gaining a sizable competitive edge and product leadership. To realize the promise and potential of AI, software needs to enable fast prototyping and experimentation at scale, and efficiently turn data into valuable insights. Over 100× performance speedup is possible with software that is well parallelized, vectorized, written with better data reuse, has cache blocking, makes good use of prefetching, and above all, is abstracted with familiar industry-standard APIs and libraries used by ML developers and data scientists. As a result of increased software performance efficiency, on the same hardware system, a customer can realize and serve higher insights per second and increase inference throughput. Our software optimizations target a general-purpose CPU, which, in addition to AI models and pipelines, can also be efficient through workload consolidation for other IIoT applications. Both general-purpose workloads and AI model inferencing run on the same hardware targets while minimizing energy, maintenance, and total cost of ownership of a heterogenous infrastructure. Edge IIoT also has unique power and space constraints that can be addressed well with multi-stream inferencing support on general-purpose CPUs. It is imperative to optimize software for all phases of the end-to-end AI pipeline to run “efficient AI” and realize quicker insights, thereby turning them into concrete business results for IIoT solutions. In this article, using optimized frameworks such as Intel Pytorch extensions, Intel Scikit-learn extensions, and Intel distribution of Modin, we get 3.6x-81 × improvement in end-to-end pipeline performance. What this means for the CNN-based anomaly detection pipeline and predictive analytics pipeline solutions is higher throughput of insights and also serving multiple streams of inference across the shop floor, thereby maximizing the potential of the IIoT AI solution. Popular and relevant IIoT AI use cases can realize significant performance improvement when software and AI implementations are purposefully optimized for the target AI hardware and systems.