Abstract
With the development of multi-/many-core processors, applications need to be written as parallel programs to improve execution efficiency. For data-intensive applications that use multiple threads to read/write files simultaneously, an I/O sub-system can easily become a bottleneck when too many of these types of threads exist; on the contrary, too few threads will cause insufficient resource utilization and hurt performance. Therefore, programmers must pay much attention to parallelism control to find the appropriate number of I/O threads for an application. This paper proposes a parallelism control mechanism named IOPA that can adjust the parallelism of applications to adapt to the I/O capability of a system and balance computing resources and I/O bandwidth. The programming interface of IOPA is also provided to programmers to simplify parallel programming. IOPA is evaluated using multiple applications with both solid state and hard disk drives. The results show that the parallel applications using IOPA can achieve higher efficiency than those with a fixed number of threads.
Highlights
In the era of big data, data intensive applications become common in both desktop and server systems, many of which need to access large amount of files including web pages, images, and videos
To solve the parallelism control problem discussed above, this paper proposes IOPA, an I/ O parallelism control mechanism that can adapt to the I/O capability of the system
The IOPA monitors the load of an I/O sub-system and correspondingly adjusts the number of I/O threads in applications
Summary
In the era of big data, data intensive applications become common in both desktop and server systems, many of which need to access large amount of files including web pages, images, and videos. Due to the performance gap between CPU and I/O sub-system, I/O bandwidth can become system bottleneck, especially for parallel applications that use multiple threads to read/write files simultaneously. In this situation, multiple I/O threads may generate too many requests that overload the I/O sub-system, and the different requests may interfere with each other in prefetching, buffering and disk actions, causing further reductions in I/O throughput. To make full use of I/O bandwidth while avoiding I/O bottlenecks, programmers must pay much attention to the I/O parallelism of a program, i.e., the number of threads that access files.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
