Thread Pool Research Articles

FAMOUS, faster: using parallel computing techniques to accelerate the FAMOUS/HadCM3 climate model with a focus on the radiative transfer algorithm

Abstract. We have optimised the atmospheric radiation algorithm of the FAMOUS climate model on several hardware platforms. The optimisation involved translating the Fortran code to C and restructuring the algorithm around the computation of a single air column. Instead of the existing MPI-based domain decomposition, we used a task queue and a thread pool to schedule the computation of individual columns on the available processors. Finally, four air columns are packed together in a single data structure and computed simultaneously using Single Instruction Multiple Data operations. The modified algorithm runs more than 50 times faster on the CELL's Synergistic Processing Element than on its main PowerPC processing element. On Intel-compatible processors, the new radiation code runs 4 times faster. On the tested graphics processor, using OpenCL, we find a speed-up of more than 2.5 times as compared to the original code on the main CPU. Because the radiation code takes more than 60 % of the total CPU time, FAMOUS executes more than twice as fast. Our version of the algorithm returns bit-wise identical results, which demonstrates the robustness of our approach. We estimate that this project required around two and a half man-years of work.

Parallelism on multicore processors using Parallel.FX

The Parallel.FX Task Parallel Library is the latest tool developed for multicore parallelism optimization using the .NET technology. It is a managed concurrency library that provides optimized managed code for multicore processors using a new thread pool that withstands cancellation, waiting and pool isolation, among many other features. The Task Parallel Library also uses dynamic work stealing techniques for superior scalability. This paper analyzes the performance improvement of using the Task Parallel Library of Parallel.FX when applying a Multi-Objective Evolutionary Algorithm to solve a timetabling problem. For comparative purposes, this algorithm has also been parallelized using threads. The results obtained show that both alternatives allow a reduction in the runtime necessary to solve this problem. However, parallelizing the code using the Task Parallel Library of Parallel.FX has the advantage of being easier and the code size is much smaller than directly programming threads.

A novel algorithm for performance prediction of web-based software systems

Performance metrics can be predicted with appropriate performance models and evaluation algorithms. The goal of our work is to adapt the Mean-Value Analysis evaluation algorithm to model the behavior of the thread pool. The computation time and the computational complexity of the proposed algorithm have been provided. The limit of the response time and the throughput sequences computed by the novel algorithm has been determined. It has been shown that the proposed algorithm can be applied to performance prediction of web-based software systems in ASP.NET environment. The proposed algorithm has been validated and the correctness of performance prediction with the novel algorithm has been verified with performance measurements. Error analysis has been performed to verify the correctness of performance prediction.

A Superscalar software architecture model for Multi-Core Processors (MCPs)

Design of high-performance servers has become a research thrust to meet the increasing demand of network-based applications. One approach to design such architectures is to exploit the enormous computing power of Multi-Core Processors (MCPs) that are envisioned to become the state-of-the-art in processor architecture. In this paper, we propose a new software architecture model, called SuperScalar, suitable for MCP machines. The proposed SuperScalar model consists of multiple pipelined thread pools, where each pipelined thread pool consists of multiple threads, and each thread takes a different role. The main advantages of the proposed model are global information sharing by the threads and minimal memory requirement due to fewer threads. We have conducted in-depth performance analyses of the proposed scheme along with three prior software architecture schemes (Multi-Process (MP), Multi-Thread (MT) and Event-Driven (ED)) via an analytical model. The performance results indicate that the proposed SuperScalar model shows the best performance across all system and workload parameters compared to the MP, MT and ED models. Although the MT model shows competitive performance with less number of processing cores and smaller data cache size, the advantage of the SuperScalar model becomes obvious as the number of processing cores increases.

Optimal estimates for the operating parameters of an information web portal

We consider the problems of state estimation and Bayesian parameter identification for a stochastic model of an observation system that describes the evolution of one parameter of a web portal. Based on the thread pool control process analysis, we offer a mathematical model for the oscillations of the operating parameters for the portal's information sources as a nonlinear stochastic observation system of a special kind. We obtain solutions of the optimal estimation problem and give numerical results.

신뢰성 향상과 안전한 웹 서비스를 위한 웹 서버 아키텍처 환경의 설계

기존의 웹 서버 아키텍처 설계에서는 웹 서비스의 신뢰성, 견고성과 안전성을 유지하기 위하여 데이터 암호화(encryption) 기법을 사용하였다. 그러나 데이터 암호화 기법의 사용은 웹 서버의 처리량(throughput) 감소와 평균 응답 시간을 증가시키면서, CPU 작업을 낭비하기기 때문에 웹 어플리케이션 서버의 성능에 부정적인 영향을 나타낸다. 또한 최근의 웹 어플리케이션들은 안전한 인터넷 통신을 위하여 보안과 안전성을 요구하고 있다. 그러므로 본 논문에서는 안전한 웹 서비스를 위하여 기존의 웹 서버에 새로운 웹 서비스 모듈들을 추가하고 쓰레드 풀(Thread pool)과 Non-blocking I/O를 이용하는 개선된 웹 서버를 제안하고, 안전한 웹 서비스 성능을 나타내기 위하여 신뢰성과 안전성을 제공한다. 그리고 본 논문에서 제안한 웹 서버 시스템의 안전성과 성능을 평가하기 위하여 기존의 전형적인 톰캣 기반의 웹 서버와 제안 시스템에 대하여 실험을 통해 안전성과 성능을 비교 평가한다. In the existing design of web server architecture, data encryption technique is used to keep the reliability, stability, and safety of web service. But the use of data encryption technique wastes the work of cpu while decreasing throughput of web server and increasing average response time so that it shows negative effect on the capacity of web application server. Also, the latest web applications require security and safety for the safe internet communication. Therefore, this paper suggests the improved web server which uses thread pool and Non-blocking I/O adding new web service modules to the existing web server for the safe web service, provides reliability and safety to show the safe web service capacity. And we compare and evaluate the safety and capacity through experiment on the existing traditional Tomcat based web server and the proposed system to evaluate the safety and capacity of the proposed web server system.

Applying control theory in the real world

There has been considerable interest in using control theory to build web servers, database managers, and other systems. We claim that the potential value of using control theory cannot be realized in practice without a methodology that addresses controller design, testing, and tuning. Based on our experience with building a controller for the .NET thread pool, we develop a methodology that: (a) designs for extensibility to integrate diverse control techniques, (b) scales the test infrastructure to enable running a large number of test cases, (c) constructs test cases for which the ideal controller performance is known a priori so that the outcomes of test cases can be readily assessed, and (d) tunes controller parameters to achieve good results for multiple performance metrics. We conclude by discussing how our methodology can be extended, especially to designing controllers for distributed systems.

통합 데이터 방송 환경을 위한 범용 서비스 인포메이션 처리 시스템

최근 혁신적인 발전을 거듭하고 있는 데이터 방송 서비스는 사용자에게 더 높은 품질의 영상과 음질은 물론 유용한 부가 정보를 함께 제공한다. 이 중 채널 종류, 프로그램 이름과 같이 단말기 및 사용자에게 필요한 정보를 서비스 인포메이션이라고 한다. 서비스 인포메이션은 테이블 형태로 전송이 이루어지는데, 각 표준별로 독자적인 테이블 구조를 추가로 정의하고 있다. 방송 사업자는 이들 데이터 방송 표준에 정의되어 있는 다양한 테이블 중 자신에게 필요한 테이블을 선택적으로 이용하기 때문에, 이는 결과적으로 사업자간 서비스의 비호환성을 야기한다. 본 논문에서는 다양한 미들웨어 표준을 모두 지원할 수 있는 통합 미들웨어를 위한 범용 서비스 인포메이션 처리 시스템을 제안한다. 제안된 시스템은 XML 기반의 테이블 기술 정보를 이용하여 별도의 수정 없이 동적으로 파싱 대상 테이블을 확장할 수 있는 구조를 갖추었다. 또한, 효율적인 파싱 및 이벤트 디스패칭 작업을 위하여 서비스 인포메이션 테이블의 특성을 고려한 쓰레드 풀 모델을 도입하였다. The data broadcasting service, which is growing remarkably today, provides viewers with useful information as well as high quality video and audio. Service information is a kind of additional data that contains a wide range of information such as channel list and/or program title. Each service information is transmitted in the form of a table. And most standard committees have specified their own table list used for carrying the service information. Consequently, It causes incompatibility among services that each broadcast operators produce because the tables that they use differ from each other. In this paper, we propose a general-purpose service information processing system for an integrated data broadcasting middleware that is compatible with heterogenous broadcasting environments. The system is able to change its target table list dynamically without any code modification. Futhermore, we also adopted a thread pool model for efficient parsing and event dispatching.

Effective performance measurement and analysis of multithreaded applications

Understanding why the performance of a multithreaded program does not improve linearly with the number of cores in a shared-memory node populated with one or more multicore processors is a problem of growing practical importance. This paper makes three contributions to performance analysis of multithreaded programs. First, we describe how to measure and attribute parallel idleness , namely, where threads are stalled and unable to work. This technique applies broadly to programming models ranging from explicit threading ( e.g. , Pthreads) to higher-level models such as Cilk and OpenMP. Second, we describe how to measure and attribute parallel overhead -- when a thread is performing miscellaneous work other than executing the user's computation. By employing a combination of compiler support and post-mortem analysis, we incur no measurement cost beyond normal profiling to glean this information. Using idleness and overhead metrics enables one to pinpoint areas of an application where concurrency should be increased (to reduce idleness), decreased (to reduce overhead), or where the present parallelization is hopeless (where idleness and overhead are both high). Third, we describe how to measure and attribute arbitrary performance metrics for high-level multithreaded programming models, such as Cilk. This requires bridging the gap between the expression of logical concurrency in programs and its realization at run-time as it is adaptively partitioned and scheduled onto a pool of threads. We have prototyped these ideas in the context of Rice University's HPCToolkit performance tools. We describe our approach, implementation, and experiences applying this approach to measure and attribute work, idleness, and overhead in executions of Cilk programs.

Parallel and distributed local search in C OMET

The availability of commodity multiprocessors and high-speed networks of workstations offer significant opportunities for addressing the increasing computational requirements of optimization applications. To leverage these potential benefits, it is important, however, to make parallel and distributed processing easily accessible to a wide audience of optimization programmers. This paper addresses this challenge by proposing parallel and distributed programming abstractions that keep the distance from sequential local search algorithms as small as possible. The abstractions, including parallel loops, interruptions, thread pools, and shared objects, are compositional and cleanly separate the optimization program and the parallel instructions. They have been evaluated experimentally on a variety of applications, including warehouse location and coloring, for which they provide significant speedups.

Needles in a Haystack

Talking politics online is not exclusively reserved for politically-orientated discussion forums, particularly the everyday political talk crucial to the public sphere. People talk politics just about anywhere online from reality TV discussion forums to numerous other forum genres. Thus, the need to tap into those discussions is important if our aim is to provide a more comprehensive overview of the online discursive landscape. However, widening our scope of analysis presents us with a new set of diffi culties, namely, how do we identify political talk within the vast pool of threads and postings, and how do we assess such talk in light of the public sphere, while at the same time, taking into account its informal nature. The aim of this article is to tackle these questions by presenting a methodological approach, which attempts to detect, describe, and assess political talk in non-political discussion forums.

Comparing the performance of web server architectures

In this paper, we extensively tune and then compare the performance of web servers based on three different server architectures. The μserver utilizes an event-driven architecture, Knot uses the highly-efficient Capriccio thread library to implement a thread-per-connection model, and WatPipe uses a hybrid of events and threads to implement a pipeline-based server that is similar in spirit to a staged event-driven architecture (SEDA) server like Haboob. We describe modifications made to the Capriccio thread library to use Linux's zero-copy sendfile interface. We then introduce the SY mmetric Multi-Processor Event Driven (SYMPED) architecture in which relatively minor modifications are made to a single process event-driven (SPED) server (the μserver) to allow it to continue processing requests in the presence of blocking due to disk accesses. Finally, we describe our C++ implementation of WatPipe, which although utilizing a pipeline-based architecture, excludes the dynamic controls over event queues and thread pools used in SEDA. When comparing the performance of these three server architectures on the workload used in our study, we arrive at different conclusions than previous studies. In spite of recent improvements to threading libraries and our further improvements to Capriccio and Knot, both the event-based μserver and pipeline-based Wat-Pipe server provide better throughput (by about 18%). We also observe that when using blocking sockets to send data to clients, the performance obtained with some architectures is quite good and in one case is noticeably better than when using non-blocking sockets.

Models for predicting the performance of ASP.NET Web applications

The most common consideration of web-based information systems is performance. Performance metrics depend on many factors. The first goal of our work is to analyse how some of them affect the response time. Our results have shown that the thread pool properties and the queue size limits are performance factors. In addition, the distribution of the response time has been determined. With the help of properly designed performance models and evaluation algorithms, the performance metrics of a system can be determined at the early stages of the development process. The second goal of our work is to predict the response time, the throughput, and the tier utilization of ASP.NET web applications, based on queueing models handling one and multiple session classes. In addition, web applications have been tested with concurrent user sessions to validate the models in different versions of the ASP.NET environment.

Failover, load sharing and server architecture in SIP telephony

We apply some of the existing web server redundancy techniques for high service availability and scalability to the relatively new IP telephony context. The paper compares various failover and load sharing methods for registration and call routing servers based on the Session Initiation Protocol (SIP). In particular, we consider SIP server failover techniques based on the clients, DNS (Domain Name Service), database replication and IP address takeover, and load sharing techniques using DNS, SIP identifiers, network address translators, and servers with same IP addresses. We describe our two-stage reliable and scalable SIP server architecture in which the first stage proxies the request to one of the second stage server group based on the destination user identifier. We quantitatively evaluate the performance improvement of the load sharing architecture using our SIP server. We quantitatively compare the effect of SIP server architecture such as event-based and thread pool. Additionally, we present an overview of the failover mechanism we implemented in our test-bed using the open source MySQL database.

Clipping 기반의 무선 인터넷 사이트 구축용 변환 서버 구현

현재 유선 인터넷 컨텐츠와 비교할 때 무선 인터넷 컨텐츠가 크게 부족하여 유선 컨텐츠들을 무선 인터넷 컨텐츠로 변환시키고자 하는 요구가 높다. 본 연구의 변환 서버는 사용자 단말기로부터 요청을 자동적으로 인식하여 대상 URL 서버상의 원본 문서를 검색하여 이들을 중간 언어인 metaXML 문서로 생성하고 이를 사용자 단말기에 적합한 무선 마크업 언어 문서로 변환 출판한다. 또한, 본 변환 서버는 이미지 변환을 위하여 이미지 변환기와 기존 웹 페이지들의 재구성 저작 도구인 Clipper와 연동하여 동작한다. 대상 URL의 정적/동적 웹 페이지를 처리하는 변환 서버의 처리 능력에 대하여 성능 실험을 수행하였으며, 동적 문서에 대한 변환 실험 결과에 따르면, 변환 서버내 스레드 풀의 스레드 개수를 5개로 유지할 때 변환 서버가 가장 나은 처리량을 나타내었다. Because a quantity wireless internet contents is much less than wired internet contents, it exist high necessities that wired internet contents should be converted into wireless internet contents. The conversion server implemented in this paper, automatically recognizes the type of user agents when they request, retrieves source documents on the web site specified by an URL, generates metaXML documents as an intermediate form, and converts them wireless markup documents appropriate for user agents. The conversion server interoperates with the image converter for image conversion and the clipper which is an authoring tool for clipping existing wired internet documents. We performed experiments about capability of the conversion server transcoding static/dynamic web pages specified by an URL. According to performance results on dynamic web pages, the conversion server showed better throughput when a thread pool in the terror maintains 5 threads compared with 1 and 10 threads.

Optimizing data aggregation for cluster-based internet services

Large-scale cluster-based Internet services often host partitioned datasets to provide incremental scalability. The aggregation of results produced from multiple partitions is a fundamental building block for the delivery of these services. This paper presents the design and implementation of a programming primitive -- Data Aggregation Call (DAC) -- to exploit partition parallelism for cluster-based Internet services. A DAC request specifies a local processing operator and a global reduction operator, and it aggregates the local processing results from participating nodes through the global reduction operator. Applications may allow a DAC request to return partial aggregation results as a tradeoff between quality and availability. Our architecture design aims at improving interactive responses with sustained throughput for typical cluster environments where platform heterogeneity and software/hardware failures are common. At the cluster level, our load-adaptive reduction tree construction algorithm balances processing and aggregation load across servers while exploiting partition parallelism. Inside each node, we employ an event-driven thread pool design that prevents slow nodes from adversely affecting system throughput under highly concurrent workload. We further devise a staged timeout scheme that eagerly prunes slow or unresponsive servers from the reduction tree to meet soft deadlines. We have used the DAC primitive to implement several applications: a search engine document retriever, a parallel protein sequence matcher, and an online parallel facial recognizer. Our experimental and simulation results validate the effectiveness of the proposed optimization techniques for reducing response time, improving throughput, and gracefully handling server unresponsiveness. We also demonstrate the ease-of use of the DAC primitive and the scalability of our architecture design.

SEDA

We propose a new design for highly concurrent Internet services, which we call the staged event-driven architecture (SEDA). SEDA is intended to support massive concurrency demands and simplify the construction of well-conditioned services. In SEDA, applications consist of a network of event-driven stages connected by explicit queues . This architecture allows services to be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. SEDA makes use of a set of dynamic resource controllers to keep stages within their operating regime despite large fluctuations in load. We describe several control mechanisms for automatic tuning and load conditioning, including thread pool sizing, event batching, and adaptive load shedding. We present the SEDA design and an implementation of an Internet services platform based on this architecture. We evaluate the use of SEDA through two applications: a high-performance HTTP server and a packet router for the Gnutella peer-to-peer file sharing network. These results show that SEDA applications exhibit higher performance than traditional service designs, and are robust to huge variations in load.

Evaluating and Optimizing Thread Pool Strategies for Real-Time CORBA

Strict control over the scheduling and execution of processor resources is essential for many fixed-priority real-time applications. To facilitate this common requirement, the Real-Time CORBA (RT-CORBA) specification defines standard middleware features that support end-to-end predictability for operations in such applications. One of the most important features in RT-CORBA is thread pools, which allow application developers and end-users to configure and control processor resources.This paper provides two contributions to the evaluation of techniques for improving the quality of implementation of RT-CORBA thread pools. First, we describe the key patterns underlying common strategies for implementing RT-CORBA thread pools. Second, we evaluate each thread pool strategy in terms of its consequences on (1) feature support, such as request buffering and thread borrowing, (2) scalability in terms of end-points and event demultiplexers required, (3) efficiency in terms of data movement, context switches, memory allocations, and synchronizations required, (4) optimizations in terms of stack and thread specific storage memory allocations, and (5) bounded and unbounded priority inversion incurred in each implementation. This paper also provides results that illustrate empirically how different thread pool implementation strategies perform in different ORB configurations.

Analysis of optimal thread pool size

The success of e-commerce, messaging middleware, and other Internet-based applications depends in part on the ability of network servers to respond in a timely and reliable manner to simultaneous service requests. Multithreaded systems, due to their efficient use of system resources and the popularity of shared-memory multi-processor architectures, have become the server implementation of choice. However, creating and destroying a thread is far from free, requiring run-time memory allocation and deallocation. These overheads become especially onerous during periods of high load and can be a major factor behind system slowdowns. A thread-pool architecture addresses this problem by prespawning and then managing a pool of threads. Threads in the pool are reused, so that thread creation and destruction overheads are incurred only once per thread, and not once per request. However, efficient thread management for a given system load highly depends on the thread pool size, which is currently determined heuristically. In this paper, we characterize several system resource costs associated with thread pool size. If the thread pool is too large, and threads go unused, then processing and memory resources are wasted maintaining the thread pool. If the thread pool is too small, then additional threads must be created and destroyed on the fly to handle new requests. We analytically determine the optimal thread pool size to maximize the expected gain of using a thread.