Large-scale Web Services Research Articles

Efficient anytime algorithm for large-scale QoS-aware web service composition

The QoS-aware web service composition (WSC) problem aims at the fully automatic construction of a composite web service with the optimal accumulated QoS value. It is, however, intractable to solve the QoS-aware WSC problem for large scale instances since the problem corresponds to a global optimisation problem. That is, in the real world, traditional algorithms can require significant amount of time to finally find the optimal solution, and such an unexpected long delay is unfavourable to users. In this paper, we propose a novel anytime algorithm using dynamic beam widths for the QoSaware WSC problem. Our algorithm generates early sub-optimal solutions and keeps improving the quality of the solution along with the execution time, up to the optimal solution if a client allows enough time. We empirically validate that our algorithm can identify composite web services with high quality much earlier than an optimal algorithm and the beam stack search.

Improving Energy Efficiency in Web Services

With the rapid increase of the energy consumption associated with IT systems and services, energy efficiency is becoming a critical issue in the design, development and management of web service systems. One of the main mechanisms that can be used to reduce the energy consumption is dynamic speed scaling which scales the frequencies of the processors of web servers at hardware level. Another approach is service selection to facilitate the use of energy through effective distribution and management of the web services. In this paper, both the web service selection and server dynamic speed scaling are optimized by maximizing the quality of service (QoS) revenue and minimizing energy costs. Stochastic models of web service systems are proposed, and techniques for quantitative analysis of the performance and energy consumption are investigated. The authors formulate the service selection and speed scaling as a Markov Decision problem, and introduce related algorithms to solve it. Furthermore, the authors build up an optimization framework using multi-agent techniques, and design efficient algorithms to solve the problem in large-scale web service systems. Finally, the effectiveness of their approach is validated by simulation experiments.

Personalized QoS-Aware Web Service Recommendation and Visualization

With the proliferation of web services, effective QoS-based approach to service recommendation is becoming more and more important. Although service recommendation has been studied in the recent literature, the performance of existing ones is not satisfactory, since (1) previous approaches fail to consider the QoS variance according to users' locations; and (2) previous recommender systems are all black boxes providing limited information on the performance of the service candidates. In this paper, we propose a novel collaborative filtering algorithm designed for large-scale web service recommendation. Different from previous work, our approach employs the characteristic of QoS and achieves considerable improvement on the recommendation accuracy. To help service users better understand the rationale of the recommendation and remove some of the mystery, we use a recommendation visualization technique to show how a recommendation is grouped with other choices. Comprehensive experiments are conducted using more than 1.5 million QoS records of real-world web service invocations. The experimental results show the efficiency and effectiveness of our approach.

QoS-aware service selection via collaborative QoS evaluation

We present in this paper a novel collaborative filtering based scheme for evaluating the QoS of large scale Web services. The proposed scheme automates the process of assessing the QoS of a priori ...

Trade-Off Analysis on QoS-Aware Dynamic Web Services Composition with Evolutionary Optimization

Nowadays web services are a distributed technology that can successfully solve integration problems between heterogeneous systems. The supporting technology for web services has been widely studied mainly focusing on the standardization of service transactions and running a single and multiple web services. In dealing with complex and large-scale web service requests, there is a foreseeable bottleneck of supporting technology. QoS-Aware web service composition is NP-hard problem and one of the most challenging problems in web services. This paper proposes an optimization solution using enhanced evolutionary algorithm (EA) for the dynamic composition of web service components and performs the trade-off analysis on dynamic composition of multi domain with multi objectives. First, we propose the Distance Function Based Evolutionary Algorithm (DFBEA) techniques to automatically select optimal combinations of web service components from available component repositories. Second, using multi domain with multi objectives case study, we demonstrate the trade-off analysis on efficiency and effectiveness of the proposed technique and algorithm through experimental evaluation of component selection, and furthermore a comparison with other optimization techniques will be proposed.

Power and performance evaluation of Memcached on the TILEPro64 architecture

Power consumption of data centers had become an important factor in the economy and sustainability of large-scale Web services. Researchers and practitioners are spending considerable effort to characterize Web-scale workloads and evaluating their applicability to alternative, more power-efficient architectures. One such workload in particular is the caching layer, which stores expensive-to-regenerate data in fast storage to reduce service times. In this paper we look at one such application, the Memcached key-value store, which is widely deployed at Facebook and other Web services, and one alternative architecture, the TILEPro64 many-core system-on-chip. We explore the performance and power characteristics of Memcached under a variety of workload variations, implementation choices, and communication protocol, and compare them to a traditional implementation on x86-based servers. Our results suggest that the TILEPro64 architecture can significantly outperform x86-based architectures in terms of throughput per Watt for the evaluated version of Memcached.

Incremental face recognition for large-scale social network services

Due to the rapid growth of social network services such as Facebook and Twitter, incorporation of face recognition in these large-scale web services is attracting much attention in both academia and industry. The major problem in such applications is to deal efficiently with the growing number of samples as well as local appearance variations caused by diverse environments for the millions of users over time. In this paper, we focus on developing an incremental face recognition method for Twitter application. Particularly, a data-independent feature extraction method is proposed via binarization of a Gabor filter. Subsequently, the dimension of our Gabor representation is reduced considering various orientations at different grid positions. Finally, an incremental neural network is applied to learn the reduced Gabor features. We apply our method to a novel application which notifies new photograph uploading to related users without having their ID being identified. Our extensive experiments show that the proposed algorithm significantly outperforms several incremental face recognition methods with a dramatic reduction in computational speed. This shows the suitability of the proposed method for a large-scale web service with millions of users.

Continuous-time MAXQ Algorithm for Web Service Composition

Web services composition present a technology to compose complex service applications from individual (atomic) services, that is, through web services composition, distributed applications and enterprise business processes can be integrated by individual service components developed independently. In this paper, we concentrate on the optimization problems of dynamic web service composition, and our goal is to find an optimal composite policy. Different from many traditional composite methods that do not scale to large continuous-time processes, we introduce a hierarchical reinforcement learning technique, i.e., a continuous-time unified MAXQ algorithm, to solve large-scale web service composition problems in the context of continuous-time semi-Markov decision process (SMDP) model under either average- or discounted-cost criteria. The proposed algorithm can avoid the “curse of modeling” and the “curse of dimensionality” existing in the optimization process. Finally, we use a travel reservation as an example to illustrate the high effectiveness of the proposed algorithm, and the simulation results show that, it has better optimization performance and faster learning speed than the flat Q-learning.

QoS-driven Global Optimization Approach for Large-scale Web Services Composition

One of the aims of SOA is to compose atomic web services into a powerful composite service. QoS based selection approaches are used to choose the best solution among candidate services with the same functionality. Due to the increasing scale of the candidate services and demands for real-time in some specific application domains, the rapid convergent algorithm for large-scale web service composition is especially important, but rare work has been done to solve the problem. This paper describes the Web services composition model and constructs the web service selection mathematical model. According to these models, service composition problem can be considered as Single-Objective Multi-Constraints optimization problem. We propose a new algorithm named GAELS (Genetic Algorithm Embedded Local Searching), which uses the strategies of enhanced initial population and mutation with local searching, to speed up the convergence. Finally, the in-depth experimental results show that the GAELS algorithm can get the non-inferior solution more quickly and more adaptively than simple genetic algorithm in large-scale web service composition.

An efficient incremental face annotation for large scale web services

This paper proposes an incremental face annotation framework for sharing and publishing photographs which contain faces under a large scale web platform such as a social network service with millions of users. Unlike the conventional face recognition environment addressed by most existing works, the image databases being accessed by the large pool of users can be huge and frequently updated. A reasonable way to efficiently annotate such huge databases is to accommodate an adaptation of model parameters without the need to retrain the model all over again when new data arrives. In this work, we are particularly interested in the following issues related to increment of data: (i) the huge number of images being added at each instant, (ii) the large number of users joining the web each day, and (iii) the large number of classification systems being added at each period. We propose an efficient recursive estimation method to handle these data increment issues. Our experiments on several databases show that our proposed method achieves an almost constant execution time with comparable accuracy relative to those state-of-the-art incremental versions of principal component analysis, linear discriminant analysis and support vector machine.

Exploring a UML Profile Approach to Modeling Web Services in Healthcare

The Web services paradigm offers numerous potential benefits to the health care area. These include interoperability, portability, scalability, and compliance with universal standards. How-ever, development of large-scale Web services applications in health care is relatively scarce. Multiple challenges are slowing the diffusion of information technology practices in health care, starting with the absence of modeling tools and methodologies that are the bases for building such technology. This article describes a UML profile approach to developing Web services applications in the health care area, emphasizing the use of such extensions as stereotyping and tagging to model the unique elements of health care processing. We apply these ideas to a health clinic example. From here, one can imagine building incrementally upon these ideas to develop frameworks for Web services applications in health care. The benefits would enhance the overall quality of health care delivery and lower costs.

Autopilot

Microsoft is rapidly increasing the number of large-scale web services that it operates. Services such as Windows Live Search and Windows Live Mail operate from data centers that contain tens or hundreds of thousands of computers, and it is essential that these data centers function reliably with minimal human intervention. This paper describes the first version of Autopilot , the automatic data center management infrastructure developed within Microsoft over the last few years. Autopilot is responsible for automating software provisioning and deployment; system monitoring; and carrying out repair actions to deal with faulty software and hardware. A key assumption underlying Autopilot is that the services built on it must be designed to be manageable. We also therefore outline the best practices adopted by applications that run on Autopilot.

Can Web services scale up?

In the past, only major Internet players such as Amazon, eBay, and Google were interested in deploying large-scale Web services. However, this is changing rapidly as all sorts of companies and governmental organizations are suddenly looking toward Web services as a platform that might support a wide range of demanding applications. This emerging trend presents developers with a new challenge: building Web services solutions that scale. In a nutshell, a scalable system is one that can flexibly accommodate growth in its client base. Such systems typically run on a clustered computer or in a large data center and must be able to handle high loads or sudden demand bursts and a vast number of users. They must reliably respond even in the event of failures or reconfiguration. Ideally, they're self-managed and automate as many routine services such as backups and component upgrades as possible. Many settings also require security against attempted intrusions and distributed denial-of-service (DDoS) attacks. At a glance, today's Web services standards seem to answer these needs. However, a more probing analysis reveals many critical limitations.

Engineering web cache consistency

Server-driven consistency protocols can reduce read latency and improve data freshness for a given network and server overhead, compared to the traditional consistency protocols that rely on client polling. Server-driven consistency protocols appear particularly attractive for large-scale dynamic Web workloads because dynamically generated data can change rapidly and unpredictably. However, there have been few reports on engineering server-driven consistency for such workloads. This article reports our experience in engineering server-driven consistency for a sporting and event Web site hosted by IBM, one of the most popular sites on the Internet for the duration of the event. We also examine an e-commerce site for a national retail store. Our study focuses on scalability and cachability of dynamic content. To assess scalability, we measure both the amount of state that a server needs to maintain to ensure consistency and the bursts of load in sending out invalidation messages when a popular object is modified. We find that server-driven protocols can cap the size of the server's state to a given amount without significant performance costs, and can smooth the bursts of load with minimal impact on the consistency guarantees. To improve performance, we systematically investigate several design issues for which prior research has suggested widely different solutions, including whether servers should send invalidations to idle clients. Finally, we quantify the performance impact of caching dynamic data with server-driven consistency protocols and the benefits of server-driven consistency protocols for large-scale dynamic Web services. We find that (i) caching dynamically generated data can increase cache hit rates by up to 10%, compared to the systems that do not cache dynamically generated data; and (ii) server-driven consistency protocols can increase cache hit rates by a factor of 1.5-3 for large-scale dynamic Web services, compared to client polling protocols. We have implemented a prototype of a server-driven consistency protocol based on our findings by augmenting the popular Squid cache.