Online Transaction Processing Research Articles

Proposal of server and storage resource partition method for mission critical system consolidation

AbstractWe propose server and storage resource partition method for mission critical system consolidation to prevent performance interference in cloud computing environment. We made experiments to allocate dedicated CPU core and SSD to a virtual machine. As a result of running a standard online transaction processing benchmark on three virtual machines, transaction performance decrease by 65% with the existing method and by only 9% with the proposed method.

Autonomic workload performance tuning in large-scale data repositories

The workload in large-scale data repositories involves concurrent users and contains homogenous and heterogeneous data. The large volume of data, dynamic behavior and versatility of large-scale data repositories is not easy to be managed by humans. This requires computational power for managing the load of current servers. Autonomic technology can support predicting the workload type; decision support system or online transaction processing can help servers to autonomously adapt to the workloads. The intelligent system could be designed by knowing the type of workload in advance and predict the performance of workload that could autonomically adapt the changing behavior of workload. Workload management involves effectively monitoring and controlling the workflow of queries in large-scale data repositories. This work presents a taxonomy through systematic analysis of workload management in large-scale data repositories with respect to autonomic computing (AC) including database management systems and data warehouses. The state-of-the-art practices in large-scale data repositories are reviewed with respect to AC for characterization, performance prediction and adaptation of workload. Current issues are highlighted at the end with future directions.

Performance prediction and adaptation for database management system workload using Case-Based Reasoning approach

Workload management in a Database Management System (DBMS) has become difficult and challenging because of workload complexity and heterogeneity. During and after execution of the workload, it is hard to control and handle the workload. Before executing the workload, predicting its performance can help us in workload management. By knowing the type of workload in advance, we can predict its performance in an adaptive way that will enable us to monitor and control the workload, which ultimately leads to performance tuning of the DBMS. This study proposes a predictive and adaptive framework named as the Autonomic Workload Performance Prediction (AWPP) framework. The proposed AWPP framework predicts and adapts the DBMS workload performance on the basis of information available in advance before executing the workload. The Case-Based Reasoning (CBR) approach is used to solve the workload management problem. The proposed CBR approach is compared with other machine learning techniques. To validate the AWPP framework, a number of benchmark workloads of the Decision Support System (DSS) and the Online Transaction Processing (OLTP) are executed on the MySQL DBMS. For preparation of training and testing data, we executed more than 1000 TPC-H and TPC-C like workloads on a standard data set. The results show that our proposed AWPP framework through CBR modeling performs better in predicting and adapting the DBMS workload. DBMSs algorithms can be optimized for this prediction and workload can be controlled and managed in a better way. In the end, the results are validated by performing post-hoc tests.

Janus: A Hybrid Scalable Multi-Representation Cloud Datastore

Cloud-based data-intensive applications have to process high volumes of transactional and analytical requests on large-scale data. Businesses base their decisions on the results of analytical requests, creating a need for real-time analytical processing. We propose Janus, a hybrid scalable cloud datastore, which enables the efficient execution of diverse workloads by storing data in different representations. Janus manages big datasets in the context of datacenters, thus supporting scaling out by partitioning the data across multiple servers. This requires Janus to efficiently support distributed transactions. In order to support the different datacenter requirements, Janus also allows diverse partitioning strategies for the different representations. Janus proposes a novel data movement pipeline to continuously ensure up to date data between the different representations. Unlike existing multi-representation storage systems and Change Data Capture (CDC) pipelines, the data movement pipeline in Janus supports partitioning and handles both distributed transactions and diverse partitioning strategies. In this paper, we focus on supporting Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) workloads, and hence use row and column-oriented representations, which are the most efficient representations for these workloads. Our evaluations over Amazon AWS illustrate that Janus can provide real-time analytical results, in addition to processing high-throughput transactional workloads.

Workload Aware Incremental Repartitioning of NoSQL for Online Transactional Processing Applications

<p><span lang="EN-US">Numerous applications are deployed on the web with the increasing popularity of internet. The applications include, 1) Banking applications,<br /> 2) Gaming applications, 3) E-commerce web applications. Different applications reply on OLTP (Online Transaction Processing) systems. OLTP systems need to be scalable and require fast response. Today modern web applications generate huge amount of the data which one particular machine and Relational databases cannot handle. The E-Commerce applications are facing the challenge of improving the scalability of the system. Data partitioning technique is used to improve the scalability of the system. The data is distributed among the different machines which results in increasing number of transactions. The work-load aware incremental repartitioning approach is used to balance the load among the partitions and to reduce the number of transactions that are distributed in nature. Hyper Graph Representation technique is used to represent the entire transactional workload in graph form. In this technique, frequently used items are collected and Grouped by using Fuzzy C-means Clustering Algorithm. Tuple Classification and Migration Algorithm is used for mapping clusters to partitions and after that tuples are migrated efficiently.</span></p>

Performance innovations in the IBM z14 platform

The IBM z14 platform brings together numerous innovations that collectively provide considerable performance gains over prior designs for the full spectrum of modern workloads being run on today's enterprise systems. These workloads range from traditional online transaction processing and batch workloads to Linux-based and cloud-based workloads, leveraging advances in analytics, mobile transaction processing, social data mining, cognitive applications, security (blockchain), and other areas. The performance gains in the z14 platform required advancements in the processor hardware microarchitecture, physical design, and instruction-set architecture, as well as in the firmware and hypervisor stacks. The result is a comprehensive platform solution that enables efficient multiprocessor scaling for large single-image partitions and mixed-workload multipartition environments within a single platform, while remaining responsive to the “bursty” demands of batch and transaction-based workloads. The performance gains present in the z14 are a direct result of the collaborative efforts between diverse teams within IBM working together to produce advancements in hardware, firmware, and software that outpace the gains of the individual contributions. In this paper, we discuss these results with a focus on the client value delivered through system performance and workload responsiveness.

BI4IPM

The Agri-Food sector is facing global challenges. The first challenge is feeding a world population that will reach 9.3 billion people in 2050, according to UN projections. The second challenge is the demand from consumers for high-quality products obtained through more sustainable, safe and clear agri-food chains. Integrated pest management (IPM) could be an important instrument for helping farmers face these challenges. IPM requires the simultaneous use of different crop protection techniques to control pests through an ecological and economic approach. This work explores the possibility of developing a framework that combines business intelligence (BI) technologies with IPM principles to support farmers in the decisional process, thereby decreasing environmental cost and improving production performance. The proposed BI system is called BI4IPM, and it combines on-line transaction processing (OLTP) with on-line analytical processing (OLAP) to verify adherence to the IPM technical specifications.

RDBMS Current Challenges and Oppurtunities with NoSQL to NewSQL

Relational Database Management Systems (RDBMS) are designed to manage well-structured data, requiring users to design a schema before storing and querying data with the help of SQL (Structured Query Language). The evolution of database management system undergoes many developments for more than forty years to meet the challenges by modern applications. In this fast Growing science and technological period, where advancements in web technology, mobile devices and sensors (IoT) need to accommodate huge amount of structured and unstructured data, called Big Data. The demand for technologies with large processing and storage requirements are rising enormously. Customized RDBMS has to confront challenges against Big Data’s scalability and performances. To meet these requirements such as volume, variety, velocity and variability, enterprises are adopting alternative technologies like NoSQL and NewSQL. Currently, NewSQL scalability is classified as modern RDBMS which carries NoSQL for OLTP (Online Transaction Processing) to read as well write data, whereas satisfying the Atomicity, Consistency, Isolation, and Durability (ACID) properties of a traditional DB system. Further my research paper introduces NoSQL and its key attributes and NewSQL (NuoDB, VoltDB)

Data Warehouse Schemas using Multidimensional Data Model for Retail

A data warehouse are central repositories of integrated data from one or more disparate sources from operational data in On-Line Transaction Processing (OLTP) system to use in decision making strategy and business intelligent using On-Line Analytical Processing (OLAP) techniques. Data warehouses support OLAP applications by storing and maintaining data in multidimensional format. Multidimensional data models as an integral part of OLAP designed to solve complex query analysis in real time.

Auto Accessories, Inc.: An Educational Case on Online Transaction Processing (OLTP) and Controls as Compared to Batch Processing and Controls

ABSTRACT The intent of this educational case is to increase students' understanding of online transaction processing (OLTP) and controls as compared to batch processing and controls. Learning concepts about batch processing is important because many entities continue to use batch processing for critical applications such as payroll, credit card processing, and Big Data. Students learn the advantages and disadvantages of batch processing and OLTP. The case provides a Microsoft Access database that includes a working batch program (module) and an online screen (form). Students use the form to update an employee table in a relational database with OLTP and use the module to update an employee master file with batch processing. Students compare the output from batch processing to the output from OLTP after processing the same input transactions and demonstrate that the outputs match when no input or processing errors exist. Students implement similar data validation edits in both the module and the form. Also, students implement run-to-run control total checks in the module and report input data errors in a batch-processed error report. Students learn processing and controls that are unique to batch processing, unique to OLTP, and common to both processing modes.

OLTP PERFORMANCE IMPROVEMENT USING FILE-SYSTEMS LAYER COMPRESSION

Database for Online Transaction Processing (OLTP) application is used by almost every corporations that has adopted computerisation to support their operational day to day business. Compression in the storage or file-systems layer has not been widely adopted for OLTP database because of the concern that it might decrease database performance. OLTP compression in the database layer is available commercially but it has a significant licence cost that reduces the cost saving of compression. In this research, transparent file-system compression with LZ4, LZJB and ZLE algorithm have been tested to improve performance of OLTP application. Using Swing-bench as the benchmark tool and Oracle database 12c, The result indicated that on OLTP workload, LZJB was the most optimal compression algorithm with performance improvement up to 49% and consistent reduction of maximum response time and CPU utilisation overhead, while LZ4 was the compression with the highest compression ratio and ZLE was the compression with the lowest CPU utilisation overhead. In terms of compression ratio, LZ4 can deliver the highest compression ratio which is 5.32, followed by LZJB, 4.92; and ZLE, 1.76. Furthermore, it is found that there is indeed a risk of reduced performance and/or an increase of maximum response time.

Service demand modeling and performance prediction with single-user tests

Performance load tests of online transaction processing (OLTP) applications are expensive in terms of manpower, time and costs. Alternative performance modeling and prediction tools are required to generate accurate outputs with minimal input sample points. Service Demands (time needed to serve 1 request at queuing stations) are typically needed as inputs by most performance models. However, as service demands vary as a function of workload (load dependent service demands), models that input singular service demands produce erroneous predictions. The alternative, which is to collect service demands at varying workloads, requires time and resource intensive load tests to estimate multiple sample points—which defeats the purpose of performance modeling for industrial use. In this paper, we propose a service demand model as a function of concurrency that can be estimated with a single-user test. Further, we analyze multiple CPU performance metrics (cache hits/misses, branch prediction, context switches and so on) using Principal Component Analysis (PCA) to extract a regression function of service demand with increasing workloads. We use the service demand models as input to performance prediction algorithms such as Mean Value Analysis (MVA), to accurately predict throughput at varying workloads. This service demand prediction model uses CPU hardware counters, which is used in conjunction with a modified version of MVA with single-user service demand inputs. The predicted throughput values are within 9% deviation with measurements procured for a variety of application/hardware configurations. Such a service demand model is a step towards reducing reliance on conventional load testing for performance assurance.

POSTER

Large-scale graphs are becoming critical in various domains such as social network, scientific application, knowledge discovery, and even system software, etc. Many of those use cases require large-scale high-performance graph databases, which are designed for serving continuous updates from the clients, and at the same time, answering complex queries towards the current graph in an on-line manner. Those operations in graph databases, also referred as OLTP (online transaction processing) operations, need specific design and implementation in graph partitioning algorithms. In this study, we designed an incremental online graph partitioning (IOGP), optimized for OLTP workloads. It is designed to achieve better locality, generate balanced partitions, and increase the parallelism for accessing hotspots of the graph. Our evaluation results on both real world and synthetic graphs in both simulation and real system confirm a better performance on graph queries (as much as 2X) with small overheads during graph insertion (less than 10%).

Toward a General Distributed Messaging Framework for Online Transaction Processing Applications

This paper presents a general distributed messaging framework for online transaction processing applications in e-business industry, and we name it as Metamorphosis (MetaQ). Specifically, this messaging framework has features of high availability, scalability, and high data throughput. The current implementation of MetaQ supports distributed XA transactions, asynchronous messaging, and multiple ways for storing message offsets. As a consequence, it is suited to the application contexts having a large quantity of messages, transaction-support, and real-time requirements. More important, another branch of MetaQ implementation, i.e., RocketMQ has been deployed and used in Taobao.com and Alipay.com. The real usages in both typical online transaction applications have proven that the nature of MetaQ can perform well for such big Internet applications.

What's Really New with NewSQL?

A new class of database management systems (DBMSs) called NewSQL tout their ability to scale modern on-line transaction processing (OLTP) workloads in a way that is not possible with legacy systems. The term NewSQL was first used by one of the authors of this article in a 2011 business analysis report discussing the rise of new database systems as challengers to these established vendors (Oracle, IBM, Microsoft). The other author was working on what became one of the first examples of a NewSQL DBMS. Since then several companies and research projects have used this term (rightly and wrongly) to describe their systems. Given that relational DBMSs have been around for over four decades, it is justifiable to ask whether the claim of NewSQL's superiority is actually true or whether it is simply marketing. If they are indeed able to get better performance, then the next question is whether there is anything scientifically new about them that enables them to achieve these gains or is it just that hardware has advanced so much that now the bottlenecks from earlier years are no longer a problem. To do this, we first discuss the history of databases to understand how NewSQL systems came about. We then provide a detailed explanation of what the term NewSQL means and the different categories of systems that fall under this definition.

在大规模系统上优化 TPC-C 评测程序

Transaction processing performance council benchmark C (TPC-C) is the de facto standard for evaluating the performance of high-end computers running on-line transaction processing applications. Differing from other standard benchmarks, the transaction processing performance council only defines specifications for the TPC-C benchmark, but does not provide any standard implementation for end-users. Due to the complexity of the TPC-C workload, it is a challenging task to obtain optimal performance for TPC-C evaluation on a large-scale high-end computer. In this paper, we designed and implemented a large-scale TPC-C evaluation system based on the latest TPC-C specification using solid-state drive (SSD) storage devices. By analyzing the characteristics of the TPC-C workload, we propose a series of system-level optimization methods to improve the TPC-C performance. First, we propose an approach based on SmallFile table space to organize the test data in a round-robin method on all of the disk array partitions; this can make full use of the underlying disk arrays. Second, we propose using a NOOP-based disk scheduling algorithm to reduce the utilization rate of processors and improve the average input/output service time. Third, to improve the system translation lookaside buffer hit rate and reduce the processor overhead, we take advantage of the huge page technique to manage a large amount of memory resources. Lastly, we propose a locality-aware interrupt mapping strategy based on the asymmetry characteristic of non-uniform memory access systems to improve the system performance. Using these optimization methods, we performed the TPC-C test on two large-scale high-end computers using SSD arrays. The experimental results show that our methods can effectively improve the TPC-C performance. For example, the performance of the TPC-C test on an Intel Westmere server reached 1.018 million transactions per minute.

STI-BT: A Scalable Transactional Index

Distributed Key-Value (DKV) stores have been intensively used to manage online transaction processing on large data-sets. DKV stores provide simplistic primitives to access data based on the primary key of the stored objects. To help programmers to efficiently retrieve data, some DKV stores provide distributed indexes. Besides that, and also to simplify programming such applications, several proposals have provided strong consistency abstractions via distributed transactions. In this paper we present STI-BT, a highly scalable, transactional index for Distributed Key-Value stores. STI-BT is organized as a distributed B $^+$ Tree and adopts an innovative design that allows to achieve high efficiency in large-scale, elastic DKV stores. As such, it provides both the desirable properties identified above, and does so in a far more efficient and scalable way than the few existing state of the art proposals that also enable programmers to have strongly consistent distributed transactional indexes. We have implemented STI-BT on top of an open-source DKV store and deployed it on a public cloud infrastructure. Our extensive study demonstrates scalability in a cluster of $100$ machines, and speed ups with respect to state of the art up to $5.4\times$ .

Eksplorasi dan Perancangan Prototipe Aplikasi Kantor Maya Untuk Meningkatkan Sustainable Competitive Advantage di Perguruan Tinggi

<p><em>Sustainable competitive advantage for universities could not be separated from the information technology. Actually, some universities have been implemented this information technology, especially virtual office applications. Eventhough, the virtual office applications were implemented, but preliminary research found that those applications used were Online Transaction Processing only, not Online Analytical Processing that can be used to make decision. Based on those explanations, this research had been done to explore and design the virtual office prototype application for universities. Three methods were used to conduct this reseach, which were explorative, validation, and application design. Finally, prototype applications of virtual office dashboard, lecturer dashboard and accreditation dashboard to improve sustainable competitive advantage for universities were presented as this study result.</em></p>

COMMERCIAL SYSTEM WITH SPECIAL REFERENCE TO E-COMMERCE, ITS PROS & CONS

E-commerce or Electronic commerce is a methodology of modern business, which addresses the requirements of business organizations. It can be broadly defined as the process of buying or selling of goods or services using an Electronic Medium such as the Internet. A person sitting on his chair in front of a computer can access all the facilities of the internet to buy or sell the products. Traditional commerce that is carried out physically with effort of a person to go and get products. E-commerce has made it easier for human to reduce physical work and to save a lot of time. Even though it gives a lot of advantages to the economy, it faces some demerits mainly in the case of security. Security is the challenge facing E-commerce today. With the invention of E-commerce there is the possibility for B to B business, business through ads in YouTube, through website, mobile commerce, Electronic Fund Transfer, supply chain management, internet marketing, online transaction process etc..Even though with merits and demerits, E-commerce is in a developing stage.

A software cybernetics approach to self-tuning performance of on-line transaction processing systems

We construct two layer fuzzy rules based on the collected data.We develop algorithms from software cybernetics view to reconfigure the performance control parameters.We show self-tuning performance for On-Line Transaction Processing. Self-tuning performance of On-Line Transaction Processing (OLTP) Systems is a challenging and time-consuming task since multiple performance parameters are needed to be automatically configured in Database Management Systems (DBMSs). In this paper, we present a software cybernetics approach to self-tune the performance of DBMSs. A DBMS is designed with an adaptive control based on fuzzy logic such that it has the capability to control objects, i.e., the performance parameters, and update the controller itself, i.e., a set of fuzzy rules in our case. The principles and concepts in software cybernetics are applied to guide the synthesis of software controllers for monitoring and adapting system behaviors. Experimental results for On-Line Transaction Processing using TPC-C, a benchmark of the Transaction Processing Performance Council, show that the proposed method is feasible and effective.