Data Transfer Management Research Articles

The structure of control and data transfer management system for the GAMMA-400 scientific complex

A description of the control and data transfer management system for scientific instrumentation involved in the GAMMA-400 space project is given. The technical capabilities of all specialized equipment to provide the functioning of the scientific instrumentation and satellite support systems are unified in a single structure. Control of the scientific instruments is maintained using one-time pulse radio commands, as well as program commands in the form of 16-bit code words, which are transmitted via onboard control system and scientific data acquisition system. Up to 100 GByte of data per day can be transferred to the ground segment of the project. The correctness of the proposed and implemented structure, engineering solutions and electronic elemental base selection has been verified by the experimental working-off of the prototype of the GAMMA-400 scientific complex in laboratory conditions.

SKMD

Heterogeneous computing on CPUs and GPUs has traditionally used fixed roles for each device: the GPU handles data parallel work by taking advantage of its massive number of cores while the CPU handles non data-parallel work, such as the sequential code or data transfer management. This work distribution can be a poor solution as it underutilizes the CPU, has difficulty generalizing beyond the single CPU-GPU combination, and may waste a large fraction of time transferring data. Further, CPUs are performance competitive with GPUs on many workloads, thus simply partitioning work based on the fixed roles may be a poor choice. In this article, we present the single-kernel multiple devices (SKMD) system, a framework that transparently orchestrates collaborative execution of a single data-parallel kernel across multiple asymmetric CPUs and GPUs. The programmer is responsible for developing a single data-parallel kernel in OpenCL, while the system automatically partitions the workload across an arbitrary set of devices, generates kernels to execute the partial workloads, and efficiently merges the partial outputs together. The goal is performance improvement by maximally utilizing all available resources to execute the kernel. SKMD handles the difficult challenges of exposed data transfer costs and the performance variations GPUs have with respect to input size. On real hardware, SKMD achieves an average speedup of 28% on a system with one multicore CPU and two asymmetric GPUs compared to a fastest device execution strategy for a set of popular OpenCL kernels.

The Globus Galaxies platform: delivering science gateways as a service

SummaryThe use of public cloud computers to host sophisticated scientific data and software is transforming scientific practice by enabling broad access to capabilities previously available only to the few. The primary obstacle to more widespread use of public clouds to host scientific software (‘cloud‐based science gateways’) has thus far been the considerable gap between the specialized needs of science applications and the capabilities provided by cloud infrastructures. We describe here a domain‐independent, cloud‐based science gateway platform, the Globus Galaxies platform, which overcomes this gap by providing a set of hosted services that directly address the needs of science gateway developers. The design and implementation of this platform leverages our several years of experience with Globus Genomics, a cloud‐based science gateway that has served more than 200 genomics researchers across 30 institutions. Building on that foundation, we have implemented a platform that leverages the popular Galaxy system for application hosting and workflow execution; Globus services for data transfer, user and group management, and authentication; and a cost‐aware elastic provisioning model specialized for public cloud resources. We describe here the capabilities and architecture of this platform, present six scientific domains in which we have successfully applied it, report on user experiences, and analyze the economics of our deployments. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

Chemical sensor platform for non-invasive monitoring of activity and dehydration.

A non-invasive solution for monitoring of the activity and dehydration of organisms is proposed in the work. For this purpose, a wireless standalone chemical sensor platform using two separate measurement techniques has been developed. The first approach for activity monitoring is based on humidity measurement. Our solution uses new humidity sensor based on a nanostructured TiO2 surface for sweat rate monitoring. The second technique is based on monitoring of potassium concentration in urine. High level of potassium concentration denotes clear occurrence of dehydration. Furthermore, a Wireless Body Area Network (WBAN) was developed for this sensor platform to manage data transfer among devices and the internet. The WBAN coordinator controls the sensor devices and collects and stores the measured data. The collected data is particular to individuals and can be shared with physicians, emergency systems or athletes' coaches. Long-time monitoring of activity and potassium concentration in urine can help maintain the appropriate water intake of elderly people or athletes and to send warning signals in the case of near dehydration. The created sensor system was calibrated and tested in laboratory and real conditions as well. The measurement results are discussed.

Field Evaluation of the Smartphone-based Travel Behaviour Data Collection App “SmartMo”

This paper outlines an innovative approach to the evaluation of a self-administered smartphone-based survey for the collection of travel behaviour data. For this approach, a traditional travel survey is modified to match mobile devices. The smartphone application “SmartMo” is designed in a multi-stage iterative development process. It is then implemented and evaluated through a number of field tests involving 97 participants. Results of the field evaluation will be discussed including the technical performance (e.g. secure data transfer and data management, energy consumption, map-matching), usability (e.g. comprehensibility, handling, joy of use) as well as user acceptance (e.g. willingness to participate, data protection and privacy). A brief overview of the SmartMo data collection system will also be provided.

Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

Graphics processors, or GPUs, have recently been widely used as accelerators in shared environments such as clusters and clouds. In such shared environments, many kernels are submitted to GPUs from different users, and throughput is an important metric for performance and total ownership cost. Despite recently improved runtime support for concurrent GPU kernel executions, the GPU can be severely underutilized, resulting in suboptimal throughput. In this paper, we propose Kernelet, a runtime system to improve the throughput of concurrent kernel executions on the GPU. Kernelet embraces transparent memory management and PCI-e data transfer techniques, and dynamic slicing and scheduling techniques for kernel executions. With slicing, Kernelet divides a GPU kernel into multiple sub-kernels (namely slices ). Each slice has tunable occupancy to allow co-scheduling with other slices for high GPU utilization. We develop a novel Markov chain-based performance model to guide the scheduling decision. Our experimental results demonstrate up to 31 percent and 23 percent performance improvement on NVIDIA Tesla C2050 and GTX680 GPUs, respectively.

Efficient fine-grained shared buffer management for multiple OpenCL devices

OpenCL programming provides full code portability between different hardware platforms, and can serve as a good programming candidate for heterogeneous systems, which typically consist of a host processor and several accelerators. However, to make full use of the computing capacity of such a system, programmers are requested to manage diverse OpenCL-enabled devices explicitly, including distributing the workload between different devices and managing data transfer between multiple devices. All these tedious jobs pose a huge challenge for programmers. In this paper, a distributed shared OpenCL memory (DSOM) is presented, which relieves users of having to manage data transfer explicitly, by supporting shared buffers across devices. DSOM allocates shared buffers in the system memory and treats the on-device memory as a software managed virtual cache buffer. To support fine-grained shared buffer management, we designed a kernel parser in DSOM for buffer access range analysis. A basic modified, shared, invalid cache coherency is implemented for DSOM to maintain coherency for cache buffers. In addition, we propose a novel strategy to minimize communication cost between devices by launching each necessary data transfer as early as possible. This strategy enables overlap of data transfer with kernel execution. Our experimental results show that the applicability of our method for buffer access range analysis is good, and the efficiency of DSOM is high.

Grade Exam Management System Design and Implementation Based on ADO.NET

The grade examination management program is various, workload is big, and efficiency is low. This paper on the basis of the existing educational system and the provisions of the state on the basis of grade examination system, developed a test management data transfer system based on ADO.NET technology, the heterogeneous data interconnection test process, the full realization of cyber source sharing, improving the efficiency of examination management. Through the introduction of test data of the design has been completed and mature and stable operation of the system, detailed description of the design and implementation, it has good application value.

Integrated LTSSM (Link Training & Status State Machine) and MAC Layer of USB 3.0 Device for Reliable SuperSpeed Data Transactions

USB (Universal Serial Bus) is the current generation of computer peripherals widely used transmission interface. User applications demand a higher performance connection between the PC and other increasingly sophisticated peripherals. USB 3.0 enables more demanding applications compared to its earlier interfaces – Fast Speed USB 1.1 (12 Mbps) and High Speed USB 2.0 (480 Mbps), because of its high bandwidth (Super Speed 4.8Gbps), efficient power management scheme and reliable data flow control. In this research we have extended our previous work and developed a synthesizable hardware of LTSSM (Link Training and Transition State Machine) for ensuring the reliable USB link for data transfer and efficient power management to reduce any power wasting conditions. We have integrated LTSSM with previously developed USB 3.0 MAC layer controller according to the constraints specified by Intel Corporation in its USB 3.0 specifications sheet. This implementation meets the required constraints and ensures the data rate of at least 4.5 Gbps.

Towards International Cooperation and Capacity Building between Space Agencies: A Case of GISTDA and NSPO

Geo-Informatics and Space Technology Development Agency (GISTDA) firmly believes in the synergy of international cooperation and capacity building and is always on a quest to build global knowledge partners. Keeping in mind the growing trend of international cooperation, GISTDA and the National Space Organization (NSPO) of Taiwan will enter into a cooperation agreement that will develop space system engineering in Thailand. The cooperation aims to promote technical activities, human resources development as well as infrastructure development in both countries. Several cooperative activities had already been initiated according to the work plan and an In-flight Systems Engineering Technical Assistance (SETA) has also been drawn up and agreed upon by both the agencies. The SETA encompasses the area of satellite operation, satellite and ground maintenance, anomaly handling, instrument calibration and satellite life extension assessment. The execution procedure and budget planning has been taken into consideration and the allotment of resources, both time and money, are planned for. GISTDA has also proposed the timeframe for personnel exchanges and training program to NSPO and the work plan will be put into action as soon as the areas are mutually agreed upon. The paper shall discuss the entire project plans divided into phases with major concentration on the e-aspect of the cooperation. Online training, remote system management and satellite data transfer over the Research and Education Network (REN) for emergency and educational purposes are but a few of the strategic areas covered in this cooperation. Mutual cooperation between the two agencies will help facilitate mission-critical agendas with no visible shortcoming. The paper shall also set precedent for future cooperation between international agencies and discuss how cost saving is achieved with the application of communication technologies.

Managing data transfers in computer clusters with orchestra

Cluster computing applications like MapReduce and Dryad transfer massive amounts of data between their computation stages. These transfers can have a significant impact on job performance, accounting for more than 50% of job completion times. Despite this impact, there has been relatively little work on optimizing the performance of these data transfers, with networking researchers traditionally focusing on per-flow traffic management. We address this limitation by proposing a global management architecture and a set of algorithms that (1) improve the transfer times of common communication patterns, such as broadcast and shuffle, and (2) allow scheduling policies at the transfer level, such as prioritizing a transfer over other transfers. Using a prototype implementation, we show that our solution improves broadcast completion times by up to 4.5X compared to the status quo in Hadoop. We also show that transfer-level scheduling can reduce the completion time of high-priority transfers by 1.7X.

Structure for control and data transfer from scientific instruments of the CORONAS-PHOTON mission

A description is given of the control structure of scientific instruments and data transfer management system involved in the CORONAS-PHOTON satellite mission. The technical capabilities of all specialized instruments to provide the function of the scientific instruments and spacecract (SC) support systems are unified in a single structure. The correctness of the proposed and implemented structure has been verified by the operation of the CORONAS-PHOTON’s complex of scientific instruments. Control of the scientific instruments was maintained using one-time and program pulse commands (OTC and PPC), as well as command data files (CDF), which are transmitted via onboard control units. Up to 30 OTC and 52 kbit of CDF can be transmitted per one control session. The date is transmitted to data-acquisition ground stations (DAGS) at Scientific center for Earth orbital monitoring. Up to 1.3 Gbyte of data can be transmitted per day. Data encryption and reception by two DAGS simultaneously was used to reduce the quantity of transmission failures. The CORONAS-PHOTON SC has been performing scientific experiments from February to December 2009. Over this period, 297 control sessions and 1050 data acquisition sessions were performed; about 5 Mbit of uplink control data were transmitted on board SC, and 250 Gbyte of scientific data were received by DAGS.

HiCUDA: High-Level GPGPU Programming

Graphics Processing Units (GPUs) have become a competitive accelerator for applications outside the graphics domain, mainly driven by the improvements in GPU programmability. Although the Compute Unified Device Architecture (CUDA) is a simple C-like interface for programming NVIDIA GPUs, porting applications to CUDA remains a challenge to average programmers. In particular, CUDA places on the programmer the burden of packaging GPU code in separate functions, of explicitly managing data transfer between the host and GPU memories, and of manually optimizing the utilization of the GPU memory. Practical experience shows that the programmer needs to make significant code changes, often tedious and error-prone, before getting an optimized program. We have designed hiCUDA}, a high-level directive-based language for CUDA programming. It allows programmers to perform these tedious tasks in a simpler manner and directly to the sequential code, thus speeding up the porting process. In this paper, we describe the hiCUDA} directives as well as the design and implementation of a prototype compiler that translates a hiCUDA} program to a CUDA program. Our compiler is able to support real-world applications that span multiple procedures and use dynamically allocated arrays. Experiments using nine CUDA benchmarks show that the simplicity hiCUDA} provides comes at no expense to performance.

Design and implementation of high-speed buffered crossbars with efficient load balancing for multi-core SoCs

A large increase of the number of devices integrated in a single chip in conjunction with the significant demands of modern applications for performance has led the designers to a system development methodology based on integrating multiple pre-verified intellectual property cores. Yet, design productivity requirements push designers to focus on key micro-architectural solutions to manage more efficiently the scaling of multi-core SoCs as well as to increase the degree of design automation, particularly as rapid prototyping using reconfigurable computing is becoming mainstream. In this paper we present a novel interconnect architecture based on optimized components to efficiently manage SoCs that follow either a multi-core based approach or are built to support SIMD-style applications that can exploit the processing power of a pool of hardware resources; first we analyze the design of a crossbar featuring shared-memory combined input-crosspoint buffering as a solution for efficient implementation of on-chip interconnection; second we describe the design of a load-balancer featuring configurable proportional allocation of on-chip resources and in-order delivery as a solution for efficient scheduling and execution of processing tasks. The main focus of the paper is to describe and evaluate the mechanisms designed to distribute and manage data transfers so as to implement an efficient interconnection of the integrated cores and control access to available (either on-chip or off-chip) resources for the implementation of a number of embedded systems and applications. Each of these challenges is handled by the proposed architecture in an efficient way in terms of performance, cost in silicon and flexibility.

Critical services in the LHC computing

The LHC experiments (ALICE, ATLAS, CMS and LHCb) rely for the data acquisition, processing, distribution, analysis and simulation on complex computing systems, running using a variety of services, provided by the experiments, the Worldwide LHC Computing Grid and the different computing centres. These services range from the most basic (network, batch systems, file systems) to the mass storage services or the Grid information system, up to the different workload management systems, data catalogues and data transfer tools, often internally developed in the collaborations. In this contribution we review the status of the services most critical to the experiments by quantitatively measuring their readiness with respect to the start of the LHC operations. Shortcomings are identified and common recommendations are offered.

An asymmetric distributed shared memory model for heterogeneous parallel systems

Heterogeneous computing combines general purpose CPUs with accelerators to efficiently execute both sequential control-intensive and data-parallel phases of applications. Existing programming models for heterogeneous computing rely on programmers to explicitly manage data transfers between the CPU system memory and accelerator memory. This paper presents a new programming model for heterogeneous computing, called Asymmetric Distributed Shared Memory (ADSM), that maintains a shared logical memory space for CPUs to access objects in the accelerator physical memory but not vice versa. The asymmetry allows light-weight implementations that avoid common pitfalls of symmetrical distributed shared memory systems. ADSM allows programmers to assign data objects to performance critical methods. When a method is selected for accelerator execution, its associated data objects are allocated within the shared logical memory space, which is hosted in the accelerator physical memory and transparently accessible by the methods executed on CPUs. We argue that ADSM reduces programming efforts for heterogeneous computing systems and enhances application portability. We present a software implementation of ADSM, called GMAC, on top of CUDA in a GNU/Linux environment. We show that applications written in ADSM and running on top of GMAC achieve performance comparable to their counterparts using programmer-managed data transfers. This paper presents the GMAC system and evaluates different design choices. We further suggest additional architectural support that will likely allow GMAC to achieve higher application performance than the current CUDA model.

An asymmetric distributed shared memory model for heterogeneous parallel systems

Heterogeneous computing combines general purpose CPUs with accelerators to efficiently execute both sequential control-intensive and data-parallel phases of applications. Existing programming models for heterogeneous computing rely on programmers to explicitly manage data transfers between the CPU system memory and accelerator memory. This paper presents a new programming model for heterogeneous computing, called Asymmetric Distributed Shared Memory (ADSM), that maintains a shared logical memory space for CPUs to access objects in the accelerator physical memory but not vice versa. The asymmetry allows light-weight implementations that avoid common pitfalls of symmetrical distributed shared memory systems. ADSM allows programmers to assign data objects to performance critical methods. When a method is selected for accelerator execution, its associated data objects are allocated within the shared logical memory space, which is hosted in the accelerator physical memory and transparently accessible by the methods executed on CPUs. We argue that ADSM reduces programming efforts for heterogeneous computing systems and enhances application portability. We present a software implementation of ADSM, called GMAC, on top of CUDA in a GNU/Linux environment. We show that applications written in ADSM and running on top of GMAC achieve performance comparable to their counterparts using programmer-managed data transfers. This paper presents the GMAC system and evaluates different design choices. We further suggest additional architectural support that will likely allow GMAC to achieve higher application performance than the current CUDA model.

Balancing internal supply chain logistics: a comparative analysis of manufacturing and service firm operations

With the advent of high speed and large volume data transfer, logistics and supply chain management techniques have made great changes in internal supply chains. The present study dealt with three large organisations, at least regionally headquartered in the Pittsburgh, PA (US) metropolitan area, within the service industry and how they strategically leverage incentives to promote SCM-related concepts to attract and retain internal and external customers. Each segment will begin with a brief synopsis of the current situation, followed by examples of how the company incorporated logistical SCM-related systems into their strategic plans. In terms of SCM initiatives, each company was documented and their relationships explored with various operational and financial performance metrics, including information sharing, product and service customisation, long-term relationships, hedging strategies, advanced planning systems, leveraging the internet, supply network structure, and distribution network structure.

E-Banking: Performance, Problems, and Potentials in Bangladesh

With the technological advancement in course of time E-banking has become a need of time around the globe. It has changed the way of doing banking business by making the whole business speedier, more flexible, and more consumers friendly. If connected with the Internet, one can bank round the clock from anywhere on the planet -with one single account only, which makes life easier. But banking online is prone to different risks related to online security i.e. risks and frauds associated with online data transfer and data management. Bangladesh, a labor surplus country, still lacks proper ICT infrastructure, IT education facilities, and the required human resources followed by the country fails to get the cent percent benefits from this technical update. However, E-banking has been started in Bangladesh since 2005 with high rate of market penetration. But risks are still at large as Bangladesh could not make significant progress on the various issues of E-banking business model like access to computer, Internet, and IT knowledge. On the other hand, poor governance along with the negative effects of automation because of E-banking like workforce cut creates employee dissatisfaction leading poor performance at the end. Therefore, the question arise will it be feasible to establish and develop E-banking in a developing country like Bangladesh and even if it is being introduced, to what extent E-banking facilities should be incorporated.

SCoPE: Towards a Systolic Array for SVM Object Detection

This paper presents SCoPE (systolic chain of processing elements), a first step towards the realization of a generic systolic array for support vector machine (SVM) object classification in embedded image and video applications. SCoPE provides efficient memory management, reduced complexity, and efficient data transfer mechanisms. The proposed architecture is generic and scalable, as the size of the chain, and the kernel module can be changed in a plug and play approach without affecting the overall system architecture. These advantages provide versatility, scalability and reduced complexity that make it ideal for embedded applications. Furthermore, the SCoPE architecture is intended to be used as a building block towards larger systolic systems for multi-input or multi-class classification. Simulation results indicate real-time performance, achieving face detection at ~33 frames per second on an FPGA prototype.