Data-intensive Activities Research Articles

Dynamic pricing on maximum concurrency for heterogeneous instances using hyperparameter optimization in dueling deep reinforcement learning in a multi-cloud scenario

Users possess the option to rent instances of various sorts, in a variety of regions, and a variety of availability zones, thanks to cloud service carriers like AWS, GCP, and Azure. In the cloud business right now, fixed price models are king when it comes to pricing. However, as the diversity of cloud providers and users grows, this approach is unable to accurately reflect the market’s current needs for cost savings. As a consequence, a dynamic pricing strategy has become a desirable tactic to better handle the erratic cloud demand. In this study, a deep learning model was used to propose a dynamic pricing structure that ensures service providers are treated fairly in a multi-cloud context. The computational optimization of DL approaches can be severely hampered by the requirement for human hyperparameter selection. Traditional automated solutions to this issue have inadequate durability or fail in specific circumstances. To choose the hyper-parameters in the Dueling Deep Q-Network (DDQN), the hybrid DL approach in this study uses the concept-based wild horse optimization (WHO) method. A community of untamed horses is evolved, and the fitness of the population is evaluated concurrently to estimate the optimum hyper-parameters. The plan changes the price appropriately to promote the use of underutilized resources and discourage the use of overutilized resources. The evaluation’s findings demonstrated that the suggested strategy can lower end-user costs while conducting compute- and data-intensive activities in a multi-cloud environment. The research was concluded by comparing current models after the results were analyzed using various performance indicators.

Tensions in Data Journey Activities: Mobilising, Processing, Producing, and Re-purposing Data in Environmental Assessment Practice

Digital data shape work practices as they travel through space, time, and social situations. In turn, workers shape the materiality of data through different data-oriented activities. In this paper, we draw on theory of data journeys to develop and present a conceptual framework of four data-intensive activities: Producing, mobilising, processing, and re-purposing. Together, those activities capture entanglements of data transformations and data-intensive practices. We develop the framework through a qualitative study of digitisation in Environmental Assessment (EA) practice. EA practitioners reflect on their experiences working with digital environmental data through ten semi-structured and two interactive interviews. The paper offers two main contributions. First, we develop a data journey theoretical framework and show that it is useful as a unit of analysis for uncovering tensions in data-oriented activities. Second, we use these insights to outline a set of tensions for future design activities in the area of data work practices.

Differential Privacy in Practice: Expose your Epsilons!

Differential privacy is at a turning point. Implementations have been successfully leveraged in private industry, the public sector, and academia in a wide variety of applications, allowing scientists, engineers, and researchers the ability to learn about populations of interest without specifically learning about these individuals. Because differential privacy allows us to quantify cumulative privacy loss, these differentially private systems will, for the first time, allow us to measure and compare the total privacy loss due to these personal data-intensive activities. Appropriately leveraged, this could be a watershed moment for privacy.
 Like other technologies and techniques that allow for a range of instantiations, implementation details matter. When meaningfully implemented, differential privacy supports deep data-driven insights with minimal worst-case privacy loss. When not meaningfully implemented, differential privacy delivers privacy mostly in name. Using differential privacy to maximize learning while providing a meaningful degree of privacy requires judicious choices with respect to the privacy parameter epsilon, among other factors. However, there is little understanding of what is the optimal value of epsilon for a given system or classes of systems/purposes/data etc. or how to go about figuring it out.
 To understand current differential privacy implementations and how organizations make these key choices in practice, we conducted interviews with practitioners to learn from their experiences of implementing differential privacy. We found no clear consensus on how to choose epsilon, nor is there agreement on how to approach this and other key implementation decisions. Given the importance of these implementation details there is a need for shared learning amongst the differential privacy community. To serve these purposes, we propose the creation of the Epsilon Registry—a publicly available communal body of knowledge about differential privacy implementations that can be used by various stakeholders to drive the identification and adoption of judicious differentially private implementations.

Big data and risk management in business processes: implications for corporate real estate

PurposeThe purpose of this paper is to improve understanding of the integration between big data (BD) and risk management (RM) in business processes (BPs), with special reference to corporate real estate (CRE).Design/methodology/approachThis conceptual study follows, methodologically, the structuring inter-textual coherence process – specifically, the synthesised coherence tactical approach. It draws heavily on theoretical evidence published, mainly, in the corporate finance and the business management literature.FindingsA new conceptual framework is presented for CRE to proactively develop insights into the potential benefits of using BD as a business strategy/instrument. The approach was found to strengthen decision-making processes and encourage better RM – with significant consequences, in particular, for business process management (BPM). Specifically, by recognising the potential uses of BD, it is also possible to redefine the processes with advantages in terms of RM.Originality/valueThis study contributes to the literature in the fields of real estate, RM, BPM and digital transformation. To the best knowledge of authors, although the literature has examined the concepts of BD, RM and BP, no prior studies have comprehensively examined these three elements and their conjoint contribution to CRE. In particular, the study highlights how the automation of data-intensive activities and the analysis of such data (in both structured and unstructured forms), as a means of supporting decision making, can lead to better efficiency in RM and optimisation of processes.

Abstract 5282: A cloud-enabled open source data management platform supporting a federated research and development organization

Abstract Biopharmaceutical R&D organizations characterize drug candidate target effects and modes of action and create molecular models of target diseases. These data-intensive activities are informed by vast data resources including publicly available data, internally generated data and partnered private data collections. However, rapid evolution in computing, data management tools, analytical and visualization methods, the complexity of data types and the data volumes that must be accommodated present significant technical and logistic hurdles to overcome. It is particularly difficult for a geographically dispersed R&D organization to make data resources easily available to scientists for search, visualization and exploration. Nevertheless, this is required for R&D scientists to gain insight into disease and drug mechanisms and to capture the knowledge needed to sustain the scientific enterprise. Standardized commercial solutions to R&D data challenges are unattractive since they require significant resource investment in platform configuration, user-training and system maintenance. This strategy necessarily creates delay in adopting newly emerging technologies and provides incentive not to adopt alternatives due to investment in existing systems. In contrast, our solution to R&D data demands was to build a cloud-deployed data platform using state of the art tools developed and maintained by the open source software community at the Apache Software Foundation. Partnering with academic data scientists, we selected the best available tools to fit our specific needs. We integrated them into a platform accessible to our federated R&D scientific community while allowing the system to be freely modified and updated on demand to meet evolving user requirements. Priorities for our data platform are to ingest, secure and index R&D source data of all types, make these indexed data assets available to computational scientists for analysis and provide faceted search capability based on a comprehensive metadata model. Three products: LabKey server, Apache OODT and ISATools have all been combined into a scientific data management system to provide a unified data resource enhanced by a search platform powered by Apache Solr. The platform supports both internally generated data and data imported from public, contracted or partnered sources. All data are available for interactive exploration by our R&D community, accessed via integrated search, analysis and visualization tools. Deployment of this system to our R&D organization has been met with enthusiastic adoption. Feedback for improvement or requests for system enhancements and additional capabilities are rapidly addressed in this open source environment, leading to further adoption among the R&D scientists and providing the basis for accessible, stable institutional knowledge collections. Citation Format: Lauren Intagliata, Selina Chu, Garth McGrath, Giuseppe Totaro, Daniel Civello, Nipurn Doshi, Shivika Thapar, Michael Livstone, Chris Mattmann, Paul Ramirez, Maureen Cronin. A cloud-enabled open source data management platform supporting a federated research and development organization. [abstract]. In: Proceedings of the 107th Annual Meeting of the American Association for Cancer Research; 2016 Apr 16-20; New Orleans, LA. Philadelphia (PA): AACR; Cancer Res 2016;76(14 Suppl):Abstract nr 5282.

Common motifs in scientific workflows: An empirical analysis

Workflow technology continues to play an important role as a means for specifying and enacting computational experiments in modern science. Reusing and re-purposing workflows allow scientists to do new experiments faster, since the workflows capture useful expertise from others. As workflow libraries grow, scientists face the challenge of finding workflows appropriate for their task, understanding what each workflow does, and reusing relevant portions of a given workflow. We believe that workflows would be easier to understand and reuse if high-level views (abstractions) of their activities were available in workflow libraries. As a first step towards obtaining these abstractions, we report in this paper on the results of a manual analysis performed over a set of real-world scientific workflows from Taverna, Wings, Galaxy and Vistrails. Our analysis has resulted in a set of scientific workflow motifs that outline (i) the kinds of data-intensive activities that are observed in workflows (Data-Operation motifs), and (ii) the different manners in which activities are implemented within workflows (Workflow-Oriented motifs). These motifs are helpful to identify the functionality of the steps in a given workflow, to develop best practices for workflow design, and to develop approaches for automated generation of workflow abstractions.

Scripting for large-scale sequencing based on Hadoop

The large volumes of data generated by modern sequencing experiments present significant challenges in their manipulation and analysis. Traditional approaches are often found to be complicated to scale. We describe our ongoing work on SeqPig, a tool that facilitates the use of the Pig Latin distributed scripting language to manipulate, analyze and query sequencing data applying the advances motivated by the “big data revolution” in data-intensive activities. SeqPig provides access to popular data formats and implements a number of custom sequencing-specific functions. Most importantly, it grants users access to the scalable Hadoop platform from a high level scripting language

DZero data-intensive computing on the Open Science Grid

High energy physics experiments periodically reprocess data, in order to take advantage of improved understanding of the detector and the data processing code. Between February and May 2007, the DZero experiment has reprocessed a substantial fraction of its dataset. This consists of half a billion events, corresponding to about 100 TB of data, organized in 300,000 files.The activity utilized resources from sites around the world, including a dozen sites participating to the Open Science Grid consortium (OSG). About 1,500 jobs were run every day across the OSG, consuming and producing hundreds of Gigabytes of data. Access to OSG computing and storage resources was coordinated by the SAM-Grid system. This system organized job access to a complex topology of data queues and job scheduling to clusters, using a SAM-Grid to OSG job forwarding infrastructure.For the first time in the lifetime of the experiment, a data intensive production activity was managed on a general purpose grid, such as OSG. This paper describes the implications of using OSG, where all resources are granted following an opportunistic model, the challenges of operating a data intensive activity over such large computing infrastructure, and the lessons learned throughout the project.

The Special Case of Pesticides: Science and Regulation

Pesticide regulation is a data-intensive activity that is strongly controlled at the federal level. The pesticide law FIFRA controls pesticide sale, use, and disposal. Normal use of pesticides is excluded under RCRA and CERCLA. There are approximately 1,000 pesticide active ingredients with highly diverse functions ranging from insecticides to disinfectants and pheromones (insect sex attractants). There is a relatively large database on many pesticides, which helps reduce uncertainty in risk assessments. Witnesses subject to Daubert should be familiar with the data as well as the computer models designed for pesticides. “Inert” ingredients that are formulated with the pesticide active ingredients can become issues in torts cases. Federal courts have ruled both in favor and against FIFRA preemption, i.e., the ability of a plaintiff to seek relief in a torts case may be blocked by FIFRA. Golf course permitting provides a special land use whereby fundamental pesticide risk and regulatory issues are debated at the level of local planning and zoning commissions.

Mining a large database with a parallel database server

Data mining is a data-intensive computation activity. Parallel processing has often been used in data mining algorithms. However, when data do not fit in memory, some solutions do not apply and a database system may be required rather than flat files. Most of the implementations use the database system loosely coupled with the data mining techniques. Hence, the database system only issues queries to be processed on the client machine. In this work, we address the data consuming activities through parallel processing on a database server providing a tight integration with data mining techniques. Experimental results showing the potential benefits of this integration were obtained. Despite the difficulties in processing a complex application, we extracted rules and obtained high performance on all the data-intensive activities such as the construction of the decision tree, pruning and rule extraction.

Research perspectives for time series management systems

Empirical research based on time series is a data intensive activity that needs a data base management system (DBMS). We investigate the special properties a time series management system (TSMS) should have. We then show that currently available solutions and related research directions are not well suited to handle the existing problems. Therefore, we propose the development of a special purpose TSMS, which will offer particular modeling, retrieval, and computation capabilities. It will be suitable for end users, offer direct manipulation interfaces, and allow data exchange with a variety of data sources, including other databases and application packages. We intend to build such a system on top of an off-the-shelf object-oriented DBMS.