Intermediate Datasets Research Articles

A comment on Dincecco et al. (2022): Pre‐colonial warfare and long‐run development in India

AbstractWe test the reproducibility and replicability of M. Dincecco, J. Fenske, A. Menon and S. Mukherjee (2022), which reports a positive relationship between pre‐colonial interstate warfare and long‐run development patterns across India. Overall, we confirm that all of the study's estimates are computationally reproducible using the provided replication package in Stata, but note that the ease of replication could be improved by the provision of code and intermediate data sets for the conflict exposure measure. We test for and find no evidence of data manipulation in the final data sets. Concerning direct replicability, we consider different ways of measuring distance to conflicts and also alternative proxies for both the dependent variable and variables that capture channels by which the main effects operate. We find that some estimates are sensitive to the type of conflict considered. Other estimates are sensitive to the time period considered, most likely due to time heterogeneity in the number of conflicts recorded. Nevertheless, most estimates are substantially in line with the original study.

Intermediate redshift calibration of gamma-ray bursts and cosmic constraints in non-flat cosmology

ABSTRACT We propose a new method to calibrate gamma-ray burst (GRB) correlations employing intermediate redshift data sets, instead of limiting to z ≃ 0 catalogues, and applied it to the well-consolidated Amati correlation. This model-independent calibration technique is based on the Bézier polynomial interpolation of the most updated observational Hubble data and baryonic acoustic oscillations (BAO) and alleviates de facto the well-known circularity problem affecting GRB correlations. In doing so, we also investigate the influence of the BAO scales rs, got from Planck results, and $r_{\rm s}^{\rm fid}$, got from the considered fiducial cosmology, by considering the cases $(r_{\rm s}/r_{\rm s}^{\rm fid})=1$ and $(r_{\rm s}/r_{\rm s}^{\rm fid})\ne 1$. We get constraints on the cosmic parameters, using Markov chain–Monte Carlo simulations, first fixing and then leaving free the spatial curvature density parameter Ωk. Compared to previous literature, we obtain tighter constraints on the mass density parameter Ωm. In particular, our findings turn out to be highly more compatible with those got from standard candle indicators within the Lambda cold dark matter paradigm. Finally, we critically re-examine the recent H0 tension in view of our outcomes.

Review on Advanced Cost Effective Approach for Privacy with Dataset in Cloud Storage

Cloud computing allows customers to run compute and data-intensive applications without the need for a large investment in infrastructure. Additionally, a significant amount of intermediate datasets are created and often saved, in order to reduce the expense of re-computing these applications. It becomes difficult to protect the privacy of intermediate datasets because attackers may be able to retrieve information that is sensitive to privacy via the analysis of several intermediate datasets. Existing techniques to deal with this problem generally endorse the use of encryption for all cloud datasets. For data-intensive applications, the time and expense of repeatedly decrypting and encrypting intermediate datasets are prohibitive; hence, encrypting all intermediate datasets does not make sense. Big heterogeneous data storage concerns and challenges, countermeasures (security and administration) and cloud storage prospects, are discussed in this article. New questions arise for cloud storage researchers, when they examine these issues in depth.

A Protection Layer over MapReduce Framework for Big Data Privacy

In many organizations, big data analytics has become a trend in gathering valuable data insights. The framework MapReduce, which is generally used for this purpose, has been accepted by most organizations for its exceptional characteristics. However, because of the availability of significant processing resources, dispersed privacy-sensitive details can be collected quickly, increasing the widespread privacy concerns. This article reviews some of the existing research articles on the MapReduce framework's privacy issues and proposes an additional layer of privacy protection over the adopted framework. The data is split into bits and processed in the clouds, and two other steps are taken. Hadoop splits the file into bits of a smaller scale. The task tracker then allocates these bits to several mappers. First, the data is split up into key-value pairs, and the intermediate data sets are generated. The efficiency of the suggested approach may then be effectively interpreted. Overall, the proposed method provides improved scalability. The following figures compare execution time with relation to file size and the number of partitions. As privacy protection technique is used, the loss of data content can be appropriately handled. It has been demonstrated that MRPL outperforms current methods in terms of CPU optimization, memory usage, and reduced information loss. Research reveals that the suggested strategy creates significant advantages for Big Data by enhancing privacy and protection. MRPL can considerably solve the privacy issues in Big Data.

Using open-source data to construct 20 metre resolution maps of children’s travel time to the nearest health facility

Physical access to health facilities is an important factor in determining treatment seeking behaviour and has implications for targets within the Sustainable Development Goals, including the right to health. The increased availability of high-resolution land cover and road data from satellite imagery offers opportunities for fine-grained estimations of physical access which can support delivery planning through the provision of more realistic estimates of travel times. The data presented here is of travel time to health facilities in Uganda, Zimbabwe, Tanzania, and Mozambique. Travel times have been calculated for different facility types in each country such as Dispensaries, Health Centres, Clinics and Hospitals. Cost allocation surfaces and travel times are provided for child walking speeds but can be altered easily to account for adult walking speeds and motorised transport. With a focus on Uganda, we describe the data and method and provide the travel maps, software and intermediate datasets for Uganda, Tanzania, Zimbabwe and Mozambique.

A New Approach for Reconstruction of IMFs of Decomposition and Ensemble Model for Forecasting Crude Oil Prices

Accurate forecasting for the crude oil price is important for government agencies, investors, and researchers. To cope with this issue, in this paper, a new paradigm is designed for the reconstruction of intrinsic mode functions (IMFs) of decomposition and ensemble models to reduce the complexity in computation and to enhance the forecasting accuracy. Decomposition and ensemble methodologies significantly enhance the forecasting accuracy under the framework of “divide and conquer” with the proposed reconstruction of IMFs method. The proposed approach used the autocorrelation at lag 1 of all IMFs for the reconstruction. The ensemble empirical mode decomposition (EEMD) technique is employed to decompose the data into different IMFs. Models that utilized the decomposed data relatively perform well, as compared to its application to the undecomposed data. However, sometimes, the decomposition may produce poor results due to the error accumulation at the end. Thus, in this study, the reconstruction of IMFs is proposed for minimizing the aforementioned error, thereby increasing the forecasting accuracy. The Brent and West Texas Intermediate (WTI) datasets (daily and weekly) are exploited to compare the forecasting performance of autoregressive integrated moving average (ARIMA) along with artificial neural network (ANN) models with the decomposed data. The results have proven that the new paradigm of reconstruction of IMFs through autocorrelation was a better and simple strategy that significantly improved the performance of single models including ARIMA and ANN. Hence, it is concluded that the proposed model takes less computational time and achieved higher forecasting accuracy with the reconstruction of IMFs as opposed to using all IMFs.

Cloud Based Privacy Preserving Dynamic Intermediate Datasets

Design an encryption of privacy preserving and scheduling of intermediate datasets in cloud. Implemenation of encryption is done as follows: Identification of intermediate datasets that needs to be encrypted. Based on frequent pattern mining the least frequent intermediate datasets are encrypted. Perform column level encryption to the sensitive information. Predicting the data based on inference analysis would not be possible. So that the data will be secure when compared to the existing system.

A Data and Task Co-Scheduling Algorithm for Scientific Cloud Workflows

Cloud computing has emerged as a promising computational infrastructure for cost-efficient workflow execution by provisioning on-demand resources in a pay-as-you-go manner. While scientific workflows require accessing community-wide resources, they usually need to be performed in collaborative cloud environments composed of multiple datacenters. Although such environments facilitate scientific collaboration, the movements of input and intermediate datasets across geographically distributed datacenters may cause intolerable latency that would hinder efficient execution of large-scale data-intensive scientific workflows. To address the problem, in this article we propose a novel multi-level K-cut graph partitioning algorithm to minimize the volume of data transfer across datacenters while satisfying load balancing and fixed data constraints. The algorithm first contracts the fixed input datasets in the same datacenter and their consuming tasks, and coarsens the contracted graph to a predefined scale in a level-by-level manner. Then, a K-cut algorithm is used to partition the resulted graph into K parts such that the cut size is minimized. After that, the partitioned graph is projected back to the original workflow graph, during which the load balancing constraint is maintained. We evaluate our algorithm using three real-world workflow applications and the results demonstrate that the proposed algorithm outperforms other state-of-the-art algorithms.

OLPSO_Iot:A Privacy Preservation Encrypted Data For Internet of Things (Iot) Data Security in Cloud Computation Environment

The Internet of Things connected devices will send data to cloud storage but cloud storage management carries their applications without any infrastructure investment by distributed computing. Therefore, manyindustries are doing their business in the cloud. For a while,the processing ofthe original data setand several intermediate data sets wasrendered by data-intensive applications. However, a challenging task is to support the privacy of the intermediate data set.Inourearlier research, optimal privacy preserving based data search in the cloud was presented using cuckoo search encryption algorithm toimprove the security.In this paper applied the orthogonal learning PSO(OLPSO) algorithm tohelp secure the IoT data in a cloud environment and improve the data transfer as well as decrease the data loss rate with efficient memory.

Production Operations Management System (POMS) for Fermilab Experiments

The Production Operations Management System (POMS) software allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs, and currently manages the majority of production computing of experiments at Fermilab. POMS provides a web service interface that enables automated job submission on distributed resources according to customers’ requests, as well as subsequent monitoring of those submissions as well as recovery of failed submissions, debugging and record keeping. POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub (for job submission), SAM (for data management), and Landscape (for monitoring), and in combination with them handles the creation of intermediate datasets of files for multistage campaigns. POMS visualizes these campaigns with an interactive GUI interface. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign. This paper will discuss the current usage, history, and some technical details of POMS, as well as planned future extensions, particularly interfacing with Rucio as well as SAM for file movement.

Structured Query Language Learning: Concept Map-Based Instruction Based on Cognitive Load Theory

Structured query language (SQL) is difficult to master because the execution process of SQL statements is invisible. When learning to construct an SQL query, learners must visualise the evolution process of the intermediate datasets of the SQL statement in working memory, which may burden learners' cognitive load and consequently jeopardise learning outcomes. This study describes the execution process of SQL statements by using concept maps to improve learners' understanding of SQL. An empirical experiment was conducted using two database courses, namely concept map-based and conventional instruction, to examine the relationship between concept maps and the understanding of SQL from a cognitive load theory perspective. The experimental results demonstrated the superiority of concept map-based instruction over conventional instruction because concept map-based instruction reduces extraneous load but increases germane load. Concept map construction facilitated learner engagement and promoted meaningful learning. Studying the instructors' concept maps helped learners follow the cognitive structures used by instructors to perform SQL queries, and enabled them to perceive the execution process of SQL queries relatively easily. These results potentially help educators understand the learning difficulties caused by the declarative nature of SQL and motivating researchers to resolve the inherent problem by considering learners' cognitive processes.

CERN Analysis Preservation and Reuse Framework: FAIR research data services for LHC experiments

In this paper we present the CERN Analysis Preservation service as a FAIR (Findable, Accessible, Interoperable and Reusable) research data preservation repository platform for LHC experiments. The CERN Analysis Preservation repository allows LHC collaborations to deposit and share the structured information about analyses as well as to capture the individual data assets associated to the analysis. We describe the typical data ingestion pipelines, through which an individual physicist can preserve and share their final n-tuples, ROOT macros, Jupyter notebooks, or even their full analysis workflow code and any intermediate datasets of interest for preservation within the restricted context of experimental collaboration. We discuss the importance of annotating the deposited content with high-level structured information about physics concepts in order to promote information discovery and knowledge sharing inside the collaboration. Finally, we describe techniques used to facilitate the reusability of preserved data assets by capturing and re-executing reproducible recipes and computational workflows using the REANA Reusable Analysis platform.

Embedding Analytics within the Curation of Scientific Workflows.

This paper reports on the ongoing activities and curation practices of the National Center for Biomolecular NMR Data Processing and Analysis1. Over the past several years, the Center has been developing and extending computational workflow management software for use by a community of biomolecular NMR spectroscopists. Previous work had been to refactor the workflow system to utilize the PREMIS framework for reporting retrospective provenance as well as for sharing workflows between scientists and to support data reuse. In this paper, we report on our recent efforts to embed analytics within the workflow execution and within provenance tracking. Important metrics for each of the intermediate datasets are included within the corresponding PREMIS intellectual object, which allows for both inspection of the operation of individual actors as well as visualization of the changes throughout a full processing workflow.These metrics can be viewed within the workflow management system or through standalone metadata widgets. Our approach is to support a hybrid approach of both automated, workflow execution as well as manual intervention and metadata management. In this combination, the workflow system and metadata widgets encourage the domain experts to be avid curators of the data which they create, fostering both computational reproducibility and scientific data reuse.

A tool for estimating antiretroviral medication coverage for HIV-infected women during pregnancy (PMTCT-ACT)

BackgroundIn the typical prevention of mother to child transmission (PMTCT) of HIV cascade of care discussion or analysis, the period of analysis begins at the first visit for antenatal care (ANC) for that pregnancy. This starting point is problematic for two reasons: (1) a large number of HIV-infected women are already on life-long antiretroviral therapy (ART) when presenting for ANC; and (2) women present to ANC at different gestational ages. The PMTCT ART Coverage Tool (PMTCT-ACT), which estimates the proportion of days covered (PDC) with ART, was developed to address each of these problems.MethodsPDC is a preferred method to measure adherence to chronic medications, such as ART. For evaluating the PMTCT cascade of care, as indicated by PDC with ART over various time periods, a “starting point” based on a specific day before delivery must be defined that applies to all women (treatment experienced or naïve at the first ANC visit at any gestational age). Using the example of 168 days prior to delivery (24 weeks), PMTCT-ACT measures PDC with ART during that period. PMTCT-ACT is provided as a STATA do-file. Using an example dataset for two women (ID1 is treatment experienced; ID2 is treatment naïve), the details of each major portion of the tool (Parts 1–5) are presented. PMTCT-ACT along with the intermediate datasets created during the analysis are provided as supplemental files.ConclusionsEvaluating the PMTCT cascade of care requires a standard definition of the follow-up period during pregnancy that applies to all HIV-infected pregnant women and a standard measure of adherence. PMTCT-ACT is a new tool that fits this purpose. PMTCT-ACT can also be easily adjusted to evaluate other ante- and post-natal periods (e.g., final 4 weeks, final 8 weeks, complete pregnancy period, initial 24 weeks postpartum, time periods consistent with infant HIV testing guidelines).

Cost-Effective Privacy Preserving of Intermediate Data using Group Search Optimization Algorithm

Cloud computing provides huge storage ability to clients, in order to convey their applications without any infrastructure investment. Alongside those applications, an extensive set of intermediate datasets will be created, and it is a challenging issue to protect the security of those intermediate datasets. Therefore, in this paper, we preserve only sensitive information which reduces the time and cost. For privacy preserving of the intermediate dataset, in this paper, we propose a combination of group search optimisation (GSO) and advanced encryption standard (AES). At first, the original dataset is split into an intermediate dataset and we select the corresponding node from the cloud for each intermediate dataset using GSO algorithm. After that, we separate the sensitive data using information gain. Finally, we apply the AES to encrypt the sensitive data. The performance of proposed methodology is in terms of encryption time and memory usage.

Graph partition–based data and task co‐scheduling of scientific workflow in geo‐distributed datacenters

SummaryMost large‐scale scientific workflows take place in multiple collaborative datacenters for access to community‐wide resources, while adhering to each datacenter's non‐uniform resource limits. However, moving both initial input datasets with predetermined locations and intermediate datasets needing placement decisions across geo‐distributed datacenters hinders efficient execution of large‐scale data‐intensive scientific workflows. Thus, scientific workflow's data and task co‐scheduling deal with situations such as pre‐placed initial input datasets, placement of intermediate datasets and each datacenter's non‐uniform computation and storage constraint, while minimizing the cross‐datacenter data transfer. Since this scheduling problem is known to be NP‐hard, here, we propose a novel approach, based on the multilevel graph coarsening and uncoarsening framework, together with a specialized hybrid genetic algorithm having distinctive graph partition driven features of repair and local improvement, for scheduling data‐intensive scientific workflows in geo‐distributed datacenters and optimizing the cross‐datacenter data transfer volume. Extensive simulations, based on four real‐world workflow traces, show that our algorithm significantly reduces the overall geo‐distributed data transfer and demonstrate its effectiveness.

Privacy preserving of intermediate dataset using hybridisation of oppositional gravitational search algorithm and elliptic curve cryptography

Distributed computing gives the gigantic capacity ability to the clients to send their applications without any infrastructure investment. Based on the application lot of intermediate dataset will be created. To protecting these intermediate dataset is a challenging task. Moreover, encrypting all dataset is a time and cost consuming. To overcome the problem, in this paper we proposed a privacy preserving of intermediate dataset using a combination of oppositional gravitational search algorithm and elliptic curve cryptography (OGSA + ECC). Initially, we split the dataset into a number of the intermediate datasets, then, we choose the node corresponding intermediate dataset from the cloud using an oppositional gravitational search algorithm (OGSA). After that, we choose the sensitive data from the dataset using the information gain measure to minimise the processing time and cost. Then, using the ECC algorithm the sensitive data is encrypted and in the cloud the secure data are stored. The experimentation is carried out in terms of encryption time and memory use.

Privacy preserving of intermediate dataset using hybridisation of oppositional gravitational search algorithm and elliptic curve cryptography

Distributed computing gives the gigantic capacity ability to the clients to send their applications without any infrastructure investment. Based on the application lot of intermediate dataset will be created. To protecting these intermediate dataset is a challenging task. Moreover, encrypting all dataset is a time and cost consuming. To overcome the problem, in this paper we proposed a privacy preserving of intermediate dataset using a combination of oppositional gravitational search algorithm and elliptic curve cryptography (OGSA + ECC). Initially, we split the dataset into a number of the intermediate datasets, then, we choose the node corresponding intermediate dataset from the cloud using an oppositional gravitational search algorithm (OGSA). After that, we choose the sensitive data from the dataset using the information gain measure to minimise the processing time and cost. Then, using the ECC algorithm the sensitive data is encrypted and in the cloud the secure data are stored. The experimentation is carried out in terms of encryption time and memory use.

Host-Based Intrusion Detection System with System Calls

In a contemporary data center, Linux applications often generate a large quantity of real-time system call traces, which are not suitable for traditional host-based intrusion detection systems deployed on every single host. Training data mining models with system calls on a single host that has static computing and storage capacity is time-consuming, and intermediate datasets are not capable of being efficiently handled. It is cumbersome for the maintenance and updating of host-based intrusion detection systems (HIDS) installed on every physical or virtual host, and comprehensive system call analysis can hardly be performed to detect complex and distributed attacks among multiple hosts. Considering these limitations of current system-call-based HIDS, in this article, we provide a review of the development of system-call-based HIDS and future research trends. Algorithms and techniques relevant to system-call-based HIDS are investigated, including feature extraction methods and various data mining algorithms. The HIDS dataset issues are discussed, including currently available datasets with system calls and approaches for researchers to generate new datasets. The application of system-call-based HIDS on current embedded systems is studied, and related works are investigated. Finally, future research trends are forecast regarding three aspects, namely, the reduction of the false-positive rate, the improvement of detection efficiency, and the enhancement of collaborative security.

Gastric Pathology Image Classification Using Stepwise Fine-Tuning for Deep Neural Networks.

Deep learning using convolutional neural networks (CNNs) is a distinguished tool for many image classification tasks. Due to its outstanding robustness and generalization, it is also expected to play a key role to facilitate advanced computer-aided diagnosis (CAD) for pathology images. However, the shortage of well-annotated pathology image data for training deep neural networks has become a major issue at present because of the high-cost annotation upon pathologist's professional observation. Faced with this problem, transfer learning techniques are generally used to reinforcing the capacity of deep neural networks. In order to further boost the performance of the state-of-the-art deep neural networks and alleviate insufficiency of well-annotated data, this paper presents a novel stepwise fine-tuning-based deep learning scheme for gastric pathology image classification and establishes a new type of target-correlative intermediate datasets. Our proposed scheme is deemed capable of making the deep neural network imitating the pathologist's perception manner and of acquiring pathology-related knowledge in advance, but with very limited extra cost in data annotation. The experiments are conducted with both well-annotated gastric pathology data and the proposed target-correlative intermediate data on several state-of-the-art deep neural networks. The results congruously demonstrate the feasibility and superiority of our proposed scheme for boosting the classification performance.