Intermediate Datasets Research Articles

Novel entropy-based approach for cost-effective privacy preservation of intermediate datasets in cloud

Cloud computing provides enormous storage capacity and huge computation power. Cloud enables users to deploy data-intensive applications without substructure investment. Intermediate data are generated under the cloud applications and stored again in cloud. An opponent may analyse multiple intermediate data sets to access the privacy sensitive information.The proposed technique to have good privacy and utility trade off, joint entropy with adaptive optimization process is used to maintain the privacy in cloud. Here optimal entropy value process using adaptive particle swarm optimization (APSO) process. After get the optimal entropy database difference model was processed, Entropy and database difference ratio is taken as the evaluation matrices and is tested using various datasets. The technique is also compared to the existing PSO optimization process on privacy preservation in cloud. An adaption feature of Particle swarm optimization show that APSO enhances privacy and preserving cost of intermediate data sets can be significantly saved.

Total pressure fluctuations and two-phase flow turbulence in self-aerated stepped chute flows

Current knowledge in high-velocity self-aerated flows continues to rely upon physical modelling. Herein a miniature total pressure probe was successfully used in both clear-water and air-water flow regions of high-velocity open channel flows on a steep stepped channel. The measurements were conducted in a large size facility (θ=45°, h=0.1m, W=0.985m) and they were complemented by detailed clear-water and air-water flow measurements using a Prandtl-Pitot tube and dual-tip phase-detection probe respectively in both developing and fully-developed flow regions for Reynolds numbers within 3.3×105 to 8.7×105. Upstream of the inception point of free-surface aeration, the clear-water developing flow was characterised by a developing turbulent boundary layer and an ideal-flow region above. The boundary layer flow presented large total pressure fluctuations and turbulence intensities, with distributions of turbulence intensity close to intermediate roughness flow data sets: i.e., intermediate between d-type and k-type. The total pressure measurements were validated in the highly-aerated turbulent shear region, since the total pressure predictions based upon simultaneously-measured void fraction and velocity data agreed well with experimental results recorded by the total pressure probe. The results demonstrated the suitability of miniature total pressure probe in both monophase and two-phase flows. Both interfacial and water phase turbulence intensities were recorded. Present findings indicated that the turbulence intensity in the water phase was smaller than the interfacial turbulence intensity.

Conditional Entropy with Swarm Optimization Approach for Privacy Preservation of Datasets in Cloud

Background/Objective: The primary intension is to provide utility trade off and good privacy for intermediate datasets in cloud. Methods: An efficient conditional entropy and database difference ratio is employed for the process. Utility is taken care with the employment of conditional entropy with the help of Swarm Optimization (PSO). Privacy handled by database difference ratio. Findings: Conditional entropy is found out between the first column and the original database and this is taken as the fitness function in Particle Swarm Optimization (PSO). Database difference is taken between the original database and convoluted database in the consequent module to yield right selection data and tuple information from the intermediate datasets in cloud. Applications/Improvements: Improved methods are required to provide privacy and utility for the right selection of datasets. This approach has better results by having higher entropy values and lower dataset difference ratio.

Improved CTT-SP Algorithm with Critical Path Method for Massive Data Storage in Scientific Workflow Systems

Scientific WorkFlows (SWFs) play significant roles in scientific research and engineering simulation, which are often data intensive and has complex data dependencies. The storage of massive intermediate datasets has great impacts on the performance and the quality of service of a SWF system, which has become a difficult and complex task. Through analyzing the cost transitive tournament shortest path (CTT-SP)-based algorithm proposed for the intermediate data storage in SWF systems, the disadvantage of the CTT-SP algorithm over its sensitivity to the main branches can be found. We proposed an improved CTT-SP algorithm based on the critical path method (CPM), aiming at reducing the sensitivity of the main branch and improving the performance of the algorithm. By adding the generation information of the datasets on the arrows, the data dependency graph of a SWF can be converted to an activity on arrow (AOA) net. Then, the critical path of the AOA net is used to be the main branch of the CTT-SP algorithm, which can reduce the impacts of the main branch and keep the quality of service of the SWF systems. Experiments are designed to test the impacts of the main branches and the performance of the improved CTT-SP algorithms. The results show the significant impacts of the main branches on the performance of the CTT-SP algorithm and the effectiveness of the CPM on improving the performance of the CTT-SP algorithm. Comparison results demonstrate the positiveness and effectiveness of the improved CTT-SP algorithm.

Effect of Water Activity on Reaction Kinetics and Intergranular Transport: Insights from the Ca(OH)2 + MgCO3→ CaCO3 + Mg(OH)2Reaction at 1·8 GPa

The kinetics of the irreversible reaction Ca(OH)2 + MgCO3 → CaCO3 + Mg(OH)2 were investigated at high pressures and temperatures relevant to metamorphic petrology, using both in situ synchrotron X-ray diffraction and post-mortem analysis of reaction rim growth on recovered samples. Reaction kinetics are found to strongly depend on water content; comparable bulk-reaction kinetics are obtained under water-saturated (excess water, c. 10 wt %) and under intermediate (0·1–1 wt % water) conditions when temperature is increased by c. 300 K. In contrast, similar reaction kinetics were observed at ∼673 K and 823 K between intermediate and dry experiments, respectively, where dry refers to a set of experiments with water activity below 1·0 (no free water), as buffered by the CaO–Ca(OH)2 assemblage. Given the activation energies at play, this gap—corresponding to the loss of no more than 1 wt % of water by the assemblage—leads to a difference of several orders of magnitude in reaction kinetics at a given temperature. Further analysis, at the microscopic scale, of the intermediate and dry condition samples, shows that intergranular transport of calcium controls the reaction progress. Grain boundary diffusivities could be retrieved from the classic treatment of reaction rim growth rate. In turn, once modeled, this rate was used to fit the bulk kinetic data derived from X-ray powder diffraction, offering an alternative means to derive calcium diffusivity data. Based on a comparison with effective grain boundary data for Ca and Mg from the literature, it is inferred that both dry and intermediate datasets are consistent with a water-saturated intergranular medium with different levels of connectivity. The very high diffusivity of Ca in the CaCO3 + Mg(OH)2 rims, in comparison with that of Mg in enstatite rims found by earlier workers, emphasizes the prominent role of the interactions between diffusing species and mineral surfaces in diffusion kinetics. Furthermore, we show that the addition of water is likely to change the relative diffusivity of Mg and Ca in carbonate aggregates. From a qualitative point of view, we confirm, in a carbonate-bearing system, that small water content variations within the 0–1 wt % range have tremendous effects on both intergranular transport mechanisms and kinetics. We also propose that the water content dependent diffusivity of major species (Mg, Ca) in low-porosity metamorphic rocks is strongly dependent on the interaction between diffusing species and mineral surfaces. This parameter, which will vary from one rock-type to another, needs also to considered when extrapolating (P, T, t, xH2O) laboratory diffusion data to metamorphic processes.

Cost-Effective Privacy Preserving Of Intermediate Data Sets in Cloud by Privacy Leakage Upper Bound Constraint-Based Approach

Cost-Effective Privacy Preserving Of Intermediate Data Sets in Cloud by Privacy Leakage Upper Bound Constraint-Based Approach

A new energy-aware task scheduling method for data-intensive applications in the cloud

Maximizing energy efficiency while ensuring the user׳s Service-Level Agreement (SLA) is very important for the purpose of environmental protection and profit maximization for the cloud service providers. In this paper, an energy and deadline aware task scheduling method for data-intensive applications is proposed. In this method, first, the datasets and tasks are modeled as a binary tree by a data correlation clustering algorithm, in which both the data correlations generated from the initial datasets and that from the intermediate datasets have been considered. Hence, the amount of global data transmission can be reduced greatly, which are beneficial to the reduction of SLA violation rate. Second, a “Tree-to-Tree” task scheduling approach based on the calculation of Task Requirement Degree (TRD) is proposed, which can improve energy efficiency of the whole cloud system by reducing the number of active machines, decreasing the global time consumption on data transmission, and optimizing the utilization of its computing resources and network bandwidth. Experiment results show that the power consumption of the cloud system can be reduced efficiently while maintaining a low-level SLA violation rate.

A Privacy Leakage Upper Bound Constraint-Based Approach for Cost-Effective Privacy Preserving of Intermediate Data Sets in Cloud

Cloud computing provides massive computation power and storage capacity which enable users to deploy computation and data-intensive applications without infrastructure investment. Along the processing of such applications, a large volume of intermediate data sets will be generated, and often stored to save the cost of recomputing them. However, preserving the privacy of intermediate data sets becomes a challenging problem because adversaries may recover privacy-sensitive information by analyzing multiple intermediate data sets. Encrypting ALL data sets in cloud is widely adopted in existing approaches to address this challenge. But we argue that encrypting all intermediate data sets are neither efficient nor cost-effective because it is very time consuming and costly for data-intensive applications to en/decrypt data sets frequently while performing any operation on them. In this paper, we propose a novel upper bound privacy leakage constraint-based approach to identify which intermediate data sets need to be encrypted and which do not, so that privacy-preserving cost can be saved while the privacy requirements of data holders can still be satisfied. Evaluation results demonstrate that the privacy-preserving cost of intermediate data sets can be significantly reduced with our approach over existing ones where all data sets are encrypted.

A semantic‐aware data generator for ETL workflows

SummaryExtract, transform, and load (ETL) processes organized as workflows play an important role in the future data integration for cloud services. ETL designers/administrators need testing data set that is aware of semantics of ETL workflow workloads to evaluate their developed ETL systems. Populating testing ETL systems with meaningful workload data is a difficult task. In this paper, we propose a semantic‐aware data generator for ETL workflows. With given ETL workflow models and workload characterizations, the generator is able to generate synthetic data that capture the semantics of ETL activities. This is carried out by a three‐staged approach. First, we derive expected cardinalities of all the source, intermediate, and target data sets involved in the ETL workflow model with some user‐specified cardinality requirements. Then, with the concept of symbolic test, symbolic data instead of concrete data involved in ETL activities are generated, and semantics of the ETL workflow models are transformed to various constraints over these symbols. At last, concrete data are derived on the basis of resolving constraints. Our generator may facilitate ETL workload test case generation for ETL toolkit performance and function evaluations as well as ETL workflow solution benchmarking. Copyright © 2013 John Wiley & Sons, Ltd.

On-demand minimum cost benchmarking for intermediate dataset storage in scientific cloud workflow systems

Many scientific workflows are data intensive: large volumes of intermediate datasets are generated during their execution. Some valuable intermediate datasets need to be stored for sharing or reuse. Traditionally, they are selectively stored according to the system storage capacity, determined manually. As doing science on clouds has become popular nowadays, more intermediate datasets in scientific cloud workflows can be stored by different storage strategies based on a pay-as-you-go model. In this paper, we build an intermediate data dependency graph (IDG) from the data provenances in scientific workflows. With the IDG, deleted intermediate datasets can be regenerated, and as such we develop a novel algorithm that can find a minimum cost storage strategy for the intermediate datasets in scientific cloud workflow systems. The strategy achieves the best trade-off of computation cost and storage cost by automatically storing the most appropriate intermediate datasets in the cloud storage. This strategy can be utilised on demand as a minimum cost benchmark for all other intermediate dataset storage strategies in the cloud. We utilise Amazon clouds’ cost model and apply the algorithm to general random as well as specific astrophysics pulsar searching scientific workflows for evaluation. The results show that benchmarking effectively demonstrates the cost effectiveness over other representative storage strategies.

A data dependency based strategy for intermediate data storage in scientific cloud workflow systems

SUMMARYMany scientific workflows are data intensive where large volumes of intermediate data are generated during their execution. Some valuable intermediate data need to be stored for sharing or reuse. Traditionally, they are selectively stored according to the system storage capacity, determined manually. As doing science in the cloud has become popular nowadays, more intermediate data can be stored in scientific cloud workflows based on a pay‐for‐use model. In this paper, we build an intermediate data dependency graph (IDG) from the data provenance in scientific workflows. With the IDG, deleted intermediate data can be regenerated, and as such we develop a novel intermediate data storage strategy that can reduce the cost of scientific cloud workflow systems by automatically storing appropriate intermediate data sets with one cloud service provider. The strategy has significant research merits, i.e. it achieves a cost‐effective trade‐off of computation cost and storage cost and is not strongly impacted by the forecasting inaccuracy of data sets' usages. Meanwhile, the strategy also takes the users' tolerance of data accessing delay into consideration. We utilize Amazon's cost model and apply the strategy to general random as well as specific astrophysics pulsar searching scientific workflows for evaluation. The results show that our strategy can reduce the overall cost of scientific cloud workflow execution significantly. Copyright © 2010 John Wiley & Sons, Ltd.

Analysis of protein complexes through model‐based biclustering of label‐free quantitative AP‐MS data

Affinity purification followed by mass spectrometry (AP-MS) has become a common approach for identifying protein–protein interactions (PPIs) and complexes. However, data analysis and visualization often rely on generic approaches that do not take advantage of the quantitative nature of AP-MS. We present a novel computational method, nested clustering, for biclustering of label-free quantitative AP-MS data. Our approach forms bait clusters based on the similarity of quantitative interaction profiles and identifies submatrices of prey proteins showing consistent quantitative association within bait clusters. In doing so, nested clustering effectively addresses the problem of overrepresentation of interactions involving baits proteins as compared with proteins only identified as preys. The method does not require specification of the number of bait clusters, which is an advantage against existing model-based clustering methods. We illustrate the performance of the algorithm using two published intermediate scale human PPI data sets, which are representative of the AP-MS data generated from mammalian cells. We also discuss general challenges of analyzing and interpreting clustering results in the context of AP-MS data.

High‐resolution fields of global runoff combining observed river discharge and simulated water balances

This paper demonstrates the potential of combining observed river discharge information with climate‐driven water balance model (WBM) outputs to develop composite runoff fields. Such combined runoff fields simultaneously reflect the numerical accuracy of the discharge measurements and preserve the spatial and temporal distribution of simulated runoff. Selected gauging stations from the World Meteorological Organization Global Runoff Data Centre (GRDC) data archive were geographically coregistered to a gridded simulated topological network at 30′ (longitude × latitude) spatial resolution (STN–30p). Interstation regions between gauging stations along the STN–30p network were identified, and annual interstation runoff was calculated. The annual interstation runoff was compared with outputs from WBM calculations, which were performed using long‐term mean monthly climate forcings (air temperature and precipitation). The simulated runoff for each cell was multiplied by the ratio of observed to simulated runoff of the corresponding interstation region from the GRDC data set to create spatially distributed runoff fields at 30′ resolution. The resulting composite runoff fields (UNH/GRDC Composite Runoff Fields V1.0) are released to the scientific community along with intermediate data sets, such as station attributes and long‐term monthly regimes of the selected gauging stations, the simulated topological network (STN–30p), STN–30p derived attributes for the selected stations, and gridded fields of the interstation regions along STN–30p. These data sets represent high‐resolution fields that are of value to a broad range of water‐related research, including terrestrial modeling, climate‐atmosphere interactions, and global water resource assessments.

ESTIMATION OF SPATIAL VARIABILITY IN PEARL MILLET GROWTH WITH NON-DESTRUCTIVE METHODS

Growth variability in pearl millet (Pennisetum glaucum) over short distances is a severe constraint on the interpretation of agricultural experiments in the West African Sahel. The purpose of this study, therefore, was to compare different non-destructive methods to estimate, spatially, millet growth and final yields. Aerial photography, georeferenced radiometric measurements and a chlorophyll meter were tested during three rainy seasons in a nitrogen rate×density×genotype experiment in western Niger. For the radiometric measurements, normalized difference vegetation indices (NDVI) obtained and calibrated for individual millet hills spaced 1.5 m apart were aggregated for the entire experiment with 6000 samples per hectare. A simple calibration procedure was used to correct for variation in soil background reflectance and incident light. For NDVI measurements of individual planting hills, the correlation between plant total dry matter (TDM), leaf weight, leaf area and NDVI was high (r2 = 0.89–0.91) and regression parameters were genotype-specific. Aggregated georeferenced NDVI measurements at the plot level correlated with grain and TDM at harvest (r2 = 0.40–0.87). The analysis of true-colour and infrared aerial photographs permitted the monitoring of millet growth and the quantitative evaluation of treatment responses throughout the growing season. The infrared images were the most efficient in the detection of vegetation followed by the normalized green band of true-colour images. The red band was the least effective because of the influence of soil albedo and image vignetting. Although chlorophyll meter measurements reflected relative differences in plant nitrogen status between treatments, their interpretation required destructive sampling and proved unsuitable to predict millet yields. The results demonstrate the potential of georeferenced radiometric data and aerial photographs to improve soil sampling strategies, sequential plant growth monitoring and the statistical design and analysis of experiments. By providing intermediate data sets, the tested tools can also help in the upscaling of ground truth to satellite data in yield prediction studies.

Observation of hysteresis between solar activity indicators andp-mode frequency shifts for solar cycle 22

Using intermediate degreep-mode frequency data sets for solar cycle 22, we find that the frequency shifts and magnetic activity indicators show a “hysteresis” phenomenon. It is observed that the magnetic indices follow different paths for the ascending and descending phases of the solar cycle while for radiative indices, the separation between the paths are well within the error limits.

Observation of Hysteresis between Solar Activity Indicators and p-mode Frequency Shifts for Solar Cycle 22

AbstractUsing intermediate degreep-mode frequency data sets for solar cycle 22, we find that the frequency shifts and magnetic activity indicators show a “hysteresis” phenomenon. It is observed that the magnetic indices follow different paths for the ascending and descending phases of the solar cycle while for radiative indices, the separation between the paths are well within the error limits.

Constraints on Primordial Non‐Gaussianity from the High‐Redshift Cluster MS 1054−03

The implications of the massive, X-ray selected cluster of galaxies MS 1054-03 at z = 0.83 are discussed in light of the hypothesis that the primordial density fluctuations may be non-Gaussian. We generalize the Press-Schechter (PS) formalism to the non-Gaussian case and calculate the likelihood that a cluster as massive as MS 1054 would have been found in the Einstein Medium Sensitvity Survey (EMSS). A flat universe (ΩM + ΩΛ = 1) is assumed, and the mass fluctuation amplitude is normalized to the present-day cluster abundance. The probability of finding an MS 1054-like cluster then depends only on ΩM and the extent of primordial non-Gaussianity. We quantify the latter by adopting a specific functional form for the probability distribution function (PDF), denoted ψλ, which tends to Gaussianity for λ 1 but is significantly non-Gaussian for λ 10, and show how λ is related to the more familiar statistic T, the probability of ≥3 σ fluctuations for a given PDF relative to a Gaussian. Special attention is given to a careful calculation of the virial mass of MS 1054 from the available X-ray temperature, galaxy velocity, and weak-lensing data. We find that Gaussian initial density fluctuations are consistent with the data on MS 1054 only if ΩM 0.2. For ΩM ≥ 0.25 a significant degree of non-Gaussianity is required, unless the mass of MS 1054 has been substantially overestimated by X-ray and weak-lensing data. The required amount of non-Gaussianity is a rapidly increasing function of ΩM for 0.25 ≤ ΩM ≤ 0.45, with λ ≤ 1 (T 7) at the upper end of this range. For a fiducial ΩM = 0.3, ΩΛ = 0.7 universe, favored by several lines of evidence we obtain an upper limit λ ≤ 10, corresponding to a T ≥ 3. This finding is consistent with the conclusions of Koyama, Soda, & Taruya, who applied the generalized PS formalism to low (z 0.1) and intermediate (z 0.6) redshift cluster data sets.

A comparative study of in-situ and remote intermediate layer measurements against wind model predictions of vertical ion drift

Abstract Measurements from a sequence of rockets launched from Wallops Island (37.95°N, 75.47°W) on a single night in 1968 still provide the only in-situ data on the midlatitude intermediate layer phenomenon. TMA vapor trail observations made during this sequence of rocket flights were the first to experimentally examine the wind shear theory for the formation and slow downward motion of the layers. Using these rocket data from solar maximum, as well as several intermediate layer data sets from the Arecibo Observatory (18.35°N, 66.75°W) at solar minimum, we compare the location and motion of the layer observations to the predictions of two widely used thermospheric wind models: the HWM-93 thermospheric wind model and the TIEGCM. HWM-93 is an empirical model derived largely from measurements that had limited capabilities for determining night-time winds at lower thermospheric altitudes, and TIEGCM is a self consistent first principles model. Like an earlier study by Szuszczewicz et al., 1995 , we find that some of our intermediate layer data agree quite well with the location of a convergent null in the meridional wind predicted by TIEGCM, but the correlation between layer altitude and the null in the vertical ion drift is in general poor. This finding contradicts theoretical expectations based on the two fluid plasma equations. We also demonstrate that uncertainties in the ion neutral collision frequency do not improve the agreement between the model vertical drifts and the observed layer behavior. Furthermore, we find some evidence that the HWM-93 model may overemphasize diurnal tidal effects in the winter, post-midnight period. The discrepancies found in this study demonstrate that our present understanding of midlatitude lower thermospheric wind systems is inadequate for predicting intermediate layer dynamics in the post-sunset period.