Out-of-core Research Articles

The multiplier effects of the Finnish sugar sector

Background information on the economic role of industries is crucial to policy and decision makers due to the involvement of public financing. At the provincial and municipal level, the economic role of industries is crucial for employment and tax revenues that sustain the livelihood of the provinces and municipalities in Finland. This study employed an input-output model (IO model) to answer the question on the linkages between the sugar sector with the different industries in the economy and how much production, value-added and employment would be lost if the sugar sector in Finland would disappear totally. The IO-model would produce the direct, indirect and induced economic effects usually referred to as the multiplier effects of the sugar sector. The gross output of sugar beet production is approximated at about € 33million, and the value-added sugar beet production is estimated at € 17.2 million. The output of the sugar factories is approximated to be € 165.7 million, and the value-added is estimated at € 39.6 million. Even though the effects of primary sugar beet production and sugar factory production including multiplier effects seem rather small compared to the total effects at the regional level, the actual effects are felt at the municipalities level in the form of decreasing tax revenues and unemployment if the sugar sector in Finland disappears and is not substituted for by other forms of production and industry. The output effects of the sugar chain production including sugar beet production, sugar factories and sugar utilizing industries totals € 14 371.0 million when indirect multiplier effects € 8 258.3 million in addition to direct effects € 6 112.7 million are taken into account. The value-added effects of the sugar chain production totals € 4 451.5 million when indirect value-added effects € 2 857.7 million in addition to direct value-added effects € 1 593.8 are taken into account. The employment effects of the sugar chain production totals 58 733 persons as the indirect employment effects 37 979 persons in addition to direct effects 20 754 persons are taken into account. As far as employment effects are concerned, the number of employees does not refer to full-time employees, but included also all those who are involved at any stages of the sugar chain production. The indirect effects include multiplier effects due to both household spending and intermediate input purchases. Hence, if the sugar utilizing industries are included in the production chain, the multiplier effects are considerable. There is data to prove that without domestic production, the sugar price is higher for Ireland compared to the other EU member states due to imperfections in the EU sugar market after the reform of the EU sugar regime in 2006. The sugar price for consumers in Ireland is the highest compared to the United Kingdom (UK), Finland and Germany. The UK and Finland are not self-sufficient in producing sugar for the domestic market, but Germany has been always producing over its self-sufficiency limit. Therefore, the consumers in Germany enjoy the lowest price for sugar compared to Ireland, Finland and the UK. Due to the existing domestic sugar production in Finland and the UK, the price of sugar is lower in these countries compared to Ireland.

Suffix Array Construction in External Memory Using D-Critical Substrings

We present a new suffix array construction algorithm that aims to build, in external memory, the suffix array for an input string of length n measured in the magnitude of tens of Giga characters over a constant or integer alphabet. The core of this algorithm is adapted from the framework of the original internal memory SA-DS algorithm that samples fixed-size d-critical substrings. This new external-memory algorithm, called EM-SA-DS, uses novel cache data structures to construct a suffix array in a sequential scanning manner with good data spatial locality: data is read from or written to disk sequentially. On the assumed external-memory model with RAM capacity Ω (( nB ) 0.5 ), disk capacity O ( n ), and size of each I/O block B , all measured in log n -bit words, the I/O complexity of EM-SA-DS is O ( n / B ). This work provides a general cache-based solution that could be further exploited to develop external-memory solutions for other suffix-array-related problems, for example, computing the longest-common-prefix array, using a modern personal computer with a typical memory configuration of 4GB RAM and a single disk.

An efficient algorithm for maximizing range sum queries in a road network.

Given a set of positive-weighted points and a query rectangle r (specified by a client) of given extents, the goal of a maximizing range sum (MaxRS) query is to find the optimal location of r such that the total weights of all the points covered by r are maximized. All existing methods for processing MaxRS queries assume the Euclidean distance metric. In many location-based applications, however, the motion of a client may be constrained by an underlying (spatial) road network; that is, the client cannot move freely in space. This paper addresses the problem of processing MaxRS queries in a road network. We propose the external-memory algorithm that is suited for a large road network database. In addition, in contrast to the existing methods, which retrieve only one optimal location, our proposed algorithm retrieves all the possible optimal locations. Through simulations, we evaluate the performance of the proposed algorithm.

Session details: Fast, cheap and out-of-core rendering

No abstract available.

A phenomenological model of Io’s UV aurora based on HST/STIS observations

We have carried out a comprehensive analysis of a large set of spatially resolved observations of Io’s OI 1304Å, OI] 1356Å, SI 1479Å and SI] 1900Å aurora taken by the Space Telescope Imaging Spectrograph (STIS) of the Hubble Space Telescope (HST) between 1997 and 2001. We find that the variability of the observed morphologies can be solely explained by the changes of the plasma and magnetic field environment of the Io torus and by the viewing perspective. The variations in brightness are strongly correlated with the periodic variations of the ambient electron density. Based on these findings we develop a phenomenological model for the spatial distribution of the oxygen and sulfur emissions in Io’s vicinity. Taking into account Io’s position with respect to the plasma torus, the orientation of Jupiter’s magnetic field and the viewing perspective of the observation, the model calculates the auroral morphology and brightness. By fitting the model parameters to the observations we find that the model is able to reproduce the main features in all images obtained over a period of five years with one parameter set for each emission multiplet. The spatial distribution of the OI] 1356Å, OI 1304Å, SI 1479Å, and SI] 1900Å multiplets are shown to be very similar. In contrast to previous investigations, the model results reveal that the majority of the radiation from the bound atmosphere is emitted within 100km above the surface. The equatorial aurora spots extend far into the wake region explaining observed features in the downstream region. The relative brightness of two the equatorial spots is best explained by our model if the emission on the day-side flank of Io is higher by a factor of ∼1.5 with respect to the nightside flank. The measured brightness during an observation in eclipse is significantly lower than expected from the fitted model. The day–night asymmetry and the brightness decrease in eclipse support the idea of a wide collapse of Io’s atmosphere in shadow. Since our phenomenological aurora model is able to reproduce the main features of the observed morphology by taking into account the variations of the magnetospheric parameters, it can be applied to predict the emission for future UV aurora observations for a given time and position of the observer.

Eliminating Indirect Energy Subsidies in Ukraine: Estimation of Environmental and Socioeconomic Effects Using Input–Output Modeling

Energy subsidies are causing adverse effects on the economy and environment stimulating inefficient resource allocation and excessive energy consumption. Therefore, multiple positive effects are likely to be obtained if subsidies are reduced or removed completely. This study estimates indirect end-user energy subsidies in gas and electricity sectors of Ukraine and assesses short-term environmental and socioeconomic effects of a hypothetical elimination of these subsidies. Consumer subsidies are quantified utilizing the price-gap approach that compares end-user prices and reference prices that would predominate in competitive markets where no subsidies are provided. Input–output analysis is undertaken to investigate impacts in the short-term. First, a price model is utilized to estimate price changes resulting from subsidy elimination and then a demand-driven IO model is adopted for estimation of associated changes of environmental and social variables. As a result of elimination of energy subsidies, total energy consumption and GHG emissions would decline by about 2.5 % and 3.6 %, respectively. In addition, distributional effects of a subsidy reform are analyzed utilizing data on consumption patterns of different income groups. JEL Classification: D57, P22, P28, Q41.

IMPROVED POINTER MACHINE AND I/O LOWER BOUNDS FOR SIMPLEX RANGE REPORTING AND RELATED PROBLEMS

We investigate one of the fundamental areas in computational geometry: lower bounds for range reporting problems in the pointer machine and the external memory models. We develop new techniques that lead to new and improved lower bounds for simplex range reporting as well as some other geometric problems. Simplex range reporting is the problem of storing n points in the d-dimensional space in a data structure such that the k points that lie inside a query simplex can be found efficiently. This is one of the fundamental and extensively studied problems in computational geometry. Currently, the best data structures for the problem achieve Q(n) + O(k) query time using [Formula: see text] space in which the [Formula: see text] notation either hides a polylogarithmic or an nε factor for any constant ε > 0, (depending on the data structure and Q(n)). The best lower bound on this problem is due to Chazelle and Rosenberg who showed any pointer machine data structure that can answer queries in O(nγ + k) time must use Ω(nd-ε-dγ) space. Observe that this bound is a polynomial factor away from the best known data structures. In this article, we improve the space lower bound to [Formula: see text]. Not only this bridges the gap from polynomial to sub-polynomial, it also offers a smooth trade-off curve. For instance, for polylogarithmic values of Q(n), our space lower bound almost equals Ω((n/Q(n))d); the latter is generally believed to be the “right” bound. By a simple geometric transformation, we also improve the best lower bounds for the halfspace range reporting problem. Furthermore, we study the external memory model and offer a new simple framework for proving lower bounds in this model. We show that answering simplex range reporting queries with Q(n)+O(k/B) I/Os requires [Formula: see text]) space or [Formula: see text] blocks, in which B is the block size.

An analysis of input-output relations in interaction with smart tangible objects

This article focuses on the conceptual relation between the user's input and a system's output in interaction with smart tangible objects. Understanding this input-output relation (IO relation) is a prerequisite for the design of meaningful interaction. A meaningful IO relation allows the user to know what to do with a system to achieve a certain goal and to evaluate the outcome. The work discussed in this article followed a design research process in which four concepts were developed and prototyped. An evaluation was performed using these prototypes to investigate the effect of highly different IO relations on the user's understanding of the interaction. The evaluation revealed two types of IO relations differing in functionality and the number of mappings between the user and system actions. These two types of relations are described by two IO models that provide an overview of these mappings. Furthermore, they illustrate the role of the user and the influence of the system in the process of understanding the interaction. The analysis of the two types of IO models illustrates the value of understanding IO relations for the design of smart tangible objects.

Linear-Space Data Structures for Range Mode Query in Arrays

A mode of a multiset S is an element a∈S of maximum multiplicity; that is, a occurs at least as frequently as any other element in S. Given an array A[1:n] of n elements, we consider a basic problem: constructing a static data structure that efficiently answers range mode queries on A. Each query consists of an input pair of indices (i,j) for which a mode of A[i:j] must be returned. The best previous data structure with linear space, by Krizanc, Morin, and Smid (Proceedings of the International Symposium on Algorithms and Computation (ISAAC), pp. 517–526, 2003), requires \(\varTheta (\sqrt{n}\log\log n)\) query time in the worst case. We improve their result and present an O(n)-space data structure that supports range mode queries in \(O(\sqrt{n/\log n})\) worst-case time. In the external memory model, we give a linear-space data structure that requires \(O(\sqrt{n/B})\) I/Os per query, where B denotes the block size. Furthermore, we present strong evidence that a query time significantly below \(\sqrt{n}\) cannot be achieved by purely combinatorial techniques; we show that boolean matrix multiplication of two \(\sqrt{n} \times \sqrt{n}\) matrices reduces to n range mode queries in an array of size O(n).Additionally, we give linear-space data structures for the dynamic problem (queries and updates in near O(n 3/4) time), for orthogonal range mode in d dimensions (queries in near O(n 1−1/2d) time) and for half-space range mode in d dimensions (queries in \(O(n^{1-1/d^{2}})\) time). Finally, we complement our dynamic data structure with a reduction from the multiphase problem, again supporting that we cannot hope for much more efficient data structures.

The Limits of Buffering: A Tight Lower Bound for Dynamic Membership in the External Memory Model

We study the dynamic membership (or dynamic dictionary) problem, which is one of the most fundamental problems in data structures. We study the problem in the external memory model with cell size $b$ bits and cache size $m$ bits. We prove that if the amortized cost of updates is at most $0.9$ (or any other constant $<1$), then the query cost must be $\Omega(\log_{b \log n} (n/m))$, where $n$ is the number of elements in the dictionary. In contrast, when the update time is allowed to be $1+o(1)$, then a bit vector or hash table gives query time $O(1)$. Thus, this is a threshold phenomenon for data structures. This lower bound answers a folklore conjecture of the external memory community. Since almost any data structure task can solve membership, our lower bound implies a dichotomy between two alternatives: (i) make the amortized update time at least $1$ (so the data structure does not buffer, and we lose one of the main potential advantages of the cache), or (ii) make the query time at least roughly logarithmic in $n$. Our result holds even when the updates and queries are chosen uniformly at random and there are no deletions; it holds for randomized data structures, holds when the universe size is $O(n)$, and does not make any restrictive assumptions such as indivisibility. All of the lower bounds we prove hold regardless of the space consumption of the data structure, while the upper bounds need only linear space. The lower bound has some striking implications for external memory data structures. It shows that the query complexities of many problems, such as one-dimensional range counting, predecessor, rank-select, and many others, are all the same in the regime where the amortized update time is less than $1$, as long as the cell size is large enough ($b=\mathrm{polylog}(n)$ suffices). The proof of our lower bound is based on a new combinatorial lemma called the lemma of surprising intersections (LOSI), which allows us to use a proof methodology where we first analyze the intersection structure of the positive queries by using encoding arguments, and then use statistical arguments to deduce properties of the intersection structure of all queries, even the negative ones. In most other data structure arguments that we know, it is difficult to argue anything about the negative queries. Therefore we believe that the LOSI and this proof methodology might find future uses for other problems.

Spatial distribution of volcanoes on Io: Implications for tidal heating and magma ascent

Extreme volcanism on Io results from tidal heating, but its tidal dissipation mechanisms and magma ascent processes are poorly constrained. Here we analyze the distribution of volcanic hotspots and paterae identified within the first 1:15,000,000-scale global geologic map of Io to characterize their patterns of spatial organization. Ionian hotspots correspond to the locations of positive thermal anomalies that have been detected since 1979, whereas paterae are caldera-like volcano-tectonic depressions that record locations of volcanic activity over a longer period of geologic time (up to ∼1 million years). Some (∼20%) of patera floor units are associated with active hotspots, but the majority appeared to be extinct or dormant at the time of observation. Volcano distributions are useful for testing interior models of Io because the relative strength of tidal heating in the asthenosphere and deep-mantle greatly affect expected patterns of surface heat flux. We examine the spatial distribution of volcanic centers using nearest neighbor (NN) statistics and distance-based clustering. Nearest neighbor analysis reveals that hotspots (i.e., sites of active volcanism) are globally random, but closer to the equator, they are uniform (i.e., more widely spaced than a random model would predict). This suggests that magma scavenging around active volcanic systems in the near-equatorial region may drive hotspots apart, whereas vigorous mantle convection and/or deep-mantle heating may reduce surface heat flux variations and promote spatial randomness on a global scale. In contrast to the hotspots, NN patera floor units tend to be clustered, which implies that multiple eruptive units tend to form in association with most volcanic systems. Generalized paterae, which represent volcanic systems, tend to be uniformly distributed, except in the northern regions, where their distribution is random. This implies that most volcanic systems interact with one another and repel, except at high northern latitudes, where they appear to form independently. Distance-based clustering results support a dominant role for asthenospheric heating within Io, but show a 30–60° eastward offset in volcano concentrations from predicted locations of maximum surface heat flux along the tidal axis. This offset may imply faster than synchronous rotation, a role for lateral advection of magma within Io's interior prior to its eruption, state of stress controls on the locations of magma ascent, and/or a missing component in existing tidal dissipation models, such as the effects of fluid tides generated within a globally extensive layer of interconnected partial melt.

A structural analysis of the A5/1 state transition graph

We describe efficient algorithms to analyze the cycle structure of the graph induced by the state transition function of the A5/1 stream cipher used in GSM mobile phones and report on the results of the implementation. The analysis is performed in five steps utilizing HPC clusters, GPGPU and external memory computation. A great reduction of this huge state transition graph of 2^64 nodes is achieved by focusing on special nodes in the first step and removing leaf nodes that can be detected with limited effort in the second step. This step does not break the overall structure of the graph and keeps at least one node on every cycle. In the third step the nodes of the reduced graph are connected by weighted edges. Since the number of nodes is still huge an efficient bitslice approach is presented that is implemented with NVIDIA's CUDA framework and executed on several GPUs concurrently. An external memory algorithm based on the STXXL library and its parallel pipelining feature further reduces the graph in the fourth step. The result is a graph containing only cycles that can be further analyzed in internal memory to count the number and size of the cycles. This full analysis which previously would take months can now be completed within a few days and allows to present structural results for the full graph for the first time. The structure of the A5/1 graph deviates notably from the theoretical results for random mappings.

Computational geometry in the parallel external memory model

Continued advances in VLSI scaling combined with unsustainable power consumption of frequency scaling resulted in parallel processors having become mainstream as demonstrated by modern multicores. Current off-the-shelf processors already contain up to 16 cores and the prototypes boast up to 48 cores on a single chip.

A scalable algorithm for maximizing range sum in spatial databases

This paper investigates the MaxRS problem in spatial databases. Given a set O of weighted points and a rectangular region r of a given size, the goal of the MaxRS problem is to find a location of r such that the sum of the weights of all the points covered by r is maximized. This problem is useful in many location-based applications such as finding the best place for a new franchise store with a limited delivery range and finding the most attractive place for a tourist with a limited reachable range. However, the problem has been studied mainly in theory, particularly, in computational geometry. The existing algorithms from the computational geometry community are in-memory algorithms which do not guarantee the scalability. In this paper, we propose a scalable external-memory algorithm ( ExactMaxRS ) for the MaxRS problem, which is optimal in terms of the I/O complexity. Furthermore, we propose an approximation algorithm ( ApproxMaxCRS ) for the MaxCRS problem that is a circle version of the MaxRS problem. We prove the correctness and optimality of the ExactMaxRS algorithm along with the approximation bound of the ApproxMaxCRS algorithm. From extensive experimental results, we show that the ExactMaxRS algorithm is two orders of magnitude faster than methods adapted from existing algorithms, and the approximation bound in practice is much better than the theoretical bound of the ApproxMaxCRS algorithm.

Hybrid Parallelism of Multifrontal Linear Solution Algorithm with Out Of Core Capability for Finite Element Analysis

Hybrid Parallelism of Multifrontal Linear Solution Algorithm with Out Of Core Capability for Finite Element Analysis

Proportional Allocation of Resources on Shared Ring Buffer for Virtualization

In virtualization, without proper scheduling and resource management, a load surge in one VM may unacceptably degrade the performance of another. Key requirement of IO performance virtualization is performance isolation. The current state of performance isolation for virtualization is more rudimentary. This paper presents a resources allocation framework, based on abstracted Xen IO model, that provides both throughput and fairness guarantees for network and storage device via 2-level resource control. The high-level tier makes guest domains perceive the states of the resource through resource control on sharing ring buffer. The high-level tier uses public/private token-bucket to support work-conserving and fairly distributes any spare bandwidth to the different VMs. The low-level tier is intended to meet the fairness guarantee, and computes token quantity value of each bucket by feedback-driven scheduler. The authors analytically and experimentally demonstrated that their framework is work-conserving, and achieves fairness, and high adaptability.

Finding maximal cliques in massive networks

Maximal clique enumeration is a fundamental problem in graph theory and has important applications in many areas such as social network analysis and bioinformatics. The problem is extensively studied; however, the best existing algorithms require memory space linear in the size of the input graph. This has become a serious concern in view of the massive volume of today's fast-growing networks. We propose a general framework for designing external-memory algorithms for maximal clique enumeration in large graphs. The general framework enables maximal clique enumeration to be processed recursively in small subgraphs of the input graph, thus allowing in-memory computation of maximal cliques without the costly random disk access. We prove that the set of cliques obtained by the recursive local computation is both correct (i.e., globally maximal) and complete. The subgraph to be processed each time is defined based on a set of base vertices that can be flexibly chosen to achieve different purposes. We discuss the selection of the base vertices to fully utilize the available memory in order to minimize I/O cost in static graphs, and for update maintenance in dynamic graphs. We also apply our framework to design an external-memory algorithm for maximum clique computation in a large graph.

Io’s atmosphere: Constraints on sublimation support from density variations on seasonal timescales using NASA IRTF/TEXES observations from 2001 to 2010

We present an analysis of 19 μm spectra of Io’s SO 2 atmosphere from the TEXES mid-infrared high spectral resolution spectrograph on NASA’s Infrared Telescope Facility, incorporating new data taken between January 2005 and June 2010 and a re-analysis of earlier data taken from November 2001 to January 2004. This is the longest set of contiguous observations of Io’s atmosphere using the same instrument and technique thus far. We have fitted all 16 detected blended absorption lines of the ν 2 SO 2 vibrational band to retrieve the subsolar values of SO 2 column abundance and the gas kinetic temperature. By incorporating an existing model of Io’s surface temperatures and atmosphere, we retrieve sub-solar column densities from the disk-integrated data. Spectra from all years are best fit by atmospheric temperatures <150 K. Best-fit gas kinetic temperatures on the anti-Jupiter hemisphere, where SO 2 gas abundance is highest, are low and stable, with a mean of 108 (±18) K. The sub-solar SO 2 column density between longitudes of 90–220° varies from a low of 0.61 (±0.145) × 10 −17 cm −2, near aphelion in 2004, to a high of 1.51 (±0.215) × 10 17 cm −2 in 2010 when Jupiter was approaching its early 2011 perihelion. No correlation in the gas temperature was seen with the increasing SO 2 column densities outside the errors. Assuming that any volcanic component of the atmosphere is constant with time, the correlation of increasing SO 2 abundance with decreasing heliocentric distance provides good evidence that the atmosphere is at least partially supported by frost sublimation. The SO 2 frost thermal inertias and albedos that fit the variation in atmospheric density best are between 150–1250 W m −2 s −1/2 K −1 and 0.613–0.425 respectively. Photometric evidence favors albedos near the upper end of this range, corresponding to thermal inertias near the lower end. This relatively low frost thermal inertia produces larger amplitude seasonal variations than are observed, which in turn implies a substantial additional volcanic atmospheric component to moderate the amplitude of the seasonal variations of the total atmosphere on the anti-Jupiter hemisphere. The seasonal thermal inertia we measure is unique both because it refers exclusively to the SO 2 frost surface component, and also because it refers to relatively deep subsurface layers (few meters) due to the timescales of many years, while previous studies have determined thermal inertias at shallower levels (few centimeters), relevant for timescales of ∼2 h (eclipse) or ∼2 days (diurnal curves).

ReCoil - an algorithm for compression of extremely large datasets of dna data

The growing volume of generated DNA sequencing data makes the problem of its long term storage increasingly important. In this work we present ReCoil - an I/O efficient external memory algorithm designed for compression of very large collections of short reads DNA data. Typically each position of DNA sequence is covered by multiple reads of a short read dataset and our algorithm makes use of resulting redundancy to achieve high compression rate.While compression based on encoding mismatches between the dataset and a similar reference can yield high compression rate, good quality reference sequence may be unavailable. Instead, ReCoil's compression is based on encoding the differences between similar or overlapping reads. As such reads may appear at large distances from each other in the dataset and since random access memory is a limited resource, ReCoil is designed to work efficiently in external memory, leveraging high bandwidth of modern hard disk drives.

EE-IO modeling of the environmental impacts of Finnish imports using different data sources

The environmental impacts of Finnish imports were modeled using alternative emission datasets. The first dataset was based on domestic technology assumptions from the Finnish environmentally extended input–output (EE-IO) model. In the second dataset, emissions of the largest volumes of imported goods were substituted for EIOLCA (USA) data and in the third option for data from Ecoinvent and the Danish LCA Food Database. Comparative results show that the environmental impacts of imports based on domestic emission intensities are mostly lower than the alternative approaches. Considering the high level of Finnish environmental protection we concluded that the impacts of imports are underestimated using domestic emission intensities. Additionally, we concluded that hybrid IO modeling will increase the accuracy of IO modeling by using process-based data for the most relevant material flows until a multi-regional IO model for Finland has been developed.