Distributed Processing Framework Research Articles

DynamoGraph: extending the Pregel paradigm for large-scale temporal graph processing

Large-scale temporal graphs can serve as a model in many application scenarios. Owing to the popularity of online social networks and increased research interest in gathering and analysing data about human behaviour and interaction reality mining temporal graphs gain attraction in social network analysis and more specifically in the analysis of dynamic processes in social networks. Originally developed for web-graph analysis, systems supporting a Pregel-style processing paradigm can be used to process large-scale social network data. However, current methods for social network analysis require data to be processed off-line, or lack support for temporal graphs, or support datasets of limited size only. In this work we present a cloud-based distributed processing framework extending the Pregel paradigm for large-scale temporal graphs. By using computing resources in the cloud this system is scalable and prepared to scale for the massive datasets that occur in social network analysis.

Similarity-based Change Detection for RDF in MapReduce

Managing and analyzing huge amount of heterogeneous data are essential for various services in the Internet of Things (IoT). Such analysis requires keeping track of data versions over time, and consequently detecting changes between them. However, it is challenging to identify the differences between datasets in Resource Description Framework (RDF), which has gained great attention as a format for the semantic annotation of sensor data. This results from the property of RDF triples as an unordered set and the existence of blank nodes. Existing change detection techniques have limitations in terms of scalability or utilization of structural RDF features. In this paper, we describe the implementation details of similarity-based RDF change detection techniques on the well-known distributed processing framework, MapReduce. In addition, we present an experimental comparison of these change detection techniques.

Thinking Like a Vertex

The vertex-centric programming model is an established computational paradigm recently incorporated into distributed processing frameworks to address challenges in large-scale graph processing. Billion-node graphs that exceed the memory capacity of commodity machines are not well supported by popular Big Data tools like MapReduce, which are notoriously poor performing for iterative graph algorithms such as PageRank. In response, a new type of framework challenges one to “think like a vertex” (TLAV) and implements user-defined programs from the perspective of a vertex rather than a graph. Such an approach improves locality, demonstrates linear scalability, and provides a natural way to express and compute many iterative graph algorithms. These frameworks are simple to program and widely applicable but, like an operating system, are composed of several intricate, interdependent components, of which a thorough understanding is necessary in order to elicit top performance at scale. To this end, the first comprehensive survey of TLAV frameworks is presented. In this survey, the vertex-centric approach to graph processing is overviewed, TLAV frameworks are deconstructed into four main components and respectively analyzed, and TLAV implementations are reviewed and categorized.

MRPack: Multi-Algorithm Execution Using Compute-Intensive Approach in MapReduce.

Large quantities of data have been generated from multiple sources at exponential rates in the last few years. These data are generated at high velocity as real time and streaming data in variety of formats. These characteristics give rise to challenges in its modeling, computation, and processing. Hadoop MapReduce (MR) is a well known data-intensive distributed processing framework using the distributed file system (DFS) for Big Data. Current implementations of MR only support execution of a single algorithm in the entire Hadoop cluster. In this paper, we propose MapReducePack (MRPack), a variation of MR that supports execution of a set of related algorithms in a single MR job. We exploit the computational capability of a cluster by increasing the compute-intensiveness of MapReduce while maintaining its data-intensive approach. It uses the available computing resources by dynamically managing the task assignment and intermediate data. Intermediate data from multiple algorithms are managed using multi-key and skew mitigation strategies. The performance study of the proposed system shows that it is time, I/O, and memory efficient compared to the default MapReduce. The proposed approach reduces the execution time by 200% with an approximate 50% decrease in I/O cost. Complexity and qualitative results analysis shows significant performance improvement.

FoxPSL: A Fast, Optimized and eXtended PSL implementation

Abstract In this paper, we describe foxPSL , a fast, optimized and extended implementation of Probabilistic Soft Logic (PSL) based on the distributed graph processing framework Signal/Collect . PSL is one of the leading formalisms of statistical relational learning, a recently developed field of machine learning that aims at representing both uncertainty and rich relational structures, usually by combining logical representations with probabilistic graphical models. PSL can be seen as both a probabilistic logic and a template language for hinge-loss Markov Random Fields, a type of continuous Markov Random fields (MRF) in which Maximum a Posteriori inference is very efficient, since it can be formulated as a constrained convex minimization problem, as opposed to a discrete optimization problem for standard MRFs. From the logical perspective, a key feature of PSL is the capability to represent soft truth values, allowing the expression of complex domain knowledge, like degrees of truth, in parallel with uncertainty. foxPSL supports the full PSL pipeline from problem definition to a distributed solver that implements the Alternating Direction Method of Multipliers (ADMM) consensus optimization. It provides a Domain Specific Language that extends standard PSL with a class system and existential quantifiers, allowing for efficient grounding. Moreover, it implements a series of configurable optimizations, like optimized grounding of constraints and lazy inference, that improve grounding and inference time. We perform an extensive evaluation, comparing the performance of foxPSL to a state-of-the-art implementation of ADMM consensus optimization in GraphLab, and show an improvement in both inference time and solution quality. Moreover, we evaluate the impact of the optimizations on the execution time and discuss the trade-offs related to each optimization.

GeoProcessing Workflow Models for Distributed Processing Frameworks

Information Systems (GIS) platforms are used to implement and deal with the massive spatial data, especially image data. Therefore, these platforms require high storage capacity and high computational power to process them. This paper aims of processing of Geodata using Distributed processing Frameworks. Large volume of Geodata cannot be processed using desktop GIS tools such as QGIS, ArcGIS, GRASS, OpenJUMP etc. Therefore, to handle and process on these types of large data, use of Hadoop Distributed processing framework needs to be deployed. GeoProcessing is a GIS operation used to manipulate spatial data. It is one of the original proposal in GIS development. Almost every GIS application is represented by a GeoProcessing Workflow. This paper explains the GeoProcessing Workflow for processing of image data. Also explains Hadoop Distributed File System (HDFS), MapReduce Programming Model and Yet Another Resource Negotiator (YARN) architecture, useful in large spatial data handling and analysis at fast rate.

City Forensics: Using Visual Elements to Predict Non-Visual City Attributes.

We present a method for automatically identifying and validating predictive relationships between the visual appearance of a city and its non-visual attributes (e.g. crime statistics, housing prices, population density etc.). Given a set of street-level images and (location, city-attribute-value) pairs of measurements, we first identify visual elements in the images that are discriminative of the attribute. We then train a predictor by learning a set of weights over these elements using non-linear Support Vector Regression. To perform these operations efficiently, we implement a scalable distributed processing framework that speeds up the main computational bottleneck (extracting visual elements) by an order of magnitude. This speedup allows us to investigate a variety of city attributes across 6 different American cities. We find that indeed there is a predictive relationship between visual elements and a number of city attributes including violent crime rates, theft rates, housing prices, population density, tree presence, graffiti presence, and the perception of danger. We also test human performance for predicting theft based on street-level images and show that our predictor outperforms this baseline with 33% higher accuracy on average. Finally, we present three prototype applications that use our system to (1) define the visual boundary of city neighborhoods, (2) generate walking directions that avoid or seek out exposure to city attributes, and (3) validate user-specified visual elements for prediction.

MOCgraph

Existing distributed graph processing frameworks, e.g. , Pregel, Giraph, GPS and GraphLab, mainly exploit main memory to support flexible graph operations for efficiency. Due to the complexity of graph analytics, huge memory space is required especially for those graph analytics that spawn large intermediate results. Existing frameworks may terminate abnormally or degrade performance seriously when the memory is exhausted or the external storage has to be used. In this paper, we propose MOCgraph, a scalable distributed graph processing framework to reduce the memory footprint and improve the scalability, based on message online computing. MOCgraph consumes incoming messages in a streaming manner, so as to handle larger graphs or more complex analytics with the same memory capacity. MOCgraph also exploits message online computing with external storage to provide an efficient out-of-core support. We implement MOCgraph on top of Apache Giraph, and test it against several representative graph algorithms on large graph datasets. Experiments illustrate that MOCgraph is efficient and memory-saving, especially for graph analytics with large intermediate results.

ASPIRE

Many vertex-centric graph algorithms can be expressed using asynchronous parallelism by relaxing certain read-after-write data dependences and allowing threads to compute vertex values using stale (i.e., not the most recent) values of their neighboring vertices. We observe that on distributed shared memory systems, by converting synchronous algorithms into their asynchronous counterparts, algorithms can be made tolerant to high inter-node communication latency. However, high inter-node communication latency can lead to excessive use of stale values causing an increase in the number of iterations required by the algorithms to converge. Although by using bounded staleness we can restrict the slowdown in the rate of convergence, this also restricts the ability to tolerate communication latency. In this paper we design a relaxed memory consistency model and consistency protocol that simultaneously tolerate communication latency and minimize the use of stale values. This is achieved via a coordinated use of best effort refresh policy and bounded staleness. We demonstrate that for a range of asynchronous graph algorithms and PDE solvers, on an average, our approach outperforms algorithms based upon: prior relaxed memory models that allow stale values by at least 2.27x; and Bulk Synchronous Parallel (BSP) model by 4.2x. We also show that our approach frequently outperforms GraphLab, a popular distributed graph processing framework.

PSPs and ERPs: Applying the dynamics of post-synaptic potentials to individual units in simulation of temporally extended Event-Related Potential reading data

The Parallel Distributed Processing (PDP) framework is built on neural-style computation, and is thus well-suited for simulating the neural implementation of cognition. However, relatively little cognitive modeling work has concerned neural measures, instead focusing on behavior. Here, we extend a PDP model of reading-related components in the Event-Related Potential (ERP) to simulation of the N400 repetition effect. We accomplish this by incorporating the dynamics of cortical post-synaptic potentials—the source of the ERP signal—into the model. Simulations demonstrate that application of these dynamics is critical for model elicitation of repetition effects in the time and frequency domains. We conclude that by advancing a neurocomputational understanding of repetition effects, we are able to posit an interpretation of their source that is both explicitly specified and mechanistically different from the well-accepted cognitive one.

Connectionism coming of age: legacy and future challenges.

Connectionism coming of age: legacy and future challenges.

BiShard Parallel Processor: A Disk-Based Processing Engine for Billion-Scale Graphs

Abstract Processing very large graphs efficiently is a challenging task. Distributed graph processing systems process the billion-scale graphs efficiently but incur overheads of partitioning and distribution of the large graph over a cluster of nodes. In order to overcome these problems a disk-based engine, GraphChi was proposed recently that processes the graph in chunks on a single PC. GraphChi significantly outperformed all the representative distributed processing frameworks. Still, we observe that GraphChi incurs some serious degradation in performance due to 1) high number of non-sequential I/Os for processing every chunk of the graph; and 2) limited parallelism to process the graph. In this paper, we propose a novel engine named BiShard Parallel Processor (BSPP) to efficiently process billions-scale graphs on a single PC. We introduce a new storage structure BiShard. BiShard divides the large graph into subgraphs and maintains the in and out edges separately. This storage mechanism significantly reduces the number of non-sequential I/Os. We implement a new processing model named BiShard Parallel (BSP) on top of Bishard. BSP exploits the properties of Bishard to enable full CPU parallelism for processing the graph. Our experiments on real large graphs show that our solution significantly outperforms GraphChi.

Theoretical analysis of word production deficits in adult aphasia

The cognitive analysis of adult language disorders continues to draw heavily on linguistic theory, but increasingly it reflects the influence of connectionist, spreading activation models of cognition. In the area of spoken word production, 'localist' connectionist models represent a natural evolution from the psycholingistic theories of earlier decades. By contrast, the parallel distributed processing framework forces more radical rethinking of aphasic impairments. This paper exemplifies these multiple influences in contemporary cognitive aphasiology. Topics include (i) what aphasia reveals about semantic-phonological interaction in lexical access; (ii) controversies surrounding the interpretation of semantic errors and (iii) a computational account of the relationship between naming and word repetition in aphasia. Several of these topics have been addressed using case series methods, including computational simulation of the individual, quantitative error patterns of diverse groups of patients and analysis of brain lesions that correlate with error rates and patterns. Efforts to map the lesion correlates of nonword errors in naming and repetition highlight the involvement of sensorimotor areas in the brain and suggest the need to better integrate models of word production with models of speech and action.

Automaticity and control in prospective memory: a computational model.

Prospective memory (PM) refers to our ability to realize delayed intentions. In event-based PM paradigms, participants must act on an intention when they detect the occurrence of a pre-established cue. Some theorists propose that in such paradigms PM responding can only occur when participants deliberately initiate processes for monitoring their environment for appropriate cues. Others propose that perceptual processing of PM cues can directly trigger PM responding in the absence of strategic monitoring, at least under some circumstances. In order to address this debate, we present a computational model implementing the latter account, using a parallel distributed processing (interactive activation) framework. In this model PM responses can be triggered directly as a result of spreading activation from units representing perceptual inputs. PM responding can also be promoted by top-down monitoring for PM targets. The model fits a wide variety of empirical findings from PM paradigms, including the effect of maintaining PM intentions on ongoing response time and the intention superiority effect. The model also makes novel predictions concerning the effect of stimulus degradation on PM performance, the shape of response time distributions on ongoing and prospective memory trials, and the effects of instructing participants to make PM responses instead of ongoing responses or alongside them. These predictions were confirmed in two empirical experiments. We therefore suggest that PM should be considered to result from the interplay between bottom-up triggering of PM responses by perceptual input, and top-down monitoring for appropriate cues. We also show how the model can be extended to simulate encoding new intentions and subsequently deactivating them, and consider links between the model’s performance and results from neuroimaging.

A Scalable XSLT Processing Framework based on MapReduce

The eX tensible Stylesheet Language Transformation (XSLT ) is a de-facto standard for XML data transforming and extracting . Efficient processing of large amounts of XML data brings challenges to conventional XSLT processors, which are designed to run in a single machine context. To solve these data-intensive problems, MapReduce paradigm in the cloud computing domain has received a comprehensive attention in both academia and IT industry recently. In this paper, a novel MapReduce-based XSLT distributed processing framework named CloudXSLT is proposed to implement efficient and scalable XML data transforming. First, the architecture of CloudXSLT framework is outlined. Subsequently, several XML data and XSLT rule representation models which are suitable for MapReduce paradigm are defined, and several MapReduce-based XSLT distributed processing algorithms are proposed. Finally, an experiment on a simulation environment with real XML datasets shows our framework is more efficient and scalable than conventional XSLT processors when processing large size of XML data.

Jobcast ^|^mdash; Parallel and distributed processing framework Data processing on a cloud style KVS database

In this paper, we propose a new architecture for parallel and distributed processing framework, gJobcasth, which enables data processing on a cloud style KVS database. Nowadays, lots of KVS (as known as Key Value Store) systems exist which achieve high scalability for data spaces among a huge number of computers. Some of KVS implementations use gconsistent hashh algorithm to identify the backend data node to store a pair of key and value. Jobcast also uses consistent hash algorithm for a distribution strategy and has a capability to store key and value pairs into huge number of computers as a KVS system. Furthermore, Jobcast also distributes gjobsh into data nodes for parallel and distributed processing. In this paper, we introduce a basic architecture of Jobcast and evaluate a data processing performance for a typical example.

Jobcast — Parallel and distributed processing framework Data processing on a cloud style KVS database

Jobcast — Parallel and distributed processing framework Data processing on a cloud style KVS database

ULTRAMAP V3 – A REVOLUTION IN AERIAL PHOTOGRAMMETRY

Abstract. In the last years, Microsoft has driven innovation in the aerial photogrammetry community. Besides the market leading camera technology, UltraMap has grown to an outstanding photogrammetric workflow system which enables users to effectively work with large digital aerial image blocks in a highly automated way. Best example is the project-based color balancing approach which automatically balances images to a homogeneous block. UltraMap V3 continues innovation, and offers a revolution in terms of ortho processing. A fully automated dense matching module strives for high precision digital surface models (DSMs) which are calculated either on CPUs or on GPUs using a distributed processing framework. By applying constrained filtering algorithms, a digital terrain model can be derived which in turn can be used for fully automated traditional ortho texturing. By having the knowledge about the underlying geometry, seamlines can be generated automatically by applying cost functions in order to minimize visual disturbing artifacts. By exploiting the generated DSM information, a DSMOrtho is created using the balanced input images. Again, seamlines are detected automatically resulting in an automatically balanced ortho mosaic. Interactive block-based radiometric adjustments lead to a high quality ortho product based on UltraCam imagery. UltraMap v3 is the first fully integrated and interactive solution for supporting UltraCam images at best in order to deliver DSM and ortho imagery.

A neurally plausible Parallel Distributed Processing model of Event-Related Potential word reading data

The Parallel Distributed Processing (PDP) framework has significant potential for producing models of cognitive tasks that approximate how the brain performs the same tasks. To date, however, there has been relatively little contact between PDP modeling and data from cognitive neuroscience. In an attempt to advance the relationship between explicit, computational models and physiological data collected during the performance of cognitive tasks, we developed a PDP model of visual word recognition which simulates key results from the ERP reading literature, while simultaneously being able to successfully perform lexical decision—a benchmark task for reading models. Simulations reveal that the model’s success depends on the implementation of several neurally plausible features in its architecture which are sufficiently domain-general to be relevant to cognitive modeling more generally.

SU‐GG‐T‐376: Medical Physics Calculations in the Cloud: A New Paradigm for Clinical Computing

Purpose: The accuracy of medical physics calculations is largely limited by the availability of fast computer hardware, which is in turn strongly limited by budget considerations. This research explores the use of the Cloud Computing paradigm to replace or supplement existing computing infrastructure for medical physics calculations. Cloud computing services allow for on‐demand virtual computing clusters with a pay‐as‐you‐go pricing model. This model has the potential to make large scale Monte Carlo simulations routine in clinical and research activities. Methods and materials: To test the feasibility of medical physics calculations in a cloud computing environment, a distributed Monte Carlo dose calculation framework was implemented with Amazon's EC2 cloud computing service, the Fluka 2008.3b Monte Carlo package, a distributed processing framework written in Python, and the Hadoop MapReduce framework. A variety of relevant calculations were performed: depth dose profiles of proton, electron, and photon beams, along with a simple proton treatment plan using a voxel patient geometry (Zubal phantom). The performance of EC2 was tested by running calculations on 1 to 200 virtual nodes with the different distributed processing frameworks. EC2 prices were compared with costs of comparable local hardware. Results: Relevant medical physics calculations were successfully carried out using the EC2 service. Heavy charged particle, electron, and photon depth dose profiles and a simple treatment plan were successfully calculated remotely. Performance data demonstrated the expected 1/n speed‐up as a function of node number. Two different distributed processing schemes were implemented in the cloud. Conclusion: Our implementations demonstrate the power of the existing cloud computing model. The relatively cheap pay‐as‐you‐go pricing for infrastructure on demand offers an extremely promising new paradigm for clinical and research computing, available to anyone with a network connection.