Preprocessing Time Research Articles

Task Partitioning Scheduling Algorithms for Heterogeneous Multi-Cloud Environment

Cloud computing is now an emerging trend for cost-effective, universal access, reliability, availability, recovery and flexible IT resources. Although cloud computing has a tremendous growth, there is a wide scope of research in different dimensions. For instance, one of the challenging topics is task scheduling problem, which is shown to be NP-Hard. Recent studies report that the tasks are assigned to clouds based on their current load, without considering the partition of a task into pre-processing and processing time. Here, pre-processing time is the time needed for initialization, linking and loading of a task, whereas processing time is the time needed for the execution of a task. In this paper, we present three task partitioning scheduling algorithms, namely cloud task partitioning scheduling (CTPS), cloud min–min task partitioning scheduling and cloud max–min task partitioning scheduling, for heterogeneous multi-cloud environment. The proposed CTPS is an online scheduling algorithm, whereas others are offline scheduling algorithm. Basically, these proposed algorithms partition the tasks into two different phases, pre-processing and processing, to schedule a task in two different clouds. We compare the proposed algorithms with four task scheduling algorithms as per their applicability. All the algorithms are extensively simulated and compared using various benchmark and synthetic datasets. The simulation results show the benefit of the proposed algorithms in terms of two performance metrics, makespan and average cloud resource utilization. Moreover, we evaluate the simulation results using analysis of variance statistical test and confidence interval.

Fast synchronization‐free algorithms for parallel sparse triangular solves with multiple right‐hand sides

SummaryThe sparse triangular solve kernels, SpTRSV and SpTRSM, are important building blocks for a number of numerical linear algebra routines. Parallelizing SpTRSV and SpTRSM on today's manycore platforms, such as GPUs, is not an easy task since computing a component of the solution may depend on previously computed components, enforcing a degree of sequential processing. As a consequence, most existing work introduces a preprocessing stage to partition the components into a group of level‐sets or colour‐sets so that components within a set are independent and can be processed simultaneously during the subsequent solution stage. However, this class of methods requires a long preprocessing time as well as significant runtime synchronization overheads between the sets. To address this, we propose in this paper novel approaches for SpTRSV and SpTRSM in which the ordering between components is naturally enforced within the solution stage. In this way, the cost for preprocessing can be greatly reduced, and the synchronizations between sets are completely eliminated. To further exploit the data‐parallelism, we also develop an adaptive scheme for efficiently processing multiple right‐hand sides in SpTRSM. A comparison with a state‐of‐the‐art library supplied by the GPU vendor, using 20 sparse matrices on the latest GPU device, shows that the proposed approach obtains an average speedup of over two for SpTRSV and up to an order of magnitude speedup for SpTRSM. In addition, our method is up to two orders of magnitude faster for the preprocessing stage than existing SpTRSV and SpTRSM methods.

Itzï (version 17.1): an open-source, distributed GIS model for dynamic flood simulation

Abstract. Worldwide, floods are acknowledged as one of the most destructive hazards. In human-dominated environments, their negative impacts are ascribed not only to the increase in frequency and intensity of floods but also to a strong feedback between the hydrological cycle and anthropogenic development. In order to advance a more comprehensive understanding of this complex interaction, this paper presents the development of a new open-source tool named Itzï that enables the 2-D numerical modelling of rainfall–runoff processes and surface flows integrated with the open-source geographic information system (GIS) software known as GRASS. Therefore, it takes advantage of the ability given by GIS environments to handle datasets with variations in both temporal and spatial resolutions. Furthermore, the presented numerical tool can handle datasets from different sources with varied spatial resolutions, facilitating the preparation and management of input and forcing data. This ability reduces the preprocessing time usually required by other models. Itzï uses a simplified form of the shallow water equations, the damped partial inertia equation, for the resolution of surface flows, and the Green–Ampt model for the infiltration. The source code is now publicly available online, along with complete documentation. The numerical model is verified against three different tests cases: firstly, a comparison with an analytic solution of the shallow water equations is introduced; secondly, a hypothetical flooding event in an urban area is implemented, where results are compared to those from an established model using a similar approach; and lastly, the reproduction of a real inundation event that occurred in the city of Kingston upon Hull, UK, in June 2007, is presented. The numerical approach proved its ability at reproducing the analytic and synthetic test cases. Moreover, simulation results of the real flood event showed its suitability at identifying areas affected by flooding, which were verified against those recorded after the event by local authorities.

Adaptive Point Location in Planar Convex Subdivisions

We present a planar point location structure for a convex subdivision [Formula: see text]. Given a query sequence of length [Formula: see text], the total running time is [Formula: see text], where [Formula: see text] is the number of vertices in [Formula: see text] and [Formula: see text] is the minimum time required by any linear decision tree for answering planar point location queries in [Formula: see text] to process the same query sequence. The running time includes the preprocessing time. Therefore, for [Formula: see text], our running time is only worse than the best possible bound by [Formula: see text] per query, which is much smaller than the [Formula: see text] query time offered by a worst-case optimal planar point location structure.

Visualization, Quantification, and Alignment of Spectral Drift in Population Scale Untargeted Metabolomics Data.

Untargeted liquid-chromatography-mass spectrometry (LC-MS)-based metabolomics analysis of human biospecimens has become among the most promising strategies for probing the underpinnings of human health and disease. Analysis of spectral data across population scale cohorts, however, is precluded by day-to-day nonlinear signal drifts in LC retention time or batch effects that complicate comparison of thousands of untargeted peaks. To date, there exists no efficient means of visualization and quantitative assessment of signal drift, correction of drift when present, and automated filtering of unstable spectral features, particularly across thousands of data files in population scale experiments. Herein, we report the development of a set of R-based scripts that allow for pre- and postprocessing of raw LC-MS data. These methods can be integrated with existing data analysis workflows by providing initial preprocessing bulk nonlinear retention time correction at the raw data level. Further, this approach provides postprocessing visualization and quantification of peak alignment accuracy, as well as peak-reliability-based parsing of processed data through hierarchical clustering of signal profiles. In a metabolomics data set derived from ∼3000 human plasma samples, we find that application of our alignment tools resulted in substantial improvement in peak alignment accuracy, automated data filtering, and ultimately statistical power for detection of metabolite correlates of clinical measures. These tools will enable metabolomics studies of population scale cohorts.

Enhancing lifetime of visual sensor networks with a preprocessing-based multi-face detection method

Recently, advances in hardware such as CMOS camera nodes have led to the development of Visual Sensor Networks (VSNs) that process sensed data and transmit the useful information to the base station for completing subsequent tasks. Today, object detection and sending useful information to the base station for object recognition is emerged as an important challenging issue in VSNs. Our investigations show that the face’s information is adequate for completing object recognition. According to literature, many approaches have been proposed for object detection and sending useful information to the base station to be completed subsequent tasks like object recognition. However, in most of them, lack of preprocessing methods in camera nodes causes network to be faced with large volume of data. For example when there is more than one object within the each camera node filed-of-view, conventional works deliver empty spaces among objects to the base station. Also, most of them send whole information about each object to the base station, while sending only face’s information of each object is adequate for completing object recognition. Therefore, in this paper, a preprocessing method in camera nodes named Preprocessing-based Multi-Face Detection (PMFD) is proposed. Our method works based on the extracting bounding box of each object’s face, using Boosting-based face detection algorithm, and sending only the faces’ information to the base station. The simulation results show that PMFD method has acceptable preprocessing time complexity and injects low volume of traffic into the network. Consequently, PMFD method prolongs the network lifetime in comparison with state-of-the-art algorithms.

Prediction of monthly electric energy consumption using pattern-based fuzzy nearest neighbour regression

Electricity demand forecasting is of important role in power system planning and operation. In this work, fuzzy nearest neighbour regression has been utilised to estimate monthly electricity demands. The forecasting model was based on the pre-processed energy consumption time series, where input and output variables were defined as patterns representing unified fragments of the time series. Relationships between inputs and outputs, which were simplified due to patterns, were modelled using nonparametric regression with weighting function defined as a fuzzy membership of learning points to the neighbourhood of a query point. In an experimental part of the work the model was evaluated using real-world data. The results are encouraging and show high performances of the model and its competitiveness compared to other forecasting models.

Complexity Reduction of Multiscale UTLM Cell Clusters

Time-domain electromagnetic simulations employing unstructured tetrahedral meshes offer smooth boundary approximations and graded meshes for multiscale problems. However, multiscale effects may arise not only as a consequence of fine geometry but also from CAD and mesh generation artifacts, and it is critical that the simulation algorithms can be employed in their presence without unduly compromising their computational performance. The ability of the unstructured transmission line modeling (UTLM) algorithm to coalesce small computational cells into larger entities is a key enabler for the approach. This paper demonstrates the use of complexity reduction techniques to both notably reduce the preprocessing time required for this and, as a consequence, substantially extend its capability.

Interactive registration between supine and prone scans in computed tomography colonography using band-height images

In computed tomographic colonography (CTC), a patient is commonly scanned twice including supine and prone scans to improve the sensitivity of polyp detection. Typically, a radiologist must manually match the corresponding areas in the supine and prone CT scans, which is a difficult and time-consuming task, even for experienced scan readers. In this paper, we propose a method of supine-prone registration utilizing band-height images, which are directly constructed from the CT scans using a ray-casting algorithm containing neighboring shape information. In our method, we first identify anatomical feature points and establish initial correspondences using local extreme points on centerlines. We then correct correspondences using band-height images that contain neighboring shape information. We use geometrical and image-based information to match positions between the supine and prone centerlines. Finally, our algorithm searches the correspondence of user input points using the matched anatomical feature point pairs as key points and band-height images. The proposed method achieved accurate matching and relatively faster processing time than other previously proposed methods. The mean error of the matching between the supine and prone points for uniformly sampled positions was 18.41±22.07mm in 20 CTC datasets. The average pre-processing time was 62.9±8.6s, and the interactive matching was performed in nearly real-time. Our supine-prone registration method is expected to be helpful for the accurate and fast diagnosis of polyps.

Wind Energy Potential Assessment and Forecasting Research Based on the Data Pre-Processing Technique and Swarm Intelligent Optimization Algorithms

Accurate quantification and characterization of a wind energy potential assessment and forecasting is significant to optimal wind farm design, evaluation and scheduling. However, wind energy potential assessment and forecasting remain difficult and challenging research topics at present. Traditional wind energy assessment and forecasting models usually ignore the problem of data pre-processing as well as parameter optimization, which leads to low accuracy. Therefore, this paper aims to assess the potential of wind energy and forecast the wind speed in four locations in China based on the data pre-processing technique and swarm intelligent optimization algorithms. In the assessment stage, the cuckoo search (CS) algorithm, ant colony (AC) algorithm, firefly algorithm (FA) and genetic algorithm (GA) are used to estimate the two unknown parameters in the Weibull distribution. Then, the wind energy potential assessment results obtained by three data-preprocessing approaches are compared to recognize the best data-preprocessing approach and process the original wind speed time series. While in the forecasting stage, by considering the pre-processed wind speed time series as the original data, the CS and AC optimization algorithms are adopted to optimize three neural networks, namely, the Elman neural network, back propagation neural network, and wavelet neural network. The comparison results demonstrate that the new proposed wind energy assessment and speed forecasting techniques produce promising assessments and predictions and perform better than the single assessment and forecasting components.

Efficient Sampling Methods for Discrete Distributions

We study the fundamental problem of the exact and efficient generation of random values from a finite and discrete probability distribution. Suppose that we are given n distinct events with associated probabilities p_1, dots , p_n. First, we consider the problem of sampling from the distribution where the i-th event has probability proportional to p_i. Second, we study the problem of sampling a subset which includes the i-th event independently with probability p_i. For both problems we present on two different classes of inputs—sorted and general probabilities—efficient data structures consisting of a preprocessing and a query algorithm. Varying the allotted preprocessing time yields a trade-off between preprocessing and query time, which we prove to be asymptotically optimal everywhere.

Influence of nature of core on vibro acoustic behavior of sandwich aerospace structures

This paper presents the study of influence of core geometry on vibration and acoustic response characteristics of sandwich panels which are used as aerospace structures. Sandwich panels considered in this research work are: (a) Honeycomb core, (b) Truss and Z core, (c) Foam core. The present study has found that (i) For a honeycomb core sandwich panel in due consideration to space constraint, the better acoustic comfort can be achieved by reducing the core height and increasing the face sheet thickness. (ii) It is demonstrated that, for a honeycomb core sandwich panel, vibration and acoustic response is not sensitive to the cell size. (iii) It is observed that, triangular core gives better acoustic comfort for the truss core sandwich panel compared to other type of core. (iv) For foam core sandwich panels, it is observed that sandwich panel with carbon-epoxy (high stiffness) face sheet radiates less sound in the lower frequency range (0–100 Hz). While the sandwich panel with Titanium (high density) face sheet radiates less sound at the higher frequencies. In order to reduce the preprocessing time and computational effort throughout the analysis in the present study, equivalent 2D elastic properties are calculated and used to find out the vibration and acoustic response characteristics.

Performance Evaluation of Various STL File Mesh Refining Algorithms Applied for FDM-RP Process

Layered manufacturing machines use the stereolithography (STL) file to build parts. When a curved surface is converted from a computer aided design (CAD) file to STL, it results in a geometrical distortion and chordal error. Parts manufactured with this file, might not satisfy geometric dimensioning and tolerance requirements due to approximated geometry. Current algorithms built in CAD packages have export options to globally reduce this distortion, which leads to an increase in the file size and pre-processing time. In this work, different mesh subdivision algorithms are applied on STL file of a complex geometric features using MeshLab software. The mesh subdivision algorithms considered in this work are modified butterfly subdivision technique, loops sub division technique and general triangular midpoint sub division technique. A comparative study is made with respect to volume and the build time using the above techniques. It is found that triangular midpoint sub division algorithm is more suitable for the geometry under consideration. Only the wheel cap part is then manufactured on Stratasys MOJO FDM machine. The surface roughness of the part is measured on Talysurf surface roughness tester.

Shedding light on Aspergillus niger volatile exometabolome.

An in-depth exploration of the headspace content of Aspergillus niger cultures was performed upon different growth conditions, using a methodology based on advanced multidimensional gas chromatography. This volatile fraction comprises 428 putatively identified compounds distributed over several chemical families, being the major ones hydrocarbons, alcohols, esters, ketones and aldehydes. These metabolites may be related with different metabolic pathways, such as amino acid metabolism, biosynthesis and metabolism of fatty acids, degradation of aromatic compounds, mono and sesquiterpenoid synthesis and carotenoid cleavage. The A. niger molecular biomarkers pattern was established, comprising the 44 metabolites present in all studied conditions. This pattern was successfully used to distinguish A. niger from other fungi (Candida albicans and Penicillium chrysogenum) with 3 days of growth by using Partial Least Squares-Discriminant Analysis (PLS-DA). In addition, PLS-DA-Variable Importance in Projection was applied to highlight the metabolites playing major roles in fungi distinction; decreasing the initial dataset to only 16 metabolites. The data pre-processing time was substantially reduced, and an improvement of quality-of-fit value was achieved. This study goes a step further on A. niger metabolome construction and A. niger future detection may be proposed based on this molecular biomarkers pattern.

Optimal Any-Angle Pathfinding In Practice

Any-angle pathfinding is a fundamental problem in robotics and computer games. The goal is to find a shortest path between a pair of points on a grid map such that the path is not artificially constrained to the points of the grid. Prior research has focused on approximate online solutions. A number of exact methods exist but they all require super-linear space and pre-processing time. In this study, we describe Anya: a new and optimal any-angle pathfinding algorithm. Where other works find approximate any-angle paths by searching over individual points from the grid, Anya finds optimal paths by searching over sets of states represented as intervals. Each interval is identified on-the-fly. From each interval Anya selects a single representative point that it uses to compute an admissible cost estimate for the entire set. Anya always returns an optimal path if one exists. Moreover it does so without any offline pre-processing or the introduction of additional memory overheads. In a range of empirical comparisons we show that Anya is competitive with several recent (sub-optimal) online and pre-processing based techniques and is up to an order of magnitude faster than the most common benchmark algorithm, a grid-based implementation of A*.

PinMesh—Fast and exact 3D point location queries using a uniform grid

This paper presents PinMesh, a very fast algorithm with implementation to preprocess a polyhedral mesh, also known as a multi-material mesh, in order to perform 3D point location queries. PinMesh combines several innovative components to efficiently handle the largest available meshes. Because of a 2-level uniform grid, the expected preprocessing time is linear in the input size, and the code parallelizes well on a shared memory machine. Querying time is almost independent of the dataset size. PinMesh uses exact arithmetic with rational numbers to prevent roundoff errors, and symbolic perturbation with Simulation of Simplicity (SoS) to handle geometric degeneracies or special cases. PinMesh is intended to be a subroutine in more complex algorithms. It can preprocess a dataset and perform 1 million queries up to 27 times faster than RCT (Relative Closest Triangle), the current fastest algorithm. Preprocessing a sample dataset with 50 million triangles took only 14 elapsed seconds on a 16-core Xeon processor. The mean query time was 0.6μs.

Forecasting business cycle with chaotic time series based on neural network with weighted fuzzy membership functions

This study presents a forecasting model of cyclical fluctuations of the economy based on the time delay coordinate embedding method. The model uses a neuro-fuzzy network called neural network with weighted fuzzy membership functions (NEWFM). The preprocessed time series of the leading composite index using the time delay coordinate embedding method are used as input data to the NEWFM to forecast the business cycle. A comparative study is conducted using other methods based on wavelet transform and Principal Component Analysis for the performance comparison. The forecasting results are tested using a linear regression analysis to compare the approximation of the input data against the target class, gross domestic product (GDP). The chaos based model captures nonlinear dynamics and interactions within the system, which other two models ignore. The test results demonstrated that chaos based method significantly improved the prediction capability, thereby demonstrating superior performance to the other methods.

Faster Lyndon factorization algorithms for SLP and LZ78 compressed text

We present two efficient algorithms which, given a compressed representation of a string w of length N, compute the Lyndon factorization of w. Given a straight line program (SLP) S of size n that describes w, the first algorithm runs in O(n2+P(n,N)+Q(n,N)nlog⁡n) time and O(n2+S(n,N)) space, where P(n,N), S(n,N), Q(n,N) are respectively the pre-processing time, space, and query time of a data structure for longest common extensions (LCE) on SLPs. Given the Lempel–Ziv 78 encoding of size s for w, the second algorithm runs in O(slog⁡s) time and space.

Enhance Inverted Index Using in Information Retrieval

This paper proposes a method to represent the first step in information retrieval (IR) (that prepare the document set (preprocessing), In Information retrieval systems, tokenization is an integral part whose prime objective is to identify the token and their count. In this paper, an effective tokenization approach which is based on proposed new method called enhance inverted index (EII). The result shows that efficiency/ effectiveness of the proposed algorithm. Tokenization on documents helps to satisfy user’s information need more precisely and reduced search sharply, believed to be a part of information retrieval. Pre-processing of input document is an integral part of Tokenization, which involves preprocessing of documents and generates its respective tokens, which is the basis of these tokens. Probabilistic IR generates its scoring and gives reduced search space. The comparative analysis based on the two parameters; reduce the time of search space, Pre-processing time

Algorithms for algebraic path properties in concurrent systems of constant treewidth components

We study algorithmic questions for concurrent systems where the transitions are labeled from a complete, closed semiring, and path properties are algebraic with semiring operations. The algebraic path properties can model dataflow analysis problems, the shortest path problem, and many other natural problems that arise in program analysis. We consider that each component of the concurrent system is a graph with constant treewidth, a property satisfied by the controlflow graphs of most programs. We allow for multiple possible queries, which arise naturally in demand driven dataflow analysis. The study of multiple queries allows us to consider the tradeoff between the resource usage of the one-time preprocessing and for each individual query. The traditional approach constructs the product graph of all components and applies the best-known graph algorithm on the product. In this approach, even the answer to a single query requires the transitive closure (i.e., the results of all possible queries), which provides no room for tradeoff between preprocessing and query time. Our main contributions are algorithms that significantly improve the worst-case running time of the traditional approach, and provide various tradeoffs depending on the number of queries. For example, in a concurrent system of two components, the traditional approach requires hexic time in the worst case for answering one query as well as computing the transitive closure, whereas we show that with one-time preprocessing in almost cubic time, each subsequent query can be answered in at most linear time, and even the transitive closure can be computed in almost quartic time. Furthermore, we establish conditional optimality results showing that the worst-case running time of our algorithms cannot be improved without achieving major breakthroughs in graph algorithms (i.e., improving the worst-case bound for the shortest path problem in general graphs). Preliminary experimental results show that our algorithms perform favorably on several benchmarks.