Amount Of Raw Data Research Articles

A Water Quality Monitoring Method Based on Fuzzy Comprehensive Evaluation in Wireless Sensor Networks

A novel water quality monitoring method named WQMMFCE is proposed based on fuzzy comprehensive evaluation model to solve the issue of high energy consumption caused by centralized approach under the background of water quality monitoring. The weights of all factors which are necessary for fuzzy comprehensive evaluation can be obtained firstly by using the binary expert evaluation. The information needs to transmit is decided according to water quality grade which is quantitatively analyzed directly by using fuzzy comprehensive evaluation rather than transmitting large amount of raw data to the monitoring center. Simulation result shows that the method has a great advantage over the centralized one on energy saving and can prolong the lifetime of the network.

Design and Implementation of Data Collection and Analysis Based on TWAMP

To analyze and evaluate IP network, large amounts of raw data measured by different protocol needs to be organized. After the network probes execute protocol issued by RFC, all raw data was stored in distributed network probes. So far system based on TWAMP is hard to find. No more details are described to teach system designers how to collect, store and analyze these data. In this paper two practical mechanisms are introduced. Data collection mechanism mainly focuses on how to effectively store information retrieved from raw TWAMP test packets. Data Analysis mechanism shows how to process raw data and retrieve network metric from these data.

Mechanical Fault Diagnosis Based on Data Mining Technology

Data mining is a variety of analysis tools using the data found in the mass relationship between model and data technology, and relationships of these models can be used to predict. Organically with the combination of expert system, can solve the problem of automatic acquisition of knowledge. It from a large number, incomplete, noisy, fuzzy, random data mining the hidden decision-making has important reference value to generate the information. Data mining technology used in machinery fault diagnosis system, the scene of large quantities of raw data into valuable knowledge for specialists to explore the information they are interested in, but also describes the evolution trend of mechanical malfunction, the expert decision-making support information. And, after years of development, data mining technology has matured and widely used. The purpose of this study is that data mining technology used in machinery fault diagnosis and to study its feasibility. If applicable, must be able to open up a mechanical fault diagnosis, a new diagnostic method, which greatly promote the development of mechanical fault diagnosis technology.

Knowledge Discovery Through Experiential Learning From Business and Other Contemporary Data Sources: A Review and Reappraisal

Every day massive amount of data is generated, collected, and stored in information repositories such as databases and data warehouses. Current information technology is sufficiently mature and powerful to store any amount of raw data in an organized manner. However, finding useful patterns, trends, rules, correlations, and deviations in large amount of data, and/or making meaningful predictions from it still remains one of the main challenges of the information era. The more data one has, the more difficult it is to analyze and draw meaningful conclusions. Knowledge discovery in databases (KDD) and data mining (DM) is a field, which uses computer-based and analytic technologies to efficiently extract intelligence from data that humans need. In this article, we review the process of knowledge discovery in databases, and describe selected methodologies, methods and tools, tasks, basic learning paradigms, and applications for knowledge generation by computer learning from data instances. We also examine the current trends in the field with respect to the data types mined, data mining methods used, classes of data mining applications, as well as the data mining software used.

Evolving spectral transformations for multitemporal information extraction using evolutionary computation

Remote sensing plays an important role in assessing temporal changes in land features. The challenge often resides in the conversion of large quantities of raw data into actionable information in a timely and cost-effective fashion. To address this issue, research was undertaken to develop an innovative methodology integrating biologically-inspired algorithms with standard image classification algorithms to improve information extraction from multitemporal imagery. Genetic programming was used as the optimization engine to evolve feature-specific candidate solutions in the form of nonlinear mathematical expressions of the image spectral channels (spectral indices). The temporal generalization capability of the proposed system was evaluated by addressing the task of building rooftop identification from a set of images acquired at different dates in a cross-validation approach. The proposed system generates robust solutions (kappa values > 0.75 for stage 1 and > 0.4 for stage 2) despite the statistical differences between the scenes caused by land use and land cover changes coupled with variable environmental conditions, and the lack of radiometric calibration between images. Based on our results, the use of nonlinear spectral indices enhanced the spectral differences between features improving the clustering capability of standard classifiers and providing an alternative solution for multitemporal information extraction.

A multiple user sharing behaviors based approach for fake file detection in P2P environments

File pollution is a problem that is threatening security and availability in peer-to-peer environments, and could eventually be fatal to the survival of these systems. Current methods of fake file detection, which are based on trust mechanisms or file characteristics, do not consider the problems such as interference of multiple user sharing behaviors, the difficulty in obtaining large amounts of raw data, and malicious fraud immediately after version publication. This is the first time that differences in the sharing habits of users and particularly long retention time for individual files have been detected, which impact fake file detection. This paper proposes a pollution prevention model based on multiple user sharing behaviors, which reduces interference in multiple user sharing behaviors and malicious fraud immediately after version publication. It designs a low-cost implementation mechanism for use in structured peer-to-peer networks, which automatically collects a large amount of user file-sharing information and alleviates difficulties in obtaining large quantities of raw data. A parameter configuration is also given in this paper. Simulation experiments use realistic system logs and prove that this approach detects fake files accurately and quickly, thus decreasing download times of fake files, effectively guaranteeing almost 100% real file downloads and resisting file pollution.

Abstract 2017: Key pathways that are altered at both gene copy number and transcript levels in ovarian cancer: An integrated analysis of TCGA data

Abstract Background One of the three cancer types that have been systematically examined at genomic and epigenomic levels by The Cancer Genome Atlas (TCGA) program is ovarian cancer, which represents a major health issue for women world-wide. Currently, a major challenge is to interrogate the vast amount of raw data to reveal fundamental knowledge regarding the etiology of these cancers. One of our goals is to identify genes and pathways that are jointly altered in terms of both gene copy number and transcript levels. We hypothesize that these genes and pathways represent the critical genomic and transcriptomic events that are important for cancer maintenance. Method Paired array Comparative Genomic Hybridization (aCGH) and transcriptome data from 345 ovarian cancer samples were obtained from TCGA. The majority of these tumors (263) are grade 3 serous adenocarcinoma. Both expression and aCGH data were lowess normalized and aCGH data further subjected to the circular binary segmentation (CBS) algorithm. Spearman's rank correlation (Rho) was computed on 15,553 genes detected for both aCGH and expression array to investigate the association between copy number and expression in the 345 ovarian tumor samples. The genes were then ranked based on Rho and significantly enriched pathways were indentified with highly ranked genes using gene set enrichment analysis (GSEA) algorithm in 380 pathways from Kyoto Encyclopedia of Genes and Genomes (KEGG) and BioCarta databases. Result Among the 15,553 genes, 5,486 genes in the cytobands of 19p13.11-p13.2, 8q24.3, 4p16.3, 16p13.3, 15q22.31, 12q24.31 and 10q24.32 exhibited high gene dosage-expression correlations (FDR < 0.1%, Rho > 0.4). Among the 5,486 dosage sensitive genes, 142 were pre-defined as cancer genes in Cancer Gene Census database. These genes include BRCA2 (Correlation P< 5.65E-18), PIK3CA (P<5.44E-37), PTEN (P<1.30E-24), AKT1 (P<1.33E-33), AKT2 (P<6.78E-17), MSH6 (P<1.18E-16), STK11 (P<6.47E-18), MLH1 (P<1.23E-24), MSH2 (P<2.63E-20), KRAS (P < 8.33E-39), and NF1 (P<2.56E-19). Further GSEA pathway analysis revealed a number of cancer related pathways that were highly enriched with dosage sensitive genes. These pathways included PI3K/Akt/mTOR pathway, Wnt-signal pathway, MAPK pathway, VEGF pathway, RAS pathway, and insulin signaling pathway. Conclusion Our integrated analysis of gene copy number and transcriptome data of ovarian cancer data in TCGA revealed key pathways that may be critical for ovarian cancer development and maintenance. Further analyses incorporating other information, including clinical data, will shed further light on the genetic and molecular basis of ovarian cancer pathophysiology and help identify the key targets for therapeutic intervention. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 101st Annual Meeting of the American Association for Cancer Research; 2010 Apr 17-21; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2010;70(8 Suppl):Abstract nr 2017.

From the 6 Ps of Planning to the 4 Ds of Digitization: Difficulties, Dilemmas, and Defective Decision Making

Interface problems have been cited as critical factor in the suboptimal performance of a large-scale digital command and control system. A live field study involving fully functioning Brigade and Battlegroup headquarters was observed. More than 3,000 communications events were extracted and analyzed in terms of their quantity, direction, and content. The effect of the interface problem was pronounced. Voice mediated communications (conducted by radio and avoiding the interface entirely) were superior at converting “data” into “information.” In cases where the interface was relied upon, users seized on a highly simplistic comms. facility and put it to use in ways that were not anticipated. The findings added significant value in terms of the phased, real-world delivery of this system. Very little human factors analysis had been performed previously. The current analysis, therefore, was able to provide a considerable amount of insight and justification for further design iterations. The take-away message: Neither networked technology nor the large quantities of raw data carried by it are sufficient to guarantee successful naturalistic decision making.

A new method to simultaneously estimate the radius of a cylindrical object and the wave propagation velocity from GPR data

We present a new method to simultaneously estimate cylindrical object radius ( R) and electromagnetic (EM) wave propagation velocity ( v) from ground penetrating radar (GPR) data. R estimation methods have been investigated since the middle of the previous decade, but studies have become more intensive and important over the last several years since they increase the utility of GPR data and enable new GPR applications. Since existing methods, according to the author's best knowledge, are based on a priori known v, the proposed method has an advantage: it eliminates the measurement of v and its influence on R estimation quality. Estimating v accurately results in better soil characterisation. Three steps are used to simultaneously estimate v and R. First, using the extracted raw data, the coordinates of the hyperbola apex ( x 0, t 0) are estimated. Second, the boundary speed ( v 0) is estimated, based on the previous results. In the final step, v is reduced from v 0 to a predefined v min. From the analysis of propagation velocity choice criterion, an optimal v is chosen, which is used to calculate a unique R. This proposed method is a nonlinear least squares fitting procedure. The method is implemented and verified, using data collected under real conditions, in a Matlab environment. A comparison of the proposed and existing methods shows that the new method is significantly more accurate and robust with regard to noise and the amount of raw data.

The smallest cells pose the biggest problems: high-performance computing and the analysis of metagenome sequence data

New high-throughput DNA sequencing technologies have revolutionized how scientists study the organisms around us. In particular, microbiology - the study of the smallest, unseen organisms that pervade our lives - has embraced these new techniques to characterize and analyze the cellular constituents and use this information to develop novel tools, techniques, and therapeutics. So-called next-generation DNA sequencing platforms have resulted in huge increases in the amount of raw data that can be rapidly generated. Argonne National Laboratory developed the premier platform for the analysis of this new data (mg-rast) that is used by microbiologists worldwide. This paper uses the accounting from the computational analysis of more than 10,000,000,000 bp of DNA sequence data, describes an analysis of the advanced computational requirements, and suggests the level of analysis that will be essential as microbiologists move to understand how these tiny organisms affect our every day lives. The results from this analysis indicate that data analysis is a linear problem, but that most analyses are held up in queues. With sufficient resources, computations could be completed in a few hours for a typical dataset. These data also suggest execution times that delimit timely completion of computational analyses, and provide bounds for problematic processes.

Integration of OLAP and data mining for analysis of results from dependability evaluation experiments

This paper proposes the application of On-Line Analytical Processing (OLAP) and data mining approaches to analyse the large amount of raw data collected in fault injection campaigns and dependability benchmarking experiments. We use data warehousing technologies to store raw results from different experiments in a multidimensional structure where raw data can be analysed by means of OLAP tools. Moreover, we present an approach for identifying the key infrastructural factors determining the behaviour of computer systems in the presence of faults by the application of data mining methods on the data sets. Results obtained with the proposed techniques identified important factors impacting performance and dependability that could not have been revealed solely by the benchmark measures.

COIFLET BASED METHODS FOR RANGE IMAGE PROCESSING

In industry applications, the range images are generally huge points arrays and are additively noised. They usually represent surfaces of 3D objects and are used for reverse engineering process in CAD/CAM domains. To compute the geometrical model of each surface present in the range image, we denoise and sub-sample the raw range data. Denoising allows us to avoid the adverse effects of the noise on the obtained result. Sub-sampling the raw range data leads to a low image processing overheads like those of segmentation process. Based on interpolation properties of particular wavelets named coiflets, we propose a method for smoothing noisy range images. The smoothed image keeps invariant the "topological characteristics" of the represented surfaces. Thereafter, we propose a method for sub-sampling dense range images which leads to the reduction of the amount of raw data by a factor of four. This method eliminates the "redundant" information, thus the obtained result describes the essential details (as the shape of the physical surface) of the initial range image. The smoothing and sub-sampling methods are designed to be easily integrated in any reconstruction algorithm to improve its result and reduce its overhead in spite of its high complexity.

A sequential injection electronic tongue employing the transient response from potentiometric sensors for anion multidetermination

Intelligent and automatic systems based on arrays of non-specific-response chemical sensors were recently developed in our laboratory. For multidetermination applications, the normal choice is an array of potentiometric sensors to generate the signal, and an artificial neural network (ANN) correctly trained to obtain the calibration model. As a great amount of information is required for the proper modelling, we proposed its automated generation by using the sequential injection analysis (SIA) technique. First signals used were steady-state: the equilibrium signal after a step-change in concentration. We have now adapted our procedures to record the transient response corresponding to a sample step. The novelty in this approach is therefore the use of the dynamic components of the signal in order to better discriminate or differentiate a sample. In the developed electronic tongue systems, detection is carried out by using a sensor array formed by five potentiometric sensors based on PVC membranes. For the developed application we employed two different chloride-selective sensors, two nitrate-selective sensors and one generic response sensor. As the amount of raw data (fivefold recordings corresponding to the five sensors) is excessive for an ANN, some feature extraction step prior to the modelling was needed. In order to attain substantial data reduction and noise filtering, the data obtained were fitted with orthonormal Legendre polynomials. In this case, a third-degree Legendre polynomial was shown to be sufficient to fit the data. The coefficients of these polynomials were the input information fed into the ANN used to model the concentrations of the determined species (Cl-, NO3- and HCO3-). Best results were obtained by using a backpropagation neural network trained with the Bayesian regularisation algorithm; the net had a single hidden layer containing three neurons with the tansig transfer function. The results obtained from the time-dependent response were compared with those obtained from steady-state conditions, showing the former superior performance. Finally, the method was applied for determining anions in synthetic samples and real water samples, where a satisfactory comparison was also achieved.

Joining high-throughput technology with in silico modelling advances genome-wide screening towards targeted discovery

Genome research has entered the functional evaluation phase now and high-throughput (HT) methods provide an enormous amount of raw data for that purpose. However, functional verification still requires experimental regimens not suitable for application in a HT style, requiring an efficient discovery and selection process pinpointing biological mechanisms and processes for subsequent targeted verification. Regulatory networks and underlying molecular mechanisms can now be deduced through the interpretation of HT-data in the context of biologic knowledge. Computational models of promoter structures are suitable for genome-wide searches and a number of recent examples demonstrate their usefulness in prediction and selection of functional targets for experimental verification.

QCDgrid: A Grid Resource for Quantum Chromodynamics

Quantum Chromodynamics (QCD) is an application area that requires access to large supercomputing resources and generates large amounts of raw data. The UK's national lattice QCD collaboration UKQCD currently stores and requires access to around five Tbytes of data, a figure that is growing dramatically as the collaboration's purpose built supercomputing system, QCDOC [P.A. Boyle, D. Chen, N.H. Christ, M. Clark, S.D. Cohen, C. Cristian, Z. Dong, A. Gara, B. Joo, C. Jung, C. Kim, L. Levkova, X. Liao, G. Liu, R.D. Mawhinney, S. Ohta, K. Petrov, T. Wettig and A. Yamaguchi, “Hardware and software status of QCDOC, arXiv: hep-lat/0309096”, Nuclear Physics. B, Proceedings Supplement, Vol. 838, pp. 129–130, 2004. See: http://www.ph.ed.ac.uk/ukqcd/community/qcdoc/; P.A. Boyle, D. Chen, N.H. Christ, M.A. Clark, S.D. Cohen, C. Cristian, Z. Dong, A. Gara, B. Joo, C. Jung, C. Kim, L.A. Levkova, X. Liao, R.D. Mawhinney, S. Ohta, K. Petrov, T. Wettig and A. Yamaguchi, “Overview of the QCDSP and QCDOC computers”, IBM Journal of Research and Development, Vol. 49, No. 2/3, p. 351, 2005] came into full production service towards the end of 2004. This data is stored on QCDgrid, a data Grid currently composed of seven storage elements at five separate UK sites.

MS-Express: data-extracting and -processing software for high-throughput experimentation with mass spectrometry.

High-throughput experiments (HTE) result in large amounts of raw data that have to be evaluated for sample classification. Especially mass spectrometry, a widely used detection method in catalytic HTE applications, produces enormous amounts of data. In the past few years, in catalysts research, several test rigs based on mass spectrometric detection have been independently reported by different groups. In a typical HTE, the catalysts are tested sequentially; the recording of the scans, however, occurs continuously. For this reason, the scans of interest have to be extracted from the raw data, and scans belonging to the same sample have to be averaged in a tedious procedure before further processing. In this publication, we present our custom-designed software MS-Express (mass spectrometry data-extracting and -processing software), an efficient tool for HTE MS data evaluation. MS-Express not only sorts the data, it also establishes statistical significance with the help of reference and blank data and provides concise information about abundance and intensity distributions of expected peaks. A special feature is that the program also reports unexpected MS signals, which potentially lead to unexpected discoveries.

An ensemble method for gene discovery based on DNA mi-croarray data

The advent of DNA microarray technology has offered the promise of casting new insights onto deciphering secrets of life by monitoring activities of thousands of genes simultaneously. Current analyses of microarray data focus on precise classification of biological types, for example, tumor versus normal tissues. A further scientific challenging task is to extract disease-relevant genes from the bewildering amounts of raw data, which is one of the most critical themes in the post-genomic era, but it is generally ignored due to lack of an efficient approach. In this paper, we present a novel ensemble method for gene extraction that can be tailored to fulfill multiple biological tasks including (i) precise classification of biological types; (ii) disease gene mining; and (iii) target-driven gene networking. We also give a numerical application for (i) and (ii) using a public microarrary data set and set aside a separate paper to address (iii).

Economic Change and the National Question in Twentieth-Century Europe. Edited by Alice Teichova, Herbert Matis, and Jaroslav Patek. Cambridge: Cambridge University Press, 2000. Pp. xvi, 433. $74.95.

The connection between nationalism and economic development is an important subject, and the contributors to the volume before us are to be commended for tackling it. But the significance of contributions to edited volumes in humanities and the social sciences rarely extends beyond their symbolic function—of serving as memorabilia to the rites and ceremonies in which scholarly conviviality finds its chief expression. Thus they are forgotten almost as soon as they appear in print, similarly to the elegant menus of formal dinners and wedding invitations, and for all intents and purposes are lost to the world of learning. Their only chance to escape this sad fate in most cases depends on the clarity, originality, and persuasiveness of the editors' vision, which may claim and hold the reader's attention, while creating a conceptual framework within which each individual essay acquires an added meaning. The editors of Economic Change and the National Question in Twentieth-Century Europe fail to provide such a framework, and the result is a collection of historical trivia with no more intellectual interest than any limited amount of raw data awaiting an interpreter.

Designing efficient user-friendly biological data management systems.

BIOINFORMATICS can refer to almost any collaborative effort between biologists or geneticists and computer scientists and thus covers a wide variety of traditional computer science domains including data modeling, data retrieval, data mining, data integration, data managing, data warehousing, data cleaning, ontologies, simulation, parallel computing, agent-based technology, grid computing, and visualization. However, applying each of these domains to biomolecular and biomedical applications raises specific and unexpectedly challenging specific research issues. The design of a biological data management system relies on the access and exploitation of information related to diseases, disorders, and condition. This information is available at multiple data sources and requires sophisticated tools for its access and analysis. Biological is growing at a rate unseen since the earliest days of the field. Gene sequencing robots, new experimental methodologies and online data collection devices are causing exponential growth in the amount of raw data on the web that is available to the life scientists. Life scientists need to exploit transparently these large datasets with various new applications to analyze, mine, cluster, and visualize this wealth of information. A transparent biological data management system should provide life scientists the ability to access data and applications despite the lack of explicit knowledge about where the data are stored, how they are structured, where the application is running. Many systems have been developed since the meetings on the Interconnection of Molecular Biology Databases (the first of the series was organized at Stanford University in the San Francisco Bay Area, August 9–12, 1994) and the list of queries of the DOE report on Genome Informatics (Robbins, 1993). Although successful, these systems are often limited and fail to meet all users’ needs while the needs and the problems to address became significantly more complex. During their development and usage, existing approaches collected fruitful experience. The analysis of the past experiences should benefit the research community. It is time to take some distance and try to get the perspective provided by the accurate insight into the specific problem being addressed by each system, why the particular architecture was chosen, its strengths, and any weaknesses it may have, to evaluate them and provide an overall summary of these approaches, and their characteristics (advantages and disadvantages). The diversity of data sources and the multiple of applications often distributed on the Internet raise complex issues related to integration. Traditional integration approaches are typically not addressing the two dimensions of the problem: multi-database systems, mediations and warehouses are data-driven whereas agent architectures (CORBA), Web services, and more recently grids are application-oriented. New approaches integrating both data and applications with the flexibility needed to accommodate life scientists are still needed. The problem is made more complex by the semantic mismatches between scientific resources. Information about scientific objects (e.g., a sequence, a gene) is typically spread over multiple data sources, each providing a different identifier for the object. A biological data management system must integrate them all and reconcile these different identifiers in order to provide life scientists a transparent access to each scientific object. Existing efforts to formalize keys for scientific objects, data formats, and

Study on Airport Change Detection

Change detection is playing more and more important role in remote sensing information processing. However, there are two aspects of shortcoming in existing change detection algorithms. One is that too large amount of raw data is usually needed to implement the change detection. The other is that it is sometimes difficult to acquire enough amount of quantitative information. In this paper, the runway in airport is detected by taking Hough transform in single-band images. The region of interest is then located by region-growing. After a coarse step of calibration, a judgement is made on whether there is change occurring within ROI based on the edge-correlation. Finally, the elastic mask matching procedure is adopted to make an explanation to the changes detected. Satisfactory results are obtained in the experiments.