XML Data Files Research Articles

Analisis Dynamic ETL Incremental Load untuk Data Integration Datawarehouse

Data integration is a combination of techniques and businesses that are used to collect data from different sources into useful and valuable information ETL process that includes extracting data from various data sources, transforming data to form and calculate data and load data on target storage, to support data warehouse need. Based on organizations and industries that have implemented data warehouse, the problem that generally arises regarding data load is the difficulty in integrating different data sources, how to form data from various data formats into uniform data, how to integrate data delta between data sources and target storage in an incremental load process so that this data synchronization process can be carried out continuously and relatively faster. ETL process requires a platform that can facilitate data integration needs, in order to run this process. SSIS (SQL Server Integration Service) is a Data Integration platform to build an enterprise-level data integration and solutions for data transformation. Integration Service can extract and change data (transform) from various sources such as XML data files, flat files, APIs, and relational data sources, and then load into one or several destination data. According to the problem related to data load, we will examine how the solution model uses SSIS for the ETL process. This paper proposed an ETL Architecture model by completing the ETL process for full & incremental load extraction and the original data layer.

Data transformation technology of 3D ship structure model for approval

Objectives Chinese ship design and approval institutions usually use different 3D design platforms, which can cause the problems of unsmooth model transfer and ineffective data conversion. This paper addresses the data conversion techniques of 3D ship structure models to ascertain whether a converted model can be directly used by classification societies for approval. Methods By analyzing the model data of the CAD and CAE systems, the required data for generating an approval-oriented 3D hull structure model is summarized. The storage format of the required data and the interface between the stored data and common 3D software platforms are introduced. Taking a cargo tank section of a ship as an example, the required data of the 3D model of the target section is stored in an XML data file. Based on this data file, the design model and FE analysis model of the target section are reconstructed through CATIA and Patran software, and equipped with a secondary developed interface. Results The reconstructed model has been submitted to Bureau Veritas (BV) for direct approval of the structural design. Conclusions The data conversion method proposed in this paper simplifies model transfer among different platforms, and can provide valuable references for realizing the paper-free approval of 3D ship models.

Handling distributed XML queries over large XML data based on MapReduce framework

With the increase in available extensible markup language (XML) documents, numerous approaches to querying have been proposed in the literature. XPath queries and Twig pattern queries are the two basic approaches, directly affecting the efficiency of XML operations. Distributive manipulation of massive XML data is challenging. This paper aims to develop an efficient distributed XML query processing method using MapReduce, which simultaneously processes several queries on large volumes of XML data. First, we split up a large-scale XML data file into file-splits and put them in a distributed storage system. Then, we present an efficient algorithm to compute different fragments of the document tree using the MapReduce framework in parallel. In order to efficiently handle a large amount of XML data, we built a partition index and used a random access mechanism for specific queries. The experiment results show that our proposed approach is efficient with good scalability.

Standardized Storage of Sports Data Based on XML

Sports data feature strong timeliness, multiple types, many specifications, large data amount and very complex storage. The information technology has been more widely applied in sports data storage, and also achieved a wealth of research findings. In terms of sports data, the paper proposes a standardized data storage solution based on XML, makes a brief description of the methods and principles of various data concerning sports training programs, and then puts forward the logical structure of data storage. The XML data file system and database system grounded on XML storage sports in- dustry is simply discussed and a high efficient, standardized storage solution is raised for sports data.

Fast and Efficient XML Data Access for Next-Generation Mass Spectrometry.

MotivationIn mass spectrometry-based proteomics, XML formats such as mzML and mzXML provide an open and standardized way to store and exchange the raw data (spectra and chromatograms) of mass spectrometric experiments. These file formats are being used by a multitude of open-source and cross-platform tools which allow the proteomics community to access algorithms in a vendor-independent fashion and perform transparent and reproducible data analysis. Recent improvements in mass spectrometry instrumentation have increased the data size produced in a single LC-MS/MS measurement and put substantial strain on open-source tools, particularly those that are not equipped to deal with XML data files that reach dozens of gigabytes in size.ResultsHere we present a fast and versatile parsing library for mass spectrometric XML formats available in C++ and Python, based on the mature OpenMS software framework. Our library implements an API for obtaining spectra and chromatograms under memory constraints using random access or sequential access functions, allowing users to process datasets that are much larger than system memory. For fast access to the raw data structures, small XML files can also be completely loaded into memory. In addition, we have improved the parsing speed of the core mzML module by over 4-fold (compared to OpenMS 1.11), making our library suitable for a wide variety of algorithms that need fast access to dozens of gigabytes of raw mass spectrometric data.AvailabilityOur C++ and Python implementations are available for the Linux, Mac, and Windows operating systems. All proposed modifications to the OpenMS code have been merged into the OpenMS mainline codebase and are available to the community at https://github.com/OpenMS/OpenMS.

Extended Term tXSchema uses SAX Parsing Partially

Wide Web Consortium recommendation for XML Schema illustrates the structure and data types of XML document. tXSchema could be a framework for the creation and validation of time variable documents. All parts of tXSchema changes over time to replicate changes reference world of information. tXSchema could be a model for making temporal Schema from base Schema, logical and Physical Annotations. We also describe how the validator can be extended (temporal validator) to validate documents in this seeming uncertain situation of data that vary across time while its corresponding schema and even its representation are also varies. Since many applications require to keep tracks of the Schema, data evaluation, which suggested a need of versioning. When system is working with huge versions of schema as well as xml data files here we are specially focusing on how to minimize processing time by using SAX parser's partial involvement with conventional DOM parser. In this paper we deal with versioning in tXSchema model, more precisely here we propose a set of Schema change primitive for the maintenance of logical and physical annotations and define their operational perspectives and also minimizes processing time.

From data to analysis: linking NWChem and Avogadro with the syntax and semantics of Chemical Markup Language

BackgroundMultidisciplinary integrated research requires the ability to couple thediverse sets of data obtained from a range of complex experiments andcomputer simulations. Integrating data requires semantically richinformation. In this paper an end-to-end use of semantically rich data incomputational chemistry is demonstrated utilizing the Chemical MarkupLanguage (CML) framework. Semantically rich data is generated by the NWChemcomputational chemistry software with the FoX library and utilized by theAvogadro molecular editor for analysis and visualization.ResultsThe NWChem computational chemistry software has been modified and coupled tothe FoX library to write CML compliant XML data files. The FoX library wasexpanded to represent the lexical input files and molecular orbitals used bythe computational chemistry software. Draft dictionary entries and a formatfor molecular orbitals within CML CompChem were developed. The Avogadroapplication was extended to read in CML data, and display molecular geometryand electronic structure in the GUI allowing for an end-to-end solutionwhere Avogadro can create input structures, generate input files, NWChem canrun the calculation and Avogadro can then read in and analyse the CML outputproduced. The developments outlined in this paper will be made available infuture releases of NWChem, FoX, and Avogadro.ConclusionsThe production of CML compliant XML files for computational chemistrysoftware such as NWChem can be accomplished relatively easily using the FoXlibrary. The CML data can be read in by a newly developed reader in Avogadroand analysed or visualized in various ways. A community-based effort isneeded to further develop the CML CompChem convention and dictionary. Thiswill enable the long-term goal of allowing a researcher to run simple“Google-style” searches of chemistry and physics and have theresults of computational calculations returned in a comprehensible formalongside articles from the published literature.Electronic supplementary materialThe online version of this article (doi:10.1186/1758-2946-5-25) contains supplementary material, which is available to authorized users.

A Query Optimization Modle and Algorithms for XML Data File

Along with the increase of XML applications, Native XML database technology is gaining the attention of the field of database, and query optimization is still one of the key research contents. This article，from a new perspective, represents the research on the relationship between XML nodes and proposes a query optimization model based on the associated elements of XML data file . In order to query the information of the related elements in the XML data file, this method takes the strategy of index; and proposed the query algorithms on the query optimization model based on associated elements.

A change detection system for unordered XML data using a relational model

The dramatic increase in the evolution of XML data available on the Internet requires a change detection system to keep track of important changes occurring during their life time. In this paper, we introduce a novel approach of detecting changes between two versions of unordered XML data stored in a traditional relational database using approaches like XRel. Most of the existing work in the area of XML change detection is mainly focused on detecting changes between two versions of XML data by constructing their Document Object Model (DOM) trees and then comparing these two tree structures based on Longest Common Sequence (LCS) using minimum edit distances. The basic tree comparison approach is not efficient in handling large XML files due to the fact that (1) an equivalent XML DOM tree will be twice as large as the original document and (2) the entire trees of both versions have to be memory resident during the comparison process. These two issues are constrained by the available main memory. In addition, existing approaches fail to detect changes among versions of XML data stored in relational databases as reverse mapping is not loss-less. We propose an efficient algorithm (XRel_Change_SQL) for detecting unordered changes between two XML data files stored in XRel as the underlying relational data model, using Structured Query Language (SQL). We compare the efficiency and quality of our change detection algorithm with existing XML change detection tools like X-Diff, DeltaXML and XANDY. We provide an experimental evaluation of the results obtained from the benchmark datasets as well as some synthetic datasets to show that our approach is highly scalable, and results in a much better efficiency and delta quality than the aforementioned approaches and tools.

The ProteoRed MIAPE web toolkit: A User-friendly Framework to Connect and Share Proteomics Standards

The development of the HUPO-PSI's (Proteomics Standards Initiative) standard data formats and MIAPE (Minimum Information About a Proteomics Experiment) guidelines should improve proteomics data sharing within the scientific community. Proteomics journals have encouraged the use of these standards and guidelines to improve the quality of experimental reporting and ease the evaluation and publication of manuscripts. However, there is an evident lack of bioinformatics tools specifically designed to create and edit standard file formats and reports, or embed them within proteomics workflows. In this article, we describe a new web-based software suite (The ProteoRed MIAPE web toolkit) that performs several complementary roles related to proteomic data standards. First, it can verify that the reports fulfill the minimum information requirements of the corresponding MIAPE modules, highlighting inconsistencies or missing information. Second, the toolkit can convert several XML-based data standards directly into human readable MIAPE reports stored within the ProteoRed MIAPE repository. Finally, it can also perform the reverse operation, allowing users to export from MIAPE reports into XML files for computational processing, data sharing, or public database submission. The toolkit is thus the first application capable of automatically linking the PSI's MIAPE modules with the corresponding XML data exchange standards, enabling bidirectional conversions. This toolkit is freely available at http://www.proteored.org/MIAPE/.

A platform to standardize, store, and visualize proteomics experimental data

With the development of functional genomics research, large-scale proteomics studies are now widespread, presenting significant challenges for data storage, exchange, and analysis. Here we present the Integrated Proteomics Exploring Database (IPED) as a platform for managing proteomics experimental data (both process and result data). IPED is based on the schema of the Proteome Experimental Data Repository (PEDRo), and complies with the General Proteomics Standard (GPS) drafted by the Proteomics Standards Committee of the Human Proteome Organization. In our work, we developed three components for the IPED platform: the IPED client editor, IPED server software, and IPED web interface. The client editor collects experimental data and generates an extensible markup language (XML) data file compliant with PEDRo and GPS; the server software parses the XML data file and loads information into a core database; and the web interface displays experimental results, to provide a convenient graphic representation of data. Given software convenience and data abundance, IPED is a powerful platform for data exchange and presents an important resource for the proteomics community. In its current release, IPED is available at http://www.biosino.org/iped2.

Improvement of engine performance using an optimization procedure

This paper presents a simple and effective approach to improve engine performance of a racing car with special requirements. Attention is focused on optimal design of the intake system, using a gradient-based approximation method of mathematical programming. Since optimization relies on accurate numerical analysis of engine processes, the sub-models and parameters needed in the analysis software are carefully determined by experiment. Subsequently, the influence of different design parameters of intake and exhaust systems on engine performance is investigated numerically. The most influencing parameters are selected to be the design variables in the optimization process. In order to improve engine power at several engine speeds, two different forms of the optimal design problem are proposed, solved, and compared as a means to identify the most appropriate one. Since the analysis software is a black-box program, the optimization procedure is implemented by employing the optimization software as a master (driver) program while the analysis software acts as the slave program. The data exchange between these programs is established by XML data files and suitable wrapper programs. The results obtained confirm the usefulness of the approach presented.

An intelligent process planning system for prismatic parts using STEP features

This paper presents an intelligent process planning system using STEP features (ST-FeatCAPP) for prismatic parts. The system maps a STEP AP224 XML data file, without using a complex feature recognition process, and produces the corresponding machining operations to generate the process plan and corresponding STEP-NC in XML format. It carries out several stages of process planning such as operations selection, tool selection, machining parameters determination, machine tools selection and setup planning. A hybrid approach of most recent techniques (neural networks, fuzzy logic and rule-based) of artificial intelligence is used as the inference engine of the developed system. An object-oriented approach is used in the definition and implementation of the system. An example part is tested and the corresponding process plan is presented to demonstrate and verify the proposed CAPP system. The paper thus suggests a new feature-based intelligent CAPP system for avoiding complex feature recognition and knowledge acquisition problems.

Mapping of industry building product model for detailed thermal simulation and analysis

Industry Foundation Classes (IFC) is an emerging data standard that promotes the interoperability of digital building product object models, or building product models (BPMs). Implementing IFC-based applications allows reducing data duplication, especially in the aspect of building geometry. With appropriate data mapping engines, an existing BPM may be readily used for performance-based simulation which requires significantly less effort compared with the traditional means. As a result, it is possible to make use of many available proprietary tools to provide additional services such as early-stage building design support. SEMPER-II (S2), which is an active, multi-domain, space-based, object-oriented design support tool for integrated building performance computing, was developed before the IFC standard was established. In order to perform building performance simulations based on the IFC compliant data models expressed through the eXtensible Markup Language (XML) based formats, an add-on data mapping engine has been developed as a front-end application in S2. The data mapping engine extracts all necessary information embedded in the XML data files, which is obtained directly from IFC compliant CAD drawings, and converts the data into the internal data format of S2.This process demonstrates a seamless link between an IFC compliant industry building product model (e.g. CAD) and the Shared Object Model (SOM) of S2 to facilitate building performance simulations.

Pedro: a configurable data entry tool for XML.

Pedro is a Java application that dynamically generates data entry forms for data models expressed in XML Schema, producing XML data files that validate against this schema. The software uses an intuitive tree-based navigation system, can supply context-sensitive help to users and features a sophisticated interface for populating data fields with terms from controlled vocabularies. The software also has the ability to import records from tab delimited text files and features various validation routines. The application, source code, example models from several domains and tutorials can be downloaded from http://pedro.man.ac.uk/.

Reusable XML technologies and the development of language learning materials

There are many CALL resources available today but often they need to be adapted for level or culture. CALL practitioners would like to reuse currently existing material rather than reinvent the wheel, but often this is not possible. Thus, they end up building CALL material, both language content and software, from scratch. This is inefficient in terms of CALL practitioners’ time and as Felix (1999) points out, there is no point ‘doing badly’ what has already been done well. Why can’t we reuse what already exists? Often the language content is hard-wired in the software and cannot be modified or the CALL material comes as an executable which is hard or impossible to change. Authoring tools can provide a degree of flexibility, but often focus on particular parts of the language learning process (e.g. interactive exercises) and the associated language content cannot, in general, easily be exported into other formats and presentation software. One solution to this problem is provided by XML technologies. They provide a strict separation between data and processing. Thus, three types of reuse are possible: reuse of the data processing engine (i.e. the XSL processing files), reuse of the language content structure (i.e. the XML data files) and reuse the linguistic resources (i.e. the language content). In this paper, an example is given of a CALL template that has been developed using XML technologies. The template provides a structure into which the language content can be slotted and a processing engine to act upon the data to create CALL material. The template was developed for the production of CALL materials for Endangered Languages (ELs), but could be used for MCTLs (Most Commonly Taught Languages) and LCTLs (Less Commonly Taught Languages) also. It has been used to develop a language learning course for Nawat (a language of El Salvador), Akan (a Ghanaian language) and Irish, demonstrating the reusability of the language content structure as well as proving the reusability of the processing engine provided by the template. A further reusable feature is the ability to create courseware in different media (Internet, CD and print) from the same source language content.