Filtering Of Raw Data Research Articles

From data to reliable conclusions: Identification and comparison of persistent micropollutants and transformation products in 37 wastewater samples by non-target screening prioritization

In this study, micropollutants in wastewater effluents were prioritized by monitoring the composition of influent and effluent wastewater by liquid chromatography – high-resolution mass spectrometry (LCHRMS) non-target screening (NTS) analysis. The study shows how important data pre-processing and filtering of raw data is to produce reliable NTS data for comparison of compounds between many samples (37 wastewater samples) analyzed at different times. Triplicate injections were critical for reducing the number of false-positive detections. Intensity drift corrections within and between batches analyzed months apart made peak intensities comparable across samples. Adjustment of the feature detection threshold was shown to be important, due to large intensity variations for low abundance compounds across batches. When the threshold correction cut-offs, or the filtering of relevant compounds by the occurrence frequency, were too stringent, a high number of false positive transformation products (TPs) were reported. We also showed that matrix effect correction by internal standards can over- or under-correct the intensity for unknown compounds, thus the TIC matrix effect correction was shown as an additional tool for a retention time dependent matrix effect correction. After these preprocessing and filtering steps, we identified 78 prioritized compounds, of which 36 were persistent compounds, defined as compounds with a reduction in peak intensity between influent and effluent wastewater <50%, and 13 compounds were defined as TPs because they occurred solely in the effluent samples. Some examples of persistent compounds are 1,3-diphenylguanidine, amisulpride and the human metabolites from losartan, verapamil and methadone. To our knowledge, nine of the identified TPs have not been previously described in effluent wastewater. The TPs were derived from metoprolol, fexofenadine, DEET and losartan. The screening of all identified compounds in effluent samples from eight wastewater treatment plants (WWTPs) showed that potential drugs of abuse, anti-psychotic and anti-depressant drugs were predominant in the capital city region, whereas the anti-epileptic agents and agricultural pesticides were dominant in more rural areas.

Object detection neural network improves Fourier ptychography reconstruction.

High resolution microscopy is heavily dependent on superb optical elements and superresolution microscopy even more so. Correcting unavoidable optical aberrations during post-processing is an elegant method to reduce the optical system's complexity. A prime method that promises superresolution, aberration correction, and quantitative phase imaging is Fourier ptychography. This microscopy technique combines many images of the sample, recorded at differing illumination angles akin to computed tomography and uses error minimisation between the recorded images with those generated by a forward model. The more precise knowledge of those illumination angles is available for the image formation forward model, the better the result. Therefore, illumination estimation from the raw data is an important step and supports correct phase recovery and aberration correction. Here, we derive how illumination estimation can be cast as an object detection problem that permits the use of a fast convolutional neural network (CNN) for this task. We find that faster-RCNN delivers highly robust results and outperforms classical approaches by far with an up to 3-fold reduction in estimation errors. Intriguingly, we find that conventionally beneficial smoothing and filtering of raw data is counterproductive in this type of application. We present a detailed analysis of the network's performance and provide all our developed software openly.

Next Generation Sequencing-Data Analysis for Cellulose- and Xylan-Degrading Enzymes from Pome Metagenome

Metagenomic DNA library from palm oil mill effluent (POME) was constructed and subjected to high-throughput screening to find genes encoding cellulose- and xylan-degrading enzymes. DNA of 30 positive fosmid clones were sequenced with next generation sequencing technology and the raw data (short insert-paired) was analyzed with bioinformatic tools. First, the quality of 64,821,599 reverse and forward sequences of 101 bp length raw data was tested using Fastqc and SOLEXA. Then, raw data filtering was carried out by trimming low quality values and short reads and the vector sequences were removed and again the output was checked and the trimming was repeated until a high quality read sets was obtained. The second step was the de novo assembly of sequences to reconstruct 2900 contigs following de Bruijn graph algorithm. Pre-assembled contigs were arranged in order, the distances between contigs were identified and oriented with SSPACE, where 2139 scaffolds have been reconstructed. 16,386 genes have been identified after gene prediction using Prodigal and putative ID assignment with Blastp vs NR protein. The acceptable strategy to handle metagenomic NGS-data in order to detect known and potentially unknown genes is presented and we showed the computational efficiency of de Bruijn graph algorithm of de novo assembly to 21 bioprospect genes encoding cellulose-degrading enzymes and 6 genes encoding xylan-degrading enzymes of 30.3% to 100% identity percentage.

An innovative Primary Surface Profile-based three-dimensional pavement distress data filtering approach for optical instruments and tilted pavement model-related noise reduction

The automatic pavement management system has the advantage of providing reliable pavement maintenance and rehabilitation strategies aiming at prolonging existing pavement service life. Therefore, the quality of noise reduction results, which is an unavoidable process of automatic pavement assessment evaluation, has a significant influence on the reliability of pavement maintenance operations suggested. The primary purpose of this paper is to propose an innovative three-dimensional (3D) pavement image-based data filtering protocol, thereby maintaining a highly functional pavement surface. First, a 3D pavement depth data collection system was developed using laser light and a charge-coupled device camera. After that, based on the analysis of Positive Noise and Negative Noise, which are optical instrument-related noises, and tilted pavement model noise, the Primary Surface Profile (PSP)-based raw data filtering approach was proposed which aims at improving the noise reduction quality. Validation experiments were conducted using both the proposed approach and the traditional data filtering method, and the results show that for the not tilted pavement surface model, the PSP-based filter method can achieve the highest noise reduction value (NRV), whereas for the tilted pavement surface model, with a slightly lower NRV than that of biphasic standard deviation average filtering, which demonstrates that the proposed data filtering method has self-adaptive and robust data filter advantages which can be incorporated into a high-performance pavement performance evaluation and management system.

New Software for Processing of LWD Extradeep Resistivity and Azimuthal Resistivity Data

Summary In petroleum exploration, reservoir navigation is used for reaching a productive reservoir and placing the borehole optimally inside the reservoir to maximize production. For proper well placement, it is necessary to calculate in real-time the parameters of the formation we are drilling in and the parameters of formations we are approaching. On the basis of these results, a decision to change the direction of drilling could be made. Modern logging-while-drilling (LWD) extra-deep and azimuthal resistivity tools acquire multicomponent, multispacing, and multifrequency data that provide sufficient information for resolving the surrounding formation parameters. These tools are generally used for reservoir navigation and real-time formation evaluation. However, real-time interpretation software is very often based on simplified resistivity models that can be inadequate and lead to incorrect geosteering decisions. The core of the newly developed software is an inversion algorithm based on transversely isotropic layered Earth with an arbitrary number of layers. The following model parameters are determined in real time: horizontal and vertical resistivities and thickness of each layer, formation dip, and azimuth. The inversion algorithm is based on the method of the most-probable parameter combination. The algorithm has good performance and excellent convergence because of its enhanced capability of avoiding local minima. This capability enables interpretation of real-time resistivity data, including azimuthal and extra-deep measurements. A graphical user interface (GUI) was developed to provide an interactive environment for each stage of the resistivity data interpretation process: preview of input resistivity logs, initial preprocessing and filtering of raw data, creation of initial guess, running inversion and viewing inversion results, and quality-control indicators. Applications of the developed software will be shown on a series of synthetic examples and field data from the North Sea and Gulf of Mexico (GOM). This newly developed software is currently in use for real-time reservoir navigation and post-well analysis.

Next-generation gene catalogues and genomics tools focused on forestry research

Gydle is developing reference gene catalogues and genomics resources targeted at forestry research, leveraging its unique bioinformatics expertise in next-generation sequencing. Next-generation sequencing technologies have revolutionized the field of genomics and have reached the capacity to generate enough quality data to sequence and assemble thousands of genomes and gene catalogues per year. These extraordinary opportunities are currently limited by challenges in data analysis which hinder the translation from raw sequencing data to applications in research and in the field. There is an enormous gap between an instrument capacity to produce millions of reads per day and the ability to analyze and translate quickly this data into usable information and value-added sets for genomics research. One particular limitation stems from the current state of assemblers which produce crude contigs and scaffolds that need considerable editing in order to obtain contiguous, gene-oriented, full-length, artifact-free annotated sequences desirable for the production of quality tools for genomics research. Researchers also lack tools to rapidly integrate and visualize their own data and collaborate in a user-friendly way. Finally, the ability to produce gene catalogues for thousands of samples calls for a new generation of tools to conduct large-scale comparative genomics studies between and within species. Gydle has developed a complete suite of proprietary bioinformatics tools to address many of the challenges associated with next-gen sequence analysis. A survey of these tools will be presented, which include innovative raw data filtering, rapid and accurate DNA alignment and gene-oriented assembly software, automated and interactive visualization tools for sequence correction and annotation. These tools enable users to process from start to finish a typical next-gen sequencing run in a user-friendly way in a matter of minutes to hours instead of weeks. Users can then visualize their results, edit sequence assemblies and annotations and identify features such as sample-specific expression patterns, SNPs, unspliced introns and alternative splicing events, handling information from hundreds of millions of reads on their workstation or laptop computer. We intend to make these bioinformatics tools available commercially in the near future. Applying these bioinformatics tools, we combined sequencing data generated by Gydle with publicly available data to produce curated reference gene catalogues for species of interest to the forestry industry, which currently 35 tree species including 7 pinus, 3 picea, 3 eucalyptus, 2 poplar and 5 oak species. Gydle gene catalogues also include over 50 other plant species of agricultural and medicinal interest as well as gene catalogues for species ecologically related to trees such as insects, endophytes, fungi, and microbiota. We intend to make these catalogues available commercially in addition to offering the possibility for researchers to integrate their own data with these catalogues privately. We used these reference genes catalogues to produce species-specific and cross-species value-added genomics resources such as: - reference gene sequences covering biochemical pathways involvedin growth, wood formation, climate adaptation, biofuel production, and secondary metabolites of interest to pest resistance and human health - curated amino-acid databases to improve proteomic and metabolomic identifications - SNP sets and designs of high-throughput genotyping assays such as Illumina GoldeGate/Infinium for marker discovery and genomic selection. - reference sets and tools for gene expression studies by RNA-seq and microarrays. Examples of gene catalogues, cross-species analysis and value-added sets will be presented.

Tomographic Approach For Tropospheric Water Vapor Detection

AbstractThe technical successes in radio navigation and the availability of nu-merical algorithms have promoted the implementation of GPS-technology to atmo-spheric sciences. The tomographical contribution of Global Satellite Navigation Sys-tems (GNSS) is possible due to the methods of high precision detection of tropospheric delays of navigation signals from satellites to receivers. The principal specific char-acter in initial constraints, data collection and assimilation methods, the obtaining of final numerical results and their interpretation make the continuation of the success story for GPS-tomography very challenging. The authors use numerical simulation as the most time- and cost-efficient way to study different processes related to tro-pospheric water vapor tomography. This paper tends to give a short overview about some known methods in GPS-tomography for detection, monitoring and modeling of the tropospheric water vapor. The possible mathematical approach to the construc-tion of virtual network of ground-based sensors (GPS-receivers) for a real geographical location and discretization of the troposphere, also some aspects of raw data filtering and analysis are described. Output of tomographical modelling of the troposphere can be used to improve the results of large-scale numerical weather prediction models and also real-time navigation. The questions of voxel geometry and methods of data processing are supposed to be the key questions in constructing an effective network of GPS-receivers for water vapor tomography.

Mass Spectrometry Based Sensor Strategies for the Authentication of Oysters According to Geographical Origin

This study was undertaken to investigate the relevance of using the pyrolysis-MS (Py-MS) technique to discriminate the production area of oysters harvested over two years and to assess from the data of the second year of harvest the potential of an alternative MS-based technique, the solid phase microextraction-MS (SPME-MS), to perform this discrimination. Oysters were harvested in various areas of France, and models of discrimination according to harvest season were built from Py-MS fingerprints and from virtual SPME-MS fingerprints obtained by summing the mass spectra generated by the SPME-GC-MS system. The treatment of the Py-MS data by a 21-12-3 artificial neural networks led to a correct classification of only 89.2% of the oyster samples according to shoreline. The misclassifications thus did not allow use of the Py-MS technique as a relevant tool for authentication of oyster origin. The assessment of the potential of the virtual SPME-MS fingerprints to discriminate the production area of oysters was undertaken on a part of the sample set. The virtual SPME-MS data were pretreated according to two methods, filtering of raw data (FRD) and comprehensive combinatory standard correction (CCSC), a recently developed chemometric method used for the correction of instrumental signal drifts in MS systems. The results obtained with the virtual SPME-MS fingerprints are promising because this technique, when the data were pretreated by the CCSC method, led to a successful discrimination of the oyster samples not only according to shoreline but also according to production region. This study confirms that an efficient correction method (CCSC) of instrumental drifts can considerably increase the discriminative information contained in the volatile fraction of food products.

Correction of the data generated by mass spectrometry analyses of biological tissues: Application to food authentication

The objective of the work was to assess the relevance for the authentication of food of a novel chemometric method developed to correct mass spectrometry (MS) data from instrumental drifts, namely, the comprehensive combinatory standard correction (CCSC). Applied to gas chromatography (GC)–MS data, the method consists in analyzing a liquid sample with a mixture of n internal standards and in using the best combination of standards to correct the MS signal provided by each compound. The paper focuses on the authentication of the type of feeding in farm animals based on the composition in volatile constituents of their adipose tissues. The first step of the work enabled on one hand to ensure the feasibility of the conversion of the adipose tissue sample into a liquid phase required for the use of the CCSC method and on the other hand, to determine the key parameters of the extraction of the volatile fraction from this liquid phase by dynamic headspace. The second step showed the relevance of the CCSC pre-processing of the MS fingerprints generated by dynamic headspace-MS analysis of lamb tissues, for the discrimination of animals fed exclusively with pasture ( n = 8) or concentrate ( n = 8). When compared with filtering of raw data, internal normalization and correction by a single standard, the CCSC method increased by 17.1-, 3.3- and 1.3-fold, respectively, the number of mass fragments which discriminated the type of feeding. The final step confirmed the advantage of the CCSC pre-processing of dynamic headspace-gas chromatography-MS data for revealing molecular tracers of the type of feeding those number ( n = 72) was greater when compared to the number of tracers obtained with raw data ( n = 42), internal normalization ( n = 63) and correction by a single standard ( n = 57). The relevance of the information gained by using the CCSC method is discussed.

Measurement Noise and Stratospheric Gravity Wave Characteristics Obtained from GPS Occultation Data

Tsuda and Hocke (2002) reduce the amount of filtering applied to raw radio occultation data in order to increase the vertical resolution of GPS based temperature retrievals in the stratosphere. The authors discuss gravitywave characteristics obtained from these optimised retrievals, including characteristics of vertical wave number spectra for vertical wavelengths below 2 km, and the meridional distribution of potential energies for such short wavelengths. The filtering of raw data is required because real measurements contain noise. In this note, we discuss the role of measurement noise for gravity wave characteristics obtained from dry temperature retrievals of GPS based radio occultation soundings. We show that for realistic noise levels, temperature fluctuations can be safely interpreted as originating from small scale atmospheric waves in the altitude region below 30 km and for wavelengths greater than 2 km, with the detailed limits depending on noise and retrieval characteristics. An optimising procedure as the one applied by Tsuda and Hocke, however, may raise noise related effects to the level of atmospheric waves for short vertical wavelengths. We further show that conclusions for vertical wavelengths less than 2 km, if at all, can only be drawn after taking the actual noise characteristics of the observations and the details of the filter procedures applied in the retrieval into account.