Statistical Filtering Research Articles

Simple, efficient and thorough shotgun proteomic analysis with PatternLab V.

Shotgun proteomics aims to identify and quantify the thousands of proteins in complex mixtures such as cell and tissue lysates and biological fluids. This approach uses liquid chromatography coupled with tandem mass spectrometry and typically generates hundreds of thousands of mass spectra that require specialized computational environments for data analysis. PatternLab for proteomics is a unified computational environment for analyzing shotgun proteomic data. PatternLab V (PLV) is the most comprehensive and crucial update so far, the result of intensive interaction with the proteomics community over several years. All PLV modules have been optimized and its graphical user interface has been completely updated for improved user experience. Major improvements were made to all aspects of the software, ranging from boosting the number of protein identifications to faster extraction of ion chromatograms. PLV provides modules for preparing sequence databases, protein identification, statistical filtering and in-depth result browsing for both labeled and label-free quantitation. The PepExplorer module can even pinpoint de novo sequenced peptides not already present in the database. PLV is of broad applicability and therefore suitable for challenging experimental setups, such as time-course experiments and data handling from unsequenced organisms. PLV interfaces with widely adopted software and community initiatives, e.g., Comet, Skyline, PEAKS and PRIDE. It is freely available at http://www.patternlabforproteomics.org .

Separation of monodisperse poly(3-hydroxybutyrate) particles by fractionation: Theory and practice

A new method is proposed in this article which is based on fractionation and results poly(3-hydroxybutyrate) particles of nearly monodisperse size distribution. This kind of separation is achieved by taking advantage of the phenomenon that the movement of particles triggered by a constant force in a homogeneous fluid have a velocity depending on the diameter of the particle. The procedure consists of two steps. First, a set of particles with a wide distribution of diameters is produced by using the emulsion-suspension method followed by their separation into fractions on hydrogel columns in the second step. The novel concept of statistical filters has been introduced in order to describe this phenomenon quantitatively. Additionally, mathematical models have been developed that allow the simulation and prediction of the outcome of particle separation. Our method was verified by model calculations and measurements; the results of the two approaches correlated closely. The proposed method is very advantageous for the preparation of drug carrier matrices, because it allows the close control and modeling of the kinetics of drug release.

New framework for upscaling gas-solid heat transfer in dense packing

The objective of this work is to provide a suitable upscaling framework for modeling gas-solid heat transfer in dense packings. In a macroscopic two-temperature model, the pore-scale heat transfer is modeled by determining a Nusselt number. We introduce a new surface statistical filter to extract the Nusselt number from pore-scale simulations. A thermal offset boundary condition is developed to deal with dense packings that exhibit high heat transfer. The new method is applied to sphere packings. Several pore-scale simulations over a range of Reynolds number (10–100) and solid volume fraction (0.1–0.6) are performed. The extracted Nusselt numbers show very good agreement with results available in the literature for lower solid volume fractions (0.1–0.5). We show that the Nusselt number to be used with the two-temperature model is the Nusselt number extracted using macroscopic (filtered) temperature. We propose a valid Nusselt number correlation for a dense packing of spheres. Nevertheless, the proposed scaling framework is not restricted to sphere packings.

Three-Dimensional Reconstruction of Rolling Contact Fatigue Characteristics

Focusing on the 3D topographic characteristics of rolling contact fatigue, a reconstruction method of the fatigue surface of roller based on point cloud data was proposed in this research. A 3D laser scanner was used to capture the data of point cloud on the surface of the fatigue roller. The gradient segmentation method was used to achieve segmentation of the fatigue contact surface, and the Kd-Tree algorithm in Statistical Outlier Removal filter was adopted to remove different types of noise. The greedy triangulation and hole repair and reconstruction of the curled point cloud were conducted. The experimental results showed that the segmentation accuracy of the fatigue contact surface was above 97.7%, the curling error rate of point cloud was 0.09%, and the maximum deviation of the reconstructed fatigue roller surface was 0.0199 mm. These methods can be applied to analyze the working conditions of roller specimen and contact fatigue.

Optimal parameters selection of the Frost filter based on despeckling efficiency prediction for Sentinel SAR images

Synthetic aperture radar (SAR) images have found numerous applications. However, further analysis of SAR images including interpretation, classification, segmentation, etc. is an extremely challenging task due to the presence of highly intensive speckle noise. Therefore, image despeckling is one of the main stages in preliminary SAR data processing. Over the past decades, a large number of different image despeckling techniques have been proposed ranging from local statistics filters to deep learning based ones. In this study, we analyze one of the most known and widely used local statistics Frost filter. Despeckling efficiency of the Frost filter significantly depends on the sliding window size and tuning (also called damping) factor. Here, we present a method for optimal parameters selection of the Frost filter for a given image based on despeckling efficiency prediction. Despeckling efficiency prediction for the Frost filter is carried out using a set of statistical and spectral input parameters and multilayer neural network. It is shown that such a prediction can be performed before applying image despeckling with a high accuracy and it is faster than despeckling itself. Both simulated speckled images and real-life Sentinel-1 SAR images have been used for extensive evaluation of the proposed method.

Automatic Deformation Extraction Method of Buildings in Mining Areas Based on TLS Point Clouds

Presently, the deformation extraction of buildings in mining areas using terrestrial laser scanner (TLS) point clouds is performed manually. Automatic deformation extraction holds great significance for building deformation monitoring in mining areas. Therefore, this study proposes an automatic extraction method for building deformation in mining areas using TLS point clouds. The corner points of doors and windows on the wall are considered as key points and the wall deformation in the mining area is extracted with minimal manual intervention. First, the input data were preprocessed, including 2D boundary point cloud acquisition and denoising (using a distance slope filter). Next, the key points were extracted via three steps: boundary line splitting, seed key point clustering, and key point judgment. Finally, the 3D coordinates of the key points and the relationship between the key points of the two phases were established to calculate the deformation value. The results confirmed negligible difference between the deformation value extracted using this method and the real value. Most of the errors were between −5 and 5 mm, and only a few exceeded ±5 mm; however, no error exceeded ±9 mm. The deformation value obtained using this method was almost identical to that obtained using the manual method, and the absolute error between them was below 8 mm. The performance verification of this method showed that the proposed distance slope filter removed the noise points more effectively compared to that of the traditional denoising filters, i.e., statistical and radius filters, establishing its suitability for the complex measurement environment of mining areas. During the automatic extraction of key point coordinates, the root mean square error (RMSE) values were below 2.0 mm; RMSE values during manual extraction were below 7.1 mm. The proposed method demonstrated greater stability than that of the manual extraction method.

A Fast and Effective Irregular Stripe Removal Method for Moon Mineralogy Mapper (M3)

Hyperspectral imagery (HSI) is one of the emerging tools to explore the physical properties and chemical composition of the lunar surface. Moon mineralogy mapper (M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) is the most widely used lunar HSI data set with the widest coverage and the excellent resolution; however, dense and nonperiodic stripes distributed across all bands in M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> images hinder visual interpretation as well as their use in subsequent applications. In this article, a fast destriping approach for M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> is proposed using the Hodrick–Prescott decomposition embedded in the low-rank framework (LRHP) to overcome this limitation. The integration of a statistical filter and variational model tackles the problem stemming from a lack of the correct residual information when certain pixels are corrupted in every band, thereby restoring severely degraded hyperspectral images (HSIs). Simulated and real experiments conducted on typical regions on the Moon with various levels of corruption demonstrate that the proposed LRHP rapidly achieves favorable performance against state-of-the-art approaches. Also, expanding tests on interference imaging spectrometer (IIM) data of Chang’E-1 and commonly used terrestrial remote sensing images show that LRHP has good generalization capability. Moreover, the integrated band depth (IBD) maps further verify the necessity of destriping and the high spectral fidelity of LRHP that benefits further applications.

Phase triangulation method with statistical filtering for measurements under conditions of random additive noise and a limited dynamic range of a photodetector

Phase triangulation method with statistical filtering for measurements under conditions of random additive noise and a limited dynamic range of a photodetector

Potential GDP and its factors assessment

The object of the study is the reserves of economic growth in the country on the example of Ukraine. One of the problems of such studies is the calculation of potential GDP, which is not observed, but is calculated on the basis of various methods. Also problematic is the choice of method/methods of calculating potential GDP and potential values of its factors. Any estimate of the potential value of a variable is based on one or more statistical relationships and therefore contains an element of uncertainty. In order to reduce uncertainty, 2 methods were used to determine the potential values of the components of GDP – the growth rate of employment, fixed capital and TFP (total factor productivity).&#x0D; The study used the methods of one-dimensional statistical filters Hodrick-Prescott and Baxter-King to estimate the potential values of GDP and the model of the production function to calculate potential GDP based on the potential values of its factors. The main reasons for the slowdown in Ukraine's GDP have been identified, the main of which is low capital productivity due to budget constraints. The second place in this ranking was taken by labor productivity, the last third – by TFP. Weak productivity and investment growth reinforced each other. Capital has the highest growth potential in Ukraine. Therefore, measures to stimulate capital investment, including in research and innovation and human capital, are important. Other factors that affect GDP through labor productivity and TFP are population aging, emigration, and tight lending conditions. To neutralize these factors, it is necessary to create new jobs, facilitate the conditions for obtaining loans by enterprises, stimulate advanced training and lifelong learning. The proposed approach to the separate calculation of potential values of GDP factors and their analysis find reserves for GDP growth. This provides the advantages of this method over other approaches.

Spatiotemporal wave propagation of the shale oil and gas development in the Marcellus Shale in the past one century and a half

ABSTRACT Dense shale oil and gas extraction activities have a long history in the Marcellus Shale and the investigation of their evolution draws increasing interest. The spatiotemporal analysis of these shale oil and gas extraction activities is essential to understand their evolution in space-time, and important for the estimation of their historical impacts on local environments and communities. These shale oil and gas extraction activities formed numerous scattered hot spots, and it demands tremendous efforts to trace them over time. This particular issue then challenges the conventional spatiotemporal analysis approaches which are effective for single or limited spatiotemporal objects. This study models the spatiotemporal developments of the shale oil and gas extraction activities as wave propagation. By using the wave representation, the complicated transmission of these shale oil and gas extraction activities is automatically traced within a sub-space-time as free propagation. To implement the proposed wave model, density surfaces of active wells from 1868 to 2018 were estimated annually. A 15-year focal statistic filter was applied to the density curve at each location to identify temporal objects. The waves were formed from temporal objects using a spatiotemporal expansion algorithm while maximally allowing 1 gap year. Finally, comprehensive attributes were derived for each wave. The results indicate that chaotic and fragmentary waves propagated at the western and northern edges of the Marcellus Shale. In contrast, large and simple wave propagation occurred in the middle of the Marcellus Shale. The propagation of large waves has a southward general trend, the trend follows the Appalachian region’s mountainous topography. Different types of drilling activities show diversified wave propagation patterns and the conventional gas wells formed the most chaotic wave propagation. The wave representation is successfully applied to the evolution of the shale oil and gas extraction activities, and this study also develops a comprehensive set of attributes to depict the wave. The scattered hot spots of dense shale oil and gas extraction activities are automatically assembled in waves. These concrete and detailed attributes of the waves are effective in summarizing the general spatiotemporal evolution pattern of chaotic and fragmentary spatiotemporal objects, e.g., the dense shale oil and gas extraction activities in this case. The developed method is also transferable and applicable to the evolution of other geographic phenomena, which may comprise of either limited or numerous spatiotemporal objects.

Real-Time Malware Process Detection and Automated Process Killing

Perimeter-based detection is no longer sufficient for mitigating the threat posed by malicious software. This is evident as antivirus (AV) products are replaced by endpoint detection and response (EDR) products, the latter allowing visibility into live machine activity rather than relying on the AV to filter out malicious artefacts. This paper argues that detecting malware in real-time on an endpoint necessitates an automated response due to the rapid and destructive nature of some malware. The proposed model uses statistical filtering on top of a machine learning dynamic behavioural malware detection model in order to detect individual malicious processes on the fly and kill those which are deemed malicious. In an experiment to measure the tangible impact of this system, we find that fast-acting ransomware is prevented from corrupting 92% of files with a false positive rate of 14%. Whilst the false-positive rate currently remains too high to adopt this approach as-is, these initial results demonstrate the need for a detection model that is able to act within seconds of the malware execution beginning; a timescale that has not been addressed by previous work.

Spatial filtering based on Riemannian distance to improve the generalization of ErrP classification

Due to the inherent non-stationarity of EEG signals, before each experimental session, BCI is usually calibrated to build the classification models, thus avoiding performance decay. This tedious re-calibration procedure is a limiting factor for real-world applications. Therefore, single-calibration or zero-calibration plays a crucial role in the use of BCIs in real contexts, outside the laboratory. Here, we propose and validate a statistical spatial filter, Riemannian Fisher criterion beamformer, based on Riemannian geometry able to use the invariance properties of Riemannian distance to handle cross-session and cross-subject generalization. The proposed method is validated with two datasets publicly available, consisting of error-related potentials. The results show that the proposed filter improves the generalization across sessions and across subjects and that it is robust to the amount of error samples used to train the classification model.

The novel noise classification techniques found on quadruple threshold statistical detection filter under fix intensity impulse outlier environment

Because of the enormous necessity of contemporary noise suppressing algorithms, this article proposes the novel noise classification technique found on QTSD filter improved from the TTSD filter. The four thresholds for each auxiliary situations are incorporated into the proposed QTSD framework for dealing with the limitation of the earlier noise classification technique. The mathematical pattern is modeled by each photograph elements and is investigated in contradiction to the 1st threshold for analyzing whether it is non-noise or noise photograph elements. Subsequently, the calculated photograph element is analyzed with the contradiction between the 2nd threshold, which is modeled by using the normal distribution (mean and variance), and is analyzed with the contradiction between the 3rd threshold, which is modeled by using the quartile distribution (median). Finally, the calculated photograph element is investigated in contradiction to the 4th threshold, which is modeled from maximum or minimum value for analyzing whether it is non-noise or noise photograph elements FIIN. For performance evaluation, extensive noisy photographs are made up of nine photographs under FIIN environment distribution, which are synthesized for investigating the proposed noise classification techniques found on QTSD filter in the objective indicators (noise classification, non-noise classification and overall classification correctness). From these results, the proposed noise classification technique can outstandingly produce the higher correctness than the earlier noise classification techniques.

A non‐Gaussian Bayesian filter for sequential data assimilation with non‐intrusive polynomial chaos expansion

AbstractNon‐Gaussian data assimilation is vital for several applications with nonlinear dynamical systems, including geosciences, socio‐economics, infectious disease modeling, and autonomous navigation. Widespread adoption of non‐Gaussian data assimilation requires easy‐to‐implement schemes. We develop, implement, and apply an efficient nonlinear non‐Gaussian data assimilation scheme using non‐intrusive stochastic collocation‐based polynomial chaos expansion (PCE) and Gaussian mixture model (GMM) priors fit to the state's uncertainty. First, we represent the uncertainty in a dynamical system using PCE and propagate it using the stochastic collocation method until an assimilation time. Then, we convert the polynomial basis prior to its equivalent Karhunen–Loeve (KL) form, fit a GMM in the subspace and perform a Bayesian filtering step. Thereafter, the posterior polynomial basis is recovered from the posterior GMM in the KL form, and uncertainty propagation is continued using the stochastic collocation method. The derivation and new equations required for the above conversions are presented. We apply the new scheme to an illustrative population growth dynamics application and a complex fluid flow problem for demonstrating its capabilities. In both cases, our filter accurately captures the non‐Gaussian statistics compared to the polynomial chaos‐ensemble Kalman filter and the polynomial chaos‐error subspace statistical estimation filter.

Space Imaging Sensor Power Supply Filtering: Improving EMC Margin Assessment with Clustering and Sensitivity Analyses

This work is dedicated to the assessment of the filtering performances of an optoelectronic sensor for space applications. Particular care is taken concerning the power supply subsystem (here voltage shifter integrated circuit), which is part of the electromagnetic compatibility (EMC) compliance of an imaging equipment embedded on spacecrafts. The proposed methodology aims at two major targets: First, evaluating the Filter Effectiveness (FE) subject concerning varying parameters (including filter topology, parasitic effects and source/load impedance variations); second, quantifying the relative importance of representative equivalent electrical components through sensitivity analyses (nominal and parasitic values). The latter point is of utmost importance considering the expected versatility of such systems, such as manufacturing tolerances, for instance. Nominal values and/or components are often badly defined for confidentiality reasons, lack of knowledge or pure ignorance of inputs. An analytical deterministic formulation (here through the transfer matrix approach) is proposed and completed with an original stochastic strategy (Reduced Order Clustering, ROC). This ensures the reliable assessment of both statistical filter performances and most influential parameters, jointly with computational resources saving relatively to brute force Monte Carlo simulations.

Enhancement of underwater optical images based on background light estimation and improved adaptive transmission fusion

Underwater optical imaging technology plays a vital role in humans' underwater activities. However, the serious quality degradation of underwater optical images hinders further development of such technology. This phenomenon is mainly caused by the absorption and scattering of light in the underwater medium. The blurred image formation model is widely used in the field of optical images and depends on two optical parameters: background light (BL) and the transmission map (TM). Therefore, we propose an underwater optical image enhancement method in the context of underwater optical image restoration and color correction. First, BL estimation based on the gray close operation, which can avoid the influence of white objects while accurately calculating BL, is proposed. Then, an improved adaptive transmission fusion (IATF) method is proposed, and the adjusted reversed saturation map (ARSM) method is applied to compensate for and refine the estimated TMs to obtain the final TMs. This paper also proposes a new underwater light attenuation prior (NULAP) method. Finally, to enhance color saturation and edge details, a statistical colorless slant correction fusion smoothing filter method is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods for dehazing, color and detail enhancement, and (uneven) light intensity.

Foreword to the Special Section on Meta-Level and Adversarial Tracking

The nine papers in this special section focus on meta-level and adversarial tracking. A plethora of well-established tracking algorithms aim to estimate, over time, the latent kinematic state (e.g., position, velocity, higher order kinematics, or any other spatiotemporal characteristic) of a single or multiple targets based on the available sensory observations, including from several sources. Here, we refer to such techniques as sensor-level trackers. Meta-level and adversarial tracking presents a shift away from the traditional viewpoint of a scene where objects move independently of one another in an unpremeditated manner and without regard to possible competition or group structures, toward an integrated viewpoint where intents, anomalies, group interactions, and characteristics of competitors/adversaries can be automatically learned. This also enables more accurate state estimation by capitalizing on inferred meta-level information. The papers included here showcase a diverse set of recent relevant technical developments and applications. It comprises of nine selected articles, drawing on recent advances in stochastic modeling, computational methods, statistical filtering, sensing systems, and others.

High-accuracy continuous mapping of surface water dynamics using automatic update of training samples and temporal consistency modification based on Google Earth Engine: A case study from Huizhou, China

Surface water is experiencing significant changes due to anthropogenic activities and climate change. Recent studies have explored long-term surface water dynamics with the development of multi-source remote sensing data and mapping technology. However, cost-effective and time-saving reliable references for mapping surface water from remote sensing images at multiple times are still lacking. Further, negligence of abnormal changes in time-series surface water maps seriously affects surface water mapping accuracy. In this study, we propose a high-accuracy method for long-term mapping of surface water by automatic update of training samples and temporal consistency modification in surface water sequences. Taking the surface water in Huizhou from 1986 to 2020 as a case study, the method is applied and tested. Temporal training samples were updated through Robust Mahalanobis distances and statistical filtering. Combined with the random forest algorithm and the Google Earth Engine platform, annual surface water maps were generated. A method for temporal consistency detection in moving window and abnormal changes modification rule with intra-annual information are proposed to improve annual surface water mapping accuracy. The accuracy of surface water dynamics and abnormal water modifications were quantified as 89.8% and 95.7%, respectively. With our proposed method, the resultant maximum annual water areas of Huizhou were detected, with a statistically significant upward trend and net increase of 103.1 km2 over 35 years. Reservoir construction and aquaculture expansion constitute the main sources of increased waters in this case study. It is demonstrated that our proposed method offers important advantages in terms of the accuracy of annual surface water maps and especially change monitoring.

KARGA: Multi-platform Toolkit for k-mer-based Antibiotic Resistance Gene Analysis of High-throughput Sequencing Data.

High-throughput sequencing is widely used for strain detection and characterization of antibiotic resistance in microbial metagenomic samples. Current analytical tools use curated antibiotic resistance gene (ARG) databases to classify individual sequencing reads or assembled contigs. However, identifying ARGs from raw read data can be time consuming (especially if assembly or alignment is required) and challenging, due to genome rearrangements and mutations. Here, we present the k-mer-based antibiotic gene resistance analyzer (KARGA), a multi-platform Java toolkit for identifying ARGs from metagenomic short read data. KARGA does not perform alignment; it uses an efficient double-lookup strategy, statistical filtering on false positives, and provides individual read classification as well as covering of the database resistome. On simulated data, KARGA's antibiotic resistance class recall is 99.89% for error/mutation rates within 10%, and of 83.37% for error/mutation rates between 10% and 25%, while it is 99.92% on ARGs with rearrangements. On empirical data, KARGA provides higher hit score (≥1.5-fold) than AMRPlusPlus, DeepARG, and MetaMARC. KARGA has also faster runtimes than all other tools (2x faster than AMRPlusPlus, 7x than DeepARG, and over 100x than MetaMARC). KARGA is available under the MIT license at https://github.com/DataIntellSystLab/KARGA.

Low-speed bearing fault diagnosis based on improved statistical filtering and convolutional neural network

Low-speed bearing fault signal is weak and overwhelmed by background noise, and it is ineffective in diagnosing bearing faults. In this study, an improved statistical filtering and convolution neural network (CNN) low-speed bearing fault diagnosis method is presented. Statistical filtering is a convenient and effective signal-filtering method that can extract fault signals from strong background noise original signals. The improved method can deal with drawbacks in selecting the standard distinction index of statistical filtering algorithm. CNN is used to extract signal features automatically and build the diagnosis model for fault classification. An engineering experiment is performed with different condition-bearing fault signals to test this presented method, and the results show that the improved statistical filter and CNN are effective in diagnosing low-speed bearing signals. Comparison with traditional methods proves that this study has superior performance in low-speed bearing fault diagnosis.