Improve Filter Performance Research Articles

Using evolutionary computation for discovering spam patterns from e-mail samples

One of the most relevant problems affecting the efficient use of e-mail to communicate worldwide is the spam phenomenon. Spamming involves flooding Internet with undesired messages aimed to promote illegal or low value products and services. Beyond the existence of different well-known machine learning techniques, collaborative schemes and other complementary approaches, some popular anti-spam frameworks such as SpamAssassin or Wirebrush4SPAM enabled the possibility of using regular expressions to effectively improve filter performance. In this work, we provide a review of existing proposals to automatically generate fully functional regular expressions from any input dataset combining spam and ham messages. Due to configuration difficulties and the low performance achieved by analysed schemes, in this work we introduce DiscoverRegex, a novel automatic spam pattern-finding tool. Patterns generated DiscoverRegex outperform those created by existing approaches (able to avoid FP errors) whilst minimising the computational resources required for its proper operation. DiscoverRegex source code is publicly available at https://github.com/sing-group/DiscoverRegex.

Accounting for Model Error from Unresolved Scales in Ensemble Kalman Filters by Stochastic Parameterization

AbstractThe use of discrete-time stochastic parameterization to account for model error due to unresolved scales in ensemble Kalman filters is investigated by numerical experiments. The parameterization quantifies the model error and produces an improved non-Markovian forecast model, which generates high quality forecast ensembles and improves filter performance. Results are compared with the methods of dealing with model error through covariance inflation and localization (IL), using as an example the two-layer Lorenz-96 system. The numerical results show that when the ensemble size is sufficiently large, the parameterization is more effective in accounting for the model error than IL; if the ensemble size is small, IL is needed to reduce sampling error, but the parameterization further improves the performance of the filter. This suggests that in real applications where the ensemble size is relatively small, the filter can achieve better performance than pure IL if stochastic parameterization methods are combined with IL.

Directly deposited optical-blocking filters for single-photon x-ray imaging spectroscopy

Directly deposited optical-blocking filters (DD OBFs) have the potential to improve filter performance and lower risk and cost for future x-ray imaging spectroscopy missions. However, they have not been fully characterized on high-performance charge coupled devices (CCDs). This paper reports the results of DD OBFs processed on high-performance photon-counting CCDs. It is found that CCD performance is not degraded by deposition of such filters. X-ray and optical transmission through the OBF is characterized and found to match theoretical expectation. Light-leaks through pinholes and the side and back surfaces are found to lower the optical extinction ratio; various coating processes are developed to resolve these issues.

Use of Graphene Sorbent in Filters for Cleaning Surface Water and Hot Water

The technology of producing foamed (thermoexpanded) graphite in the form of anisotropic particles with a size of about 100 microns and a thickness of about 5 atomic layers has been developed. The BET surface area of 12–15 m 2 /g, the calculated surface area at a thickness of 1,5 nm is about 800 m2/g, which allowed it to be used in filters for drinking water purification. Water testing of Lake Ladoga showed that even after 500 m 3 of filtered water the turbidity did not exceed 0,5 mg/l, chromaticity 13 degrees, iron 0,11 mg/l and oxidizability 3,1 mg O/l. At the same time, the resource for turbidity and iron is not yet exhausted. The advantages of the sorbent GS are shown in the example of hot water treatment. In the Ice Palace of St. Petersburg a system of two filters "Hercules" was designed to clean hot water to fill the ice cover of the ice rink, instead of filtering the cold water and then heating it, which saved more than 3 million rubles a year. To improve filter performance, a new design was developed and tested, characterized in that it is not made in the form of a traditional cartridge, but is a backfill. The drop in the performance of such a filter to a critical level of 1,2 m 3 /h occurs with a filtration of 480 m 3 , which is 4 times greater than the volume of water obtained from the coaxial design of the filter (120 m 3 ). Filters with uneven compaction of GS are proposed to be used for cleaning surface water in the manufacture of a modular mobile unit with a capacity of up to 10 m 3 /h, which is designed to produce drinking water in populated areas where there are no coagulation plants and it is impossible to reach a water conduit with drinking water, Collections, youth forums and in cases of disasters.

Compact sext‐band bandpass filter based on single multimode resonator with high band‐to‐band isolations

On the basis of a single multimode resonator (MMR), a miniaturised microstrip sext-band bandpass filter (BPF) with high band-to-band isolations has been presented. The proposed MMR is an stepped impedance resonator (SIR) with one short-circuited stub loaded in the middle and two identical open-circuited stubs located at two sides. Transmission poles (TPs) and transmission zeros (TZs) can be excited when the electrical lengths of two open stubs increase. The TZs introduced by open stubs separate the TPs into multiple groups; therefore, the design of a sext-band BPF becomes available. Moreover, two input/output tapped branches and radical stub-loaded short-circuited lines are treated as perturbation structures in order to increase design freedoms for improving filter performance. Good agreement between simulated and measured results validates the design method.

Single-Mode Multicore Fibers With Integrated Bragg Filters

Fiber Bragg gratings (FBGs) in single-mode fibers, used as narrowband filters, have reached a high maturity level, especially for applications in telecommunications and sensing. In spectroscopic applications, e.g., in Raman spectroscopy, FBG filters can be used to separate laser probing light from generated sensor light which simplifies Raman probes. For this purpose, light from a sample has to be collected with high efficiency into a fiber. The light collection efficiency in single-mode fibers is, however, strongly limited due to the small fiber core area. Therefore, we have investigated the possibility to use single-mode multicore fibers to increase the light collection efficiency while maintaining Bragg filter performance. An optical density filtering effect of 1.5, which is equivalent to a transmission of -15 dB, has been achieved with a FBG in a 61-core fiber. Simulations have been performed in order to clarify the influence of the fiber geometry and refractive index distribution on grating performance. The application of chirped gratings in the multicore fibers improves filter performance in comparison to homogeneous gratings.

A pedestrian dead-reckoning system that considers the heel-strike and toe-off phases when using a foot-mounted IMU

In this paper, we propose an advanced pedestrian dead-reckoning (PDR) algorithm that considers the heel-strike and toe-off phases. Generally, PDR systems that use a foot-mounted inertial measurement unit are based on an inertial navigation system with an extended Kalman filter (EKF). To reduce the influence of the bias and white noises in the gyroscope and accelerometer signals, a zero-velocity update is often adopted at the stance phase. However, transient and large acceleration, which cannot be measured by the accelerometer used in pedestrian navigation, occur momentarily in the heel-strike phase. The velocity information from integration of the acceleration is not reliable because the acceleration is not measured in the heel-strike phase. Therefore, the designed EKF does not correctly reflect the actual environment, because conventional algorithms do not take the non-measurable acceleration into consideration. In order to reflect the actual environment, we propose a PDR system that considers the non-measurable acceleration from the heel-strike impact. To improve the PDR system’s performance, the proposed algorithm uses a new velocity measurement obtained using the constraint between the surface and the foot during the toe-off phase. The experimental results show improved filter performance after comparison of the proposed algorithm and a conventional algorithm.

An adaptive covariance relaxation method for ensemble data assimilation

For ensemble filters, accounting for unrepresented errors by inflating the ensemble perturbations can help improve filter performance. However, tuning the inflation factor can be costly, thus demanding adaptive covariance inflation (ACI) algorithms that give an online estimate of a temporally varying inflation factor. Additionally, a spatially varying inflation factor should be used to account for an irregular observation network. Anderson's adaptive inflation method offers a spatially and temporally varying inflation factor estimated from innovation statistics using a hierarchical Bayesian approach. In this study, we propose an alternative adaptive covariance relaxation (ACR) method that estimates a relaxation parameter online. Instead of treating inflation parameters as spatially varying random variables as in Anderson's method, the relaxation‐to‐prior‐spread method provides an ensemble spread reduction term that serves as a spatial mask to account for an irregular observation network. We demonstrate with a set of experiments using the 40‐variable Lorenz model that the ACR method is able to improve filter performance with the presence of sampling/model errors over a range of severity. Its reliability and ease of implementation suggest potential for future applications with atmospheric models.

A Principal Component Analysis Based Data Fusion Method for Estimation of Respiratory Volume

Impedance plethysmography (IP) is widely used in pulmonary volume measurement in recent years. Previous researches mainly focused on improving respiratory volume measurement accuracy by improving filter performance, electrode configuration, and so on, ignoring the influence of sleep posture changes. To solve this problem, we presented a principal component analysis (PCA)-based data fusion algorithm to minimize the effects of sleep posture changes on pulmonary volume measurement using a new dual-channel IP system. In situ experiments with ten subjects indicated that the PCA-based data fusion method improved the performance with the mean absolute error decreased $\sim 25$ %. Thus, the novel method potentially achieves a higher sensitivity of the sleep respiratory function diagnosis.

Variable step‐size sign algorithm against impulsive noises

This paper proposes a new variable step-size sign algorithm through the minimisation of mean-square deviation (MSD). As it is difficult to obtain the MSD accurately, the upper bound of the MSD is derived for calculating the step size at each iteration. The proposed algorithm is not only robust to impulsive noises, but also has improved filter performance in aspects of convergence rate and steady-state estimation error owing to the proposed variable step-size strategy. The simulation results verify that the proposed algorithm has better performance than the existing algorithms in a system-identification scenario in the presence of impulsive noises.

Directional weighted Median filter for salt and pepper noise and its comparison with other filters

Current switching-based median filters are commonly found to be incapable to find exact noise density variations and prone to misclassifying pixel characteristics at high noise density interference. Therefore there is a need of having a sophisticated switching weighted median filter. In this paper, we propose a directional based detection technique called the directional weighted median filter (DWMF), to achieve much improved filtering performance in terms of effectiveness in removing impulse noise while preserving signal details and robustness in combating noise density variations.

Enhancing Cryptosporidium parvum recovery rates for improved water monitoring

Water monitoring is essential to ensure safe drinking water for consumers. However existing methods have several drawbacks, particularly with regard to the poor recovery of Cryptosporidium due to the inability to efficiently elute Cryptosporidium oocysts during the established detection process used by water utilities. Thus the development of new inexpensive materials that could be incorporated into the concentration and release stage that would control Cryptosporidium oocysts adhesion would be beneficial. Here we describe improved filter performance following dip-coating of the filters with a “bioactive” polyacrylate. Specifically 69% more oocysts were eluted from the filter which had been coated with a polymer than on the naked filter alone.

Comparative Study of Filter Performance for Separation of Singing Voice from Music Accompaniment

An audio signal is a representation of sound. Audio signals have frequency range 20 to 20 kHz. Audio signals may be synthesized directly. A mixture refers to the physical combination of two or more substances on which the identities and are mixed in the form to separate out. An audio signal classification system should be able to categorize different audio input formats (speech, background noise, and music). Audio signal classification system analyzes the input audio signal and describes the signal at the output. These are used to characterize both music and speech signals. The categorization can be done on the basis of pitch, music content, music tempo and rhythm. From the comparative results it is observed that the wiener filter is better for noise reduction than others. We refer SEGSNR parameter for study because of its improved filter performance. Separating singing voice from music is very useful in many applications.

Hexavalent Chromium Removal Through Conventional Filtration Treatment Processes

A pilot‐scale study was conducted to evaluate the efficacy of reduction/coagulation/filtration (RCF) integration into conventional filtration treatment processes for hexavalent chromium (CrVI) and total chromium removal from surface water. RCF was effective for CrVI and total chromium removals of > 99 and > 93%, respectively, from surface water spiked with 25 μg/L CrVI. However, longer filter ripening and higher headloss accumulation rates were observed. Increased aluminum sulfate (alum) and filter aid dosages were assessed to improve filter performance; alum slightly improved performance, while filter aid reduced performance. Increased chlorine contact time with low chlorine dosages enhanced iron–chromium floc formation and reduced filter ripening and headloss accumulation rates resulting in longer filter run times. CrVI removals decreased from > 99 to > 98% while total chromium removals increased to > 98%. These results demonstrate that RCF integration effectively decreased CrVI and total chromium in surface water while maintaining regulatory compliance.

Computationally Efficient Extended Kalman Filter for Nonlinear Systems

A computationally efficient extended Kalman filter is developed for nonlinear estimation problems in this paper. The filter is performed in three stages. First, the state predictions are evaluated by the dynamic model of the system. Then, the dynamic equations of the rectification quantities for the predicted states are designed. Finally, the state estimations are updated by the predicted states with the rectification quantities multiplied by a single scale factor. One advantage of the filter is that the computational cost is reduced significantly, because the matrix coefficients of the rectified equations are constant. It doesnt need to evaluate the Jacobian matrixes and the matrix inversion for updating the gain matrix neither. Another advantage is that a single scale factor is introduced to scale the model approximated error, leading to an improved filter performance. The excellent performance of the proposed filter is demonstrated by an example with the application to the estimation problems for the sensorless permanent magnet synchronous motor direct torque control system.

Optimal selection of harmonic filter branch parameters using PSO and differential evolution algorithm

Passive and Resonant Harmonic Filters (RHF) are the effective devices installed in distribution systems for reducing distortions caused by nonlinear loads by means of creating a low impedance path. Achieving both harmonic reduction and filter performance is complicated and contradictory. To attain optimum filter performance, a compromise is to be made between these two conflicting goals. Optimal selection of filter parameters reduces supply current harmonics and improves filter performance. This paper examines Passive Harmonic Filter and RHF for three-phase rectifier system acting as a nonlinear load. It seeks to find optimal values of filter branch parameters to diminish harmonics using Evolutionary Algorithms such as Particle Swarm Optimization and Differential Evolution. Also, it improves system power factor. The detailed simulation results using MATLAB are presented to support the performance of the proposed optimized filters. The actual measurement of the harmonic levels from the hardware implementation validates the superiority of the optimized filter over conventional filter. Copyright © 2013 John Wiley & Sons, Ltd.

An assessment of long-term biosand filter use and sustainability in the Artibonite Valley near Deschapelles, Haiti

A non-randomized assessment of long-term biosand filter (BSF) use and sustainability in the Artibonite Valley near Deschapelles, Haiti was conducted during March, 2011. Of the 55 BSFs visited, 47% were no longer in use. Filter lifespan ranged from <1 year to systems still in use after 12 years. Interviews with BSF owners revealed problems related to intermittent filter use due to travel for employment or personal matters; broken or missing filter parts; and fears that the filter would not be effective against cholera. In addition, 17 BSF field studies were reviewed to identify common issues impacting usage. Culturally appropriate technologies and education materials explaining proper maintenance and operation are essential for improved filter performance and sustainability. For Haiti, education materials should be provided in Creole and French and should include, (1) diagrams and descriptions of how the BSF works, (2) how to troubleshoot common problems, (3) how to properly maintain filters, and (4) a contact in case of questions. Operational problems can be minimized by providing long-term technical support, periodic water quality monitoring, and maintenance assistance for filter users.

Experiments on Cake Development in Crossflow Filtration for High Level Waste

Crossflow filtration is a key process step in many operating and planned waste treatment facilities to separate undissolved solids from supernatant slurries. This separation technology generally has the advantage of self-cleaning through the action of wall shear stress created by the flow of waste slurry through the filter tubes. However, the ability of filter wall self-cleaning depends on the slurry being filtered. Many of the alkaline radioactive wastes are extremely challenging to filtration, e.g., those containing compounds of aluminum and iron having particles whose particle size and morphology reduce cake permeability. Low filter flux can be a bottleneck in waste processing facilities such as the Salt Waste Processing Facility at the Savannah River Site and the Waste Treatment Plant at the Hanford Site. To date, increased rates are generally realized by either increasing the crossflow filter axial flow rate, limited by pump capacity, or by increasing filter surface area limited by space and increasing the required pump load. The Savannah River National Laboratory (SRNL) set up both dead-end and crossflow filter tests to better understand filter performance based on filter media structure, flow conditions, and filter cleaning. Using nonradioactive simulated wastes, both chemically and physically similar to the actual radioactive wastes, the authors performed several tests to demonstrate increases in filter performance. With the proper use of filter flow conditions, filter flow rates can be increased over rates currently realized today. This paper describes the selection of a challenging simulated waste and crossflow filter tests to demonstrate how performance can be improved by varied filter operation methods. Those methods were a slow startup to better develop the filter cake and scouring the filter wall. The results showed that for salt waste and metal oxide hydroxide sludges, the process of backpulsing is not necessary to maintain a good filter flux, and the process of periodically scouring the filter improves filter performance. The results also imply that initial filter operation is important to develop a filter cake that minimized pressure drop, the presence of a filter cake can lead to improved solids separation, and a well-developed cake with periodic scouring may allow a good filter flux to be maintained for long periods of time.

A four‐dimensional asynchronous ensemble square‐root filter (4DEnSRF) algorithm and tests with simulated radar data

AbstractA four‐dimensional ensemble square‐root filter algorithm (4DEnSRF) is designed to assimilate high‐frequency asynchronous observations distributed over time. Given the serial nature of the EnSRF, the 4DEnSRF algorithm pre‐calculates observation priors from ensemble model states at observation times and updates the observation priors at asynchronous observational times using the filter. These updated observation posteriors are used to update model state variables at the analysis time. Such an algorithm is able to utilize more observations collected over time with fewer analysis cycles, thereby reducing computational costs and potentially improving filter performance. The 4DEnSRF algorithm is tested using simulated Doppler radar data for a convective storm. The radar data are simulated elevation‐by‐elevation, grouped into batches with different time intervals and then assimilated with analysis cycles of the same lengths. Parallel sets of experiments using 4DEnSRF and the regular EnSRF are performed for comparison, with varying data batch or cycle lengths of 1 to 20 min. For longer time intervals, EnSRF either assumes that all data collected within the time window are valid at the same analysis time, or uses only elevations collected within a shorter time interval centered at the analysis time. Results show that 4DEnSRF outperforms EnSRF when the cycle length is more than 1 min. Observation timing error is the main cause of the performance degradation with EnSRF for both analysis and forecast; the longer the cycle length, the worse the degradation. For long cycle lengths, 4DEnSRF improves the analysis by utilizing more data whereas the EnSRF performs well only when data far away from the analysis time are discarded. Assimilating only a couple of scan elevations at a time using EnSRF with very short cycles can introduce imbalances into the model state that degrades the subsequent analyses and forecasts. Copyright © 2012 Royal Meteorological Society

Phosphorus Removal Potential Using Biogenic Iron Oxides

Biogenic iron oxides produced by circumneutral iron-oxidizing bacteria were evaluated for phosphate removal. Traditional batch equilibrium experiments were coupled with bench-scale flow through filters to accomplish this task. Results from batch equilibrium experiments show that biogenic iron oxides can bind between 6.5 and 25.4 mg P/g solids, values that met or exceeded the sorption capacity of other iron rich substrates. Desorption of phosphate ranged from 10 to 40% for saline and potassium chloride solutions. When basic conditions were used to remove bound phosphate, 100 to 250% of applied phosphate was recovered, due to phosphate previously bound to iron oxides and that which was released from intact microbial cells. Flow through filter experiments with biogenic iron oxides highlighted technology potential and limitations. Several filter runs achieved effluent concentrations less than 0.2 mg P/L for more than 4 days, but most trials yielded moderate removal (70%). Paradoxically, the biologic origin of the iron oxides proved to be inhibitory during filtration, while this aspect of the technology would likely be the reason it could be an attractive alternative (matrix potentially re-grown in-place). Detergents and physical separation by a sand support both effectively improved filter performance by dispersing interconnected biogenic iron oxides. Collectively, results suggest biogenic iron oxides could be a reasonable phosphate filter material if a better understanding of growth conditions and requirements is obtained.This title belongs to WERF Research Report Series.ISBN: 9781843396154 (eBook)