Noise Correction Research Articles

NewsPanda: Media Monitoring for Timely Conservation Action

Non-governmental organizations for environmental conservation have a significant interest in monitoring conservation-related media and getting timely updates about infrastructure construction projects as they may cause massive impact to key conservation areas. Such monitoring, however, is difficult and time-consuming. We introduce NewsPanda, a toolkit which automatically detects and analyzes online articles related to environmental conservation and infrastructure construction. We fine-tune a BERT-based model using active learning methods and noise correction algorithms to identify articles that are relevant to conservation and infrastructure construction. For the identified articles, we perform further analysis, extracting keywords and finding potentially related sources. NewsPanda has been successfully deployed by the World Wide Fund for Nature teams in the UK, India, and Nepal since February 2022. It currently monitors over 80,000 websites and 1,074 conservation sites across India and Nepal, saving more than 30 hours of human efforts weekly. We have now scaled it up to cover 60,000 conservation sites globally.

CRISPR-Analytics (CRISPR-A): A platform for precise analytics and simulations for gene editing.

Gene editing characterization with currently available tools does not always give precise relative proportions among the different types of gene edits present in an edited bulk of cells. We have developed CRISPR-Analytics, CRISPR-A, which is a comprehensive and versatile genome editing web application tool and a nextflow pipeline to give support to gene editing experimental design and analysis. CRISPR-A provides a robust gene editing analysis pipeline composed of data analysis tools and simulation. It achieves higher accuracy than current tools and expands the functionality. The analysis includes mock-based noise correction, spike-in calibrated amplification bias reduction, and advanced interactive graphics. This expanded robustness makes this tool ideal for analyzing highly sensitive cases such as clinical samples or experiments with low editing efficiencies. It also provides an assessment of experimental design through the simulation of gene editing results. Therefore, CRISPR-A is ideal to support multiple kinds of experiments such as double-stranded DNA break-based engineering, base editing (BE), primer editing (PE), and homology-directed repair (HDR), without the need of specifying the used experimental approach.

Probing sub-galactic mass structure with the power spectrum of surface-brightness anomalies in high-resolution observations of galaxy-galaxy strong gravitational lenses – I. Power-spectrum measurement and feasibility study

ABSTRACT While the direct detection of the dark-matter particle remains very challenging, the nature of dark matter could be possibly constrained by comparing the observed abundance and properties of small-scale sub-galactic mass structures with predictions from the phenomenological dark-matter models, such as cold, warm, or hot dark matter. Galaxy-galaxy strong gravitational lensing provides a unique opportunity to search for tiny surface-brightness anomalies in the extended lensed images (i.e. Einstein rings or gravitational arcs), induced by possible small-scale mass structures in the foreground lens galaxy. In this paper, the first in a series, we introduce and test a methodology to measure the power spectrum of such surface-brightness anomalies from high-resolution Hubble Space Telescope (HST) imaging. In particular, we focus on the observational aspects of this statistical approach, such as the most suitable observational strategy and sample selection, the choice of modelling techniques, and the noise correction. We test the feasibility of the power-spectrum measurement by applying it to a sample of galaxy-galaxy strong gravitational lens systems from the Sloan Lens ACS Survey, with the most extended, bright, high-signal-to-noise-ratio lensed images, observed in the rest-frame ultraviolet. In the companion paper, we present the methodology to relate the measured power spectrum to the statistical properties of the underlying small-scale mass structures in the lens galaxy and infer the first observational constraints on the sub-galactic matter power spectrum in a massive elliptical (lens) galaxy.

Improving label quality in crowdsourcing using deep co-teaching-based noise correction

Improving label quality in crowdsourcing using deep co-teaching-based noise correction

A high-speed transient waveform sampling SCA ASIC prototype in 180 nm CMOS technology

Switched capacitor array (SCA) is an important method of waveform digitization in high-energy physics experiments. This paper presents the design and test results of an eight-channel SCA ASIC for high-speed waveform sampling. The ASIC is designed and fabricated in 180 nm CMOS technology. The sampling rate of up to 14.25 Gsps is achieved by adopting the proposed dual-chain interpolating delay cell in DLL. The ASIC can be configured to operate in two modes: long-chain mode for deeper sampling depth and ping-pong mode for recording two waveforms in a short time. The bandwidth is over 1.1 GHz. By optimizing the sampling order, the distortion caused by the bandwidth discontinuity is reduced. Deep N-well is adopted to isolate the interference between digital and analog signals. The test results show that the random noise is approximately 0.87 mV RMS after the fixed-pattern noise correction. The time resolution is better than 5 ps RMS over the time measurement range of approximately 73μs.

Dark Current Measurement and Noise Correction Method for LWIR QWIP Detection System Based on Focal-Plane Temperature

The performance of long-wave infrared (LWIR) quantum well (QWIP) detection systems is seriously affected by the dark current of the detectors. Tiny variations in the focal-plane temperature of the devices cause fluctuations in the dark current, which in turn generate temporal noise. It is difficult to measure the dark current accurately after the detector assembly is packaged. To address the above problems, a QWIP dark current measurement method based on focal-plane temperature is proposed, as well as a method to reduce dark current noise. First, the response model of the LWIR QWIP detection system was established, and the dark current model was introduced. Then, the detection system components were introduced, chiller calibration experiments were carried out, and the dark current values of the QWIP at different temperatures were measured by combining the system design and parameters. Next, the dark current noise correction method was proposed, the target data were collected, and experiments were carried out to correct them. Finally, after the calculation, the temporal noise was reduced by 57.69% after the correction, which is proof of a significant effect. This method can obtain the real-time dark current value by collecting the focal-plane temperature data, and reduce the dark current temporal noise (difficult to eliminate using conventional methods), which is beneficial for promoting the application of QWIPs in LWIR remote sensing detection.

Neighborhood Weighted Voting-Based Noise Correction for Crowdsourcing

In crowdsourcing scenarios, we can obtain each instance’s multiple noisy labels set from different crowd workers and then use a ground truth inference algorithm to infer its integrated label. Despite the effectiveness of ground truth inference algorithms, a certain level of noise still remains in the integrated labels. To reduce the impact of noise, many noise correction algorithms have been proposed in recent years. To the best of our knowledge, however, nearly all existing noise correction algorithms only exploit each instance’s own multiple noisy label sets but ignore the multiple noisy label sets of its neighbors. Here neighbors refer to the nearest instances found in the feature space based on the distance metric learning. In this article, we propose neighborhood weighted voting-based noise correction (NWVNC). In NWVNC, we at first take advantage of the multiple noisy label sets of each instance’s neighbors (including itself) to estimate the probability that it belongs to its integrated label. Then, we use the estimated probability to identify and filter noise instances and thus obtain a clean set and a noise set. Finally, we train three heterogeneous classifiers on the clean set and correct the noise instances by the consensus voting of three trained classifiers. The experimental results on 34 simulated and two real-world crowdsourced datasets show that NWVNC significantly outperforms all the other state-of-the-art noise correction algorithms used for comparison.

Multiphase Flow Simulation of ITTC Standard Cavitator for Underwater Radiated Noise Prediction

This work focuses on the main issues related to noise measurements in cavitation tunnels. The scope of the paper is to twofold: to obtain a better understanding on the main phenomena underlying experiments and to define consistent cavitation tunnel measurement corrections for background noise, wall reflections, and distance normalisation. To this aim, the acoustic field generated by the ITTC standard cavitator model inside a cavitation tunnel is predicted by Lighthill’s acoustic analogy and solved through a finite element method that inherently accounts for the presence of the walls. Sources of sound detection relies on two multiphase CFD solvers, namely, the homogeneous mixture model—Volume of Fluid method and the Euler–Euler formulations. Starting from the computation of the sound pressure level in the free field with the assumption of spherical spreading without absorption, corrections from losses and spreading are detected by the above approach. Background-corrected sound pressure levels are identified and then compared with the source levels measured in the cavitation tunnel of the Potsdam Model Basin (SVA). It is found that free-field computations corrected by tunnel-induced effects match well with experiments up to 100 Hz (in the one-third octave band), whereas relevant discrepancies arise out of this range that need further investigations.

Real Image Denoising via Guided Residual Estimation and Noise Correction

Deep learning-based methods have dominated the field of image denoising with their superior performance. Most of them belong to the non-blind denoising approaches assuming that the noise is known at a specific level. However, real-world noise is complex and usually unknown. Since the distribution and level of noise are often unavailable, it will lead to severe performance degradation for non-blind denoising methods. Therefore, introducing noise levels is crucial for the challenging real-world denoising problem. Meanwhile, we observe that noise level mismatch will bring some artifacts to the denoised images. An intuitive solution is using the intermediate denoised images to correct the inaccurate noise level maps. Thus, we introduce an iterative correction scheme, yielding better results than direct noise prediction. We further propose an effective guided feature domain denoising residual network that can promote denoising for various real-world noises using iteratively denoised features, initial image features, and noise level maps. Experimental results on real-world image datasets show that the proposed method can provide excellent visual and objective performance for the real-world denoising task.

A point cloud preprocessing method for complicated thin-walled ring parts with irregular section

Metallic W-ring is a typically complicated thin-walled ring part with irregular section (CTWRPIC). However, there is a lack of effective measurement methods to inspect its geometric quality in the forming process, which leads to the failure of finding the forming defects in time. Due to the smooth metal surface, complex section shape, small section size and annular closed overall structure, point cloud of the section profile obtained by the line laser scanners has serious noise, outliers and missing. To this problem, this article proposes a point cloud preprocessing method for CTWRPIC. In this method, historical measurement data based outliers statistics, initial-fine rigid registration and non-rigid registration of CAD model are introduced. Firstly, the optimal alignment of point cloud from the CAD model and measurement point cloud is realized by initial-fine rigid registration. Secondly, the point cloud of CAD model is transformed by non-rigid registration to approximate the measurement profile points, the transformed point cloud of CAD model is considered as the real profile. Thirdly, based on the transformed point cloud of CAD model and distance threshold, the adaptive removal of outliers and repair of missing are realized. Finally, the correction of profile point cloud noise is completed by combining the Alpha Shapes algorithm and the proposed repair method. Based on the proposed preprocessing method and a measurement platform equipped with line laser scanners, an on-machine measurement method for metallic W-ring is proposed, and experimental results demonstrate the advantage of the proposed method. Compared with the destructive inspection method, the mean value, standard deviation and root mean square of the distance deviation of the preprocessed point cloud are all within 11 μm.

Multi-angle lensless ptychographic imaging via adaptive correction and the Nesterov method.

Lensless systems based on ptychographic imaging can simultaneously achieve a large field of view and high resolution while having the advantages of small size, portability, and low cost compared to traditional lensed imaging. However, lensless imaging systems are susceptible to environmental noise and have a lower resolution of individual images than lens-based imaging systems, which means that they require a longer time to obtain a good result. Therefore, in this paper, to improve the convergence rate and robustness of noise in lensless ptychographic imaging, we propose an adaptive correction method, in which we add an adaptive error term and noise correction term in lensless ptychographic algorithms to reach convergence faster and create a better suppression effect on both Gaussian noise and Poisson noise. The Wirtinger flow and the Nesterov algorithms are used in our method to reduce computational complexity and improve the convergence rate. We applied the method to phase reconstruction for lensless imaging and demonstrated the effectiveness of the method by simulation and experiment. The method can be easily applied to other ptychographic iterative algorithms.

STRisk: A Socio-Technical Approach to Assess Hacking Breaches Risk

Data breaches have begun to take on new dimensions and their prediction is becoming of great importance to organizations. Prior work has addressed this issue mainly from a technical perspective and neglected other interfering aspects such as the social media dimension. To fill this gap, we propose STRisk which is a predictive system where we expand the scope of the prediction task by bringing into play the social media dimension. We study over 3800 US organizations including both victim and non-victim organizations. For each organization, we design a profile composed of a variety of externally measured technical indicators and social factors. In addition, to account for unreported incidents, we consider the non-victim sample to be noisy and propose a noise correction approach to correct mislabeled organizations. We then build several machine learning models to predict whether an organization is exposed to experience a hacking breach. By exploiting both technical and social features, we achieve a Area Under Curve (AUC) score exceeding 98%, which is 12% higher than the AUC achieved using only technical features. Furthermore, our feature importance analysis reveals that open ports and expired certificates are the best technical predictors, while spreadability and agreeability are the best social predictors.

Evolving Error Feedback Fuzzy Model for Improved Robustness Under Measurement Noise

In this article, we propose a new variant of evolving fuzzy model for regression problems, which is based on error feedback integration in order to compensate measurement noise and to achieve more robust predictions; thus, it is termed as evolving error feedback fuzzy model. Thereby, we define an autoregressive-based noise model, localized per rule for being able to model different possible noise behaviors in different parts of the input space. Its predictions are added to the real fuzzy model predictions for noise correction purposes. This additive aspect leads to an extended version of the recursive fuzzily weighted least squares estimator (RFWLS-N = RFWLS with noise compensation), inheriting all its favorable convergence properties. For rule antecedent learning, we perform an evolving clustering process with the integration of a new forgetting strategy to be able to achieve more flexible updates in order to account for possible drifts. Therefore, a specific sample weighting concept is designed, with which different forgetting types can be achieved, ranging from (slow) logarithmic forgetting to (uniformly) linear forgetting and to (fast) exponential forgetting. The integration of the weights into the updates of rule centers and inverse covariance matrices is underlined with a theoretical analysis of their sample-wise contributions in the updates. Results on several noise-affected real-world datasets and system identification problems showed significantly improved model performances, compared to related evolving fuzzy methods, to the conventional RFWLS estimator and to the recursive correntropy approach. Antecedent forgetting helped us to improve the error trends in the case of abrupt drifts, while it never worsened the trends in the case of no drift.

Agile Micro-nano Satellite Attitude Estimation with Lost Measurements

Aiming at the problem of lost measurement in attitude determination process of the agile micro-nano satellite, a novel algorithm is proposed employing the Bernoulli process to describe the loss process. Firstly,the model of gyro and star sensors as well as the satellite attitude estimation system simplified with gyro is given. Since the missing observations, the modified EKF algorithm is derived by combining a Bernoulli random variable and a real-time measurement noise correction algorithm. Then the convergence of the improved algorihtm is analyzed, and the bottom boundary condition for the observation arrival rate is presented. Finally, the improved method is compared with the conventional one through digital simulation, and the experimental results show that the preciseness of the privious one is one or two orders improved compared with the latter.

Systematic noise removal from analytical ultracentrifugation data with UltraScan.

A method for removing time- and radially invariant noise from sedimentation velocity and sedimentation equilibrium experiments performed in an analytical ultracentrifuge is presented. The method averages repeat radial incident light measurements as a function of the photomultiplier response at different wavelengths to remove the majority of the time-invariant noise contributions from intensity data measurements. The results of this method are compared to traditional absorbance data generated with a buffer reference and the Beckman Optima AUC data acquisition program, and with the standard UltraScan refinement workflow. The method avoids the amplification of stochastic noise inherent in the absorbance scan subtraction traditionally employed in sedimentation velocity and equilibrium data. In addition, the collection of intensity data frees up the reference channel for additional samples, doubling the capacity of the instrument. In comparison to absorbance data, the residual mean square deviation of a fitted sedimentation velocity experiment without additional noise correction by UltraScan was improved by a factor of 4.5 when using the new method. This improvement benefits sedimentation equilibrium experiments as well as analytical buoyant density equilibrium experiments where routine time-invariant noise correction calculations cannot be performed.

Infrared Image Pre-Processing and IR/RGB Registration with FPGA Implementation

Infrared imaging sensors are frequently used in thermal signature detection applications in industrial, automotive, military and many other areas. However, advanced infrared detectors are generally associated with high costs and complexity. Infrared detectors usually necessitate a thermoelectric heater–cooler for temperature stabilization and various computationally complex preprocessing algorithms for fixed pattern noise (FPN) correction. In this paper, we leverage the benefits of uncooled focal plane arrays and describe a complete digital circuit design for Field Programmable Gate Array (FPGA)-based infrared image acquisition and pre-processing. The proposed design comprises temperature compensation, non-uniformity correction, defective pixel correction cores, spatial image transformation and registration with RGB images. When implemented on Xilinx Ultrascale+ FPGA, the system achieves a throughput of 30 frames per second using the Fraunhofer IMS Digital 17 μm QVGA-IRFPA with a microbolometer array size of 320 × 240 pixels and an RGB camera with a 1024 × 720 resolution. The maximum ratio of the standard deviation to the mean of 0.35% was achieved after FPN correction.

Receiver noise correction in semi-airborne transient electromagnetic method with an electric source

Semi-airborne transient electromagnetic method (SATEM) has been widely used in geophysical investigations. This method uses a grounded-wire to transmit the source signal and installs the receivers on an aircraft so that the data with a large depth of investigation can be acquired efficiently. However, the instability of aircraft when collecting the data will introduce the non-geological factors that distort the recorded data sets, which further affect the precision of the investigation result greatly. The researching of the mechanism and the correction method of these factors, including geometric effect, geomagnetic effect and mutual inductance effect between transmitter and receivers, is of great significance to the SATEM method. Thus, we proposed a thorough scheme to suppress noise generated by the non-geological factors of the SATEM configuration. We first derived the formulas for calculating the mutual inductance coupling between transmitter and receiver. The formulas of the geometric noise and geomagnetic noise that are affected by the roll, pitch, and yaw of the aircraft are further derived to analysis the influence interval of the motion noise. Then a hybrid correction scheme that uses a two-stage strategy to correct the receiver noise is put forward. The results show that the geomagnetic field and the mutual conductance contribute more noises than the geometric effect. The geomagnetic field will distort the data at all time channel, while the effect of the mutual inductance mainly focused on the early time channels. A field example demonstrated that the proposed method could well suppress the non-geological noise and improving the signal to noise ratio.

A review of addressing class noise problems of remote sensing classification

The development of image classification is one of the most important research topics in remote sensing. The prediction accuracy depends not only on the appropriate choice of the machine learning method but also on the quality of the training datasets. However, real-world data is not perfect and often suffers from noise. This paper gives an overview of noise filtering methods. Firstly, the types of noise and the consequences of class noise on machine learning are presented. Secondly, class noise handling methods at both the data level and the algorithm level are introduced. Then ensemble-based class noise handling methods including class noise removal, correction, and noise robust ensemble learners are presented. Finally, a summary of existing data-cleaning techniques is given.

Noise correction of atomic force microscopy-based integrin frequency analysis

As the most common joint disorder, Osteoarthritis (OA) is problematic for countries with an aging population. In OA, the cartilage degrades which causes bone on bone contact which results in constant pain for those affected. Treatments for OA are centered around symptom relief since there is no drug approved that will slow/stop OA disease progression. The cells that make up the cartilage are called chondrocytes. These chondrocytes must regulate themselves to build and break down the cartilage when necessary through cell signaling. When this signaling isn’t done properly, the chondrocytes may break down too much cartilage and result in OA. To better understand the signaling within the chondrocytes, an Atomic Force Microscope (AFM) was used to measure the movement and forces of mechanical receptor molecules (integrins) on the surface of the cell. However, because the measurements were near the sensitivity limit of the AFM, there was not enough certainty that noise (i.e., thermal, instrument and building) was not a dominant factor within the data. The focus of this project was to subtract model-based noise out of the displacement data to correct the data for further analysis. Instrument noise and building noise was found from the negative control, bare silica. Random thermal noise curves were generated based on theoretical thermal noise RMS values. A Fast Fourier Transform (FFT) was applied to each individual curve and each noise FFT was subtracted from the chondrocyte displacement curve in Python code. Inverse FFTs were applied to the resulting curves, resulting in noise-corrected time domain displacement curves.

Robust fused hypergraph neural networks for multi-label classification

Deep neural networks have been adopted in multi-label classification for their excellent performance, however, existing methods fail to comprehensively utilize the high-order correlations between instances and the high-order correlations between labels, and these methods are difficult to deal with label noise effectively. We propose a novel end-to-end deep framework named Robust Fused Hypergraph Neural Networks for Multi-Label Classification (RFHNN), which can effectively utilize the two kinds of high-order correlations and adopt them to mitigate the impact of label noise. In RFHNN, Hypergraph Neural Networks (HNNs) are adopted to mine and utilize the high-order correlations of the instances in the feature space and the label space respectively. The high-order correlations of the instances can not only improve the accuracy of the classification and the discrimination of the proposed model, but also lay the foundation for the subsequent noise correction module. Meanwhile, a hypergraph construction method based on the Apriori algorithm is proposed to realize Hypergraph Neural Networks (HNNs), which can mine robust second-order and high-order label correlations effectively. Effective classifiers are learned based on the correlations between the labels, which will not only improve the accuracy of the model, but can also enhance the subsequent noise correction module. In addition, we have designed a noise correction module in the networks. With the help of the high-order correlations among the instances and the effective classifier, the framework can effectively correct the noise and improve the robustness of the model. Extensive experimental results on datasets demonstrate that our proposed approach is better than the state-of-the-art multi-label classification algorithms. When dealing with the multi-label training datasets with noise in the label space, our proposed method also has great performance.