Correction Algorithm Research Articles

GCphase: an SNP phasing method using a graph partition and error correction algorithm

BackgroundThe utilization of long reads for single nucleotide polymorphism (SNP) phasing has become popular, providing substantial support for research on human diseases and genetic studies in animals and plants. However, due to the complexity of the linkage relationships between SNP loci and sequencing errors in the reads, the recent methods still cannot yield satisfactory results.ResultsIn this study, we present a graph-based algorithm, GCphase, which utilizes the minimum cut algorithm to perform phasing. First, based on alignment between long reads and the reference genome, GCphase filters out ambiguous SNP sites and useless read information. Second, GCphase constructs a graph in which a vertex represents alleles of an SNP locus and each edge represents the presence of read support; moreover, GCphase adopts a graph minimum-cut algorithm to phase the SNPs. Next, GCpahse uses two error correction steps to refine the phasing results obtained from the previous step, effectively reducing the error rate. Finally, GCphase obtains the phase block. GCphase was compared to three other methods, WhatsHap, HapCUT2, and LongPhase, on the Nanopore and PacBio long-read datasets. The code is available from https://github.com/baimawjy/GCphase.ConclusionsExperimental results show that GCphase under different sequencing depths of different data has the least number of switch errors and the highest accuracy compared with other methods.

Aberration correction in 3D transthoracic echocardiography

Three-dimensional cardiac imaging has been available in the clinic for more than two decades. Continuous improvement in image quality has occurred in this period due to the development of probe technology and beamforming techniques. The purpose of this article is to quantitatively and qualitatively analyze the effect of a commercially available aberration correction algorithm on clinically acquired 3D transthoracic echocardiography (3D TTE) images. Clinical triplane and 3D volume cineloops of at least one cardiac cycle of pre-beamformed channel data were captured from 50 patients using a GE HealthCare Vivid E95 system with the 4Vc-D matrix array probe. This resulted in a total of 3200 vol and 3136 triplane frames. The data were post-processed with and without aberration correction. Quantitatively, assessed by an image quality parameter based on coherence, all recordings were improved by aberration correction compared to those without aberration correction. Triplane data obtained a larger improvement in image quality than volume data. Qualitatively, as demonstrated by case examples, aberration-corrected images appear sharper, have a brighter tissue signal compared to the cavity, and provide better delineation of cardiac structures. 3D rendering of valves can also be significantly improved. In general, aberration correction provides a systematic improvement in clinical cardiac triplane and 3D volume images.

Geometric Factor Correction Algorithm Based on Temperature and Humidity Profile Lidar

Due to the influence of geometric factors, the temperature and humidity profile of lidar’s near-field signal was warped when sensing the air environment. In order to perform geometric factor correction on near-field signals, this article proposes different correction solutions for the Mie and Raman scattering channels. Here, the Mie scattering channel used the Raman method to invert the aerosol backscatter coefficient and correct the extinction coefficient in the transition zone. The geometric factor was the ratio of the measured signal to the forward-computed vibration Raman scattering signal. The aerosol optical characteristics were reversed using the corrected echo signal, and the US standard atmospheric model was added to the missing signal in the blind zone, reflecting the aerosol evolution process. The stability and dependability of the proposed algorithm were validated by the consistency between the visibility provided by the Environmental Protection Agency and the visibility acquired via lidar retrieval data. The near-field humidity data were supplemented by the interpolation method in the Raman scattering channel to reflect the water vapor transfer process in the temporal dimension. The measured transmittance curve of the filter, the theoretical normalized spectrum, and the sounding data were used to compute the delay geometric factor. The temperature was retrieved and the near-field signal distortion issue was resolved by applying the corrected quotient of the temperature channel. The proposed algorithm exhibited robustness and universality, enhancing the system’s detection accuracy compared to the temperature and humidity data constantly recorded by the probes in the meteorological gradient tower, which have a high correlation with the lidar observation data. The comparison between lidar data and instrument monitoring data showed that the proposed algorithm could effectively correct distorted echo signals in the transition zone, which was of great value for promoting the application of lidar in the meteorological monitoring of the urban canopy layer.

A new method for the rapid identification of external water types in rainwater pipeline networks using UV–Vis absorption spectroscopy

Ultraviolet–visible (UV–Vis) absorption spectroscopy, due to its high sensitivity and capability for real-time online monitoring, is one of the most promising tools for the rapid identification of external water in rainwater pipe networks. However, difficulties in obtaining actual samples lead to insufficient real samples, and the complex composition of wastewater can affect the accurate traceability analysis of external water in rainwater pipe networks. In this study, a new method for identifying external water in rainwater pipe networks with a small number of samples is proposed. In this method, the Generative Adversarial Network (GAN) algorithm was initially used to generate spectral data from the absorption spectra of water samples; subsequently, the multiplicative scatter correction (MSC) algorithm was applied to process the UV–Vis absorption spectra of different types of water samples; following this, the Variational Mode Decomposition (VMD) algorithm was employed to decompose and recombine the spectra after MSC; and finally, the long short-term memory (LSTM) algorithm was used to establish the identification model between the recombined spectra and the water source types, and to determine the optimal number of decomposed spectra K. The research results show that when the number of decomposed spectra K is 5, the identification accuracy for different sources of domestic sewage, surface water, and industrial wastewater is the highest, with an overall accuracy of 98.81%. Additionally, the performance of this method was validated by mixed water samples (combinations of rainwater and domestic sewage, rainwater and surface water, and rainwater and industrial wastewater). The results indicate that the accuracy of the proposed method in identifying the source of external water in rainwater reaches 98.99%, with detection time within 10 s. Therefore, the proposed method can become a potential approach for rapid identification and traceability analysis of external water in rainwater pipe networks.

Three-dimensional drift correction of localised non-raster scanning on atomic force microscopy

Abstract Non-raster scanning can increase the scanning frame rate and measurement speed of atomic force microscopes (AFMs). It is also possible to correct the 3D drift during the non-raster scanning. However, the algorithm for the drift correction depends upon the properties of each scan pattern. While localised non-raster scanning using a rosette scan may be faster than the frequently used Lissajous scanning patterns, the drift correction is more challenging because the scan has crossing points only in local neighbouring segments where there are short temporal and spatial separations of the crossing paths. This design note presents a novel solution that successfully overcomes this problem and extends a drift correction method previously developed for Lissajous scans to the 3D drift correction of localised non-raster scanning using a rosette scan trajectory. The drift in the X, Y and Z axes can be determined using the crossing points and locally repeated scans of the same features. The general procedure is presented together with experiments using rosette scans of a two-dimensional lateral calibration standard. Experimental results have demonstrated that the method can effectively correct both the drift in the three axes and sample tilt, leading to significantly improved images. The method requires only localised crossing points in the scan and does not need additional scans to determine the three-dimensional drift based on cross-correlation and least squares techniques, and it can be used with any AFMs capable of rosette scanning.

A simple and cost-effective strategy for electrochemiluminescence spectral determination

BackgroundElectrochemiluminescence (ECL), as a unique and powerful analytical technique, has been widely used in various fields. The determination of ECL spectra plays a crucial role in understanding ECL reaction mechanisms and conducting spectra-resolved ECL analysis. ECL intensity is typically detected using a photomultiplier tube, which offers high sensitivity for detecting extremely weak light signals but does not allow for spectral identification. Due to the time-dependent variation of ECL intensity caused by the applied potential and electrochemical reaction processes, it is challenging to perform ECL spectral detection using conventional wavelength-scanning spectrometers. ResultsIn this study, we present a straightforward and cost-effective ECL spectral detection strategy by incorporating an automatically controlled tunable optical filter device between a commonly used PMT detector and a specially designed ECL reaction cell. The effectiveness of this approach was confirmed through initial validation, where the spectrum of a green LED spotlight was measured and compared with a commercial spectrometer. In a dynamic system with stable ECL signals, the ECL spectrum of the typical Ru(bpy)32+/TPA system was rapidly acquired by adjusting the bandpass filters. To account for time-varying ECL signals in practical measurements, time-based correction algorithms were implemented to rectify variations in ECL intensity. By integrating time-based correction algorithms and an automatically controlled tunable optical filter device into a commonly utilized PMT detector, the rapid and sensitive ECL spectra determination was achieved. Experimental results demonstrated the reliability of the proposed strategy. SignificanceThis strategy is based on the widely used high-sensitivity PMT detection component, enabling the rapid and sensitive measurement of ECL spectra without altering the ECL detection hardware. It is simple, fast, efficient, and cost-effective, with the potential to be widely used for rapid ECL spectral detection and spectra-resolved ECL analysis.

An Interactive Error-correcting Approach for IoT-sourced Event Logs

Although Internet of Things (IoT) systems are widely used in various industries, they are prone to data collection errors due to device limitations and environmental factors. These errors can significantly degrade the quality of collected data and the event log extracted from raw sensor readings, impact data analysis and lead to inaccurate or distorted results. This article emphasizes the importance of evaluating data quality and errors before proceeding with analysis. The effectiveness of three error correction methods, a rule-based method and a Process Mining (PM)-based method which are adjusted for a smart home use case, and their combination was also investigated in resolving log errors. The study found that understanding different types and sources of errors, and adapting the error correction algorithm based on this knowledge of error sources, can greatly improve the algorithm’s efficiency in addressing various error types.

A Method for Correction of Dynamic Errors When Measuring Flat Surfaces.

This paper presents a new method for correction of dynamic errors occurring when measuring flat surfaces in the presence of mechanical effects. Mechanical effects cause inertial forces and moments that affect the moving components of measuring instruments, thereby causing dynamic errors. The study proposes a mathematical model, on the basis of which algorithms for correction of dynamic errors can be developed. The basic concept of the model is based on determining the optimal estimate in the current coordinate point on the basis of the theoretical model of the measured surface and the information from the measurement that contains errors caused by internal and external factors. Based on this model, an algorithm for real-time data processing has been developed. The algorithm works in "predictor-corrector" mode at each step of which the best estimate is obtained. The estimate is based on minimizing the variance of a random component in which the main values are formed from the accumulated statistical data of the error of the model and the measurement error. This paper presents the results of experimental studies, carried out with simulations of mechanical effects in four modes. The results confirm the high efficiency of the algorithm for high-accuracy measurement of flat surfaces in the presence of mechanical effects.

Determination of malathion content in sorghum grains using hyperspectral imaging technology combined with stacked machine learning models

The rapid and precise detection of pesticide residues remains a pressing safety issue in the food industry. In this study, a rapid method for analyzing pesticide residues in sorghum was developed, combining hyperspectral imaging (HSI) technology with stacking ensemble learning (SEL) models. The HSI spectral data were preprocessed using the multivariate scatter correction (MSC) algorithm. The gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost), light gradient boosting machine (LGBM), and categorical boosting (CatBoost) algorithms were employed to identify the feature wavelengths with high contributions to the predictive model, and the performances of SEL, GBDT, XGBoost, LGBM, and CatBoost to accurately predict the pesticide residues in sorghum samples were compared. The SEL model constructed using the characteristic wavelength selected by CatBoost has the best predictive performance, with RMSEP, RP2, and RPD values of 0.6940 mg/kg, 0.9798, and 7.029, respectively. The study demonstrated that the combination of HSI and SEL enabled the accurate analysis of pesticide residues in sorghum, providing a reference for the utilization of HSI methods to accurately measure the concentrations of pesticide residues in sorghum and other food products.

MRI retrospective respiratory gating and cardiac sensing by CW Doppler radar: A feasibility study.

This study investigates the feasibility of non-contact retrospective respiratory gating and cardiac sensing using continuous wave Doppler radar deployed in an MRI system. The proposed technique can complement existing sensors which are difficult to apply for certain patient populations. We leverage a software-defined radio for continuous wave radar at 2.4 GHz to detect in-vivo respiratory and cardiac timescrolled signals. In-bore radar signal demodulation is verified with full electromagnetic simulations, and its functionality is validated on a test bench and within the MR bore with four normal subjects. Radar sensing was compared against well-known references: electrocardiography on a test bench, system bellows, and pulsed plethysmography sensors with in the MRI bore. The feasibility of noncontact cardiac rate sensing, dynamic breathing sequence synchronization, and in-bore motion correction for retrospective respiratory gating applications was demonstrated. Optimal radar front-end system arrangement, along with spectral isolation and narrow bandwidth of operation, enable MRI-compatible and interference-free motion sensing. The signal-to-noise-ratio degradation by the radar integration was within 4.5% on phantom images. We confirmed that in-bore retrospective motion correction using CW Doppler radar is feasible without MRI system constraints. Non-contact motion correction sensing in MRI may provide better patient handling and through put by complementing existing system sensors and motion correction algorithms.

Automated shot-to-shot optimization of the plasma start-up scenario in the TCV tokamak

Plasma start-up is typically achieved manipulating poloidal magnetic fields, gas injection and possibly auxiliary heating. Model-based design techniques have been gaining increasing attention in view of future large tokamaks which have more stringent constraints and less room for trial-and-error. In this paper, we formulate the tokamak start-up scenario design problem as a constrained optimization problem and introduce a novel shot-to-shot correction algorithm, based on the Iterative Learning Control concept, to compensate for unavoidable modeling errors based on experimental data. The effectiveness of the approach is demonstrated in experiments on the TCV tokamak showing that the target ramp-up scenario could be obtained in a small number of shots with a rough electromagnetic model.

A Parameterization Scheme for Correcting All‐Sky Surface Longwave Downward Radiation Over Rugged Terrain

AbstractAccurate surface longwave downward radiation (SLDR) is crucial for understanding mountain climate dynamics. While existing algorithms notably improve the accuracy of clear‐sky SLDR, a terrain correction algorithm that can correct remotely sensed and model‐simulated all‐sky SLDR on a large scale remains largely unexplored. Here, we propose a parameterization scheme for estimating all‐sky SLDR in rugged terrain. We primarily improve the estimation of nearby terrain thermal contribution by considering topographic asymmetry and incorporate the effects of ice cloud thermal scattering under low water vapor conditions. We validate the reliability of our model using the Discrete Anisotropic Radiative Transfer (DART) model, demonstrating a good agreement with a bias value of −12.8 W/m2 and a RMSE value of 28.2 W/m2. Further evaluation against the Essential Thermal Infrared Remote Sensing (ELITE) SLDR product at three TIPEX‐III in situ sites, located near the bottom of deep valleys with predominantly flat surfaces, indicates significant improvement in our model, reducing the mean bias by 7.4 W/m2 and the mean RMSE by 4.1 W/m2. Post‐terrain correction, the ELITE SLDR difference map exhibits a spatial pattern of “small in the northwest and large in the southeast” in the study area, with the maximum differences reaching 67 W/m2 in the daytime and 54 W/m2 at nighttime. Comparison with existing methods reveals similar improvements due to the consideration of terrain effects. Overall, our SLDR correction model shows enormous potential for correcting remotely sensed and model‐simulated SLDR products on a large scale.

Optimizing thermal efficiency: Advancements in flat plate heat exchanger performance through baffle integration

Compact heat exchangers have been gaining popularity in many industrial applications. Various types of passive turbulising structures such as corrugations, protrusions and ribs are introduced in the flow path to increase the effective heat transfer area and the level of turbulence in the flow path. This study investigates the impact of introduction of baffles on the performance of PHEs in terms of flow characteristics, pressure drop, and heat transfer. Two distinct types of baffle structures, namely wedge and aerofoil configurations, were introduced at varying numbers - 1, 3, and 5. An extensive experimentation is conducted for a FPHE in a thermal power plant of 500 × 2 MW and various flow and thermal parameters are measured. Computational Fluid Dynamics is utilized in this study to find the optimum baffle configuration. A detailed validation study is executed to obtain the correct computational algorithm, that is the right mesh count, optimum turbulence model, and precise numerical algorithm by comparing the numerical results with the available experimental results. Wedge- type baffles create increased turbulence and pressure drop, while aerofoil-type baffles minimize stagnation and exhibit lower pressure drop. Both baffle configurations lead to a substantial increase in heat transfer, with the 5-wedge-baffle setup showing the highest up to a 55% enhancement of Nusselt number. The Performance Evaluation Criterion (PEC) of wedge and aerofoil type is about 1.24 to 1.3 and 1.22 to 1.24 to that of conventional one respectively.

X-ray spectrometry for calculating conversion coefficients from air kerma to operational quantities in radiation protection

This study explores the conversion coefficients from air kerma to operational quantities for radiation protection, using x-ray spectrometry for the narrow-beam qualities below 300 keV as defined by ISO 4037-1. By employing custom spectral correction algorithms combined with modern cadmium telluride (CdTe) semiconductor detectors, we effectively corrected spectral distortions caused by detection processes, ensuring more reliable measurements. These measurements are crucial for meeting radiation protection standards. The study also analyses the sources of uncertainty associated with the determination of conversion coefficients, thereby providing improved accuracy and reproducibility in photon dosimetry.

Rock: Cleaning Data with both ML and Logic Rules

We demonstrate Rock, a system for cleaning relational data. Rock highlights the following unique features: (1) it extends logic rules by embedding machine learning models as predicates, to benefit from both ML and logic deduction; (2) it supports entity resolution, conflict resolution, timeliness deduction and missing data imputation in a unified process; and (3) it provides parallelly scalable algorithms for rule discovery, error detection and error correction, in batch and incremental modes. We will demonstrate Rock for its (a) easy-to-use interface, (b) scalability when cleaning large datasets, (c) accuracy for detecting and correcting errors across multiple tables, and (d) applications at banks and HR departments.

Town gas daily load forecasting based on machine learning combinatorial algorithms: A case study in North China

Town gas daily load forecasting based on machine learning combinatorial algorithms: A case study in North China

Three-dimensional forward modeling and quantitative assessment of electrode offset effects in ERT

Resistivity data has important applications in geophysical exploration, but the impact of electrode offsets on resistivity response characteristics remains unclear. This study aims to explore the influence of horizontal electrode offset angles and vertical offsets caused by topographical variations on the forward modeling of resistivity data. By analyzing experimental models with different measurement arrays, the paper revealed their influence laws on the buried depth of the target body and resistivity resolution. Utilizing tools like ZondRes3D, we conducted 3D resistivity forward modeling and analyzed the results in detail. It is found that horizontal electrode offsets lead to pseudo-anomalies in the apparent resistivity response, which is related to the offset angles and the number of electrodes. Under different conditions, the horizontal electrode offsets exhibit a “gradient variation” pattern. In addition, topographical variations can also cause distortions and offsets in the apparent resistivity curves and the locations of the anomaly response. Specifically, the measuring lines near the edge of the target bodies are more susceptible to these effects. Based on the comprehensive experimental results, we have drawn several conclusions regarding the impact of electrode offsets and topographical variations, including the effects of offset angles on the pseudo-anomalies, the anomalous response laws under different topographic conditions, as well as anomalous situations under specific angles. These findings provide crucial insights for interpreting resistivity data in geophysical exploration and addressing practical engineering problems, and offer guidance for optimizing measuring line layouts and post-processing terrain correction algorithms.

Research and Application of Destriping Algorithm based on Weighted Block Sparse Representation to FengYun Satellite Imager Data

Abstract In remote sensing imaging systems, stripe noise is a pervasive issue primarily caused by the inconsistent response of multiple detectors. Stripe noise not only affects image quality but also severely hinders subsequent quantitative derived products and applications. Therefore, it is crucial to eliminate stripe noise while preserving detailed structure information in order to enhance image quality. Although existing destriping methods have achieved certain effects to some extent, they still face problems such as loss of image details, image blur, and ringing artifacts. To address these issues, this study proposes an image stripe correction algorithm based on weighted block sparse representation. This research applies techniques such as differential low-rank constraint and edge weight factor to remove stripe noise while retaining image detail information. The algorithm also uses the alternating direction method of multipliers (ADMM) to solve the MCP regularized least squares optimization problem model, improving the processing efficiency of the model. The results of this study have been applied and validated in imager data from the Medium Resolution Spectral Imager (MERSI-II) onboard FengYun-3D satellite, the Multi-channel Scanning Radiometer (AGRI) onboard FengYun-4A satellite and Precipitation Microwave Radiometer (MWRI–RM) onboard FengYun-3G. Compared to typical stripe correction methods, the proposed method achieves better stripe removal while preserving image detail information. The destriped image data can be used to generate high-quality quantitative products for various applications. Overall, by combining insights from prior research and innovative techniques, this study provides a more effective and robust solution to the stripe noise problem in remote sensing and weather forecast.

To Reconstruct or Discard: A Comparison of Additive and Subtractive Charge Sharing Correction Algorithms at High and Low X-ray Fluxes.

Effective X-ray photon-counting spectral imaging (x-CSI) detector design involves the optimisation of a wide range of parameters both regarding the sensor (e.g., material, thickness and pixel pitch) and electronics (e.g., signal-processing chain and count-triggering scheme). Our previous publications have looked at the role of pixel pitch, sensor thickness and a range of additive charge sharing correction algorithms (CSCAs), and in this work, we compare additive and subtractive CSCAs to identify the advantages and disadvantages. These CSCAs differ in their approach to dealing with charge sharing: additive approaches attempt to reconstruct the original event, whilst subtractive approaches discard the shared events. Each approach was simulated on data from a wide range of x-CSI detector designs (pixel pitches 100-600 µm, sensor thickness 1.5 mm) and X-ray fluxes (106-109 photons mm-2 s-1), and their performance was characterised in terms of absolute detection efficiency (ADE), absolute photopeak efficiency (APE), relative coincidence counts (RCC) and binned spectral efficiency (BSE). Differences between the two approaches were explained mechanistically in terms of the CSCA's effect on both charge sharing and pule pileup. At low X-ray fluxes, the two approaches perform similarly, but at higher fluxes, they differ in complex ways. Generally, additive CSCAs perform better on absolute metrics (ADE and APE), and subtractive CSCAs perform better on relative metrics (RCC and BSE). Which approach to use will, thus, depend on the expected operating flux and whether dose efficiency or spectral efficiency is more important for the application in mind.

Optimisation of Target Heart Rate for Coronary CT Angiography (CCTA) Performed on a Dedicated Cardiac CT Scanner Using a Second-Generation Motion Correction Algorithm

Optimisation of Target Heart Rate for Coronary CT Angiography (CCTA) Performed on a Dedicated Cardiac CT Scanner Using a Second-Generation Motion Correction Algorithm