Batch Alignment Research Articles

Modality Blur and Batch Alignment Learning for Twin Noisy Labels-based Visible–infrared Person Re-identification

The issue of Twin Noisy Labels, known as Noisy Annotations and Noisy Correspondences, increases the challenge in the engineering application of Visible–Infrared Person Re-identification (VI-ReID). This paper proposes an novel Modality Blur and Batch Alignment (MBBA) framework to address this issue in the practical application of VI-ReID. The MBBA consists of the Label Confidence Learning (LCL) module, the Modality Blur Learning (MBL) module, and the Batch Alignment Learning (BAL) module. The LCL utilizes the memorization effect of deep neural networks to estimate the confidence level of identity labels and rectify the noisy annotations and the noisy correspondences. The MBL uses the noiseless modality labels and the center loss to blur the boundary among modality data to make the discrimination of the learned latent feature space robust. Based on the designed alignment loss, the BAL employs the self-attention mechanism to align the significant prediction distributions among the cross-modal sample pairs from a batch-size perspective. The mean Average Precision (mAP) is improved by 4.06% and 5.34% compared to state-of-the-art methods on the 20% noisy RegDB dataset. And our MBBA exhibits contemporary state-of-the-art performance on Twin Noisy Labels based VI-ReID. Our MBBA is available at https://github.com/SWU-CS-MediaLab/MBBA.

A Shape-Based Monitoring Method for Variable Trajectories of Finishing Mill Processes

As the most critical procedure in the hot strip mill process (HSMP), the performance of the finishing mill process (FMP) has a significant influence on the key performance indicators (KPIs). Considering that the multivariate statistical process monitoring (MSPM) methods and batch alignment methods (which are implemented in a point-to-point manner) cannot effectively handle the uneven-length and multiphase characteristics of FMP data, we propose a method to monitor the shape of the variable trajectories that are closely associated with KPIs. First, the historical FMP data characterized by different grades and thickness is utilized to build monitoring models based on the Shape-Based Distance (SBD) to obtain two statistics, i.e. the SBDs and the corresponding offsets when aligned with the central trajectories. Then, by comparing the statistics of the new coming trajectories with the control limits obtained from the training set, the new coming batches can be monitored. In the end, the proposed method is tested on the real FMP dataset to validate its effectiveness.

Batch alignment of single-cell transcriptomics data using deep metric learning

scRNA-seq has uncovered previously unappreciated levels of heterogeneity. With the increasing scale of scRNA-seq studies, the major challenge is correcting batch effect and accurately detecting the number of cell types, which is inevitable in human studies. The majority of scRNA-seq algorithms have been specifically designed to remove batch effect firstly and then conduct clustering, which may miss some rare cell types. Here we develop scDML, a deep metric learning model to remove batch effect in scRNA-seq data, guided by the initial clusters and the nearest neighbor information intra and inter batches. Comprehensive evaluations spanning different species and tissues demonstrated that scDML can remove batch effect, improve clustering performance, accurately recover true cell types and consistently outperform popular methods such as Seurat 3, scVI, Scanorama, BBKNN, Harmony et al. Most importantly, scDML preserves subtle cell types in raw data and enables discovery of new cell subtypes that are hard to extract by analyzing each batch individually. We also show that scDML is scalable to large datasets with lower peak memory usage, and we believe that scDML offers a valuable tool to study complex cellular heterogeneity.

TraceTrack, an open-source software for batch processing, alignment and visualization of sanger sequencing chromatograms.

Despite the advent of next-generation sequencing technology and its widespread applications, Sanger sequencing remains instrumental for molecular biology subcloning work in biological and medical research and indispensable for drug discovery campaigns. Although Sanger sequencing technology has been long established, existing software for processing and visualization of trace file chromatograms is limited in terms of functionality, scalability and availability for commercial use. To fill this gap, we developed TraceTrack, an open-source web application tool for batch alignment, analysis and visualization of Sanger trace files. TraceTrack offers high-throughput matching of trace files to reference sequences, rapid identification of mutations and an intuitive chromatogram analysis. Comparative analysis between TraceTrack and existing software tools highlights the advantages of TraceTrack with regards to batch processing, visualization and export functionalities. TraceTrack is available at https://github.com/MSDLLCpapers/TraceTrack and as a web application at https://tracetrack.dichlab.org. TraceTrack is a web application for batch processing and visualization of Sanger trace file chromatograms that meets the increasing demand of industrial sequence validation workflows in pharmaceutical settings. Supplementary data are available at Bioinformatics Advances online.

GANDAFL: Dataflow Acceleration for Short Read Alignment on NGS Data

DNA read alignment is an integral part of genome study, which has been revolutionised thanks to the growth of Next Generation Sequencing (NGS) technologies. The inherent computational intensity of string matching algorithms such as Smith-Waterman (SmW) and the vast amount of NGS input data, create a bottleneck in the workflows. Accelerated reconfigurable computing has been extensively leveraged to alleviate this bottleneck, focusing on high-performance albeit standalone implementations. In existing accelerated solutions effective co-design of NGS short-read alignment still remains an open issue, mainly due to narrow view on real integration aspects, such as system wide communication and accelerator call overheads. In this paper, we first propose <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GANDAFL</b> , a novel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> enome <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> ligNment <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DA</b> ta- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FL</b> ow architecture for SmW Matrix-fill and Traceback stages to perform high throughput short-read alignment on NGS data. We then propose a radical software restructuring to widely-used Bowtie2 aligner that allows read alignment by batches to expose acceleration capabilities. Batch alignment minimizes calling overhead of the accelerators whereas moving both Matrix-fill and Traceback on chip extinguishes the communication data overheads. The standalone solution delivers up to ×116 and ×2 speedup over state-of-the-art software and hardware accelerators respectively and GANDAFL-enhanced Bowtie2 aligner delivers a ×1.9 speedup.

BatchDTA: implicit batch alignment enhances deep learning-based drug-target affinity estimation.

Candidate compounds with high binding affinities toward a target protein are likely to be developed as drugs. Deep neural networks (DNNs) have attracted increasing attention for drug-target affinity (DTA) estimation owning to their efficiency. However, the negative impact of batch effects caused by measure metrics, system technologies and other assay information is seldom discussed when training a DNN model for DTA. Suffering from the data deviation caused by batch effects, the DNN models can only be trained on a small amount of 'clean' data. Thus, it is challenging for them to provide precise and consistent estimations. We design a batch-sensitive training framework, namely BatchDTA, to train the DNN models. BatchDTA implicitly aligns multiple batches toward the same protein through learning the orders of candidate compounds with respect to the batches, alleviating the impact of the batch effects on the DNN models. Extensive experiments demonstrate that BatchDTA facilitates four mainstream DNN models to enhance the ability and robustness on multiple DTA datasets (BindingDB, Davis and KIBA). The average concordance index of the DNN models achieves a relative improvement of 4.0%. The case study reveals that BatchDTA can successfully learn the ranking orders of the compounds from multiple batches. In addition, BatchDTA can also be applied to the fused data collected from multiple sources to achieve further improvement.

Batch alignment via retention orders for preprocessing large-scale multi-batch LC-MS experiments.

Meticulous selection of chromatographic peak detection parameters and algorithms is a crucial step in preprocessing liquid chromatography-mass spectrometry (LC-MS) data. However, as mass-to-charge ratio and retention time shifts are larger between batches than within batches, finding apt parameters for all samples of a large-scale multi-batch experiment with the aim of minimizing information loss becomes a challenging task. Preprocessing independent batches individually can curtail said problems but requires a method for aligning and combining them for further downstream analysis. We present two methods for aligning and combining individually preprocessed batches in multi-batch LC-MS experiments. Our developed methods were tested on six sets of simulated and six sets of real datasets. Furthermore, by estimating the probabilities of peak insertion, deletion and swap between batches in authentic datasets, we demonstrate that retention order swaps are not rare in untargeted LC-MS data. kmersAlignment and rtcorrectedAlignment algorithms are made available as an R package with raw data at https://metabocombiner.img.cas.cz. Supplementary data are available at Bioinformatics online.

Coarse-to-Fine Image Registration for Multi-Temporal High Resolution Remote Sensing Based on a Low-Rank Constraint

For multi-temporal high resolution remote sensing images, the image registration is important but difficult due to the high resolution and low-stability land-cover. Especially, the changing of land-cover, solar altitude angle, radiation intensity, and terrain fluctuation distortion in the overlapping areas can represent different image characteristics. These time-varying properties cause traditional registration methods with known reference information to fault. Therefore, in this paper we propose a comprehensive coarse-to-fine registration (CCFR) algorithm. First, we design a low-rank constraint-based batch reference extraction (LRC-BRE) method. Under the low-rank constraint, the stable features with highly spatial co-occurrence can be reconstructed via matrix decomposition, and are set as reference images to batch registration. Second, we improve the general feature registration with block feature matching and local linear transformation (BFM-LLT) operators including match outlier filtering (MOF) on regional mutual information and dual-weighted block fitting (DWBF). Finally, based on combining LRC-BRE and BFM-LLT, CCFR is integrated. Experimental results show that the proposed method has a good batch alignment effect, especially in the registration of large difference image pairs. The proposed CCFR achieves a significant performance improvement over many state-of-the-art registration algorithms.

Integration, exploration, and analysis of high-dimensional single-cell cytometry data using Spectre.

As the size and complexity of high-dimensional (HD) cytometry data continue to expand, comprehensive, scalable, and methodical computational analysis approaches are essential. Yet, contemporary clustering and dimensionality reduction tools alone are insufficient to analyze or reproduce analyses across large numbers of samples, batches, or experiments. Moreover, approaches that allow for the integration of data across batches or experiments are not well incorporated into computational toolkits to allow for streamlined workflows. Here we present Spectre, an R package that enables comprehensive end-to-end integration and analysis of HD cytometry data from different batches or experiments. Spectre streamlines the analytical stages of raw data pre-processing, batch alignment, data integration, clustering, dimensionality reduction, visualization, and population labelling, as well as quantitative and statistical analysis. Critically, the fundamental data structures used within Spectre, along with the implementation of machine learning classifiers, allow for the scalable analysis of very large HD datasets, generated by flow cytometry, mass cytometry, or spectral cytometry. Using open and flexible data structures, Spectre can also be used to analyze data generated by single-cell RNA sequencing or HD imaging technologies, such as Imaging Mass Cytometry. The simple, clear, and modular design of analysis workflows allow these tools to be used by bioinformaticians and laboratory scientists alike. Spectre is available as an R package or Docker container. R code is available on Github (https://github.com/immunedynamics/spectre).

CBA: Cluster-Guided Batch Alignment for Single Cell RNA-seq

The power of single-cell RNA sequencing (scRNA-seq) in detecting cell heterogeneity or developmental process is becoming more and more evident every day. The granularity of this knowledge is further propelled when combining two batches of scRNA-seq into a single large dataset. This strategy is however hampered by technical differences between these batches. Typically, these batch effects are resolved by matching similar cells across the different batches. Current approaches, however, do not take into account that we can constrain this matching further as cells can also be matched on their cell type identity. We use an auto-encoder to embed two batches in the same space such that cells are matched. To accomplish this, we use a loss function that preserves: (1) cell-cell distances within each of the two batches, as well as (2) cell-cell distances between two batches when the cells are of the same cell-type. The cell-type guidance is unsupervised, i.e., a cell-type is defined as a cluster in the original batch. We evaluated the performance of our cluster-guided batch alignment (CBA) using pancreas and mouse cell atlas datasets, against six state-of-the-art single cell alignment methods: Seurat v3, BBKNN, Scanorama, Harmony, LIGER, and BERMUDA. Compared to other approaches, CBA preserves the cluster separation in the original datasets while still being able to align the two datasets. We confirm that this separation is biologically meaningful by identifying relevant differential expression of genes for these preserved clusters.

BBKNN: fast batch alignment of single cell transcriptomes.

Increasing numbers of large scale single cell RNA-Seq projects are leading to a data explosion, which can only be fully exploited through data integration. A number of methods have been developed to combine diverse datasets by removing technical batch effects, but most are computationally intensive. To overcome the challenge of enormous datasets, we have developed BBKNN, an extremely fast graph-based data integration algorithm. We illustrate the power of BBKNN on large scale mouse atlasing data, and favourably benchmark its run time against a number of competing methods. BBKNN is available at https://github.com/Teichlab/bbknn, along with documentation and multiple example notebooks, and can be installed from pip. Supplementary data are available at Bioinformatics online.

MetFlow: an interactive and integrated workflow for metabolomics data cleaning and differential metabolite discovery.

Mass spectrometry-based metabolomics aims to profile the metabolic changes in biological systems and identify differential metabolites related to physiological phenotypes and aberrant activities. However, many confounding factors during data acquisition complicate metabolomics data, which is characterized by high dimensionality, uncertain degrees of missing and zero values, nonlinearity, unwanted variations and non-normality. Therefore, prior to differential metabolite discovery analysis, various types of data cleaning such as batch alignment, missing value imputation, data normalization and scaling are essentially required for data post-processing. Here, we developed an interactive web server, namely, MetFlow, to provide an integrated and comprehensive workflow for metabolomics data cleaning and differential metabolite discovery. The MetFlow is freely available on http://metflow.zhulab.cn/. Supplementary data are available at Bioinformatics online.

Toward Personalized Modeling: Incremental and Ensemble Alignment for Sequential Faces in the Wild

Fitting facial landmarks on unconstrained videos is a challenging task with broad applications. Both generic and joint alignment methods have been proposed with varying degrees of success. However, many generic methods are heavily sensitive to initializations and usually rely on offline-trained static models, which limit their performance on sequential images with extensive variations. On the other hand, joint methods are restricted to offline applications, since they require all frames to conduct batch alignment. To address these limitations, we propose to exploit incremental learning for personalized ensemble alignment. We sample multiple initial shapes to achieve image congealing within one frame, which enables us to incrementally conduct ensemble alignment by group-sparse regularized rank minimization. At the same time, incremental subspace adaptation is performed to achieve personalized modeling in a unified framework. To alleviate the drifting issue, we leverage a very efficient fitting evaluation network to pick out well-aligned faces for robust incremental learning. Extensive experiments on both controlled and unconstrained datasets have validated our approach in different aspects and demonstrated its superior performance compared with state of the arts in terms of fitting accuracy and efficiency.

Speeding up Batch Alignment of Large Ontologies Using MapReduce.

Real-world ontologies tend to be very large with several containing thousands of entities. Increasingly, ontologies are hosted in repositories, which often compute the alignment between the ontologies. As new ontologies are submitted or ontologies are updated, their alignment with others must be quickly computed. Therefore, aligning several pairs of ontologies quickly becomes a challenge for these repositories. We project this problem as one of batch alignment and show how it may be approached using the distributed computing paradigm of MapReduce. Our approach allows any alignment algorithm to be utilized on a MapReduce architecture. Experiments using four representative alignment algorithms demonstrate flexible and significant speedup of batch alignment of large ontology pairs using MapReduce.

Evaluation of heterogeneous measurement outlier rejection schemes for robotic planetary surface mapping

In this paper, we describe the development and evaluation of a core algorithmic component for robust robotic planetary surface mapping. In particular, we consider the issue of outlier measurements when utilizing both odometry and sparse features for laser scan alignment. Due to the heterogeneity of the measurements and the relative scarcity of distinct geometric features in the planetary environment, we have found that the conventional outlier rejection methods in the current literature do not produce satisfactory classifications for accurate mapping performance. In light of these limitations, we develop a new approach capable of addressing these concerns. This includes a family of four outlier classification algorithms, which are incorporated through iterative reclassification into the batch alignment framework to provide robust surface mapping performance. Characterization of these outlier rejection schemes is presented using a combination of simulated data and real-world testing with an indoor rover.

Three‐dimensional SLAM for mapping planetary work site environments

AbstractIn this paper, we present a robust framework suitable for conducting three‐dimensional simultaneous localization and mapping (3D SLAM) in a planetary work site environment. Operation in a planetary environment imposes sensing restrictions, as well as challenges due to the rugged terrain. Utilizing a laser rangefinder mounted on a rover platform, we have demonstrated an approach that is able to create globally consistent maps of natural, unstructured 3D terrain. The framework presented in this paper utilizes a sparse‐feature‐based approach and conducts data association using a combination of feature constellations and dense data. Because of feature scarcity, odometry measurements are also incorporated to provide additional information in feature‐poor regions. To maintain global consistency, these measurements are resolved using a batch alignment algorithm, which is reinforced with heterogeneous outlier rejection to improve its robustness to outliers in either measurement type (i.e., laser or odometry). Finally, a map is created from the alignment estimates and the dense data. Extensive validation of the framework is provided using data gathered at two different planetary analogue facilities, which consist of 50 and 102 3D scans, respectively. At these sites, root‐mean‐squared mapping errors of 4.3 and 8.9 cm were achieved. Relative metrics are utilized for localization accuracy and map quality, which facilitate detailed analysis of the performance, including failure modes and possible future improvements. © 2012 Wiley Periodicals, Inc.

Assembly of laser-fiber arrays

This paper describes techniques and machinery for first-level packaging of laser-to-fiber interconnects using batch assembly and active alignment. These techniques are economical because the costs of labor-intensive processes, such as parts handling, fiber-polishing, and alignment, are shared among the many elements of the array. Using these techniques, prototype packages have been built consisting of 32-wide arrays of 1300 nm lasers coupled to 32-wide arrays of single-mode fiber-optic stubs. Building the package involves three steps: (1) preparing the laser array for handling and alignment by soldering it to a metallized silicon carrier; (2) building the fiber array by stripping the fibers, placing them precisely into etched silicon V-grooves, bonding them in place, and polishing both ends; and (3) aligning the two arrays to each other using batch, active alignment. Batch alignment succeeds to the extent that each array is collinear, so the collinearity achieved in processes (1) and (2) is crucial. Typically, collinearity of /spl plusmn/1 /spl mu/m or better is achieved for each array. When such arrays are batch aligned using the techniques described herein, the average coupling efficiency is about 88% as large as it would be if each laser-fiber pair were optimally aligned individually. >