Pipeline Framework Research Articles

Efficient cooling strategies for liquid hydrogen pipelines: A comparative analysis

Liquid hydrogen (LH2) stands out for its high energy density, efficient transportation, and safe low-pressure storage. However, the challenge lies in achieving rapid cooling of LH2 pipelines while minimizing hydrogen loss. This paper introduces a robust three-dimensional model for simulating the unsteady-state heat transfer and the gas-liquid two-phase flow within LH2 pipelines. Throughout the cooling cycle, the pipe undergoes transitions in flow patterns, including stratified smooth flow, wavy flow, and intermittent flow, in which wavy flow dominates due to the extremely low liquid-to-gas density ratio of hydrogen. A comprehensive comparative analysis is conducted for three cooling modes: constant flow, variable flow, and pulsing flow. Finally, an evaluative framework for distinct pipeline cooling modes has been proposed. A reasonable variable flow cooling mode exhibits certain advantages in both cooling time and liquid hydrogen consumption when compared with the constant flow cooling mode, and the pulse flow cooling method adeptly capitalizes on the performance benefits derived from intermittent flow. Specifically, within the cooling range of 40 K–20 K, it substantially economizes liquid hydrogen consumption (15%–20 %), albeit concomitant with an extended total cooling time (40%–60 %). This study can offer theoretical insights to guide the high-efficiency cooling of liquid hydrogen pipelines.

A Tidy Framework and Infrastructure to Systematically Assemble Spatio-temporal Indexes from Multivariate Data

Indexes are useful for summarizing multivariate information into single metrics for monitoring, communicating, and decision-making. While most work has focused on defining new indexes for specific purposes, more attention needs to be directed toward making it possible to understand index behavior in different data conditions, and to determine how their structure affects their values and the variability therein. Here we discuss a modular data pipeline recommendation to assemble indexes. It is universally applicable to index computation and allows investigation of index behavior as part of the development procedure. One can compute indexes with different parameter choices, adjust steps in the index definition by adding, removing, and swapping them to experiment with various index designs, calculate uncertainty measures, and assess indexes’ robustness. The article presents three examples to illustrate the usage of the pipeline framework: comparison of two different indexes designed to monitor the spatio-temporal distribution of drought in Queensland, Australia; the effect of dimension reduction choices on the Global Gender Gap Index (GGGI) on countries’ ranking; and how to calculate bootstrap confidence intervals for the Standardized Precipitation Index (SPI). The methods are supported by a new R package, called tidyindex. Supplemental materials for the article are available online.

MolPipeline: A Python Package for Processing Molecules with RDKit in Scikit-learn.

The open-source package scikit-learn provides various machine learning algorithms and data processing tools, including the Pipeline class, which allows users to prepend custom data transformation steps to the machine learning model. We introduce the MolPipeline package, which extends this concept to cheminformatics by wrapping standard RDKit functionality, such as reading and writing SMILES strings or calculating molecular descriptors from a molecule object. We aimed to build an easy-to-use Python package to create completely automated end-to-end pipelines that scale to large data sets. Particular emphasis was put on handling erroneous instances, where resolution would require manual intervention in default pipelines. MolPipeline provides the building blocks to enable seamless integration of common cheminformatics tasks within scikit-learn's pipeline framework, such as scaffold splits and molecular standardization, making pipeline building easily adaptable to diverse project requirements.

Data pipeline approaches in serverless computing: a taxonomy, review, and research trends

Serverless computing has gained significant popularity due to its scalability, cost-effectiveness, and ease of deployment. With the exponential growth of data, organizations face the challenge of efficiently processing and analyzing vast amounts of data in a serverless environment. Data pipelines play a crucial role in managing and transforming data within serverless architectures. This paper provides a taxonomy of data pipeline approaches in serverless computing. Classification is based on architectural features, data processing techniques, and workflow orchestration mechanisms, these approaches are categorized into three primary methods: heuristic-based approach, Machine learning-based approach, and framework-based approach. Furthermore, a systematic review of existing data pipeline frameworks and tools is provided, encompassing their strengths, limitations, and real-world use cases. The advantages and disadvantages of each approach, also the challenges and performance metrics that influence their effectuality have been examined. Every data pipeline approach has certain advantages and disadvantages, whether it is framework-based, heuristic-based, or machine learning-based. Each approach is suitable for specific use cases. Hence, it is crucial assess the trade-offs between complexity, performance, cost, and scalability, while selecting a data pipeline approach. In the end, the paper highlights a number of open issues and future investigations directions for data pipeline in the serverless computing, which involve scalability, fault tolerance, data real time processing, data workflow orchestration, function state management with performance and cost in the serverless computing environments.

Exergy and dynamics analyses in centrifugal turbomachinery pressurized long-distance natural gas pipelines based on Hamiltonian model

Natural gas pipelines represent intricate systems comprising diverse components, which is crucial to industry, yet demanding substantial energy consumption. However, the nuanced energy and exergy transfer dynamics within these pipelines, particularly during dynamic processes, remain relatively unexplored. Given the distributed nature and complicated component coupling among pipeline elements, coolers, and compressors, traditional energy analysis tools face challenges in their application. In this study, we introduce a novel approach utilizing Hamiltonian formalism to establish an energy and exergy analysis framework for natural gas pipelines. Specifically, we develop Hamiltonian energy functions for various components, including pipes, centrifugal compressors, and coolers, accounting for dynamic mechanisms. Through comprehensive simulations covering static and dynamic scenarios, we elucidate energy and exergy transfer processes, leveraging Hamiltonian energy function terms. Our findings validate the efficacy of the proposed analysis framework in accurately characterizing energy transfer and dissipation processes. Furthermore, validation with field data demonstrates the capability of our method to online exhibit energy transfer features effectively.

A Semantic Spatial Structure-Based Loop Detection Algorithm for Visual Environmental Sensing

Loop closure detection is an important component of the Simultaneous Localization and Mapping (SLAM) algorithm, which is utilized in environmental sensing. It helps to reduce drift errors during long-term operation, improving the accuracy and robustness of localization. Such improvements are sorely needed, as conventional visual-based loop detection algorithms are greatly affected by significant changes in viewpoint and lighting conditions. In this paper, we present a semantic spatial structure-based loop detection algorithm. In place of feature points, robust semantic features are used to cope with the variation in the viewpoint. In consideration of the semantic features, which are region-based, we provide a corresponding matching algorithm. Constraints on semantic information and spatial structure are used to determine the existence of loop-back. A multi-stage pipeline framework is proposed to systematically leverage semantic information at different levels, enabling efficient filtering of potential loop closure candidates. To validate the effectiveness of our algorithm, we conducted experiments using the uHumans2 dataset. Our results demonstrate that, even when there are significant changes in viewpoint, the algorithm exhibits superior robustness compared to that of traditional loop detection methods.

Common Metadata Framework: Integrated Framework for Trustworthy Artificial Intelligence Pipelines

Common Metadata Framework: Integrated Framework for Trustworthy Artificial Intelligence Pipelines

A computational framework based on the dynamic pipeline approach

Stream processing has inspired new computational approaches to facilitate effectiveness and efficiency. One such approach is the dynamic pipeline, which serves as a powerful computational model for stream processing. It is particularly well suited for solving problems that require incremental generation of results, making it an approach for scenarios where real-time analysis and responsiveness are critical. This paper aims to address a family of problems using the Dynamic Pipeline approach, and as a first step, we provide a comprehensive characterization of this problem family. In addition, we present the definition of a Dynamic Pipeline framework. To demonstrate the practicality of this framework, we present a proof of concept through its implementation and perform an empirical performance study. To this end, we focus on solving the problem of enumerating or listing the weakly connected components of a graph within the proposed framework. We provide two implementations of this algorithm to demonstrate the computational power and continuous behavior of the Dynamic Pipeline framework. The first implementation serves as a baseline for our experiments, representing an ad hoc solution based on the Dynamic Pipeline approach. In contrast, the second implementation is built on top of the developed framework. The observed results strongly support the suitability and effectiveness of the Dynamic Pipeline framework for implementing graph stream processing problems, especially those where continuous and real-time result generation is essential.

DeepAUP: A Deep Neural Network Framework for Abnormal Underground Heat Transport Pipelines

DeepAUP: A Deep Neural Network Framework for Abnormal Underground Heat Transport Pipelines

Temporal teacher with masked transformers for semi-supervised action proposal generation

By conditioning on unit-level predictions, anchor-free models for action proposal generation have displayed impressive capabilities, such as having a lightweight architecture. However, task performance depends significantly on the quality of data used in training, and most effective models have relied on human-annotated data. Semi-supervised learning, i.e., jointly training deep neural networks with a labeled dataset as well as an unlabeled dataset, has made significant progress recently. Existing works have either primarily focused on classification tasks, which may require less annotation effort, or considered anchor-based detection models. Inspired by recent advances in semi-supervised methods on anchor-free object detectors, we propose a teacher-student framework for a two-stage action detection pipeline, named Temporal Teacher with Masked Transformers (TTMT), to generate high-quality action proposals based on an anchor-free transformer model. Leveraging consistency learning as one self-training technique, the model jointly trains an anchor-free student model and a gradually progressing teacher counterpart in a mutually beneficial manner. As the core model, we design a Transformer-based anchor-free model to improve effectiveness for temporal evaluation. We integrate bi-directional masks and devise encoder-only Masked Transformers for sequences. Jointly training on boundary locations and various local snippet-based features, our model predicts via the proposed scoring function for generating proposal candidates. Experiments on the THUMOS14 and ActivityNet-1.3 benchmarks demonstrate the effectiveness of our model for temporal proposal generation task.

Improving Solar Energetic Particle Event Prediction through Multivariate Time Series Data Augmentation

Solar energetic particles (SEPs) are associated with extreme solar events that can cause major damage to space- and ground-based life and infrastructure. High-intensity SEP events, particularly ∼100 MeV SEP events, can pose severe health risks for astronauts owing to radiation exposure and affect Earth’s orbiting satellites (e.g., Landsat and the International Space Station). A major challenge in the SEP event prediction task is the lack of adequate SEP data because of the rarity of these events. In this work, we aim to improve the prediction of ∼30, ∼60, and ∼100 MeV SEP events by synthetically increasing the number of SEP samples. We explore the use of a univariate and multivariate time series of proton flux data as input to machine-learning-based prediction methods, such as time series forest (TSF). Our study covers solar cycles 22, 23, and 24. Our findings show that using data augmentation methods, such as the synthetic minority oversampling technique, remarkably increases the accuracy and F1-score of the classifiers used in this research, especially for TSF, where the average accuracy increased by 20%, reaching around 90% accuracy in the ∼100 MeV SEP prediction task. We also achieved higher prediction accuracy when using the multivariate time series data of the proton flux. Finally, we build a pipeline framework for our best-performing model, TSF, and provide a comprehensive hierarchical classification of the ∼100, ∼60, and ∼30 MeV and non-SEP prediction scenarios.

Speculation and negation identification via unified Machine Reading Comprehension frameworks with lexical and syntactic data augmentation

Speculation and Negation Identification focuses on the extraction of speculative and negative cues and scopes. Previous work relied on complete syntactic trees or simply fed sentences into pre-trained language models, which were confronted with poor generalization within and across datasets, and the limitations of training samples. Accordingly, we build a complete pipeline framework that firstly detects cues and then extracts scopes, and propose unified Machine Reading Comprehension paradigms for both cue detection and scope resolution on several datasets. To tackle the insufficiency of training sets and produce more useful samples with appropriate amount of lexical and syntactic knowledge, we apply data augmentation integrating lexical and syntactic features for scope resolution. Experimental results show that our model achieves higher performance than baselines on several publicly accessible corpora.

Computer vision-based intelligent elevator information system for efficient demand-based operation and optimization

Occupancy and behavior in buildings are considered to be key factors influencing energy consumption. Residential areas contribute substantially to energy consumption in urban environments and have more regular users. Developing methods that can monitor building space utilization patterns and meet occupant comfort requirements is essential for urban energy efficiency. In this work, we propose an innovative computer vision-based Intelligent Elevator Information System (IEIS), which is based on a pipeline framework that sequentially implements elevator door state judgment, passenger attribute recognition, and duplicate-free traffic counting through re-recognition in order to give demand-based operational strategies and recommendations. By monitoring the elevator occupancy on a typical weekday, the elevator operation pattern is classified into three periods including peak period, idle period and balance period, and the demand-driven operation optimization is guided based on their corresponding characteristics. Experiments show that the proposed deep learning approach exhibits high accuracy in all stages of the pipeline. IEIS can facilitate energy-efficient management of elevators, ensure safety, provide human-centered services, and reduce the cost of human intervention while achieving efficient operation, and is expected to migrate to other scenario applications, bringing multiple potential benefits to the society.

Retention and Educational Inequalities in the U.S.

Many U.S. schools utilize grade retention (repeating grades when not meeting academic benchmarks) to allow more time for students to learn grade level material. However, some research suggests retention may increase inequalities and not help students progress. We use national data (Future of Families and Child Wellbeing Study 2014–2017) and logistic regression to examine what predicts the likelihood of elementary school retention and whether retention is associated with long term outcomes. We find that race and family income did not predict who was most likely to be retained. As expected, boys were more likely to be retained than girls. Most importantly, we show that, of students in major metropolitan areas, retention did not predict long-term academic outcomes (regardless of race, sex, or familial income). Retention did predict long-term exclusionary discipline outcomes for Black students only supporting the School to Prison Pipeline framework.

SGSI – A Scalable GPU-Friendly Subgraph Isomorphism Algorithm

Due to the inherent hardness of subgraph isomorphism, the performance is often a bottleneck in various real-world applications. We address this by designing an efficient subgraph isomorphism algorithm leveraging features of GPU architecture. Existing GPU-based solutions adopt two-step output scheme, performing the same join twice in order to write intermediate results concurrently. They also lack GPU architecture-aware optimizations that allow scaling to large graphs. In this paper, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</i> calable <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</i> PU-friendly <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</i> ubgraph <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</i> somorphism algorithm, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SGSI</i> . SGSI incorporates a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Prealloc-Combine</i> strategy based on the vertex-oriented framework, which avoids joining-twice in existing solutions. It uses a GPU-friendly data structure (called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PCSR</i> ) to represent an edge-labeled graph. We also study fine-grained load balance strategies and discuss how to handle enormous graphs that cannot be resident in GPU memory. A partition-based pipeline framework is proposed. Extensive experiments on both synthetic and real graphs show that SGSI outperforms the state-of-the-art algorithms by up to several orders of magnitude and has a good scalability with graph size scaling to billions of edges.

A risk-based approach to inspection planning for pipelines considering the coupling effect of corrosion and dents

RBI, referring to a risk-based approach to inspection planning, is an established pipeline integrity management method. Both corrosion and dents are the primary threats to pipeline integrity. However, they are often treated separately in RBI without considering their interactions. This coupling may lead to a synergic effect on integrity degradation. The present study proposes an RBI planning framework for pipelines considering external corrosion and dents. Time-dependent pipeline deterioration by dents and corrosion is modeled probabilistically using a Dynamic Bayesian Network (DBN), in-line inspection (ILI) data, and corrosion propagation knowledge. Two failure scenarios (leakage and burst) are considered. The hybrid method, integrating Monte Carlo Simulation (MCS) and Latin Hypercube Sampling (LHS) technique, estimates the pipeline's Probability of Failure (PoF) over time. The pipeline failure risk is quantified by monetizing the Consequence of Failure (CoF). An optimization model of loss-maintenance total expected cost is introduced to determine the optimum inspection period using maximum acceptable risk (MAR) and the lowest total expected cost. A cost-benefit analysis (CBA) is finally implemented to choose appropriate risk reduction measures. The proposed framework is robust and well-validated by a case study on an in-service pipeline.

ROUGE-SEM: Better evaluation of summarization using ROUGE combined with semantics

With the development of pre-trained language models and large-scale datasets, automatic text summarization has attracted much attention from the community of natural language processing, but the progress of automatic summarization evaluation has stagnated. Although there have been efforts to improve automatic summarization evaluation, ROUGE has remained one of the most popular metrics for nearly 20 years due to its competitive evaluation performance. However, ROUGE is not perfect, there are studies have shown that it is suffering from inaccurate evaluation of abstractive summarization and limited diversity of generated summaries, both caused by lexical bias. To avoid the bias of lexical similarity, more and more meaningful embedding-based metrics have been proposed to evaluate summaries by measuring semantic similarity. Due to the challenge of accurately measuring semantic similarity, none of them can fully replace ROUGE as the default automatic evaluation toolkit for text summarization. To address the aforementioned problems, we propose a compromise evaluation framework (ROUGE-SEM) for improving ROUGE with semantic information, which compensates for the lack of semantic awareness through a semantic similarity module. According to the differences in semantic similarity and lexical similarity, summaries are classified into four categories for the first time, including good-summary, pearl-summary, glass-summary, and bad-summary. In particular, the back-translation technique is adopted to rewrite pearl-summary and glass-summary that are inaccurately evaluated by ROUGE to alleviate lexical bias. Through this pipeline framework, summaries are first classified by candidate summary classifier, then rewritten by categorized summary rewriter, and finally scored by rewritten summary scorer, which are efficiently evaluated in a manner consistent with human behavior. When measured using Pearson, Spearman, and Kendall rank coefficients, our proposal achieves comparable or higher correlations with human judgments than several state-of-the-art automatic summarization evaluation metrics in dimensions of coherence, consistency, fluency, and relevance. This also suggests that improving ROUGE with semantics is a promising direction for automatic summarization evaluation.

A Reusable Model-agnostic Framework for Faithfully Explainable Recommendation and System Scrutability

State-of-the-art industrial-level recommender system applications mostly adopt complicated model structures such as deep neural networks. While this helps with the model performance, the lack of system explainability caused by these nearly blackbox models also raises concerns and potentially weakens the users’ trust in the system. Existing work on explainable recommendation mostly focuses on designing interpretable model structures to generate model-intrinsic explanations. However, most of them have complex structures, and it is difficult to directly apply these designs onto existing recommendation applications due to the effectiveness and efficiency concerns. However, while there have been some studies on explaining recommendation models without knowing their internal structures (i.e., model-agnostic explanations), these methods have been criticized for not reflecting the actual reasoning process of the recommendation model or, in other words,faithfulness. How to develop model-agnostic explanation methods and evaluate them in terms of faithfulness is mostly unknown. In this work, we propose a reusable evaluation pipeline for model-agnostic explainable recommendation. Our pipeline evaluates the quality of model-agnostic explanation from the perspectives of faithfulness and scrutability. We further propose a model-agnostic explanation framework for recommendation and verify it with the proposed evaluation pipeline. Extensive experiments on public datasets demonstrate that our model-agnostic framework is able to generate explanations that are faithful to the recommendation model. We additionally provide quantitative and qualitative study to show that our explanation framework could enhance the scrutability of blackbox recommendation model. With proper modification, our evaluation pipeline and model-agnostic explanation framework could be easily migrated to existing applications. Through this work, we hope to encourage the community to focus more on faithfulness evaluation of explainable recommender systems.

MEDIPIPE: an automated and comprehensive pipeline for cfMeDIP-seq data quality control and analysis.

Cell-free methylated DNA immunoprecipitation and high-throughput sequencing (cfMeDIP-seq) has emerged as a promising liquid biopsy technology to detect cancers and monitor treatments. While several bioinformatics tools for DNA methylation analysis have been adapted for cfMeDIP-seq data, an end-to-end pipeline and quality control framework specifically for this data type is still lacking. Here, we present the MEDIPIPE, which provides a one-stop solution for cfMeDIP-seq data quality control, methylation quantification, and sample aggregation. The major advantages of MEDIPIPE are: (i) ease of implementation and reproducibility with Snakemake containerized execution environments that will be automatically deployed via Conda; (ii) flexibility to handle different experimental settings with a single configuration file; and (iii) computationally efficiency for large-scale cfMeDIP-seq profiling data analysis and aggregation. This pipeline is an open-source software under the MIT license and it is freely available at https://github.com/pughlab/MEDIPIPE.

Optimizing DNNs Model Partitioning for Enhanced Performance on Edge Devices

Deep Neural Networks (DNNs) have proven effective in various applications due to their dominant performance. However, integrating DNNs into edge devices remains challenging due to the large size of the DNN model, which requires efficient model parallelization and workload partitioning. Previous attempts to address these challenges have focused on data and model parallelism but have fallen short in terms of finding the optimal DNN model partitions for efficient distribution, considering available resources. This paper presents a pipelined DNN model parallelism framework that improves the performance of DNNs on edge devices. The framework optimizes DNN model training by determining the optimal number of partitions based on available edge resources. This is achieved through a combination of data and model parallelism techniques, which efficiently distribute the workload across multiple processors to reduce training time. The framework also includes a task controller to manage computing resources effectively. The experimental results demonstrate the effectiveness of the proposed approach, showing a significant reduction in the model training time compared to a baseline model AlexNet.