Input Pairs Research Articles

PPCon 1.0: Biogeochemical-Argo profile prediction with 1D convolutional networks

Abstract. Effective observation of the ocean is vital for studying and assessing the state and evolution of the marine ecosystem and for evaluating the impact of human activities. However, obtaining comprehensive oceanic measurements across temporal and spatial scales and for different biogeochemical variables remains challenging. Autonomous oceanographic instruments, such as Biogeochemical (BGC)-Argo profiling floats, have helped expand our ability to obtain subsurface and deep-ocean measurements, but measuring biogeochemical variables, such as nutrient concentration, still remains more demanding and expensive than measuring physical variables. Therefore, developing methods to estimate marine biogeochemical variables from high-frequency measurements is very much needed. Current neural network (NN) models developed for this task are based on a multilayer perceptron (MLP) architecture, trained over point-wise pairs of input–output features. Although MLPs can produce smooth outputs if the inputs change smoothly, convolutional neural networks (CNNs) are inherently designed to handle profile data effectively. In this study, we present a novel one-dimensional (1D) CNN model to predict profiles leveraging the typical shape of vertical profiles of a variable as a prior constraint during training. In particular, the Predict Profiles Convolutional (PPCon) model predicts nitrate, chlorophyll, and backscattering (bbp700) starting from the date and geolocation and from temperature, salinity, and oxygen profiles. Its effectiveness is demonstrated using a robust BGC-Argo dataset collected in the Mediterranean Sea for training and validation. Results, which include quantitative metrics and visual representations, prove the capability of PPCon to produce smooth and accurate profile predictions improving upon previous MLP applications.

A Cryo-CMOS 10-bit 60-MS/s SAR ADC with common-mode variation suppression switching scheme and gain boosting dynamic comparator

This paper presents a 10-bit 60-MS/s SAR ADC using an energy-efficient common-mode variation suppression (CMVS) switching scheme. The proposed CMVS switching scheme reduces energy consumption by about 92 % compared to the conventional scheme. Also, it narrows the common-mode variation to 16.6 % VDD. It improves the accuracy of the SAR ADC and makes the comparator and SAR ADC operate at the best-performance region. The proposed comparator adopts a gain boosting dynamic capacitive pre-amplifier to enhance the amplification and accelerate the comparison speed. The regeneration latch keeps the cross-coupled inverter structure to ensure high-speed regeneration. The input pair of the latch suppresses the through current while decreasing the regeneration speed. Therefore, an auxiliary input pair is added to enhance the regeneration speed. The SAR ADC is designed and simulated using 65-nm Cryo-CMOS technology with VDD = 1.2 V. At T = 300 K, it achieves a FoM of 15.39 fJ/conversion-step with 55-MS/s. At T = 4 K, it achieves a FoM of 14.15 fJ/conversion-step with 60-MS/s.

ML-GAP: machine learning-enhanced genomic analysis pipeline using autoencoders and data augmentation.

The advent of RNA sequencing (RNA-Seq) has significantly advanced our understanding of the transcriptomic landscape, revealing intricate gene expression patterns across biological states and conditions. However, the complexity and volume of RNA-Seq data pose challenges in identifying differentially expressed genes (DEGs), critical for understanding the molecular basis of diseases like cancer. We introduce a novel Machine Learning-Enhanced Genomic Data Analysis Pipeline (ML-GAP) that incorporates autoencoders and innovative data augmentation strategies, notably the MixUp method, to overcome these challenges. By creating synthetic training examples through a linear combination of input pairs and their labels, MixUp significantly enhances the model's ability to generalize from the training data to unseen examples. Our results demonstrate the ML-GAP's superiority in accuracy, efficiency, and insights, particularly crediting the MixUp method for its substantial contribution to the pipeline's effectiveness, advancing greatly genomic data analysis and setting a new standard in the field. This, in turn, suggests that ML-GAP has the potential to perform more accurate detection of DEGs but also offers new avenues for therapeutic intervention and research. By integrating explainable artificial intelligence (XAI) techniques, ML-GAP ensures a transparent and interpretable analysis, highlighting the significance of identified genetic markers.

A High Gain, Low Power Operational Transconductance Amplifier Capable of Recording and Processing ECG Signals

Based on high output resistor current mirrors and differential pair inputs, a new operational transconductance amplifier (OTA) is designed in this paper that can collect, filter, denoise and amplify electrocardiogram (ECG) signals. The OTA is designed on a 0.18μm CMOS process with a supply voltage of 1.8V. In this experiment, the ECG signal generated by the human model is first simulated, which has a noise of 50 Hz from electromagnetic interference, especially the electrical power lines. The differential input pair is then connected to the model, and it is proved that it can denoise the common-mode noise. Finally, the designed OTA is connected to the manikin, and the experimental results are compared with the traditional OTA. The design achieves a differential-mode gain of 43.56dB and a common-mode gain of -31.36dB while maintaining a differential input voltage swing of 1Vpp. Experimental results show that the OTA can denoise and amplify the ECG signal without distortion.

Ultra-low-power super class-AB adaptive biasing operational transconductance amplifier with enhanced gain for biomedical applications

The operational transconductance amplifier (OTA) proposed in this article is a bulk-driven (BD), single-stage, super-class-AB, adaptive biasing, functioning in the subthreshold region (ST) with an enormously low power supply of ± 0.25 V, providing high-gain. The input core of the OTA circuit is composed of adaptively biased BD differential input pairs based on flipped voltage follower (FVF), which drive in class-AB mode with a partial positive feedback (PPF) approach. The circuit additionally employs FVF and self-cascode (SC)-based low-power current mirror loads at its output to obtain significantly high gain and unity gain frequency. In addition, using adaptive loads based on source-degenerated metal oxide semiconductor (MOS) resistors raises dynamic current more efficiently, consequently improving the slew rate and unity gain frequency (UGF) without drawing additional power. Employing the cadence spectre tool and the UMC 0.18 μm complementary metal oxide semiconductor (CMOS) process technology, the designed OTA has been simulated. The simulation outcomes substantiate that the amplifier provides high open loop DC gain of 75 dB, 18.75 kHz UGF with a phase margin of 63.93º, and input-referred noise (IRN) of 0.734 µV/Hz0.5 at 1 kHz frequency. The proposed OTA consumes just 60.15 nW of power. The performance results confirmed that the proposed OTA circuit is appropriate in biomedical applications.

Predicting multiphase flow behavior of methane in shallow unconfined aquifers using conditional deep convolutional generative adversarial network

Numerical modeling is an essential tool for geoscience applications involving multiphase flow behavior. Performing numerical simulation is, however, computationally intensive due to multi-scale, non-linear, and multi-physics nature of mechanisms controlling multiphase flow dynamics. Additionally, the computational challenges associated with traditional numerical simulations limit their applicability for repetitive simulations required in inverse modeling, uncertainty analysis, and optimization tasks. To overcome these limitations, surrogate modeling has recently emerged as a viable solution. A surrogate model, which functions as a regression model, is trained using numerical simulation data. It approximates the input–output relationships within a dynamic fluid environment. We design surrogate models, based on conditional deep convolutional generative adversarial network (cDC-GAN), to map the cross-domain between input and output pairs in a multiphase multicomponent system, focusing on methane (CH4) migration in shallow unconfined aquifers. The cDC-GAN technique is selected due to its ability to generate high-fidelity surrogate models that effectively capture complex spatial distributions and heterogeneities. This technique excels in handling high-dimensional data, learning the intricate relationships between inputs and outputs, and producing realistic representations of subsurface phenomena. The trained cDC-GANs consider capillary entry pressure heterogeneity as input because this geologic attribute places first-order control on CH4 migration in subsurface. The outputs include CH4 saturation, water saturation, residual saturation of CH4, CH4 mole fraction, and CH4 molality. For each output, a separate cDC-GAN model is trained. Once trained, cDC-GANs can at any given time determine CH4 distribution in a shallow unconfined aquifer, both qualitatively and quantitatively. The cDC-GAN approach can be considered as a valuable complement to numerical simulations for predicting multiphase flow behavior in other geoscience-, energy-, and environmental applications such as contaminant transport, hydrocarbon production, and geological carbon dioxide and hydrogen storage.

Advancing Ionic Liquid Research with pSCNN: A Novel Approach for Accurate Normal Melting Temperature Predictions.

Ionic liquids (ILs), known for their distinct and tunable properties, offer a broad spectrum of potential applications across various fields, including chemistry, materials science, and energy storage. However, practical applications of ILs are often limited by their unfavorable physicochemical properties. Experimental screening becomes impractical due to the vast number of potential IL combinations. Therefore, the development of a robust and efficient model for predicting the IL properties is imperative. As the defining feature, it is of practice significance to establish an accurate yet efficient model to predict the normal melting point of IL (T m), which may facilitate the discovery and design of novel ILs for specific applications. In this study, we presented a pseudo-Siamese convolution neural network (pSCNN) inspired by SCNN and focused on the T m. Utilizing a data set of 3098 ILs, we systematically assess various deep learning models (ANN, pSCNN, and Transformer-CNF), along with molecular descriptors (ECFP fingerprint and Mordred properties), for their performance in predicting the T m of ILs. Remarkably, among the investigated modeling schemes, the pSCNN, coupled with filtered Mordred descriptors, demonstrates superior performance, yielding mean absolute error (MAE) and root-mean-square error (RMSE) values of 24.36 and 31.56 °C, respectively. Feature analysis further highlights the effectiveness of the pSCNN model. Moreover, the pSCNN method, with a pair of inputs, can be extended beyond ionic liquid melting point prediction.

A light gradient boosting machine-based method for predicting the dynamic response of functionally graded plates

The primary objective of this paper is to efficiently predict the dynamic response of functionally graded plates using LightGBM – a light gradient boosting machine, without reliance on supplementary analysis tools. To obtain the optimal LightGBM model, a dataset comprising 1,000 pairs of input and output is generated through iterations using a combination of isogeometric analysis (IGA) and third-order shear deformation plate theory (TSDT). In this model, the input is represented by a power index which governs the material distribution of the plate, and the output comprises 200 values illustrating deflection over time. To demonstrate the effectiveness of LightGBM in terms of accuracy and computational time, the results obtained by the proposed model are compared to those achieved with the optimal ANN, XGBoost models, and IGA. ® 2024 Journal of Science and Technology - NTTU

16.2-μW Super Class-AB OTA with current-reuse technique achieving 130.3-μA/μA FoM

In this paper, a single-stage Super Class-AB operational transconductance amplifier (OTA) using current reuse technique is presented. It is based on two scaled current mirrors located at the tail current nodes of input pairs, reusing the current through the cross-coupled input pairs by a current gain factor of α, and delivering new currents to the output branch, enhancing the overall transconductance (Gm) by a factor of (1+α). With adaptive biasing and local common mode feedback (LCMFB), slew rate (SR), open loop gain (AOL) and gain-bandwidth product (GBW) are improved. An prototype has been designed in TSMC 0.18-μm CMOS process. Post-simulation shows the positive and negative slew rate of 22V/μs and −38.3V/μs, respectively, a GBW of 3.72 MHz, a DC gain of 65.6 dB with a phase margin of 69.2° for a capacitive load of 70 pF. The proposed OTA consumes a total power of 16.2 μW under a 1.8V supply voltage.

Evaluating language models for mathematics through interactions

There is much excitement about the opportunity to harness the power of large language models (LLMs) when building problem-solving assistants. However, the standard methodology of evaluating LLMs relies on static pairs of inputs and outputs; this is insufficient for making an informed decision about which LLMs are best to use in an interactive setting, and how that varies by setting. Static assessment therefore limits how we understand language model capabilities. We introduce CheckMate, an adaptable prototype platform for humans to interact with and evaluate LLMs. We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics, with a mixed cohort of participants from undergraduate students to professors of mathematics. We release the resulting interaction and rating dataset, MathConverse. By analyzing MathConverse, we derive a taxonomy of human query behaviors and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness in LLM generations, among other findings. Further, we garner a more granular understanding of GPT-4 mathematical problem-solving through a series of case studies, contributed by experienced mathematicians. We conclude with actionable takeaways for ML practitioners and mathematicians: models that communicate uncertainty, respond well to user corrections, and can provide a concise rationale for their recommendations, may constitute better assistants. Humans should inspect LLM output carefully given their current shortcomings and potential for surprising fallibility.

Learning about structural errors in models of complex dynamical systems

Complex dynamical systems are notoriously difficult to model because some degrees of freedom (e.g., small scales) may be computationally unresolvable or are incompletely understood, yet they are dynamically important. For example, the small scales of cloud dynamics and droplet formation are crucial for controlling climate, yet are unresolvable in global climate models. Semi-empirical closure models for the effects of unresolved degrees of freedom often exist and encode important domain-specific knowledge. Building on such closure models and correcting them through learning the structural errors can be an effective way of fusing data with domain knowledge. Here we describe a general approach, principles, and algorithms for learning about structural errors. Key to our approach is to include structural error models inside the models of complex systems, for example, in closure models for unresolved scales. The structural errors then map, usually nonlinearly, to observable data. As a result, however, mismatches between model output and data are only indirectly informative about structural errors, due to a lack of labeled pairs of inputs and outputs of structural error models. Additionally, derivatives of the model may not exist or be readily available. We discuss how structural error models can be learned from indirect data with derivative-free Kalman inversion algorithms and variants, how sparsity constraints enforce a “do no harm” principle, and various ways of modeling structural errors. We also discuss the merits of using non-local and/or stochastic error models. In addition, we demonstrate how data assimilation techniques can assist the learning about structural errors in non-ergodic systems. The concepts and algorithms are illustrated in two numerical examples based on the Lorenz-96 system and a human glucose-insulin model.

Offline Handwriting Writer Identification using Depth-wise Separable Convolution with Siamese Network

Offline handwriting writer identification has significant implications for forensic investigations and biometric authentication. Handwriting, as a distinctive biometric trait, provides insights into individual identity. Despite advancements in handcrafted algorithms and deep learning techniques, the persistent challenges related to intra-variability and inter-writer similarity continue to drive research efforts. In this study, we build on well-separated convolution architectures like the Xception architecture, which has proven to be robust in our previous research comparing various deep learning architectures such as MobileNet, EfficientNet, ResNet50, and VGG16, where Xception demonstrated minimal training-validation disparities for writer identification. Expanding on this, we use a model based on similarity or dissimilarity approaches to identify offline writers' handwriting, known as the Siamese Network, that incorporates the Xception architecture. Similarity or dissimilarity measurements are based on the Manhattan or L1 distance between representation vectors of each input pair. We train publicly available IAM and CVL datasets; our approach achieves accuracy rates of 99.81% for IAM and 99.88% for CVL. The model was evaluated using evaluation metrics, which revealed only two error predictions in the IAM dataset, resulting in 99.75% accuracy, and five error predictions for CVL, resulting in 99.57% accuracy. These findings modestly surpass existing achievements, highlighting the potential inherent in our methodology to enhance writer identification accuracy. This study underscores the effectiveness of integrating the Siamese Network with depth-wise separable convolution, emphasizing the practical implications for supporting writer identification in real-world applications.

Online learning of stable robust adaptive controllers design based on data-dependent feedback linearization with application to rotary inverted pendulum

This study introduces an online (supervised) learning method to design nonlinear auto-regressive moving average (NARMA) controllers for feedback-linearized nonlinear single-input single-output (SISO) systems. The algorithm ensures Schur stability of the overall closed-loop system and provides adaptiveness and robustness for the NARMA controllers. The first stage of the method derives, in a data-dependent way, a feedback-linearized model of the nonlinear plant by using its input and output sample pairs. The method’s second stage, which constitutes the novel part of the presented study, builds up an online learning scheme for the linear auto-regressive moving average (ARMA) controller based on an already learned feedback-linearized model of the nonlinear plant. During online supervised learning, ARMA parameters of the feedback-linearized SISO plant model and the closed-loop ARMA model are computed by minimizing the plant identification and the closed-loop system tracking errors. Both errors are defined as ℓ1,ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1,{\\varvec{\\varepsilon}}}$$\\end{document}, namely ε-insensitive loss functions that provide NARMA controller the robustness against noise and outliers. The proposed online learning control algorithm is applied to a rotary inverted pendulum model and to a real rotary inverted pendulum setup. The tracking performance of the developed controller is compared with those of the linear quadratic regulator and coupled sliding mode controller in terms of mean square error.

A machine learning surrogate model for time of flight diffraction measurements of rough defects

Understanding the uncertainty in nondestructive evaluation measurements of rough defects requires a stochastic analysis due to the random variation between the morphology of different defects. In previous studies, large numbers of finite element models of randomly generated rough defects have been run in order to gain an insight into the statistics of the uncertain results. This approach is limited due to the time taken to run each individual model (typically of the order of minutes per model) and so the total number of models run is limited. In this paper, a surrogate model approach is proposed which produces close approximations to an original finite element model of a time-of-flight diffraction measurement, but in a small fraction of the time (<1 ms). The surrogate model is trained on pairs of input (rough defect) and output (A-scan) data from the finite element model; the surrogate then learns the relationship between inputs and outputs and is then able to generate new results. The architecture employed is a machine learning sequence-to-sequence approach similar to those used in natural language algorithms, based on the idea of ‘translating’ between defect to A-scan. In this study, a surrogate model was trained on 2160 finite element results which spanned across a range of ultrasonic incidence angles, rough defect correlation length and root-mean-square roughness. The surrogate model is shown to produce A-scans with close agreement to those produced by the original finite element model; taking approximately 0.425 ms instead of approximately 8 min. Millions of results are then possible in seconds, rather than the decades that would be required for finite element studies; more complete stochastic analysis are therefore now viable.

Metric learning for monotonic classification: turning the space up to the limits of monotonicity

This paper presents, for the first time, a distance metric learning algorithm for monotonic classification. Monotonic datasets arise in many real-world applications, where there exist order relations in the input and output variables, and the outputs corresponding to ordered pairs of inputs are also expected to be ordered. Monotonic classification can be addressed through several distance-based classifiers that are able to respect the monotonicity constraints of the data. The performance of distance-based classifiers can be improved with the use of distance metric learning algorithms, which are able to find the distances that best represent the similarities among each pair of data samples. However, learning a distance for monotonic data has an additional drawback: the learned distance may negatively impact the monotonic constraints of the data. In our work, we propose a new model for learning distances that does not corrupt these constraints. This methodology will also be useful in identifying and discarding non-monotonic pairs of samples that may be present in the data due to noise. The experimental analysis conducted, supported by a Bayesian statistical testing, demonstrates that the distances obtained by the proposed method can enhance the performance of several distance-based classifiers in monotonic problems.

Index analysis: An approach to understand signal transduction with application to the EGFR signalling pathway.

In systems biology and pharmacology, large-scale kinetic models are used to study the dynamic response of a system to a specific input or stimulus. While in many applications, a deeper understanding of the input-response behaviour is highly desirable, it is often hindered by the large number of molecular species and the complexity of the interactions. An approach that identifies key molecular species for a given input-response relationship and characterises dynamic properties of states is therefore highly desirable. We introduce the concept of index analysis; it is based on different time- and state-dependent quantities (indices) to identify important dynamic characteristics of molecular species. All indices are defined for a specific pair of input and response variables as well as for a specific magnitude of the input. In application to a large-scale kinetic model of the EGFR signalling cascade, we identified different phases of signal transduction, the peculiar role of Phosphatase3 during signal activation and Ras recycling during signal onset. In addition, we discuss the challenges and pitfalls of interpreting the relevance of molecular species based on knock-out simulation studies, and provide an alternative view on conflicting results on the importance of parallel EGFR downstream pathways. Beyond the applications in model interpretation, index analysis is envisioned to be a valuable tool in model reduction.

A High CMRR Differential Difference Amplifier Employing Combined Input Pairs for Neural Signal Recordings.

This article introduces a Combined .symmetrical and complementary Input Pairs (CIP) of a Differential Difference Amplifier (DDA), to boost the total Common-Mode Rejection Ratio (CMRR) for multi-channel neural signal recording. The proposed CIP-DDA employs three input pairs (transconductors). The dc-coupled input neural signal connection, via the gate terminal of the first transconductor, yields a high input impedance. The high-pass corner frequency and dc quiescent operation point are stabilized by the second transconductor. The calibration path of differential-mode gain and Common-Mode Feedback (CMFB) is provided by the proposed third transconductor. The parallel connection has no need for extra voltage headroom of input and output. The proposed CIP-DDA is targeted at integrated circuit realization and designed in a 0.18-μm CMOS technology. The proposed CIP-DDAs with system CMFB achieve an average CMRR of 103 dB, and each channel consumes circa 3.6 μW power consumption.

Private bipartite consensus control for multi-agent systems: A hierarchical differential privacy scheme

This paper addresses the differentially private bipartite consensus control problem for general linear discrete-time multi-agent systems (MASs), where the initial states of the agents are the sensitive data to be protected from potential eavesdroppers. In response to the cooperative–competitive relationship among the agents, a novel hierarchical differential privacy mechanism (HDPM) is proposed, which consists of a classifier and a pair of input and output noise injectors. The HDPM and the bipartite consensus controller are co-designed to ensure the ultimate mean-square bipartite consensus while preserving differential privacy. Finally, numerical simulations are conducted to illustrate the effectiveness of the proposed algorithm.

On‐Chip Optical Power Limiter for Quantum Communications

AbstractThis article presents an on‐chip optical power limiter that utilizes the thermo‐optical defocusing effect. A pair of input and output waveguides is designed to mimic emitting and receiving antennas. The waveguides are separated by a free‐space region filled with poly‐methyl‐meth‐acrylate (PMMA) material, which has a negative thermal‐optic coefficient that causes a decrease in refractive index with an increase in temperature. As the power in the input waveguide increases, the refractive index of the free‐space region decreases, which in turn increases the radiated beam's divergence angle with respect to input power. The empirical findings demonstrate that the non‐linear divergence angle can be written as , where θ0 represents the divergence angle of the equivalent Gaussian beam, k is a waveguide‐specific constant, and P is the input power. The edge of the receiving waveguide is tapered to adjust the coupling of the divergent beam to the output waveguide. The taper width is optimized to minimize insertion loss. The devices are two orders lengthwise smaller compared to the bulk demonstration, and they exhibit low loss ranging from 0.2 to 10 dB.

GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification.

A major challenge in applying deep learning to medical imaging is the paucity of annotated data. This study explores the use of synthetic images for data augmentation to address the challenge of limited annotated data in colonoscopy lesion classification. We demonstrate that synthetic colonoscopy images generated by Generative Adversarial Network (GAN) inversion can be used as training data to improve polyp classification performance by deep learning models. We invert pairs of images with the same label to a semantically rich and disentangled latent space and manipulate latent representations to produce new synthetic images. These synthetic images maintain the same label as the input pairs. We perform image modality translation (style transfer) between white light and narrow-band imaging (NBI). We also generate realistic synthetic lesion images by interpolating between original training images to increase the variety of lesion shapes in the training dataset. Our experiments show that GAN inversion can produce multiple colonoscopy data augmentations that improve the downstream polyp classification performance by 2.7% in F1-score and 4.9% in sensitivity over other methods, including state-of-the-art data augmentation. Testing on unseen out-of-domain data also showcased an improvement of 2.9% in F1-score and 2.7% in sensitivity. This approach outperforms other colonoscopy data augmentation techniques and does not require re-training multiple generative models. It also effectively uses information from diverse public datasets, even those not specifically designed for the targeted downstream task, resulting in strong domain generalizability. Project code and model: https://github.com/DurrLab/GAN-Inversion.