Toy Dataset Research Articles

How to Run Linear Mixed Effects Analysis for Pairwise Comparisons? A Tutorial and a Proposal for the Calculation of Standardized Effect Sizes.

This tutorial provides guidelines for conducting linear mixed effects (LME) analyses for simple designs, aimed at researchers familiar with t-tests, analysis of variance (ANOVA) and linear regression. First, we compare LME analyses with traditional methods when participants are the only source of random variation. We show that LME analysis is more interesting as soon as you have more than one observation per participant per condition. The second section discusses studies where both participants and stimuli are used as sources of random variation, ensuring robust generalization beyond the specific stimuli tested. In our search for standardized effect sizes, we saw that partial eta squared is even less informative for LME than for ANOVA. We present eta squared within as an alternative, to be used in combination with the traditional measure eta squared (also in ANOVA). To facilitate implementation, we analyze toy datasets with R and jamovi. This tutorial gives researchers a good foundation for LME analyses of simple 2 × 2 designs and paves the way for tackling more complicated designs.

Konnektor: A Framework for Using Graph Theory to Plan Networks for Free Energy Calculations.

Alchemical free energy campaigns can be planned using graph theory by building networks that contain nodes representing molecules that are connected by possible transformations as edges. We introduce Konnektor, an open-source Python package, for systematically planning, modifying, and analyzing free energy calculation networks. Konnektor is designed to aid in the drug discovery process by enabling users to easily setup free energy campaigns using complex graph manipulation methods. The package contains functions for network operations including concatenation of networks, deletion of transformations, and clustering of molecules along with a framework for combining these tools with existing network generation algorithms to enable the development of more complex methods for network generation. A comparison of the various network layout features offered is carried out using toy data sets. Additionally, Konnektor contains visualization and analysis tools, making the investigation of network features much simpler. Besides the content of the package, the paper also offers application examples, demonstrating how Konnektor can be used and how the different networks perform from a graph theory perspective. Konnektor is freely available via GitHub at https://github.com/OpenFreeEnergy/konnektor under the permissive MIT License.

Model-independent extrapolation of MUonE data with Padé and D-Log approximants

The MUonE experiment is designed to extract the hadronic contribution to the electromagnetic coupling in the spacelike region Δαhad(t) from elastic eμ scattering. The leading-order hadronic vacuum polarization contribution to the muon g−2, aμHVP,LO, can then be obtained from a weighted integral over Δαhad(t). This, however, requires knowledge of Δαhad(t) in the whole domain of integration, which cannot be achieved by experiment. In this work, we propose to use Padé and D-Log Padé approximants as a systematic and model-independent method to fit and reliably extrapolate the future MUonE experimental data, extracting aμHVP,LO with a conservative but competitive uncertainty, using no or very limited external information. The method relies on fundamental analytic properties of the two-point correlator underlying aμHVP,LO and provides lower and upper bounds for the result for aμHVP,LO. We demonstrate the reliability of the method using toy datasets generated from a model for Δαhad(t) reflecting the expected statistics of the MUonE experiment. Published by the American Physical Society 2024

Bidirectional Probabilistic Subspaces Approximation for Multiview Clustering.

The existing multiview clustering models learn a consistent low-dimensional embedding either from multiple feature matrices or multiple similarity matrices, which ignores the interaction between the two procedures and limits the improvement of clustering performance on multiview data. To address this issue, a bidirectional probabilistic subspaces approximation (BPSA) model is developed in this article to learn a consistently orthogonal embedding from multiple feature matrices and multiple similarity matrices simultaneously via the disturbed probabilistic subspace modeling and approximation. A skillful bidirectional fusion strategy is designed to guarantee the parameter-free property of the BPSA model. Two adaptively weighted learning mechanisms are introduced to ensure the inconsistencies among multiple views and the inconsistencies between bidirectional learning processes. To solve the optimization problem involved in the BPSA model, an iterative solver is derived, and a rigorous convergence guarantee is provided. Extensive experimental results on both toy and real-world datasets demonstrate that our BPSA model achieves state-of-the-art performance even if it is parameter-free.

Conditional advancement of machine learning algorithm via fuzzy neural network

Improving overall performance is the ultimate goal of any machine learning (ML) algorithm. While it is a trivial task to explore multiple individual validation measurements, evaluating and monitoring overall performance can be complicated due to the highly nonlinear nature of the functions describing the relationships among different validation metrics, such as the Dice Similarity Coefficient (DSC) and Jaccard Index (JI). Therefore, it is naturally desirable to have a reliable validation algorithm or model that can integrate all existing validation metrics into a single value. This consolidated metric would enable straightforward assessment of an ML algorithm’s performance and identify areas for improvement. To deal with such a complex nonlinear problem, this study suggests a novel parameterized model named Adaptive Neuro-Fuzzy Inference Systems (ANFIS), which takes any set of input–output precise-imprecise data and uses a neuro-adaptive learning strategy to tune the parameters of the pre-defined membership functions. Our method can be accepted as an elegant and the state-of-the-art method for the nonlinear function approximation, which could be added directly to any convolutional neural networks (CNN) loss functions as the regularization term to generate a constrained-CNN-FUZZY model optimization. To demonstrate the ability of the purposed method and provide a practical explanation of the capability of ANFIS, we use deep CNN as a testing platform to consider the fact that one of the biggest challenges CNN-developers faced today is to reduce the mismatching between the provided input data and the predicted results monitored by different validation metrics. We first create a toy dataset using MNIST and investigate the properties of the proposed model. We then use a medical dataset to demonstrate our method’s efficacy on brain lesion segmentation. In both datasets, our method shows reliable validation results to guide researchers towards choosing performance metrics in a problem-aware manner, especially when the results of different validation metrics are too similar among models to determine the best one.

MethParquet: an R package for rapid and efficient DNA methylation association analysis adopting Apache Parquet.

Genome-wide DNA methylation (DNAm) profiling is indispensable for unveiling how DNAm regulates biological pathways and individual phenotypes. However, managing and analyzing extensive DNAm data generated from large cohort studies present computational obstacles. Apache Parquet is a data file format that allows for efficient data storage, retrieval, and manipulation, alleviating computational hurdles associated with conventional row-based formats. We here introduce MethParquet, the first R package leveraging the columnar Parquet format for efficient DNAm data analysis. It can be used for data extraction, methylation risk score calculation, epigenome-wide association analyses, and other standard post-quality control tasks. The package flexibly implements diverse regression models. Via a public methylation dataset, we show the efficiency of this package in reducing running time and RAM usage in large-scale EWAS. The MethParquet R package is publicly available on the GitHub repository https://github.com/ZWangTen/MethParquet. It includes a vignette and a toy dataset derived from a public resource.

Functional Relation Field: A Model-Agnostic Framework for Multivariate Time Series Forecasting

In multivariate time series forecasting, the most popular strategy for modeling the relationship between multiple time series is the construction of graph, where each time series is represented as a node and related nodes are connected by edges. However, the relationship between multiple time series is typically complicated, e.g. the sum of outflows from upstream nodes may be equal to the inflows of downstream nodes. Such relations widely exist in many real-world scenarios for multivariate time series forecasting, yet are far from well studied. In these cases, graph might be insufficient for modeling the complex dependency between nodes. To this end, we explore a new framework to model the inter-node relationship in a more precise way based our proposed inductive bias, Functional Relation Field, where a group of functions parameterized by neural networks are learned to characterize the dependency between multiple time series. Essentially, these learned functions then form a “field”, i.e. a particular set of constraints, to regularize the training loss of the backbone prediction network and enforce the inference process to satisfy these constraints. Since our framework introduces the relationship bias in a data-driven manner, it is flexible and model-agnostic such that it can be applied to any existing multivariate time series prediction networks for boosting performance. The experiment is conducted on one toy dataset to show our approach can well recover the true constraint relationship between nodes. And various real-world datasets are also considered with different backbone prediction networks. Results show that the prediction error can be reduced remarkably with the aid of the proposed framework.

Large-scale self-normalizing neural networks

Self-normalizing neural networks (SNN) regulate the activation and gradient flows through activation functions with the self-normalization property. As SNNs do not rely on norms computed from minibatches, they are more friendly to data parallelism, kernel fusion, and emerging architectures such as ReRAM-based accelerators. However, existing SNNs have mainly demonstrated their effectiveness on toy datasets and fall short in accuracy when dealing with large-scale tasks like ImageNet. They lack the strong normalization, regularization, and expression power required for wider, deeper models and larger-scale tasks. To enhance the normalization strength, this paper introduces a comprehensive and practical definition of the self-normalization property in terms of the stability and attractiveness of the statistical fixed points. It is comprehensive as it jointly considers all the fixed points used by existing studies: the first and second moment of forward activation and the expected Frobenius norm of backward gradient. The practicality comes from the analytical equations provided by our paper to assess the stability and attractiveness of each fixed point, which are derived from theoretical analysis of the forward and backward signals. The proposed definition is applied to a meta activation function inspired by prior research, leading to a stronger self-normalizing activation function named “bi-scaled exponential linear unit with backward standardized” (bSELU-BSTD). We provide both theoretical and empirical evidence to show that it is superior to existing studies. To enhance the regularization and expression power, we further propose scaled-Mixup and channel-wise scale & shift. With these three techniques, our approach achieves 75.23% top-1 accuracy on the ImageNet with Conv MobileNet V1, surpassing the performance of existing self-normalizing activation functions. To the best of our knowledge, this is the first SNN that achieves comparable accuracy to batch normalization on ImageNet.

HyperFast: Instant Classification for Tabular Data

Training deep learning models and performing hyperparameter tuning can be computationally demanding and time-consuming. Meanwhile, traditional machine learning methods like gradient-boosting algorithms remain the preferred choice for most tabular data applications, while neural network alternatives require extensive hyperparameter tuning or work only in toy datasets under limited settings. In this paper, we introduce HyperFast, a meta-trained hypernetwork designed for instant classification of tabular data in a single forward pass. HyperFast generates a task-specific neural network tailored to an unseen dataset that can be directly used for classification inference, removing the need for training a model. We report extensive experiments with OpenML and genomic data, comparing HyperFast to competing tabular data neural networks, traditional ML methods, AutoML systems, and boosting machines. HyperFast shows highly competitive results, while being significantly faster. Additionally, our approach demonstrates robust adaptability across a variety of classification tasks with little to no fine-tuning, positioning HyperFast as a strong solution for numerous applications and rapid model deployment. HyperFast introduces a promising paradigm for fast classification, with the potential to substantially decrease the computational burden of deep learning. Our code, which offers a scikit-learn-like interface, along with the trained HyperFast model, can be found at https://github.com/AI-sandbox/HyperFast.

Instance transfer partial least squares for semi-supervised adaptive soft sensor

In order to deal with the unknown data drift problem caused by the change of working conditions in the process of industrial soft sensor modeling, this paper proposes a semi-supervised soft sensor modeling algorithm based on instance transfer learning, named instance transfer partial least squares. Under the framework of nonlinear partial least squares, the algorithm introduces the instance transfer inner model, and realizes the distribution adaptation of the modeling data and the data to be predicted through sample importance weighting. First, a nonlinear partial least squares model is constructed by using labeled samples in the source domain, which consists of a linear outer model and a nonlinear inner model. The linear outer model maps the original data to the hidden variable space, and the nonlinear inner model realizes the nonlinear modeling of hidden variables through kernel ridge regression. Second, a small number of labeled samples from the target domain are used to update the model weights of the nonlinear inner model, enabling adaptation of the latent variable distribution. The effectiveness of the algorithm is verified on a toy dataset, a simulation dataset and a real pulverized coal power plant dataset.

Characterization of stability of dynamic particle ensemble systems using topological data analysis.

Holes are ubiquitous structures in phase space, and their time evolution could indicate an instability in the dynamics of the system. However, the properties of these holes are difficult to study directly due to their theoretical complexity and lack of computational tools. This study proposes the use of persistent homology (PH), a technique from topological data analysis, as a computational tool for analyzing the properties of these phase-space holes, or more formally the H1 homology class according to PH. Initially, by using a toy data set, it is shown that the time evolution and the growth rate of a H1 class in phase space could be obtained by PH. For further validation, PH is applied to particle ensemble systems, such as the Hamiltonian flow and the two-stream instability (TSI). Both the stable case, where no H1 forms, and the unstable case, where H1 forms, were analyzed. It was shown that PH can distinguish between the stable and unstable cases purely from the phase-space time evolution plots. In unstable TSI, the PH also distinguished the transition of the H1 class from linear to non-linear growth. The growth rate, thus, obtained is in excellent agreement with the growth rate of the particle energy in the TSI system.

Robust object detection for autonomous driving based on semi-supervised learning

Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled data. How can people utilize the unlabeled data effectively? Research has focused on unsupervised and semi-supervised learning to solve such a problem. Some theoretical and empirical studies have proved that unlabeled data can help boost the generalization ability and robustness under adversarial attacks. However, current theoretical research on the relationship between robustness and unlabeled data limits its scope to toy datasets. Meanwhile, the visual models in autonomous driving need a significant improvement in robustness to guarantee security and safety. This paper proposes a semi-supervised learning framework for object detection in autonomous vehicles, improving the robustness with unlabeled data. Firstly, we build a baseline with the transfer learning of an unsupervised contrastive learning method—Momentum Contrast (MoCo). Secondly, we propose a semi-supervised co-training method to label the unlabeled data for retraining, which improves generalization on the autonomous driving dataset. Thirdly, we apply the unsupervised Bounding Box data augmentation (BBAug) method based on a search algorithm, which uses reinforcement learning to improve the robustness of object detection for autonomous driving. We present an empirical study on the KITTI dataset with diverse adversarial attack methods. Our proposed method realizes the state-of-the-art generalization and robustness under white-box attacks (DPatch and Contextual Patch) and black-box attacks (Gaussian noise, Rain, Fog, and so on). Our proposed method and empirical study show that using more unlabeled data benefits the robustness of perception systems in autonomous driving.

Synthesis-based imaging-differentiation representation learning for multi-sequence 3D/4D MRI

Multi-sequence MRIs can be necessary for reliable diagnosis in clinical practice due to the complimentary information within sequences. However, redundant information exists across sequences, which interferes with mining efficient representations by learning-based models. To handle various clinical scenarios, we propose a sequence-to-sequence generation framework (Seq2Seq) for imaging-differentiation representation learning. In this study, not only do we propose arbitrary 3D/4D sequence generation within one model to generate any specified target sequence, but also we are able to rank the importance of each sequence based on a new metric estimating the difficulty of a sequence being generated. Furthermore, we also exploit the generation inability of the model to extract regions that contain unique information for each sequence. We conduct extensive experiments using three datasets including a toy dataset of 20,000 simulated subjects, a brain MRI dataset of 1251 subjects, and a breast MRI dataset of 2101 subjects, to demonstrate that (1) top-ranking sequences can be used to replace complete sequences with non-inferior performance; (2) combining MRI with our imaging-differentiation map leads to better performance in clinical tasks such as glioblastoma MGMT promoter methylation status prediction and breast cancer pathological complete response status prediction. Our code is available at https://github.com/fiy2W/mri_seq2seq.

Learning Kernel Stein Discrepancy for Training Energy-Based Models

The primary challenge in unsupervised learning is training unnormalized density models and then generating similar samples. Few traditional unnormalized models know what the quality of the trained model is, as most models are evaluated by downstream tasks and often involve complex sampling processes. Kernel Stein Discrepancy (KSD), a goodness-of-fit test method, can measure the discrepancy between the generated samples and the theoretical distribution; therefore, it can be employed to measure the quality of trained models. We first demonstrate that, under certain constraints, KSD is equal to Maximum Mean Discrepancy (MMD), a two-sample test method. PT KSD GAN (Kernel Stein Discrepancy Generative Adversarial Network with a Pulling-Away Term) is produced to compel generated samples to approximate the theoretical distribution. The generator, functioning as an implicit generative model, employs KSD as loss to avoid tedious sampling processes. In contrast, the discriminator is trained to identify the data manifold, also known as an explicit energy-based model. To demonstrate the effectiveness of our approach, we undertook experiments on two-dimensional toy datasets. Our results highlight that our generator adeptly captures the accurate density distribution, while the discriminator proficiently recognizes the unnormalized approximate distribution shape. When applied to linear Independent Component Analysis datasets, the log likelihoods of PT KSD GAN improve by about 5‰ over existing methods when the data dimension is less than 30. Furthermore, our tests on image datasets reveal that the PT KSD GAN excels in navigating high-dimensional challenges, yielding authentically genuine samples.

Weighing risks and benefits in the presence of competing risks.

When competing events occur, there are two main options for handling them analytically that invoke different assumptions: 1) censor person-time after a competing event (which is akin to assuming they could be prevented) to calculate a conditional risk; or 2) do not censor them (allow them to occur) to calculate an unconditional risk. The choice of estimand has implications when weighing the relative frequency of a beneficial outcome and an adverse outcome in a risk-benefit analysis. We review the assumptions and interpretations underlying the two main approaches to analyzing competing risks. Using a popular metric in risk-benefit analyses, the Benefit-Risk Ratio, and a toy dataset, we demonstrated that conclusions about whether a treatment was more beneficial or more harmful can depend on whether one uses conditional or unconditional risks. We argue that unconditional risks are more relevant to decision-making about exposures with competing outcomes than conditional risks.

CMVAE: Causal Meta VAE for Unsupervised Meta-Learning

Unsupervised meta-learning aims to learn the meta knowledge from unlabeled data and rapidly adapt to novel tasks. However, existing approaches may be misled by the context-bias (e.g. background) from the training data. In this paper, we abstract the unsupervised meta-learning problem into a Structural Causal Model (SCM) and point out that such bias arises due to hidden confounders. To eliminate the confounders, we define the priors are conditionally independent, learn the relationships between priors and intervene on them with casual factorization. Furthermore, we propose Causal Meta VAE (CMVAE) that encodes the priors into latent codes in the causal space and learns their relationships simultaneously to achieve the downstream few-shot image classification task. Results on toy datasets and three benchmark datasets demonstrate that our method can remove the context-bias and it outperforms other state-of-the-art unsupervised meta-learning algorithms because of bias-removal. Code is available at https://github.com/GuodongQi/CMVAE.

Neural Integro-Differential Equations

Modeling continuous dynamical systems from discretely sampled observations is a fundamental problem in data science. Often, such dynamics are the result of non-local processes that present an integral over time. As such, these systems are modeled with Integro-Differential Equations (IDEs); generalizations of differential equations that comprise both an integral and a differential component. For example, brain dynamics are not accurately modeled by differential equations since their behavior is non-Markovian, i.e. dynamics are in part dictated by history. Here, we introduce the Neural IDE (NIDE), a novel deep learning framework based on the theory of IDEs where integral operators are learned using neural networks. We test NIDE on several toy and brain activity datasets and demonstrate that NIDE outperforms other models. These tasks include time extrapolation as well as predicting dynamics from unseen initial conditions, which we test on whole-cortex activity recordings in freely behaving mice. Further, we show that NIDE can decompose dynamics into their Markovian and non-Markovian constituents, via the learned integral operator, which we test on fMRI brain activity recordings of people on ketamine. Finally, the integrand of the integral operator provides a latent space that gives insight into the underlying dynamics, which we demonstrate on wide-field brain imaging recordings. Altogether, NIDE is a novel approach that enables modeling of complex non-local dynamics with neural networks.

Progressive Deep Multi-View Comprehensive Representation Learning

Multi-view Comprehensive Representation Learning (MCRL) aims to synthesize information from multiple views to learn comprehensive representations of data items. Prevalent deep MCRL methods typically concatenate synergistic view-specific representations or average aligned view-specific representations in the fusion stage. However, the performance of synergistic fusion methods inevitably degenerate or even fail when partial views are missing in real-world applications; the aligned based fusion methods usually cannot fully exploit the complementarity of multi-view data. To eliminate all these drawbacks, in this work we present a Progressive Deep Multi-view Fusion (PDMF) method. Considering the multi-view comprehensive representation should contain complete information and the view-specific data contain partial information, we deem that it is unstable to directly learn the mapping from partial information to complete information. Hence, PDMF employs a progressive learning strategy, which contains the pre-training and fine-tuning stages. In the pre-training stage, PDMF decodes the auxiliary comprehensive representation to the view-specific data. It also captures the consistency and complementarity by learning the relations between the dimensions of the auxiliary comprehensive representation and all views. In the fine-tuning stage, PDMF learns the mapping from the original data to the comprehensive representation with the help of the auxiliary comprehensive representation and relations. Experiments conducted on a synthetic toy dataset and 4 real-world datasets show that PDMF outperforms state-of-the-art baseline methods. The code is released at https://github.com/winterant/PDMF.

Statistical Machine-Learning Methods for Genomic Prediction Using the SKM Library.

Genomic selection (GS) is revolutionizing plant breeding. However, because it is a predictive methodology, a basic understanding of statistical machine-learning methods is necessary for its successful implementation. This methodology uses a reference population that contains both the phenotypic and genotypic information of genotypes to train a statistical machine-learning method. After optimization, this method is used to make predictions of candidate lines for which only genotypic information is available. However, due to a lack of time and appropriate training, it is difficult for breeders and scientists of related fields to learn all the fundamentals of prediction algorithms. With smart or highly automated software, it is possible for these professionals to appropriately implement any state-of-the-art statistical machine-learning method for its collected data without the need for an exhaustive understanding of statistical machine-learning methods and programing. For this reason, we introduce state-of-the-art statistical machine-learning methods using the Sparse Kernel Methods (SKM) R library, with complete guidelines on how to implement seven statistical machine-learning methods that are available in this library for genomic prediction (random forest, Bayesian models, support vector machine, gradient boosted machine, generalized linear models, partial least squares, feed-forward artificial neural networks). This guide includes details of the functions required to implement each of the methods, as well as others for easily implementing different tuning strategies, cross-validation strategies, and metrics to evaluate the prediction performance and different summary functions that compute it. A toy dataset illustrates how to implement statistical machine-learning methods and facilitate their use by professionals who do not possess a strong background in machine learning and programing.

Code and Data Repository for Black-Box Attack-Based Security Evaluation Framework for Credit Card Fraud Detection Models

The security of credit card fraud detection (CCFD) models based on machine learning is important but rarely considered in the existing research. To this end, we propose a black-box attack-based security evaluation framework for CCFD models. Under this framework, the semi-supervised learning technique and transfer-based black-box attack are combined to construct two versions of semi-supervised transfer black-box attack (STBA) algorithm. Moreover, we introduce a new nonlinear optimization model to generate the adversarial examples against CCFD models and a security evaluation index to quantitatively evaluate the security of them. This project contains four folders: data, results, src, scripts. data：include two datasets used in the paper and a toy dataset for debugging. results: include the experimental results. src: include the source code. scripts: include two scripts for evaluating the security of machine learning models based on substitute models LR and SVM.