Machine Learning Tasks Research Articles

Measuring functional connectivity in frequency-domain helps to better characterize brain function.

Resting-state functional connectivity (FC) is widely used in multivariate pattern analysis of functional magnetic resonance imaging (fMRI), including identifying the locations of putative brain functional borders, predicting individual phenotypes, and diagnosing clinical mental diseases. However, limited attention has been paid to the analysis of functional interactions from a frequency perspective. In this study, by contrasting coherence-based and correlation-based FC with two machine learning tasks, we observed that measuring FC in the frequency domain helped to identify finer functional subregions and achieve better pattern discrimination capability relative to the temporal correlation. This study has proven the feasibility of coherence in the analysis of fMRI, and the results indicate that modeling functional interactions in the frequency domain may provide richer information than that in the time domain, which may provide a new perspective on the analysis of functional neuroimaging.

Molecular set representation learning

Computational representation of molecules can take many forms, including graphs, string encodings of graphs, binary vectors or learned embeddings in the form of real-valued vectors. These representations are then used in downstream classification and regression tasks using a wide range of machine learning models. However, existing models come with limitations, such as the requirement for clearly defined chemical bonds, which often do not represent the true underlying nature of a molecule. Here we propose a framework for molecular machine learning tasks based on set representation learning. We show that learning on sets of atom invariants alone reaches the performance of state-of-the-art graph-based models on the most-used chemical benchmark datasets and that introducing a set representation layer into graph neural networks can surpass the performance of established methods in the domains of chemistry, biology and material science. We introduce specialized set representation-based neural network architectures for reaction-yield and protein–ligand binding-affinity prediction. Overall, we show that the technique we denote molecular set representation learning is both an alternative and an extension to graph neural network architectures for machine learning tasks on molecules, molecule complexes and chemical reactions.

Hilbert Curve Projection Distance for Distribution Comparison.

Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling. In this study, we propose a novel metric, called Hilbert curve projection (HCP) distance, to measure the distance between two probability distributions with low complexity. In particular, we first project two high-dimensional probability distributions using Hilbert curve to obtain a coupling between them, and then calculate the transport distance between these two distributions in the original space, according to the coupling. We show that HCP distance is a proper metric and is well-defined for probability measures with bounded supports. Furthermore, we demonstrate that the modified empirical HCP distance with the Lp cost in the d-dimensional space converges to its population counterpart at a rate of no more than O(n-1/2max{d,p}). To suppress the curse-of-dimensionality, we also develop two variants of the HCP distance using (learnable) subspace projections. Experiments on both synthetic and real-world data show that our HCP distance works as an effective surrogate of the Wasserstein distance with low complexity and overcomes the drawbacks of the sliced Wasserstein distance.

Compact: Approximating Complex Activation Functions for Secure Computation

Secure multi-party computation (MPC) techniques can be used to provide data privacy when users query deep neural network (DNN) models hosted on a public cloud. State-of-the-art MPC techniques can be directly leveraged for DNN models that use simple activation functions (AFs) such as ReLU. However, these techniques are ineffective and/or inefficient for the complex and highly non-linear AFs used in cutting-edge DNN models. We present Compact, which produces piece-wise polynomial approximations of complex AFs to enable their efficient use with state-of-the-art MPC techniques. Compact neither requires nor imposes any restriction on model training and results in near-identical model accuracy. To achieve this, we design Compact with input density awareness, and use an application specific simulated annealing type optimization to generate computationally more efficient approximations of complex AFs. We extensively evaluate Compact on four different machine-learning tasks with DNN architectures that use popular complex AFs silu, gelu, and mish. Our experimental results show that Compact incurs negligible accuracy loss while being 2x-5x computationally more efficient than state-of-the-art approaches for DNN models with large number of hidden layers. Our work accelerates easy adoption of MPC techniques to provide user data privacy even when the queried DNN models consist of a number of hidden layers, and trained over complex AFs.

An energy-saving joint resource allocation strategy for mobile edge computing

Mobile Cloud-Edge Collaboration (MCEC) views in the main of converting the site for user electronics. By naturally integrating mobile devices with cloud computing (CC) resources at the edge of the scheme, this mutual paradigm improves storage, processing, and communication capabilities. This cooperation increases the performance of user electronics, delivering users responsive and resource-efficient knowledge. Offloading in Mobile Cloud-Edge Collaboration (MCEC) is a strategic device that recovers computational efficiency and resource energy for mobile devices. By reasonably moving computation tasks from mobile devices to the edge or cloud servers, offloading declines the load on the limited processing and energy capabilities of mobile devices. This joint method influences the stable computing power and storage aptitude accessible in the cloud-edge structure, confirming that resource-intensive uses like complex data processing or machine learning (ML) tasks can be implemented professionally. Offloading not only increases the receptiveness and performance of mobile users but also contributes to energy conservation, extending the battery time of mobile devices. This study proposes an African Vultures Optimizer algorithm-based Offloading Strategy for Mobile Cloud-Edge Collaboration (AVOAOS-MCEC) approach for consumer electronics. The AVOAOS-MCEC technique is based on the nature of AVOA is a new nature-based system, which is inspired by the unusual behavior of African vultures in foraging and navigation. In addition, the AVOAOS-MCEC technique designs a task offloading process to reduce the total energy utilization with the fulfillment of capacity and delay requirements. The experimental validation of the AVOAOS-MCEC method is verified utilizing distinct measures. An extensive comparison study stated that the AVOAOS-MCEC technique outperforms the other models in terms of several performance measures.

Beyond simulation: Unlocking the frontiers of humanoid robot capability and intelligence with Pepper's open-source digital twin

This research paper presents a high-fidelity, open-source digital-twin of the Pepper robot developed within the framework of the Robot Operating System 2 (ROS 2) for better simulation realism in complex tasks of machine learning. We developed a dedicated, custom ROS 2 package with modern simulation tools, such as Gazebo Sim, MoveIt 2, Rviz2, that brings complete, realistic environments in line with the exact behaviors and interactions of robots in reality. Better accuracy of the physical movement of Pepper robot's simulation was shown on the digital twin, validated by the Choregraphe software and real robot performance, to be a strong platform of collaboration and further research by the community. This development greatly pushes the envelope of human-like humanoid robotics further by offering a scaled, flexible, and plausible training environment conducive to integrating complex algorithms of robot learning and interaction capabilities.

Weakly Supervised AUC Optimization: A Unified Partial AUC Approach.

Since acquiring perfect supervision is usually difficult, real-world machine learning tasks often confront inaccurate, incomplete, or inexact supervision, collectively referred to as weak supervision. In this work, we present WSAUC, a unified framework for weakly supervised AUC optimization problems, which covers noisy label learning, positive-unlabeled learning, multi-instance learning, and semi-supervised learning scenarios. Within the WSAUC framework, we first frame the AUC optimization problems in various weakly supervised scenarios as a common formulation of minimizing the AUC risk on contaminated sets, and demonstrate that the empirical risk minimization problems are consistent with the true AUC. Then, we introduce a new type of partial AUC, specifically, the reversed partial AUC (rpAUC), which serves as a robust training objective for AUC maximization in the presence of contaminated labels. WSAUC offers a universal solution for AUC optimization in various weakly supervised scenarios by maximizing the empirical rpAUC. Theoretical and experimental results under multiple settings support the effectiveness of WSAUC on a range of weakly supervised AUC optimization tasks.

Stereoisomers Are Not Machine Learning's Best Friends.

This study addresses the challenge of accurately identifying stereoisomers in cheminformatics, which originates from our objective to apply machine learning to predict the association constant between cyclodextrin and a guest. Identifying stereoisomers is indeed crucial for machine learning applications. Current tools offer various molecular descriptors, including their textual representation as Isomeric SMILES that can distinguish stereoisomers. However, such representation is text-based and does not have a fixed size, so a conversion is needed to make it usable to machine learning approaches. Word embedding techniques can be used to solve this problem. Mol2vec, a word embedding approach for molecules, offers such a conversion. Unfortunately, it cannot distinguish between stereoisomers due to its inability to capture the spatial configuration of molecular structures. This study proposes several approaches that use word embedding techniques to handle molecular discrimination using stereochemical information on molecules or considering Isomeric SMILES notation as a text in Natural Language Processing. Our aim is to generate a distinct vector for each unique molecule, correctly identifying stereoisomer information in cheminformatics. The proposed approaches are then compared to our original machine learning task: predicting the association constant between cyclodextrin and a guest molecule.

Quantum machine learning with Qiskit: Evaluating regression accuracy and noise impact

AbstractQuantum machine learning (QML) can be employed in solving complicated machine learning tasks although the performance in examining the regression processes is only barely understood. Knowledge gaps are intended to be closed by studying modelling performance of QML in regression tasks, with emphasis being dedicated to scaling up and ability to resist noise. The regression part offers the following functions that include straight line and complex operations. Furthermore, the authors employ quantum neural networks generated using Qiskit to perform experiments. The results demonstrate that QML has a remarkable level of accuracy in basic regressions, reaching a maximum of 97%. Nevertheless, there are difficulties in representing intricate functions, such as 5 × cos(x), which results in a noticeable decline in performance. The work deals with the influence of noise and IERs from imperfect hardware on the efficiency of QML algorithms providing insight into the core obstacles. The result of a detailed examination of the results that have tested the powers and limits of QML in the development of regression applications is represented. The future direction of research and development will be defined by the results obtained in it.

Gaussian process learning of nonlinear dynamics

One of the pivotal tasks in scientific machine learning is to represent underlying dynamical systems from time series data. Many methods for such dynamics learning explicitly require the derivatives of state data, which are not directly available and can be approximated conventionally by finite differences. However, the discrete approximations of time derivatives may result in poor estimations when state data are scarce and/or corrupted by noise, thus compromising the predictiveness of the learned dynamical models. To overcome this technical hurdle, we propose a new method that learns nonlinear dynamics through a Bayesian inference of characterizing model parameters. This method leverages a Gaussian process representation of states, and constructs a likelihood function using the correlation between state data and their derivatives, yet prevents explicit evaluations of time derivatives. Through a Bayesian scheme, a probabilistic estimate of the model parameters is given by the posterior distribution, and thus a quantification is facilitated for uncertainties from noisy state data and the learning process. Specifically, we will discuss the applicability of the proposed method to several typical scenarios for dynamical systems: identification and estimation with an affine parametrization, nonlinear parametric approximation without prior knowledge, and general parameter estimation for a given dynamical system.

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

Feature selection is predominantly used in machine learning tasks, such as classification, regression, and clustering. It selects a subset of features (relevant attributes of data points) from a larger set that contributes as optimally as possible to the informativeness of the model. There are exponentially many subsets of a given set, and thus, the exhaustive search approach is only practical for problems with at most a few dozen features. In the past, there have been attempts to reduce the search space using dynamic programming. However, models that consider similarity in pairs of features alongside the quality of individual features do not provide the required optimal substructure. As a result, algorithms, which we will call suboptimal dynamic programming algorithms, find a solution that may deviate significantly from the optimal one. In this paper, we propose an iterative dynamic programming algorithm, which invertsthe order of feature processing in each iteration. Such an alternating approach allows for improving the optimization function by using the score from the previous iteration to estimate the contribution of unprocessed features. The iterative process is proven to converge and terminates when the solution does not change in three successive iterations or when the number of iterations reaches the threshold. Results in more than 95% of tests align with those of the exhaustive search approach, being competitive and often superior to the reference greedy approach. Validation was carried out by comparing the scores of output feature subsets and examining the accuracy of different classifiers learned on these features across nine real-world applications, considering different scenarios with various numbers of features and samples. In the context of feature selection, the proposed algorithm can be characterized as a robust filter method that can improve machine learning models regardless of dataset size. However, we expect that the idea of alternating suboptimal optimization will soon be generalized to tasks beyond feature selection.

Exploring latent discrimination through an Object-Relational Causal Inference method

This paper addresses the critical but often overlooked problem of exploring latent sensitive attributes in machine-learning datasets where they are not explicitly present. We propose a method named Object-Relational Causal Inference (ORCI), solving the problem with good reusability. We mathematically demonstrate the enhancement of data fairness by uncovering latent sensitive attributes and build a framework to tackle the latent discrimination exploration challenge. Firstly, we highlight the limitations of classical causal discovery methods and introduce an approximate exhaustive search algorithm called BIC-based Exhaustive Search (BICES). Then, we integrate the Object Relational (OR) approach into causal modeling, enabling the subsequent structured probabilistic inference based on the OR causal model to derive fair and effective latent sensitive attributes. We conduct simulations and experiments on synthetic and real-world cases across various machine-learning tasks, with discrete and continuous data, and in diverse settings. Results show that our method successfully uncovers latent sensitive attributes, enhancing the data fairness while preserving data utility. Besides, our detailed theoretical derivations and rich experiments demonstrate the generalization of our method.

LLM is All You Need : How Do LLMs Perform On Prediction and Classification Using Historical Data

This study investigates the utility of large language models (LLMs) in performing traditional machine learning tasks such as prediction, and explores the potential of refinement architectures to enhance their effectiveness in these roles. Utilizing the Titanic survival dataset, we conducted a com- parative analysis using both conventional machine learning tools and LLM-based approaches. Our findings indicate that while LLMs differ fundamentally from traditional ML models in prediction tasks, there exist specific architectural modifications, termed Thought Refinement Architectures, which can significantly improve their performance. These results highlight the potential for inte- grating LLMs into traditional ML workflows, thereby expanding their applicability and enhancing predictive accuracy.

NetDiffus: Network traffic generation by diffusion models through time-series imaging

While Machine-Learning based network data analytics are now commonplace for many networking solutions, nonetheless, limited access to appropriate networking data has been an enduring challenge for many networking problems. Causes for lack of such data include complexity of data gathering, commercial sensitivity, as well as privacy and regulatory constraints. To overcome these challenges, we present a Diffusion-Model (DM) based end-to-end framework, NetDiffus, for synthetic network traffic generation which is one of the emerging topics in networking and computing system. NetDiffus first converts one-dimensional time-series network traffic into two-dimensional images, and then synthesizes representative images for the original data. We demonstrate that NetDiffus outperforms the state-of-the-art traffic generation methods based on Generative Adversarial Networks (GANs) by providing 66.4% increase in the fidelity of the generated data and an 18.1% increase in downstream machine learning tasks. We evaluate NetDiffus on seven diverse traffic traces and show that utilizing synthetic data significantly improves several downstream ML tasks including traffic fingerprinting, anomaly detection and traffic classification. The code has been included at https://github.com/Nirhoshan/NetDiffus.

Intelligent mechanical metamaterials towards learning static and dynamic behaviors

The exploration of intelligent machines has recently spurred the development of physical neural networks, a class of intelligent metamaterials capable of learning, whether in silico or in situ, from observed data. In this study, we introduce a back-propagation framework for lattice-based mechanical neural networks (MNNs) to achieve prescribed static and dynamic performance. This approach leverages the steady states of nodes for back-propagation, efficiently updating the learning degrees of freedom without prior knowledge of input loading. One-dimensional MNNs, trained with back-propagation in silico, can exhibit the desired behaviors on demand function as intelligent mechanical machines. The framework is then employed for the precise morphing control of the two-dimensional MNNs subjected to different static loads. Moreover, the intelligent MNNs are trained to execute classical machine learning tasks such as regression to tackle various deformation control tasks. Finally, the disordered MNNs are constructed and trained to demonstrate pre-programmed wave bandgap control ability, illustrating the versatility of the proposed approach as a platform for physical learning. Our approach presents an efficient pathway for the design of intelligent mechanical metamaterials for a wide range of static and dynamic target functionalities, positioning them as powerful engines for physical learning.

Multi-view clustering via latent consistency multi-graph fusion

Multi-view clustering (MVC) algorithms are prevalent in machine learning tasks due to their superiority in mining complementary information from multi-view data. Despite their widespread use, existing MVC approaches ignore the presence of inconsistencies in the multi-view features during multi-view data fusion, which leads to suboptimal clustering performance. To address this challenge, the paper presents a novel approach called multi-view clustering via latent consistency multi-graph fusion. Specifically, a novel latent representation learning model is devised to learn a unified representation from multi-view features, simultaneously capturing the high-order relationships within the multi-view data. Additionally, to alleviate the adverse effects caused by the inconsistency multi-view feature, a consistent multi-graph fusion module is proposed to provide consistent constraints for latent representations. To be specific, the graphs that reflect the manifold structure for each view are fused to guide a consistent latent representation learning. After that, an alternating optimization algorithm is designed to update variables iteratively and alternatively, among which the fusion weights are assigned adaptively. Finally, comprehensive experiments conducted on seven widely recognized benchmark multi-view datasets affirm the efficacy of the proposed method, showcasing its superiority over state-of-the-art approaches.

MIRACLE—a microphone array impulse response dataset for acoustic learning

This work introduces a large dataset comprising impulse responses of spatially distributed sources within a plane parallel to a planar microphone array. The dataset, named MIRACLE, encompasses 856,128 single-channel impulse responses and includes four different measurement scenarios. Three measurement scenarios were conducted under anechoic conditions. The fourth scenario includes an additional specular reflection from a reflective panel. The source positions were obtained by uniformly discretizing a rectangular source plane parallel to the microphone for each scenario. The dataset contains three scenarios with a spatial resolution of 23mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$23\\,\ extrm{mm}$$\\end{document} at two different source-plane-to-array distances, as well as a scenario with a resolution of 5mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5\\,\ extrm{mm}$$\\end{document} for the shorter distance. In contrast to existing room impulse response datasets, the accuracy of the provided source location labels is assessed and additional metadata, such as the directivity of the loudspeaker used for excitation, is provided. The MIRACLE dataset can be used as a benchmark for data-driven modelling and interpolation methods as well as for various acoustic machine learning tasks, such as source separation, localization, and characterization. Two timely applications of the dataset are presented in this work: the generation of microphone array data for data-driven source localization and characterization tasks and data-driven model order reduction.

Data oversampling and imbalanced datasets: an investigation of performance for machine learning and feature engineering

The classification of imbalanced datasets is a prominent task in text mining and machine learning. The number of samples in each class is not uniformly distributed; one class contains a large number of samples while the other has a small number. Overfitting of the model occurs as a result of imbalanced datasets, resulting in poor performance. In this study, we compare different oversampling techniques like synthetic minority oversampling technique (SMOTE), support vector machine SMOTE (SVM-SMOTE), Border-line SMOTE, K-means SMOTE, and adaptive synthetic (ADASYN) oversampling to address the issue of imbalanced datasets and enhance the performance of machine learning models. Preprocessing significantly enhances the quality of input data by reducing noise, redundant data, and unnecessary data. This enables the machines to identify crucial patterns that facilitate the extraction of significant and pertinent information from the preprocessed data. This study preprocesses the data using various top-level preprocessing steps. Furthermore, two imbalanced Twitter datasets are used to compare the performance of oversampling techniques with six machine learning models including random forest (RF), SVM, K-nearest neighbor (KNN), AdaBoost (ADA), logistic regression (LR), and decision tree (DT). In addition, the bag of words (BoW) and term frequency and inverse document frequency (TF-IDF) features extraction approaches are used to extract features from the tweets. The experiments indicate that SMOTE and ADASYN perform much better than other techniques thus providing higher accuracy. Additionally, overall results show that SVM with ’linear’ kernel tends to attain the highest accuracy and recall score of 99.67% and 1.00% on ADASYN oversampled datasets and 99.57% accuracy on SMOTE oversampled dataset with TF-IDF features. The SVM model using 10-fold cross-validation experiments achieved 97.40 mean accuracy with a 0.008 standard deviation. Our approach achieved 2.62% greater accuracy as compared to other current methods.

The shift-dimension of multipersistence modules

We present the shift-dimension of multipersistence modules and investigate its algebraic properties. This gives rise to a new invariant of multigraded modules over the multivariate polynomial ring arising from the hierarchical stabilization of the zeroth total multigraded Betti number. We give a fast algorithm for the computation of the shift-dimension of interval modules in the bivariate case. We construct multipersistence contours that are parameterized by multivariate functions and hence provide a large class of feature maps for machine learning tasks.

Optimization of array encoding for ultrasound imaging

Objective. The transmit encoding model for synthetic aperture imaging is a robust and flexible framework for understanding the effects of acoustic transmission on ultrasound image reconstruction. Our objective is to use machine learning (ML) to construct scanning sequences, parameterized by time delays and apodization weights, that produce high-quality B-mode images. Approach. We use a custom ML model in PyTorch with simulated RF data from Field II to probe the space of possible encoding sequences for those that minimize a loss function that describes image quality. This approach is made computationally feasible by a novel formulation of the derivative for delay-and-sum beamforming. Main results. When trained for a specified experimental setting (imaging domain, hardware restrictions, etc), our ML model produces optimized encoding sequences that, when deployed in the REFoCUS imaging framework, improve a number of standard quality metrics over conventional sequences including resolution, field of view, and contrast. We demonstrate these results experimentally on both wire targets and a tissue-mimicking phantom. Significance. This work demonstrates that the set of commonly used encoding schemes represent only a narrow subset of those available. Additionally, it demonstrates the value for ML tasks in synthetic transmit aperture imaging to consider the beamformer within the model, instead of purely as a post-processing step.