Machine Learning Tasks Research Articles

Image authentication method based on Fourier zero-frequency replacement and single-pixel self-calibration imaging by diffractive deep neural network.

The diffractive deep neural network is a novel network model that applies the principles of diffraction to neural networks, enabling machine learning tasks to be performed through optical principles. In this paper, a fully optical authentication model is developed using the diffractive deep neural network. The model utilizes terahertz light for propagation and combines it with a self-calibration single-pixel imaging model to construct a comprehensive optical authentication system with faster authentication speed. The proposed system filters the authentication images, establishes an optical connection with the Fourier zero-frequency response of the illumination pattern, and introduces the signal-to-noise ratio as a criterion for batch image authentication. Computer simulations demonstrate the fast speed and strong automation performance of the proposed optical authentication system, suggesting broad prospects for the combined application of diffractive deep neural networks and optical systems.

Testing the consistency of performance scores reported for binary classification problems

Binary classification is a fundamental task in machine learning, with applications across various scientific domains. Whether conducting fundamental research or refining practical applications, scientists typically assess and rank classification techniques based on performance metrics such as accuracy, sensitivity, and specificity. However, reported performance scores may not always provide a reliable basis for research ranking. The unreliability can be attributed to undisclosed or unconventional practices related to cross-validation, typographical errors, and other factors.In a given experimental setup with a specific number of positive and negative test items, performance scores can only assume specific, interrelated values. Based on this observation, in this paper, we introduce numerical techniques to assess the consistency of reported performance scores with the assumed experimental setup. Importantly, the proposed approaches do not rely on statistical inference. Instead, they use numerical methods (interval computing and integer linear programming) to identify inconsistencies with certainty.Through three applications in different fields of medicine, we demonstrate how the proposed tests can detect inconsistencies, thereby safeguarding the integrity of research fields. The power analyses of the tests in these applications show at least 71% of power when the performance scores are reported to four decimal places. In the investigated areas, the tests have so far identified inconsistencies in more than 100 scientific papers. To benefit the scientific community, we have made the consistency tests available in the open-source Python package mlscorecheck.

Online Learning With Uncertain Feedback Graphs.

Online learning with expert advice is widely used in various machine learning tasks. It considers the problem where a learner chooses one from a set of experts to take advice and make a decision. In many learning problems, experts may be related, henceforth the learner can observe the losses associated with a subset of experts that are related to the chosen one. In this context, the relationship among experts can be captured by a feedback graph, which can be used to assist the learner's decision-making. However, in practice, the nominal feedback graph often entails uncertainties, which renders it impossible to reveal the actual relationship among experts. To cope with this challenge, the present work studies various cases of potential uncertainties and develops novel online learning algorithms to deal with uncertainties while making use of the uncertain feedback graph. The proposed algorithms are proved to enjoy sublinear regret under mild conditions. Experiments on real datasets are presented to demonstrate the effectiveness of the novel algorithms.

Photon discerner: adaptive quantum optical sensing near the shot noise limit

Photon statistics of an optical field can be used for quantum optical sensing in low light level scenarios free of bulky optical components. However, photon-number-resolving detection to unravel the photon statistics is challenging. Here, we propose a novel detection approach, that we call ‘photon discerning’, which uses adaptive photon thresholding for photon statistical estimation without recording exact photon numbers. Our photon discerner is motivated by the field of neural networks where tunable thresholds have proven efficient for information extraction in machine learning tasks. The photon discerner maximizes Fisher information per photon by iteratively choosing the optimal threshold in real-time to approach the shot noise limit. Our proposed scheme of adaptive photon thresholding leads to unique remote-sensing applications of quantum degree of linear polarization camera and quantum LiDAR. We investigate optimal thresholds and show that the optimal photon threshold can be counter-intuitive (not equal to 1) even for weak signals (mean photon number much less than 1), due to the photon bunching effect. We also put forth a superconducting nanowire realization of the photon discerner which can be experimentally implemented in the near-term. We show that the adaptivity of our photon discerner enables it to beat realistic photon-number-resolving detectors with limited photon-number resolution in certain applications. Our work suggests a new class of detectors for information-theory driven, compact, and learning-based quantum optical sensing.

Measuring functional connectivity in frequency-domain helps to better characterize brain function.

Resting-state functional connectivity (FC) is widely used in multivariate pattern analysis of functional magnetic resonance imaging (fMRI), including identifying the locations of putative brain functional borders, predicting individual phenotypes, and diagnosing clinical mental diseases. However, limited attention has been paid to the analysis of functional interactions from a frequency perspective. In this study, by contrasting coherence-based and correlation-based FC with two machine learning tasks, we observed that measuring FC in the frequency domain helped to identify finer functional subregions and achieve better pattern discrimination capability relative to the temporal correlation. This study has proven the feasibility of coherence in the analysis of fMRI, and the results indicate that modeling functional interactions in the frequency domain may provide richer information than that in the time domain, which may provide a new perspective on the analysis of functional neuroimaging.

Molecular set representation learning

Computational representation of molecules can take many forms, including graphs, string encodings of graphs, binary vectors or learned embeddings in the form of real-valued vectors. These representations are then used in downstream classification and regression tasks using a wide range of machine learning models. However, existing models come with limitations, such as the requirement for clearly defined chemical bonds, which often do not represent the true underlying nature of a molecule. Here we propose a framework for molecular machine learning tasks based on set representation learning. We show that learning on sets of atom invariants alone reaches the performance of state-of-the-art graph-based models on the most-used chemical benchmark datasets and that introducing a set representation layer into graph neural networks can surpass the performance of established methods in the domains of chemistry, biology and material science. We introduce specialized set representation-based neural network architectures for reaction-yield and protein–ligand binding-affinity prediction. Overall, we show that the technique we denote molecular set representation learning is both an alternative and an extension to graph neural network architectures for machine learning tasks on molecules, molecule complexes and chemical reactions.

Hilbert Curve Projection Distance for Distribution Comparison.

Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling. In this study, we propose a novel metric, called Hilbert curve projection (HCP) distance, to measure the distance between two probability distributions with low complexity. In particular, we first project two high-dimensional probability distributions using Hilbert curve to obtain a coupling between them, and then calculate the transport distance between these two distributions in the original space, according to the coupling. We show that HCP distance is a proper metric and is well-defined for probability measures with bounded supports. Furthermore, we demonstrate that the modified empirical HCP distance with the Lp cost in the d-dimensional space converges to its population counterpart at a rate of no more than O(n-1/2max{d,p}). To suppress the curse-of-dimensionality, we also develop two variants of the HCP distance using (learnable) subspace projections. Experiments on both synthetic and real-world data show that our HCP distance works as an effective surrogate of the Wasserstein distance with low complexity and overcomes the drawbacks of the sliced Wasserstein distance.

Compact: Approximating Complex Activation Functions for Secure Computation

Secure multi-party computation (MPC) techniques can be used to provide data privacy when users query deep neural network (DNN) models hosted on a public cloud. State-of-the-art MPC techniques can be directly leveraged for DNN models that use simple activation functions (AFs) such as ReLU. However, these techniques are ineffective and/or inefficient for the complex and highly non-linear AFs used in cutting-edge DNN models. We present Compact, which produces piece-wise polynomial approximations of complex AFs to enable their efficient use with state-of-the-art MPC techniques. Compact neither requires nor imposes any restriction on model training and results in near-identical model accuracy. To achieve this, we design Compact with input density awareness, and use an application specific simulated annealing type optimization to generate computationally more efficient approximations of complex AFs. We extensively evaluate Compact on four different machine-learning tasks with DNN architectures that use popular complex AFs silu, gelu, and mish. Our experimental results show that Compact incurs negligible accuracy loss while being 2x-5x computationally more efficient than state-of-the-art approaches for DNN models with large number of hidden layers. Our work accelerates easy adoption of MPC techniques to provide user data privacy even when the queried DNN models consist of a number of hidden layers, and trained over complex AFs.

An energy-saving joint resource allocation strategy for mobile edge computing

Mobile Cloud-Edge Collaboration (MCEC) views in the main of converting the site for user electronics. By naturally integrating mobile devices with cloud computing (CC) resources at the edge of the scheme, this mutual paradigm improves storage, processing, and communication capabilities. This cooperation increases the performance of user electronics, delivering users responsive and resource-efficient knowledge. Offloading in Mobile Cloud-Edge Collaboration (MCEC) is a strategic device that recovers computational efficiency and resource energy for mobile devices. By reasonably moving computation tasks from mobile devices to the edge or cloud servers, offloading declines the load on the limited processing and energy capabilities of mobile devices. This joint method influences the stable computing power and storage aptitude accessible in the cloud-edge structure, confirming that resource-intensive uses like complex data processing or machine learning (ML) tasks can be implemented professionally. Offloading not only increases the receptiveness and performance of mobile users but also contributes to energy conservation, extending the battery time of mobile devices. This study proposes an African Vultures Optimizer algorithm-based Offloading Strategy for Mobile Cloud-Edge Collaboration (AVOAOS-MCEC) approach for consumer electronics. The AVOAOS-MCEC technique is based on the nature of AVOA is a new nature-based system, which is inspired by the unusual behavior of African vultures in foraging and navigation. In addition, the AVOAOS-MCEC technique designs a task offloading process to reduce the total energy utilization with the fulfillment of capacity and delay requirements. The experimental validation of the AVOAOS-MCEC method is verified utilizing distinct measures. An extensive comparison study stated that the AVOAOS-MCEC technique outperforms the other models in terms of several performance measures.

Beyond simulation: Unlocking the frontiers of humanoid robot capability and intelligence with Pepper's open-source digital twin

This research paper presents a high-fidelity, open-source digital-twin of the Pepper robot developed within the framework of the Robot Operating System 2 (ROS 2) for better simulation realism in complex tasks of machine learning. We developed a dedicated, custom ROS 2 package with modern simulation tools, such as Gazebo Sim, MoveIt 2, Rviz2, that brings complete, realistic environments in line with the exact behaviors and interactions of robots in reality. Better accuracy of the physical movement of Pepper robot's simulation was shown on the digital twin, validated by the Choregraphe software and real robot performance, to be a strong platform of collaboration and further research by the community. This development greatly pushes the envelope of human-like humanoid robotics further by offering a scaled, flexible, and plausible training environment conducive to integrating complex algorithms of robot learning and interaction capabilities.

Weakly Supervised AUC Optimization: A Unified Partial AUC Approach.

Since acquiring perfect supervision is usually difficult, real-world machine learning tasks often confront inaccurate, incomplete, or inexact supervision, collectively referred to as weak supervision. In this work, we present WSAUC, a unified framework for weakly supervised AUC optimization problems, which covers noisy label learning, positive-unlabeled learning, multi-instance learning, and semi-supervised learning scenarios. Within the WSAUC framework, we first frame the AUC optimization problems in various weakly supervised scenarios as a common formulation of minimizing the AUC risk on contaminated sets, and demonstrate that the empirical risk minimization problems are consistent with the true AUC. Then, we introduce a new type of partial AUC, specifically, the reversed partial AUC (rpAUC), which serves as a robust training objective for AUC maximization in the presence of contaminated labels. WSAUC offers a universal solution for AUC optimization in various weakly supervised scenarios by maximizing the empirical rpAUC. Theoretical and experimental results under multiple settings support the effectiveness of WSAUC on a range of weakly supervised AUC optimization tasks.

Stereoisomers Are Not Machine Learning's Best Friends.

This study addresses the challenge of accurately identifying stereoisomers in cheminformatics, which originates from our objective to apply machine learning to predict the association constant between cyclodextrin and a guest. Identifying stereoisomers is indeed crucial for machine learning applications. Current tools offer various molecular descriptors, including their textual representation as Isomeric SMILES that can distinguish stereoisomers. However, such representation is text-based and does not have a fixed size, so a conversion is needed to make it usable to machine learning approaches. Word embedding techniques can be used to solve this problem. Mol2vec, a word embedding approach for molecules, offers such a conversion. Unfortunately, it cannot distinguish between stereoisomers due to its inability to capture the spatial configuration of molecular structures. This study proposes several approaches that use word embedding techniques to handle molecular discrimination using stereochemical information on molecules or considering Isomeric SMILES notation as a text in Natural Language Processing. Our aim is to generate a distinct vector for each unique molecule, correctly identifying stereoisomer information in cheminformatics. The proposed approaches are then compared to our original machine learning task: predicting the association constant between cyclodextrin and a guest molecule.

Quantum machine learning with Qiskit: Evaluating regression accuracy and noise impact

AbstractQuantum machine learning (QML) can be employed in solving complicated machine learning tasks although the performance in examining the regression processes is only barely understood. Knowledge gaps are intended to be closed by studying modelling performance of QML in regression tasks, with emphasis being dedicated to scaling up and ability to resist noise. The regression part offers the following functions that include straight line and complex operations. Furthermore, the authors employ quantum neural networks generated using Qiskit to perform experiments. The results demonstrate that QML has a remarkable level of accuracy in basic regressions, reaching a maximum of 97%. Nevertheless, there are difficulties in representing intricate functions, such as 5 × cos(x), which results in a noticeable decline in performance. The work deals with the influence of noise and IERs from imperfect hardware on the efficiency of QML algorithms providing insight into the core obstacles. The result of a detailed examination of the results that have tested the powers and limits of QML in the development of regression applications is represented. The future direction of research and development will be defined by the results obtained in it.

Gaussian process learning of nonlinear dynamics

One of the pivotal tasks in scientific machine learning is to represent underlying dynamical systems from time series data. Many methods for such dynamics learning explicitly require the derivatives of state data, which are not directly available and can be approximated conventionally by finite differences. However, the discrete approximations of time derivatives may result in poor estimations when state data are scarce and/or corrupted by noise, thus compromising the predictiveness of the learned dynamical models. To overcome this technical hurdle, we propose a new method that learns nonlinear dynamics through a Bayesian inference of characterizing model parameters. This method leverages a Gaussian process representation of states, and constructs a likelihood function using the correlation between state data and their derivatives, yet prevents explicit evaluations of time derivatives. Through a Bayesian scheme, a probabilistic estimate of the model parameters is given by the posterior distribution, and thus a quantification is facilitated for uncertainties from noisy state data and the learning process. Specifically, we will discuss the applicability of the proposed method to several typical scenarios for dynamical systems: identification and estimation with an affine parametrization, nonlinear parametric approximation without prior knowledge, and general parameter estimation for a given dynamical system.

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

Feature selection is predominantly used in machine learning tasks, such as classification, regression, and clustering. It selects a subset of features (relevant attributes of data points) from a larger set that contributes as optimally as possible to the informativeness of the model. There are exponentially many subsets of a given set, and thus, the exhaustive search approach is only practical for problems with at most a few dozen features. In the past, there have been attempts to reduce the search space using dynamic programming. However, models that consider similarity in pairs of features alongside the quality of individual features do not provide the required optimal substructure. As a result, algorithms, which we will call suboptimal dynamic programming algorithms, find a solution that may deviate significantly from the optimal one. In this paper, we propose an iterative dynamic programming algorithm, which invertsthe order of feature processing in each iteration. Such an alternating approach allows for improving the optimization function by using the score from the previous iteration to estimate the contribution of unprocessed features. The iterative process is proven to converge and terminates when the solution does not change in three successive iterations or when the number of iterations reaches the threshold. Results in more than 95% of tests align with those of the exhaustive search approach, being competitive and often superior to the reference greedy approach. Validation was carried out by comparing the scores of output feature subsets and examining the accuracy of different classifiers learned on these features across nine real-world applications, considering different scenarios with various numbers of features and samples. In the context of feature selection, the proposed algorithm can be characterized as a robust filter method that can improve machine learning models regardless of dataset size. However, we expect that the idea of alternating suboptimal optimization will soon be generalized to tasks beyond feature selection.

Exploring latent discrimination through an Object-Relational Causal Inference method

This paper addresses the critical but often overlooked problem of exploring latent sensitive attributes in machine-learning datasets where they are not explicitly present. We propose a method named Object-Relational Causal Inference (ORCI), solving the problem with good reusability. We mathematically demonstrate the enhancement of data fairness by uncovering latent sensitive attributes and build a framework to tackle the latent discrimination exploration challenge. Firstly, we highlight the limitations of classical causal discovery methods and introduce an approximate exhaustive search algorithm called BIC-based Exhaustive Search (BICES). Then, we integrate the Object Relational (OR) approach into causal modeling, enabling the subsequent structured probabilistic inference based on the OR causal model to derive fair and effective latent sensitive attributes. We conduct simulations and experiments on synthetic and real-world cases across various machine-learning tasks, with discrete and continuous data, and in diverse settings. Results show that our method successfully uncovers latent sensitive attributes, enhancing the data fairness while preserving data utility. Besides, our detailed theoretical derivations and rich experiments demonstrate the generalization of our method.

LLM is All You Need : How Do LLMs Perform On Prediction and Classification Using Historical Data

This study investigates the utility of large language models (LLMs) in performing traditional machine learning tasks such as prediction, and explores the potential of refinement architectures to enhance their effectiveness in these roles. Utilizing the Titanic survival dataset, we conducted a com- parative analysis using both conventional machine learning tools and LLM-based approaches. Our findings indicate that while LLMs differ fundamentally from traditional ML models in prediction tasks, there exist specific architectural modifications, termed Thought Refinement Architectures, which can significantly improve their performance. These results highlight the potential for inte- grating LLMs into traditional ML workflows, thereby expanding their applicability and enhancing predictive accuracy.

NetDiffus: Network traffic generation by diffusion models through time-series imaging

While Machine-Learning based network data analytics are now commonplace for many networking solutions, nonetheless, limited access to appropriate networking data has been an enduring challenge for many networking problems. Causes for lack of such data include complexity of data gathering, commercial sensitivity, as well as privacy and regulatory constraints. To overcome these challenges, we present a Diffusion-Model (DM) based end-to-end framework, NetDiffus, for synthetic network traffic generation which is one of the emerging topics in networking and computing system. NetDiffus first converts one-dimensional time-series network traffic into two-dimensional images, and then synthesizes representative images for the original data. We demonstrate that NetDiffus outperforms the state-of-the-art traffic generation methods based on Generative Adversarial Networks (GANs) by providing 66.4% increase in the fidelity of the generated data and an 18.1% increase in downstream machine learning tasks. We evaluate NetDiffus on seven diverse traffic traces and show that utilizing synthetic data significantly improves several downstream ML tasks including traffic fingerprinting, anomaly detection and traffic classification. The code has been included at https://github.com/Nirhoshan/NetDiffus.

Intelligent mechanical metamaterials towards learning static and dynamic behaviors

The exploration of intelligent machines has recently spurred the development of physical neural networks, a class of intelligent metamaterials capable of learning, whether in silico or in situ, from observed data. In this study, we introduce a back-propagation framework for lattice-based mechanical neural networks (MNNs) to achieve prescribed static and dynamic performance. This approach leverages the steady states of nodes for back-propagation, efficiently updating the learning degrees of freedom without prior knowledge of input loading. One-dimensional MNNs, trained with back-propagation in silico, can exhibit the desired behaviors on demand function as intelligent mechanical machines. The framework is then employed for the precise morphing control of the two-dimensional MNNs subjected to different static loads. Moreover, the intelligent MNNs are trained to execute classical machine learning tasks such as regression to tackle various deformation control tasks. Finally, the disordered MNNs are constructed and trained to demonstrate pre-programmed wave bandgap control ability, illustrating the versatility of the proposed approach as a platform for physical learning. Our approach presents an efficient pathway for the design of intelligent mechanical metamaterials for a wide range of static and dynamic target functionalities, positioning them as powerful engines for physical learning.

Multi-view clustering via latent consistency multi-graph fusion

Multi-view clustering (MVC) algorithms are prevalent in machine learning tasks due to their superiority in mining complementary information from multi-view data. Despite their widespread use, existing MVC approaches ignore the presence of inconsistencies in the multi-view features during multi-view data fusion, which leads to suboptimal clustering performance. To address this challenge, the paper presents a novel approach called multi-view clustering via latent consistency multi-graph fusion. Specifically, a novel latent representation learning model is devised to learn a unified representation from multi-view features, simultaneously capturing the high-order relationships within the multi-view data. Additionally, to alleviate the adverse effects caused by the inconsistency multi-view feature, a consistent multi-graph fusion module is proposed to provide consistent constraints for latent representations. To be specific, the graphs that reflect the manifold structure for each view are fused to guide a consistent latent representation learning. After that, an alternating optimization algorithm is designed to update variables iteratively and alternatively, among which the fusion weights are assigned adaptively. Finally, comprehensive experiments conducted on seven widely recognized benchmark multi-view datasets affirm the efficacy of the proposed method, showcasing its superiority over state-of-the-art approaches.