Norm--Free Radon--Nikodym Approach to Machine Learning

Abstract

For Machine Learning (ML) classification problem, where a vector of 𝒙--observations (values of attributes) is mapped to a single 𝐲 value (class label), a generalized Radon--Nikodym type of solution is proposed. Quantum--mechanics like probability states ψ²(𝒙) are considered and Cluster Centers'', corresponding to the extremums of 〈𝐲ψ²(𝒙)〉/〈ψ²(𝒙)〉, are found from generalized eigenvalues problem. The eigenvalues give possible 𝐲 outcomes and corresponding to them eigenvectors ψ(𝒙) define the Cluster Centers''. The projection of a ψ state, localized at given 𝒙 to classify, on these eigenvectors define the probability of 𝐲 outcome, thus avoiding using a norm (L² or other types), required for a quality criteria in a typical Machine Learning technique. A coverage of each Cluster Center'' is calculated, what potentially allows to separate system properties (described by 𝐲 outcomes) and system testing conditions (described by 𝐂 coverage). As an example of such application 𝐲 distribution estimator is proposed in a form of pairs (𝐲,𝐂), that can be considered as Gauss quadratures generalization. This estimator allows to perform 𝐲 probability distribution estimation in a strongly non--Gaussian case.