CodPy: a tutorial

Abstract

CodPy stands for “Curse of dimensionality in Python” and is a Support Vector Machine (SVM), application oriented, Python library, providing tools for machine learning, statistical learning and numerical simulations, based over a partial differential equations approach. It consists of our implementation of a new strategy based on the theory of Reproducing Kernel Hilbert Spaces (RKHS). We started to develop this method several years ago, mainly for algorithmic applications in mathematical finance. The present tutorial is intended to explain the basic concept of this SVM approach, which uses Python, R code, and Latex combined together with the help of the software R markdown1. The later is similar to a Jupyter notebook, and in particular is fully reproducible by any user2. We introduce here the user to the basic notions of SVMs in CodPy, and then move on to an academic test of pattern recognition, specifically the MNIST test. A companion tutorial, later on, will supplement this first presentation and introduce the user to the discretization of differential operators, a notion that is naturally associated with any SVMs. These operators allow us to discretize many partial differential operator arising in applications. Our motivation in releasing our techniques and tools is that, to our opinion, the proposed SVMs have the potential to compete advantageously with classical learning machine platforms, such as the ones devoted in Deep Learning using neural network techniques. These traditional approaches to artificial intelligence have shown some weaknesses and limitations, and sometimes leads to non-converging methods, algorithmic inefficiency, or lack of theoretical foundations. As mathematicians, we consider that there is an urgent need to seek to tackle these questions. In the past few years, the proposed SVM approach has revealed itself as being very robust and versatile, and the present tutorial should provide a way to discuss this alternative approach with the scientific community, and share ideas in order to eventually reach a possible standardization of SVM tools for machine learning.