In heavy-ion collisions at large particle colliders, such as LHC or RHIC, heavy-flavour (charm and beauty) quarks are produced mainly through initial hard scatterings. Therefore, they can serve as the probes of properties of the hot medium created in such collisions. Additionally, in small collision systems, such as d/p+Au collisions, cold nuclear matter effects can also affect the charm quark production with respect to p+p collisions. Hadrons, that contain heavy-flavour quarks, could not be directly detected, thus they are measured via reconstruction of their decay products. However, due to a large number of particles produced in such collisions, separation of the decay products from combinatorial background is challenging and advanced statistical analysis is needed. In this article, we exploit D0 (D0)→K- π+ (K+ π-) decay in order to investigate performance of several machine learning algorithms with different implementation approaches to find the most effective way how to separate signal from random combinatorial background. For this study, we use HIJING and STAR detector simulation of d+Au collisions at √(sNN)=200 GeV embedded to the collisions recorded with the STAR. In this paper we compare deep neural network implemented using Keras with TensorFlow backend, random forest model implemented using scikit-learn and boosted decision trees implemented by means of the Toolkit for Multivariate Data Analysis with ROOT. Described methods might be applied on reconstruction of any two-body decay in high-energy physics experiments.