The article proposes an algorithm based on intelligent methods for the early diagnosis of hepatocellular carcinoma (HCC), known as liver cancer, which is rated third cause of cancer deaths in the world. Initial diagnosis of HСC is based on laboratory studies, computer tomography and X-ray examination. However, in some cases, identifying cancerous tissues as similar non-cancerous tissues (cirrhotic tissues and normal tissues) made it necessary to perform gene analysis for the diagnosis. To predict HCC based on such numerous, diverse and heterogeneous unstructured data, preference is given to the method of artificial intelligence, i.e., machine learning. It shows the possibility of applying machine learning methods to solve the problem of accurate identification of HCC due to the compatibility of HCC tissues with identical CwoHCC non-cancerous tissues. The technology of gene pair profiling using relevant peer databases is described and the Within-Sample Relative Expression Orderings (REO) technique is used to determine the gene pair’s similarity. The article also presents a new approach based on The Within-Sample Relative Expression Orderings technique for determining the gene pair’s similarity, Incremental feature selection method for feature selection, and Support Vector Machine methods for gene pair classification. The proposed approach constitutes the methodological basis of a decision support system for the early diagnosis of HCC, and the development of such a system may be beneficial for physician decision support in the relevant field