Hepatocellular carcinoma (HCC) is one of the most common cancers and an important medical problem with poor prognosis. The role of messenger RNA (mRNA) has been broadly researched in the progression of different human cancers. Microarray analysis has demonstrated that kynurenine 3-monooxygenase (KMO) expression is lower in HCC, but the mechanism of KMO in regulating the development of HCC remains unknown. Through comprehensive bioinformatics analysis of GSE101728 and GSE88839, including Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment, protein-protein interaction (PPI) network analysis, gene expression, and overall survival (OS) analysis, KMO was selected as the candidate molecular marker in HCC. The expression of KMO at the protein and RNA level was evaluated by Western blotting (WB) and quantitative real-time polymerase chain reaction (qRT-PCR). Furthermore, the cell proliferation, migration, invasion, and apoptosis, and the protein levels of epithelial-mesenchymal transition (EMT) markers were examined with Cell Counting Kit 8 (CCK-8) assays, Transwell assay, flow cytometry, and WB. Through comprehensive bioinformatics analysis, we found that the low expression of KMO in HCC is not conducive to a good prognosis of HCC. Then, through in vitro cell experiments, we found that low expression of KMO promoted HCC proliferation, invasion, metastasis, EMT, and cell apoptosis. Additionally, hsa-miR-3613-5p was found to be highly expressed in HCC cells and could negatively regulate the expression of KMO. Moreover, hsa-miR-3613-5p was found to be the target microRNA (miRNA) of KMO according to qRT-PCR verification. KMO plays an important role in the early diagnosis, prognosis, occurrence, and development of liver cancer, and may target miR-3613-5p to function. This represents a novel insight into understanding the molecular mechanisms of HCC.