PurposeThis study aims to propose an optimization framework using deep neural networks (DNN) coupled with nondominated sorting genetic algorithm II and technique for order preference by similarity to an ideal solution method to improve the tribological properties of camshaft bearing pairs of internal combustion engine.Design/methodology/approachA lubrication model based on the theory of elastohydrodynamic lubrication and flexible multibody dynamics was developed for a V6 diesel engine. Setting DNN model as fitness function, the multi-objective optimization genetic algorithm and decision-making method were used to optimize the bearing pair structure with the goal of minimizing the total friction loss and the difference of the average values of minimum oil film thickness.FindingsThe results show that the lubrication state corresponding to the optimized bearing pair structure is elastohydrodynamic lubrication. Compared with the original structure, the optimized structure significantly reduces the total friction loss.Originality/valueThe optimized performance and corresponding structural parameters are obtained, and the optimization results were verified through multibody dynamics simulation.Peer reviewThe peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-12-2023-0417/