Using Quinolin-65 (Q-65) and Violantrone (V-79) as model molecules for polar heavy hydrocarbons and resins, the nanosize effects of NiO nanoparticles on the competitive molecular adsorption on nanoparticles were conducted using multiwavelength UV–visible derivative spectrophotometry method. This is important for understanding the role of nanoparticles in oil upgrading and recovery processes. Computer simulations were also carried out to validate the experimental findings and provide more insights on the interactions between the Q-65 and/or V79 molecules and the NiO nanoparticle surface. Different-sized NiO nanoparticles (5, 40, and >100 nm) were synthesized by controlled thermal dehydroxylation of Ni(OH)2. Nanoparticles were characterized using XRD, BET, FTIR, and XPS. Macroscopic adsorption isotherms of Q-65 and V-79 molecules over 5 and 40 nm NiO nanoparticles were evaluated in toluene-based solutions as individual and binary solutions. On a normalized surface area basis, the number of Q-65 and V-79 molecules adsorbed per nm2 of the NiO surface was the highest for 40 nm NiO nanoparticles and lowest for 5 nm NiO. Results also indicated that NiO nanoparticles were more prone to adsorb V-79 than Q-65. Equilibrium binding constants for V-79 and Q-65, determined by the Langmuir adsorption model, showed the binding affinity for V-79 is size dependent. Simultaneous competitive adsorption between V-79 and Q-65 showed that V-79 concentration is an important factor influencing the uptake of both V-79 and Q-65, whereas the concentration of Q-65 affects only Q-65 uptake. Computational modeling results were consistent with the experimental results, confirming our conclusions about the adsorption process in this binary system.