A series of adsorbents was developed by physical (CO2) and chemical (KOH) activation of two bio-residues: olive stones (OS) and wood from olive tree pruning (OTP). The physicochemical properties of such materials were determined and correlated with their adsorptive performance in the removal of phenolic compounds of olive mill wastewater (OMW). Adsorption isotherms and kinetics of single phenolic acids, as well as the kinetics for competitive multi-compound adsorption, were fitted by applying different models, though Langmuir and pseudo-second order models fitted better the experimental results, respectively. The intraparticle diffusion model pointed out that mesoporosity reduces the influence of phenolic compounds’ restrictions in the external film diffusion of the adsorbent particle–solution interphase, but adsorption capacity linearly increases with the micropore volume accessible to N2 at −196 °C (and also with BET surface area), while diffusion into ultramicropores (<0.7 nm, determined by CO2-adsorption) is slow and presents minor influence on the total adsorption capacity. After saturation, thermal regeneration of spent adsorbents allows the removal of adsorbed products, enabling the reuse of samples whilst maintaining a significant performance.