The addition of carbon-rich materials, such as raw feedstocks (RAW) and biochars, to agricultural soils is on the rise. This activity has many advantages, such as improving fertility, increasing water retention, and sequestering carbon. However, they can also increase the sorption of residual herbicides in the soil, reducing the effectiveness of weed control. Thus, the objective of this study was to evaluate soil improvement and the sorption–desorption process of diclosulam in soil unamended and amended with RAW from olive stone and their biochars produced in two pyrolysis temperatures (300 and 500 °C). Oxisol was used in this study, unamended and amended with RAW and biochars (BC300 and BC500) in a rate of 10% (w w−1). The sorption–desorption process was assessed by batch-equilibrium experiments and the analysis was performed using high-performance liquid chromatography (HPLC). The addition of the three materials to the soil increased the contents of pH, organic carbon, P, K, Ca, Mg, Zn, Fe, Mn, Cu, B, cation exchange capacity, base saturation and decreased H + Al. The unamended soil had Kf (Freundlich sorption coefficient) values of diclosulam sorption and desorption of 1.56 and 12.93 mg(1 − 1/n) L1/n Kg−1, respectively. Unamended soil sorbed 30.60% and desorbed 13.40% of herbicide. Soil amended with RAW, BC300, and BC500 sorbed 31.92, 49.88, and 30.93% of diclosulam and desorbed 13.33, 11.67, and 11.16%, respectively. The addition of RAW and biochars from olive stone has the potential to change the soil fertility, but does not interfere with the bioavailability of diclosulam in weed control under field conditions, since the materials slightly influenced or did not alter the sorption–desorption of diclosulam.