This study discusses soil fertility under perennial cash crop farming (para rubber, Hevea brasiliensis; black pepper, Piper nigrum; oil palm, Elaeis guineensis) conducted by local farmers and an oil palm estate in an upland area of Sarawak, Malaysia, in comparison with the surrounding secondary forests. In the farmlands of the local farmers, rubber farming was conducted without fertilizer application, while 2–5 t ha −1 of NPK compounds were applied annually on pepper farms. Soils under rubber farming were acidic with poor nutrient contents, resembling soils in secondary forests. In pepper farms, soils were less acidic and showed high nutrient contents, especially with respect to available P and exchangeable Ca. This trend became stronger with increasing farming duration. Fertilizers applied around pepper vines appeared to migrate and spread across the fields. Bulk density and hardness of surface soils were higher in pepper farms than in secondary forests, indicating soil compaction due to field works. In the oil palm estate, annual fertilizer application rates were moderate at 0.4–0.8 t ha −1 of NPK compound fertilizers. However, the soil properties in the oil palm estate were similar to those of the small-scale pepper farms. Close to the bases of the palms where fertilizers usually are applied, the contents of exchangeable Ca and available P were high. Nutrient uptake by the dense root systems of the palms seemed to prevent excessive loss of nutrients through leaching. Loss of soil organic matter and deterioration of soil physical properties were brought about by terrace bench construction, but the soils seemed to recover to some extent over time. In conclusion, technologies such as intercropping and the appropriate allocation of different crops to specific locations as well as the proper selection and dosage of fertilizers should be developed and adopted to improve fertilizer efficiency and prevent water pollution due to fertilizer wash-off from farmlands.