In this work, the effects of torrefaction on the physiochemical properties of empty fruit bunches (EFB), palm mesocarp fiber (PMF) and palm kernel shell (PKS) are investigated. The change of properties of these biomass residues such as CHNS mass fraction, gross calorific value (GCV), mass and energy yields and surface structure when subjected to torrefaction process are studied. In this work, these materials with particle size in the range of 355–500 μm are torrefied under light torrefaction conditions (200, 220 and 240 °C) and severe torrefaction conditions (260, 280 and 300 °C). TGA is used to monitor the mass loss during torrefaction while tube furnace is used to produce significant amount of products for chemical analyses. In general, the study reveals torrefaction process of palm oil biomass can be divided into two main stages through the observation on the mass loss distribution. The first stage is the dehydration process at the temperature below than 105 °C where the mass loss is in the range of 3–5%. In the second stage, the decomposition reaction takes place at temperature of 200–300 °C. Furthermore, the study reveals that carbon mass fraction and gross calorific value (GCV) increase with the increase of torrefaction temperature but the O/C ratio, hydrogen and oxygen mass fractions decrease for all biomass. Among the biomass, torrefied PKS has the highest carbon mass fraction of 55.6% when torrefied at 300 °C while PMF has the highest GCV of 23.73 MJ kg−1 when torrefied at the same temperature. Both EFB and PMF produce lower mass fraction than PKS when subjected to same torrefaction temperature. In terms of energy yield, PKS produces 86–92% yield when torrefied at light to severe torrefaction conditions, until 280 °C. However, both EFB and PMF only produce 70–78% yield at light torrefaction conditions, until 240 °C. Overall, the mass loss of 45–55% of these biomasses is observed when subjected to torrefaction process. Moreover, SEM images reveal that torrefaction has more severe impact on surface structure of EFB and PMF than that of PKS especially under severe torrefaction conditions. The study concludes that the torrefaction process of these biomass has to be optimized based on the type of the biomass in order to offset the mass loss of these materials through the process and increase the energy value of the solid product.