The supplementary cementitious material (SCM) has been introduced as a cement replacement in the concrete or mortar mix design for reducing the emission of carbon gases during the manufacturing of cement. Palm oil fuel ash (POFA) is one of the potential agricultural wastes SCM that is rich in silica content and can be used as cement replacement. However, the inconsistent particle characteristic of the raw POFA due to the uncontrolled incineration condition brings deterioration effect in the concrete’s fresh and mechanical properties. Therefore, many papers conducted fineness reduction approach and other treatment processes to produce the micro and nano scale POFA as a suitable cement replacement. This article aims to provide a comprehensive understanding of the impact of micro and nano palm ash on cementitious materials properties, focusing on the potential of nano palm ash for future work. A detailed comparison is made between the treatment process, chemical and physical properties of micro and nano palm ash, later focuses on the fresh and hardened properties of various sizes of POFA-based concrete and mortar. Nano palm ash has a smaller particle size, higher silica content, and lower unburnt carbon than micro palm ash, resulting in improved fresh properties and early age-hardened properties of concrete when using nano palm ash.