Abstract Gaharu is the most expensive resinous wood fragrance making it a very highly valuable product. Its oleoresin was produced using ethanol reflux extraction method. In optimization study, the influence of process parameters on the extraction of oleoresin from dried gaharu particles was investigated. The parameters include particle sizes, raw material to solvent ratio and extraction duration. With the aid of regression analysis using response surface methodology (RSM), the results suggested that the optimal set of extraction condition to be consists of particle size of 0.5–1.0 mm (S3), raw material to ethanol ratio of 1:29.9 g/ml and extraction time of 4.97 h. Under this condition, the highest gaharu oleoresin yield was obtained, i.e. 7.63% (w/w) with total resin content of 6.90% (w/w). In brief, higher yield of oleoresin can be extracted from smaller particle size of dried gaharu in shorter extraction time and lower raw material to solvent ratio. In the comparison study, the optimized reflux extraction produced higher quality of gaharu oleoresin compared to reflux using WHO protocol, nevertherless just as comparable to the conventional Soxhlet extraction. Further investigation on the quality of extracts resulted to six chemical compounds determined in gaharu oleoresin using GC-MS. The outcomes of this study indicated an improvement of 62.97% of overall chemical compositions in gaharu oleoresin extract which includes 4-phenyl-2-butanone, β-guaiene, agarospirol, α-bisabolene epoxide, alloaromadendrene oxide and aromadenderence oxide compared to the conventional Soxhlet method.