Abstract Orange is one of the most popular fruits in the world. Oranges originate from South-east Asia, India, and southern China. Nowadays, they are grown in warm regions around the world. Oranges are consumed either fresh or in the form of fruit juice. De-pending on the type of orange, when ripe, its cover is orange or green, with a sweet or slightly sour taste. In Vietnam, oranges have significant export value. Therefore, en-suring consistent quality of oranges before export is an important requirement. Cur-rently in Vietnam, this process is done manually or semi-automatically, resulting in low productivity and inadequate output for export purposes. This article introduces a new design of an automatic orange sorting machine. The proposed machine is equipped with functions to classify the size and defects on the orange cover through image processing technique. The experiment results show that the proposed machine works effectively and can replace the manual sorting process with high accuracy.