Open sun drying has been used for many years for grain drying. Unfortunately, in open sun drying, solar energy is improperly utilized, and it takes a longer time to dry, leading to a compromise in product quality. Also, it requires larger floor space, continuous observation for drying, and suffers from quality impairments. In this work, a compact solar grain dryer design is being proposed and optimized for higher energy efficiency. Different designs were tested using CFD simulations for their performance. To make this grain dryer energy efficient, part of the hot air was recirculated, and simulations were performed to optimize the sizing and location of the solar fan. Several other parameters that determine the overall performance was considered, such as air flow rate, temperature, and pressure. An orifice was introduced at the inlet of the pipe to increase the quantity of recycled air. The effect of the diameter of the orifice was studied, and its optimum was determined. Finally, two dryer designs were proposed, one without fins and the other with fins.