The grass spikelet is a unique inflorescence structure that determines grain size. Although many genetic factors have been well characterized for grain size and glume development, the underlying molecular mechanisms in rice are far from established. Here, we isolated rice gene, AGL1 that controlled grain size and determines the fate of the sterile lemma. Loss of function of AGL1 produced larger grains and reduced the size of the sterile lemma. Larger grains in the agl1 mutant were caused by a larger number of cells that were longer and wider than in the wild type. The sterile lemma in the mutant spikelet was converted to a rudimentary glume-like organ. Our findings showed that the AGL1 (also named LAX1) protein positively regulated G1 expression, and negatively regulated NSG1 expression, thereby affecting the fate of the sterile lemma. Taken together, our results revealed that AGL1 played a key role in negative regulation of grain size by controlling cell proliferation and expansion, and supported the opinion that rudimentary glume and sterile lemma in rice are homologous organs.