Phosphorus (P) is one kind of growth-limiting nutrient for plants, which is taken up by root tissue from the environment as inorganic phosphate (Pi). To maintain an appropriate status of cellular Pi, plants have developed sophisticated strategies to sense Pi level and modulate their root system architecture (RSA) under the ever-changing growth condition. However, the molecular basis underlying the mechanism remains elusive. Inositol polyphosphate kinase (IPK2) is a key enzyme in the inositol phosphate metabolism pathway, which catalyzes the phosphorylation of IP3 into IP5 by consuming ATP. In this study, the function of a rice inositol polyphosphate kinase gene (OsIPK2) in plant Pi homeostasis and thus physiological response to Pi signal were characterized. As a biosynthetic gene for phytic acid in rice, overexpression of OsIPK2 led to distinct changes of inositol polyphosphate profiles and an excessive accumulation of Pi level in transgenic rice under Pi-sufficient conditions. The inhibitory effects of OsIPK2 on root growth were alleviated by Pi-deficient treatment compared with wild-type plants, suggesting the involvement of OsIPK2 in the Pi-regulated reconstruction of RSA. In OsIPK2-overexpressing plants, the altered acid phosphatase (APases) activities and misregulation of Pi starvation induced (PSI) genes were observed in roots under different Pi supply conditions. Notably, the expression of OsIPK2 also altered the Pi homeostasis and root system architecture in transgenic Arabidopsis. Taken together, our findings demonstrated that OsIPK2 plays an important role in Pi homeostasis and root system architecture adjustment in response to different environmental Pi levels in plant.