In regenerative cryocoolers, magnetic specific heat is often utilized for heat regeneration at cryogenic temperature, such as below 20 K. Magnetic regenerator materials of rare earth compound which have sufficient specific heat enabled cryocooler to reach below 4 K. However, the magnetic noise emitted from the magnetic materials during the refrigeration cycle affects the performance of noise-sensitive devices. In this study, we propose a non-magnetic regenerator material having “optical phonon degree of freedom” even at cryogenic temperatures, in which silver oxide (Ag2O) is selected. Ag2O decomposes at temperature above 410 K in ambient atmosphere, whereas we succeeded in fabricating a sintered Ag2O bulk and small particles of the size of 0.5 mm with high-density more than 90%. Specific heat measurement down to 100 mK was performed for the sintered bulk sample of Ag2O. Below 10 K, a non-Debye behavior was observed, suggesting the contribution from optical phonon modes. Calculation of the phonon density of state (phDOS) performed by the Evolution Strategy algorithm in addition to the first-principles calculation reveals that phDOS has several distinct peaks at low energies, and the low-energy optical phonons contribute to the non-Debye behavior. The low temperature specific heat of Ag2O enhanced by the optical phonons is larger than conventional non-magnetic regenerator materials when compared in terms of specific heat per volume. Moreover, the calculation of cooling performance of cryocooler using Ag2O particles is found to be reached below 4.2 K. This study shows that non-magnetic material with low temperature specific heat enhanced by optical phonon mode can be used as a regenerator material for regenerative cryocoolers.