Rate-splitting multiple access (RSMA) softly reconciles and decodes the extreme interference by non-orthogonal transmission, which can remarkably solve the spectrum scarcity for future six-generation (6G) low earth orbits (LEO) satellite communication system. In this letter, we investigate the power allocation problem in LEO satellite networks with RSMA mechanism based on the deep reinforcement learning (DRL) technique. Specifically, in order to achieve better RSMA performance, the LEO satellite base station (SBS) has to effectively allocate transmit power to common and private streams, which is very challenging due to the uncertain and limited information of the channel distribution. To solve this problem, a highly-effective proximal policy optimization (PPO) based scheme is further proposed, which enables the LEO SBS to learn an optimal power allocation strategy to maximize the sum rate of the system without knowing any prior information. Simulation results prove that the proposed scheme significantly outperforms the other three baseline schemes in terms of the sum rate metric with low computation complexity.