Due to the increasing proportion of renewable energy, a multi-layered and multi-timescale energy market has emerged in many countries such as China. In the meanwhile, power generation companies must develop more intelligent and dynamic offer strategies to adapt to today's intricate energy trading. Because of the difficulty in describing the dynamic trading environment caused by the uncertainty of renewable energy, previous studies have not fully explored the offer strategy especially in both short-term and medium-term electricity markets. In response to this challenge, this research introduces a novel biding strategy framework leveraging a Asynchronous Advantage Actor-Critic (A3C) algorithm, which can effectively address the decision making in dynamic and uncertain energy markets. The framework focuses on intra-monthly transaction clearing mechanisms with the aim of optimally enhancing earnings. The research formulates an offer model both for thermal and renewable power generation enterprises, which is applicable to medium-term monthly and intra-monthly trading. The study then validates this framework through three distinct analyses: the returns of various bid methods under standard scenarios, the offer strategies return of power generation companies with diverse cost profiles, and the impact of varying renewable energy proportions. The multi-angle simulations confirm that the model presented in this paper offers a scientific basis for the development of offer strategies for power generation companies and enable power generating firms to effectively adopt to the current power market.