Background: For the traditional Fourier Transform (FT), it cannot effectively detect the frequency of non-stationary signals with time. Analyzing the local characteristics of time-varying signal by using FT is hard to achieve. Therefore, many time-frequency analysis methods which can meet different needs have been proposed on the basis of the traditional Fourier transform, like the Short Time Fourier Transform (STFT), the widely used Continuous Wavelet Transform (CWT), Wigner-Ville Distribution (WVD) and so on. However, the best time and frequency resolution cannot be achieved at the same time due to the uncertainty criterion. Methods: From the point of view of optimizing time-frequency performance, a new Generalized S Transform (GST) window function optimization method is proposed by combining time frequency aggregation with an improved genetic algorithm in this paper. Results: In the noiseless condition, the Linear Frequency Modulation (LFM), Nonlinear Frequency Modulation (NLFM) and binary Frequency Shift Keying (2FSK) signals are simulated. The simulation results prove that the method can improve the time-frequency concentration and decreasing Rényi entropy effectively. Compared with other four traditional time-frequency analysis methods, the improved GST has more advantages. Conclusion: In this paper, a new method of optimizing the window function in GST based on improved GA is proposed in this paper. Experiments on LFM, NLFM and 2FSK signals show that the proposed method not only has the advantages of high resolution, but also determines the optimal parameters via the time frequency focusing criterion and the Rényi entropy. Compared with the other four kinds of time-frequency analysis methods, the optimized GST based on improved GA in this paper has the best time-frequency focusing.