Efficient and accurate fishery forecasting is of great significance in ensuring the efficiency of fishery operations. This paper proposes a fishery forecasting method using a brightness temperature (TB) time series spatial feature extraction and fusion model. Using Indian Ocean bigeye tuna fishery data from 2009 to 2021 as a reference, this paper discusses the feasibility of fishery forecasting using FY-3 Microwave Radiation Imager (MWRI) Level 1 TB data. For this paper, we designed a deep learning network model for radiometer TB time series feature extraction (TimeTB-FishNet) based on the Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and Attention mechanism. After expanding the dimensions of TB features, the model uses them together with spatiotemporal feature factors (year, month, longitude, and latitude) as features. By adding the GRU and Attention to the CNN, the CNN-GRU-Attention model architecture is established and can extract deep time series spatial features from the data to achieve the best results. In the model validation experiments, the TimeTB-FishNet model performed optimally, with a coefficient of determination (R2) of 0.6643. In the generalization experiments, the R2 also reached 0.6261, with a root mean square error (RMSE) of 46.6031 kg/1000 hook. When the sea surface height (SSH) was introduced, the R2 further reached 0.6463, with a lower RMSE of 45.1318 kg/1000 hook. The experimental results show that the proposed method and model are feasible and effective. The proposed model can directly use enhanced radiometer TB data without relying on lagging ocean environmental product data, performing deep temporal and spatial feature extraction for fishery forecasting. This method can provide a reference for the fishing of bigeye tuna in the Indian Ocean.