In this paper, we present a method for retrieving sea surface wind field (SSWF) from HaiYang-2B (HY-2B) scatterometer data. In contrast to the conventional algorithm, i.e., using a point-to-point (P2P) method based on geophysical model functions (GMF) to retrieve SSWF by spaceborne scatterometer, we introduce a more accurate field-to-field (F2F) retrieval method based on convolutional neural network (CNN). We fully consider the spatial correlation and continuity between adjacent observation points, and input the observation data of continuous wind field within a certain range into the neural network to construct the neural network model, and then synchronously obtain the wind field within the range. The wind field obtained by our retrieval method maintains its continuity and solves the problem of ambiguity removal in traditional wind direction retrieval methods. Comparing the retrieval results with the buoy data, the results show that the root mean square errors (RMSE) of wind direction and wind speed are less than 0.18 rad (10.31°) and 0.75 m/s, respectively. The retrieval accuracy is better than the L2B product of HY-2B.