Worn tools might lead to substantial detrimental implications on the surface integrity of workpieces for precision/ultra-precision machining. Most previous research has heavily relied on singular information, which might not be appropriate enough to ascertain tool conditions and guarantee the accuracy of workpieces. This paper proposes a CNN-LSTM hybrid model directly utilizing tool images to predict surface roughness on machined parts for tool condition assessment. This work first performs pruning based on UNet3+ architecture to eliminate redundant structures while integrating attention mechanisms to enhance the model's focus on the target region. On this basis, tool wear region information is intensely mined and heterogeneous data is optimized using Spearman correlation analysis. Subsequently, we innovatively proposed a hybrid model that integrates CNN and RNN, endowing the model with the ability to process spatial and sequential information. The effectiveness of the proposed methodology is validated using the practical data obtained from cutting experiments. The results indicate that the proposed tool condition assessment methodology significantly improves the segmentation accuracy of the tool wear region to 94.52 % (Dice coefficient) and predicts the surface roughness of machined parts with an accuracy exceeding 93.1 % (R2). It can be observed that the developed methodology may provide an effective solution for accurate tool condition assessment and the implementation of tool health management.