Estimating the State of Health (SOH) of lithium batteries is crucial for the safety and stability of the microgrids. Currently, existing data-driven methods struggle to predict across multiple groups of lithium batteries varying in type, capacity, and degree of degradation, as the prediction process requires the collection of extensive operational data for processing and computation. This paper analyzes the characteristics of lithium battery storage units within the microgrids and proposes a novel prediction method based on an improved attention mechanism. In the operational process, we collected the data under constant current (CC) and constant voltage (CV) charging modes to obtain health indicators with high correlation. We proposed a novel optimized multi-head attention method, designated OM-Attention, which employs a multi-head self-attention-based model to compute multiple sets of lithium batteries within a network node in a simultaneous manner. To validate the effectiveness of the OM-Attention method, we randomly selected four lithium batteries from the NASA public dataset for multiple experiments. Compared to SOH prediction methods, our approach exhibits superior performance with reduced Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Square Error (MSE) to 4.16%, 5.02%, and 0.302×10−4, respectively. Additionally, the R2-score achieved by OM-Attention is 97.7%.