The characteristics of electric buses make it difficult to estimate the energy consumption and mean that they are prone to battery loss; as such, fuel bus scheduling methods are no longer fully applicable. In current studies, the influence of these factors is ignored. This paper proposes an electric bus scheduling optimization model based on energy consumption and battery loss. Firstly, the LSTM (long short-term memory) is used to estimate trip energy consumption. Subsequently, these results are combined with the optimization objectives of minimizing the fleet size and battery loss amount. Limitations on the buses’ number, travel time, battery safety thresholds, remaining charge, and total charge are also considered. By controlling the different battery charge and discharge thresholds to minimize battery losses, the goal of sustainability is achieved. NSGA-II (non-dominated sorting genetic algorithm-II) is used to solve the model. The corresponding scheduling and charging scheme are determined. Electric bus route A is taken to validate the predictions. The results show that the annual fleet battery loss value decreases as the fleet size increases. The company has the lowest annual operating cost when the battery charge and discharge thresholds are set to [25%, 85%]. Optimizing the scheduling and charging scheme for electric bus can effectively reduce the operating cost.