• This is the first report on applying estimation of distribution algorithm (EDA) to the studied problem. • A kind of PN-based deadlock controllers for FMSs is imbedded to exclude infeasible individuals. • An effective voting procedure is adopted to construct the probabilistic model of EDA. • The longest common subsequence is also embedded in the model for mining excellent genes. • A new modified variable neighborhood search is developed as an efficiency enhancement of EDA. Based on the place-timed Petri net models of flexible manufacturing systems (FMSs), this paper proposes a novel effective estimation of distribution algorithm (EDA) for solving the scheduling problem of FMSs. A candidate solution is represented as an individual with two sections: the first contains the route information while the second is a permutation with repetition for parts. The feasibility of individuals is checked and guaranteed by a highly permissiveness deadlock controller. A feasible individual is interpreted into a deadlock-free schedule while the infeasible ones are amended. The probabilistic model in EDA is constructed via a voting procedure. An offspring individual is then produced based on the model from a seed individual, and the set of seed individuals is extracted by a roulette method from the current population. The longest common subsequence is also embedded into the probabilistic model for mining good genes. A modified variable neighborhood search is applied on offspring individuals to obtain better solutions in their neighbors and hence to improve EDA’s performance. Computational results show that our proposed algorithm outperforms all the existing ones on benchmark examples for the studied problem. It is of important practice significance for the manufacturing of time-critical and multi-type products.