How to balance the satisfaction of constraints and the optimization of objective functions is one of the key issues to solve constrained multi-objective optimization problems (CMOPs), especially when the constraints are complex. Although many algorithms have been designed to handle this, most of them are still unable to effectively handle CMOPs with complex constraints. Based on the above issue, this paper proposes a framework for bi-directional search, which evolves two populations (P1 and P2). P1 aims to search the constrained Pareto front (CPF) from the infeasible side of the objective space, and P2 from the feasible side, aiming to achieve a more efficient and comprehensive bi-directional search for the CPF. To ensure the diversity, we adopt a preferred weight vector selection strategy to choose potential mating parents, which improves the search capability for the marginal CPF. Furthermore, to coordinate the interaction between the two populations, we propose an environmental selection strategy to select the offspring generated by P1 and P2 under the same weight vector with the better fitness to update the populations respectively, and the fitness is evaluated based on the different constraint relaxations of the two populations, to update them respectively. Extensive experiments indicate that our proposed algorithm has significantly better results or was at least competitive when compared to eight state-of-the-art methods.