The effective action of string theory has both bulk and boundary terms if the spacetime is an open manifold. Recently, the known classical effective action of string theory at the leading order of alpha ' and its corresponding boundary action have been reproduced by constraining the effective actions to be invariant under gauge transformations and under string duality transformations. In this paper, we use this idea to find the classical effective action of the O-plane and its corresponding boundary terms in type II superstring theories at order alpha '^2 and for NS–NS couplings. We find that these constraints fix the bulk action and its corresponding boundary terms up to one overall factor. They also produce three multiplets in the boundary action that their coefficients are independent of the bulk couplings under the string dualities.