We study by Monte Carlo simulation the absorbing phase transition of the contact process (CP) on a four-dimensional hypercubic lattice with quenched impurity. The critical behavior of CP with quenched impurity has been predicted by an application of the Harris criterion established in equilibrium spin system to nonequilibrium absorbing phase transition and a mapping of disordered CP onto random quantum magnets. Harris criterion suggested that the pure fixed point is unstable if dν⊥<2, implying that any amount of impurity added to the system changes the critical behavior, where d and ν⊥ are, respectively, the substrate dimensionality and correlation-length exponent of a pure system. On the other hand, in the random transverse-field magnets the critical behavior is controlled by the infinite randomness fixed point in any dimensions, suggesting that CP with randomness follows the same scenario in any dimensions. For d<4, both expectations are valid, and the critical behavior of disordered CP is known to exhibit activated scaling. However, in four dimensions, dν⊥=2, and the stability of pure fixed point suggests that the disorder is irrelevant according to the Harris criterion. Our simulation results in four dimensions showed that the CP with quenched impurity exhibited the same critical behavior as the clean CP as long as the density of impurity sites x<xc, where xc is the critical density above which pure lattice sites cannot form an infinite cluster. At xc, the unusual nonuniversal power-law behavior was observed in the subcritical region and the activated scaling was found at the critical point.