Polymer matrix filled with ferromagnetic particles is a class of smart materials whose mechanical properties can be changed under different magnetic field. They are usually referred to as magnetorheological elastomers (MREs). A finite element simulation was presented to describe the mechanical behavior of MREs with the nonlinearity of the particle magnetization being incorporated. By introducing the Maxwell stress tensor, a representative volume element (RVE) was proposed to calculate the Young's modulus and shear modulus of MREs due to the applied magnetic field. The influences of the applied magnetic field and the particle volume fractions in the shear modulus and Young's modulus were studied. Results show that the shear modulus increases with the magnitude of the applied magnetic field, while the Young's modulus decreases.