Hydraulic fracture technology has been widely applied to improve unconventional oil and gas production. The prevailing numerical analysis for hydraulic fracture technology is mainly based on the assumption of a homogeneous reservoir. However, unconventional reservoirs usually have complicated geologic conditions and the hypothesis of the homogeneous reservoir can strongly affect the accuracy of fracture simulation. To better understand the influence of heterogeneity to hydraulic fracture development, the effects of inclusions and heterogeneous stress fields are investigated by using the extended finite-element method. The heterogeneous stress field with fracture processing is developed, and the corresponding interaction between fracture and inclusion is investigated. The effects of different inclusions positions, opening and rotation angles, fractures lengths, and injected fluid viscosities to the hydraulic fracture development are studied based on the developed numerical model. Compared with the homogeneous stress field, numerical analysis indicates that the heterogeneous stress field could affect fracture behaviors and change the fracture energy distribution. In addition, the effects of inclusion can be restricted to some extent with higher injected fluid viscosity. The “stress shadow” effect with multiple fractures can weaken the influence of inclusions with properly designed perforation locations, which may be applied to optimize the hydraulic fracture development.