The paper proposes mathematical models of the scalar and vector magnetic hysteresis to solve the problems of calculating the three-dimensional magnetic field in electrical devices by the method of spatial integral equations. A system of spatial integral equations is presented describing processes in the electrical devices. Basic and modified algorithms of the Jiles --- Atherton scalar model implementation are considered. The model parameters were tuned using a genetic algorithm. Algorithms for constructing the Jiles --- Atherton direct and inverse vector model in the three-dimensional setting were presented in accordance with the Mayergoyz approach. Generalized model of the vector 3D hysteresis makes it possible to describe the nonlinear properties of ferromagnetic medium for the isotropic and anisotropic materials. An example of the magnetic system calculation is provided taking into account the vector magnetic hysteresis. The simulation model parameters were determined from experimental data of the limited scope. Adequacy of the computational models was checked using a test problem. Results of the numerical studies demonstrated that the models were quite accurately reproducing real hysteresis dependences for various ferromagnetic materials. Proposed models and numerical algorithms implementing them were used as the component models that specified nonlinear characteristics of the magnetic materials in a software package that implemented the spatial integral equations method