The paper describes a new type of soft magnetic nanocomposite (SMN) based on a transient network of wormlike surfactant micelles with embedded oppositely charged submicron particles of magnetite acting as cross-linking agents. We study the change of the rheological properties of the SMNs with different contents of particles in response to magnetic field. We show that even at low field strengths the system acquires solid-like behavior, which can be attributed to the aggregation of particles into chain-like/column structures. A solid-like behavior appears at a rather small volume fraction of particles (0.002–0.04) indicating weak restrictions imposed by the matrix to the reorganization of particles under magnetic field, which can be due to the self-assembled structure of the micellar network. In the oscillatory rheological measurements, SMNs show a linear viscoelastic response in an unusually wide region of values of strain, magnetic field strength and content of particles, which is caused by the viscoelastic contribution of the micellar network. Upon gradual increase of magnetic field strength H, the dynamic moduli G′ and G′′ demonstrate slow growth followed by a sharp rise with a scaling law H3.0 and reach a plateau at 0.15 T. The highest values of the storage modulus G′ in SMNs are close to those in magnetorheological fluids with liquid Newtonian carrier, where particles move freely and the G′ value is defined by the interactions of magnetized particles and chain-like/columns structures. SMNs have a yield stress, which grows with the increase of magnetic field strength and finally levels off just at the same magnetic field strength at which the G′ and G′′ values reach a plateau indicating the saturation of the particles magnetization. The concentration dependencies of the elastic modulus and yield stress suggest the transition from chain-like to columnar structures of the particles. The new SMNs possessing the features of both magnetic fluids and magnetic gels have promising potential in a wide range of applications requiring responsiveness to magnetic field.