The effective electrical conductivity of multi-walled carbon nanotube/polydimethylsiloxane composites with chain-structured ferromagnetic particles has been investigated by experiments and micromechanics-based modeling. A multi-scale modeling approach is used to consider different size of fillers of multi-walled carbon nanotubes and particles as well as their distribution in the matrix. At nanoscale, for multi-walled carbon nanotube/polydimethylsiloxane composite, eight-chain model and influence of waviness of multi-walled carbon nanotube are considered to render an effective electrical conductivity. At microscale, ferromagnetic particles are aligned in the matrix made of the multi-walled carbon nanotube/polydimethylsiloxane composite, and an analytical model is established based on representative volume element. The influence of inter-particle distance is evaluated. The proposed analytic results agree well with the experimental results. The present model can be a useful tool for design and analysis of these composites for sensing applications considering their percolation threshold and overall electrical conductivity.