A novel optimization algorithm, called the Magnetic Optimization Algorithms (MOAs), is proposed in this paper which is inspired by the principles of magnetic field theory. In MOA, the possible solutions are some magnetic particles scattered in the search space. In this respect, each magnetic particle has a measure of mass and magnetic field according to its fitness. In this scheme, the fitter magnetic particles are more massive, with stronger magnetic field. In terms of interaction, these particles are located in a structured population and apply a long range force of attraction to their neighbors. Ten different structures are proposed for the algorithm and the structure that offers the best performance is found. Also, to improve the exploration ability of the algorithm, several operators are proposed: a repulsive short-range force, an explosion operator, a combination of short-range force and explosion operator and a crossover interaction between the neighboring particles. In order to test the proposed algorithm and the proposed operators, the algorithm is compared with a variety of existing algorithms on 21 numerical benchmark functions. The experimental results suggest that the proposed algorithm outperforms some of the existing algorithms.