The physical properties of the Ni-Mn-In-B alloys synergistically modulated by atomic occupancy and magnetic states are systematically investigated. The results indicate that B atoms are indirectly substituted at the Mn2 position in the Ni2Mn1.5In0.5 alloy (i.e., B→In→Mn2). When B is doped in interstitial positions, the B atoms tend to be closer to Mn atoms rather than In atoms. Additionally, the higher the concentration of Mn around the B atom, the lower the formation energy of the austenite (A) phase. The lattice parameters of the A phase decreases with increasing x content, while c/a and phase stability gradually decrease for the 4O, 6M, and NM martensites in the ferrimagnetic (FIM) state. The magnetostrain of the alloy shows an increasing trend with increasing B content due to the decrease in c/a resulting in a decrease in the tangent variable S= |c/a-a/c|; however, the elastocaloric properties decrease due to the decrease in the volume change ΔV/VA after the transformation. The Ni2Mn1.5-xIn0.5Bx (0≤x≤0.042) alloys undergo an A→6MFIM→NMFIM transformation. The total magnetic moment of the A phase decreases, whereas those of the 4O, 6M, and NM phases increase with increasing x content. The phase stability is explained by analyzing the total densities of states (TDOS) from an electronic structure perspective.