Minimizing lattice thermal conductivity (κL) is one of the most successful strategies for enhancing thermoelectrics. Existing experimental approach for κL-determination largely depend on model approximations for the electronic component (κE) with a subtraction from the measured total thermal conductivity (κ = κL+κE). However, the approximations (e.g. Wiedemann-Franz law) may not always be accurate particularly for materials with complicated band structures. Accurate κL-determination, not only characterizing the fundamental capability in heat conduction but also providing the insight of thermal functional materials, is still challenging. In this work, we utilize the magneto-thermoelectric effect under a magnetic field to suppress κE whereas κL is nearly unaffected. This allows for direct κL-measurement in materials that particularly have high carrier mobility. It is demonstrated in Bi1-xSbx single crystals that longitudinal κE can be sufficiently minimized to enable direct measurement in longitudinal κL at T < 100 K under a magnetic field of 5 T. It is revealed that the commonly used single Kane band (SKB) model approximations would lead to a 30 % overestimation in κL. The method proposed here enables a pathway of κL-determination for understanding the phonon transport in materials with complicated band structure.