Heat dissipation is one of the most serious problems in modern integrated electronics with the continuously decreasing devices size. Large portion of the consumed power is inevitably dissipated in the form of waste heat which not only restricts the device energy-efficiency performance itself, but also leads to severe environment problems and energy crisis. Thermoelectric Seebeck effect is a green energy-recycling method, while thermoelectric Peltier effect can be employed for heat management by actively cooling overheated devices, where passive cooling by heat conduction is not sufficiently enough. However, the technological applications of thermoelectricity are limited so far by their very low conversion efficiencies and lack of deep understanding of thermoelectricity in microscopic levels. Probing and managing the thermoelectricity is therefore fundamentally important particularly in nanoscale. In this short review, we will first briefly introduce the microscopic techniques for studying nanoscale thermoelectricity, focusing mainly on scanning thermal microscopy (SThM). SThM is a powerful tool for mapping the lattice heat with nanometer spatial resolution and hence detecting the nanoscale thermal transport and dissipation processes. Then we will review recent experiments utilizing these techniques to investigate thermoelectricity in various nanomaterial systems including both (two-material) heterojunctions and (single-material) homojunctions with tailored Seebeck coefficients, and also spin Seebeck and Peltier effects in magnetic materials. Next, we will provide a perspective on the promising applications of our recently developed Scanning Noise Microscope (SNoiM) for directly probing the non-equilibrium transporting hot charges (instead of lattice heat) in thermoelectric devices. SNoiM together with SThM are expected to be able to provide more complete and comprehensive understanding to the microscopic mechanisms in thermoelectrics. Finally, we make a conclusion and outlook on the future development of microscopic studies in thermoelectrics.