The realization of remarkable thermoelectric (TE) properties in a novel single-crystalline quantum material is a topic of prime interest in the field of thermoelectricity. It necessitates a proper understanding of transport properties under magnetic field and magnetic properties at low field. We report polarized Raman spectroscopic study, TE properties, and magneto-resistance (MR) along with magnetic characterization of single-crystalline Bi2Se3. Polarized Raman spectrum confirms the strong polarization effect of A1g1 and A1g2 phonon modes, which verifies the anisotropic nature of the Bi2Se3 single crystal. Magnetization measurement along the in-plane direction of single crystal divulges a cusp-like paramagnetic response in susceptibility plot, indicating the presence of topological surface states (TSSs) in the material. In-depth MR studies performed in different configurations also confirm the presence of anisotropy in the single-crystalline Bi2Se3 sample. A sharp rise in MR value near zero magnetic field and low-temperature regime manifests a weak anti-localization (WAL) effect, depicting the quantum origin of the conductivity behavior at low temperature. Moreover, in-plane magneto-conductivity data at low-temperature (up to 5 K) and low-field region (≤15 kOe) confirm the dominance of the WAL effect (due to TSS) with a negligible bulk contribution. Quantum oscillation (SdH) in magneto-transport data also exhibits the signature of TSS. Additionally, an exceptional TE power factor of ∼950 μW m−1 K−2 at 300 K is achieved, which is one of the highest values reported for pristine Bi2Se3. Our findings pave the way for designing single crystals, which give dual advantages of being a good TE material along with a topological insulator bearing potential application.