Emerging chalcogenide-based two-dimensional (2D) materials possess various unique yet fully explored properties and are thus considered promising candidates for next-generation optoelectronic and energy conversion applications. Here, TeSe2 crystals with interesting thermoelectric features were synthesized using a simple solid-state reaction. High-resolution transmission electron microscopy reveals that TeSe2 stabilizes in a 2D atomic structure with helical chains, resembling 2D tellurene. The thermoelectric properties were measured from 2 to 390 K in a polycrystalline pellet, showing an ultralow thermal conductivity below 0.25 W m-1 K-1 and a very high positive Seebeck coefficient of up to 865 μV K-1. Particularly, the thermal conductivity shows a hysteresis effect upon temperature cycling, which is tentatively explained as cracks opening and partially closing. Optical measurements indicate that TeSe2 is a semiconductor with two bandgaps at 1.43 and 1.65 eV. These results highlight that TeSe2 is an intriguing 2D semiconductor with complex thermoelectric properties, which provides a platform to further study the interplay of emerging 2D structure, thermal, and electronic properties.