While two levels systems (TLSs) are ubiqitous in solid state systems, microscopic understanding of their nature remains an outstanding problem. Conflicting phenomenological models are used to describe TLSs in seemingly similar materials when probed with different experimental techniques. Specifically, bulk measurements in amorphous solids have been interpreted using the model of a tunneling atom or group of atoms, whereas TLSs observed in the insulating barriers of Josephson junction qubits have been understood in terms of tunneling of individual electrons. Motivated by recent experiments studying TLSs in Josephson junctions, especially the effects of elastic strain on TLS properties, we analyze interaction of the electronic TLS with phonons. We demonstrate that strong polaronic effects lead to dramatic changes in TLS properties. Our model gives a quantitative understanding of the TLS relaxation and dephasing as probed in Josephson junction qubits, while providing an alternative interpretation of bulk experiments. We demonstrate that a model of polaron dressed electronic TLS leads to estimates for the density and distribution of parameters of TLSs consistent with bulk experiments in amorphous solids. This model explains such surprising observations of recent experiments as the existence of minima in the energy of some TLSs as a function of strain and makes concrete predictions for the character of TLS dephasing near such minima. We argue that better understanding of the microscopic nature of TLSs can be used to improve properties of quantum devices, from an enhancement of relaxation time of TLSs, to creating new types of strongly interacting optomechanical systems.