A phase reference has been a standard requirement in continuous-variable quantum sensing and communication protocols. However, maintaining a phase reference is challenging due to environmental fluctuations, preventing quantum phenomena such as entanglement and coherence from being utilized in many scenarios. We show that quantum communication and entanglement-assisted communication without a phase reference are possible, when a short-time memory effect is present. The degradation in the communication rate of classical or quantum information transmission decreases inversely with the correlation time. Exact solutions of the quantum capacity and entanglement-assisted classical and quantum capacity for pure dephasing channels are derived, where non-Gaussian multipartite-entangled states show strict advantages over usual Gaussian sources. For thermal-loss dephasing channels, lower bounds of the capacities are derived. The lower bounds also extend to scenarios with fading effects in the channel. In addition, for entanglement-assisted communication, the lower bounds can be achieved by a simple phase-encoding scheme on two-mode squeezed vacuum sources, when the noise is large.