The optimal coding scheme for Additive White Gaussian noise (AWGN) channels with noisy output feedback has been unknown for several decades. The best-known linear scheme is by Chance and Love, where the coefficients of the linear scheme are numerically optimized based on unique observations. In this paper, we introduce a new class of linear coding schemes, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sequential linear schemes</i> , where the encoder sequentially updates a linear state process based on feedback. We then derive the optimal scheme within this class, in a closed form, by formulating a novel Markov decision process and solving it via dynamic programming. We demonstrate that our scheme outperforms the Chance-Love scheme for channels with noisy feedback and coincides with the Shalkwijk-Kailath scheme for channels with noiseless feedback. This problem is an instance of decentralized control <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">without any common information</i> and, to the best of our knowledge, the first such scenario where we can derive analytical solutions using dynamic programming.