Two-particle pseudorapidity correlations, measured using charged particles with pT>0.5 GeV and |η|<2.4, from sNN=2.76 TeV Pb+Pb collisions collected in 2010 by the ATLAS experiment at the LHC are presented. The correlation function CN(η1,η2) is measured for different centrality intervals as a function of the pseudorapidity of the two particles, η1 and η2. The correlation function shows an enhancement along η−≡η1−η2≈0 and a suppression at large η− values. The correlation function also shows a quadratic dependence along the η+≡η1+η2 direction. These structures are consistent with a strong forward-backward asymmetry of the particle multiplicity that fluctuates event to event. The correlation function is expanded in an orthonormal basis of Legendre polynomials, Tn(η1)Tm(η2), and corresponding coefficients an,m are measured. These coefficients are related to mean-square values of the Legendre coefficients, an, of the single particle longitudinal multiplicity fluctuations: an,m=〈anam〉. Significant values are observed for the diagonal terms 〈an2〉 and mixed terms 〈anan+2〉. Magnitude of 〈a12〉 is the largest and the higher order terms decrease quickly with increase in n. The centrality dependence of the leading coefficient 〈a12〉 is compared to that of the mean-square value of the asymmetry of the number of participating nucleons between the two colliding nuclei, and also to the 〈a12〉 calculated from HIJING.