Synaptically coupled neurons show in-phase or antiphase synchrony depending on the chemical and dynamical nature of the synapse. Deterministic theory helps predict the phase differences between two phase-locked oscillators when the coupling is weak. In the presence of noise, however, deterministic theory faces difficulty when the coexistence of multiple stable oscillatory solutions occurs. We analyze the solution structure of two coupled neuronal oscillators for parameter values between a subcritical Hopf bifurcation point and a saddle node point of the periodic branch that bifurcates from the Hopf point, where a rich variety of coexisting solutions including asymmetric localized oscillations occurs. We construct these solutions via a multiscale analysis and explore the general bifurcation scenario using the lambda-omega model. We show for both excitatory and inhibitory synapses that noise causes important changes in the phase and amplitude dynamics of such coupled neuronal oscillators when multiple oscillatory solutions coexist. Mixed-mode oscillations occur when distinct bistable solutions are randomly visited. The phase difference between the coupled oscillators in the localized solution, coexisting with in-phase or antiphase solutions, is clearly represented in the stochastic phase dynamics.