Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the stochastic opening and closing of ion channels and the resulting fluctuations in membrane current. When the microscopic transitions are fast, the macroscopic fluctuations are nearly uncorrelated and can be fully characterized by their mean and noise intensity. We show how, for an arbitrary Markov chain, the noise intensity can be determined from an algebraic equation, based on the transition rate matrix; these results are in agreement with earlier results from the theory of zero-frequency noise in quantum mechanical and classical systems. We demonstrate the validity of the theory using an analytically tractable two-state Markovian dichotomous noise, an eight-state model for a calcium channel subunit (De Young-Keizer model), and Markov models of the voltage-gated sodium and potassium channels as they appear in a stochastic version of the Hodgkin-Huxley model.