The maximum expected reward vector that arises in continuous parameter Markov decision problems is frequently characterized as the unique solution of a certain Cauchy problem. This paper generalizes this characterization by viewing the maximum expected reward vector as a nonlinear semigroup in an appropriate Banach space. This perspective has several advantages. First, the semi-group may exist even though the corresponding Cauchy problem does not have a solution. Second, this approach is often useful in showing when the Cauchy problem does have a solution. Third, these methods are useful in the study of the method of successive approximations. Finally, these methods appear likely to unify some diverse results in Markov decision theory. The results in this paper are very general. First, sufficient conditions are given for the semigroup to exist. The discounted reward case is studied next ; a certain operator is shown to have a unique singular point that is the strong limit of the semigroup as the parameter $t \to \infty $. The asymptotic properties of the semigroup in the case of undiscounted rewards are studied with the aid of some fixed point theorems for monotone and nonexpansive operators; the transient, positive, negative and optimal stopping cases are studied in this context. The paper concludes with two examples. The first is a controlled diffusion process on a compact interval of the real line. The second is a controlled jump process with general state and action spaces.