Markov decision processes with arbitrary real-valued criteria

We consider discrete time infinite horizon non-stationary Markov decision models with Borel state and action spaces. A criterion is a real-valued function defined on the space of strategic measures. We obtain general results and then use them to study the following criterions and their combinations: the expected total reward criterion, the expected utility criterion, the expected average criterion, the asymptotic reward criterion.