Markov controlled models with countable state space and continuous time

Following general lines of [10], we expand the Blackwell–Strauch dynamic programming theory, which takes into account policies depending on the whole past, to continuous time Markov decision processes with countable state and Borel action spaces. Nonhomogeneous processes with finite and infinite horizon and non-randomized policies are treated. An optimality equation is obtained for negative models and models with value not equal to $-\infty$. The existence of Markovian $\varepsilon$-optimal policies is proved for models with bounded value and small positive share of far future. The semicontinuous case is also considered.