On a problem of D. Blackwell from the theory of dynamic programming

In this paper the positive case of a dynamic programming problem is considered. We prove that, for any probability $p$ on the set of states $S$ and $\lambda<1$, there exists a stationary policy $\pi^*$ such that
$$ p\{I^{\pi^*}\ge\lambda\sup_\pi I^\pi\}=1, $$
where $I^\pi$ is the mean reward.