Pure stationary nash equilibria for discounted stochastic positional games

A discounted stochastic positional game is a stochastic game with discounted payoffs in which the set of states is divided into several disjoint subsets such that each subset represents the position set for one of the player and each player control the Markov decision process only in his position set. In such a game each player chooses actions in his position set in order to maximize the expected discounted sum of his stage rewards. We show that an arbitrary discounted stochastic positional game with finite state and action spaces possesses a Nash equilibrium in pure stationary strategies. Based on the proof of this result we present conditions for determining all optimal pure stationary strategies of the players.