Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes

The classical Fatou lemma states that the lower limit of a sequence of integrals of functions is greater than or equal to the integral of the lower limit. It is known that Fatou's lemma for a sequence of weakly converging measures states a weaker inequality because the integral of the lower limit is replaced with the integral of the lower limit in two parameters, where the second parameter is the argument of the functions. In the present paper, we provide sufficient conditions when Fatou's lemma holds in its classical form for a sequence of weakly converging measures. The functions can take both positive and negative values. Similar results for sequences of setwise converging measures are also proved. We also put forward analogies of Lebesgue's and the monotone convergence theorems for sequences of weakly and setwise converging measures. The results obtained are used to prove broad sufficient conditions for the validity of optimality equations for average-cost Markov decision processes.