Classification of multi-agent reinforcement learning problems

With the advent of deep single-agents reinforcement learning (SARL), multi-agent reinforcement learning (MARL) has received a new impetus for development in the form of deep multi-agent reinforcement learning (MDRL). The active development of methods in this area over the past few years has actualized the issues of their systematization and classification. Existing works use the mechanisms used in the corresponding MDRL methods as classification signs. However, the applicability of a particular method is determined not only by the class of the method, but also by the class of the MARL problem. The purpose of this work is to formalize and classify MARL tasks. To achieve the goal, the mathematical formalization and generalization of the existing classifications of SARL tasks is carried out. The peculiarities arising in the transition from the SARL problem to the MARL problem are considered and mathematically formalized. The essential features are highlighted and the classification of MARL tasks is carried out on the basis of the set-theoretic approach. The use of the set-theoretic approach made it possible to identify classes of MARL problems, generalized in other similar works, but possessing specific properties, which can be used to develop more efficient methods for solving such MARL problems. It is expected that the proposed formalism and classification of MARL problems will be useful for researchers as a tool for setting a problem and determining the place of research in the general structure of MARL methods and tasks, and will also be useful for developers for a reasonable choice of MARL methods based on the class of the problem being solved.