Rating and ranking players in incomplete tournaments

Suppose that player A has beaten a strong opponent, while player B has beaten a weaker opponent, but twice. Who did better? Questions like this arise not only in sports, but also in psychology, sociology, political science, economics, expert judgments, the evaluation of network nodes, etc. The paper focuses on statistical and non-statistical methods for rating and ranking players in arbitrary incomplete tournaments, as well as on some approaches to comparing these methods. Among others, we discuss the generalized row sum method proposed by the author.