Mayer's transfer operator approach to Selberg's zeta function

These notes are based on three lectures given by the second author at Copenhagen University (October 2009) and at Aarhus University, Denmark (December 2009). Mostly, a survey of the results of Dieter Mayer on relationships between Selberg and Smale–Ruelle dynamical zeta functions is presented. In a special situation, the dynamical zeta function is defined for a geodesic flow on a hyperbolic plane quotient by an arithmetic cofinite discrete group. More precisely, the flow is defined for the corresponding unit tangent bundle. It turns out that the Selberg zeta function for this group can be expressed in terms of a Fredholm determinant of a classical transfer operator of the flow. The transfer operator is defined in a certain space of holomorphic functions, and its matrix representation in a natural basis is given in terms of the Riemann zeta function and the Euler gamma function.