Problem of stability of two-link trajectories in a multidimensional Birkhoff billiard

A linearized problem of stability of simple periodic motions with elastic reflections is considered: a particle moves along a straight-line segment that is orthogonal to the boundary of a billiard at its endpoints. In this problem issues from mechanics (variational principles), linear algebra (spectral properties of products of symmetric operators), and geometry (focal points, caustics, etc.) are naturally intertwined. Multidimensional variants of Hill's formula, which relates the dynamic and geometric properties of a periodic trajectory, are discussed. Stability conditions are expressed in terms of the geometric properties of the boundary of a billiard. In particular, it turns out that a nondegenerate two-link trajectory of maximum length is always unstable. The degree of instability (the number of multipliers outside the unit disk) is estimated. The estimates are expressed in terms of the geometry of the caustic and the Morse indices of the length function of this trajectory.