Practical challenges in non-convex optimization

In this talk, I will discuss several topics. First, is the optimization over low-rank matrix and tensor manifolds, which often appear in applications. Low-rank approximation of matrices is one of the rare examples when a non-convex problem can be solved in a numerically exact way by using singular value decomposition (SVD). There also exists a large class of methods for solving optimization with low-constraints.
In the second part of the talk (if time permits), I will discuss the peculiarities of optimization with deep neural networks. The theory of such optimization is still a big mystery, with a lot of empirical results and theoretical results under unrealistic assumptions. Here I plan to highlight the main points and research directions.