Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Archive
Impact factor

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS


Zh. Vychisl. Mat. Mat. Fiz.:
Year:
Volume:
Issue:
Page:
Find




Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register

Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, 2024, Volume 64, Number 4, Pages 587–626
DOI: https://doi.org/10.31857/S0044466924040028 (Mi zvmmf11729) 		 
Optimal control

On some works of Boris Teodorovich Polyak on the convergence of gradient methods and their development

S. S. Ablaevab, A. N. Beznosikovac, A. V. Gasnikovacd, D. M. Dvinskikhacd, A. V. Lobanovad, S. M. Puchinina, F. S. Stonyakinab

a Moscow Institute of Physics and Technology, 141700, Dolgoprudny, Moscow oblast, Russia
b Crimean Federal University, 295007, Simferopol, Republic of Crimea, Russia
c Institute for Information Transmission Problems, Russian Academy of Sciences, 127051, Moscow, Russia
d Institute for System Programming, Russian Academy of Sciences, 109004, Moscow, Russia
DOI: https://doi.org/10.31857/S0044466924040028
Abstract: The paper presents a review of the current state of subgradient and accelerated convex optimization methods, including the cases with the presence of noise and access to various information about the objective function (function value, gradient, stochastic gradient, higher derivatives). For nonconvex problems, the Polyak–Lojasiewicz condition is considered and a review of the main results is given. The behavior of numerical methods in the presence of a sharp minimum is considered. The aim of this review is to show the influence of the works of B.T. Polyak (1935–2023) on gradient optimization methods and their surroundings on the modern development of numerical optimization methods.
Key words: gradient descent, gradient dominance condition (Polyak–Lojasiewicz), sharp minimum, subgradient Polyak–Shor method, early stopping condition, Polyak heavy ball method, stochastic gradient descent.
Funding agency Grant number
Ministry of Science and Higher Education of the Russian Federation FSMG-2024-0011
The research was supported by the Ministry of Science and Higher Education of the Russian Federation (state assignment), project no. FSMG-2024-0011.
Received: 15.09.2023
Revised: 16.12.2023
Accepted: 17.11.2023
English version:
Computational Mathematics and Mathematical Physics, 2024, Volume 64, Issue 4, Pages 635–675
DOI: https://doi.org/10.1134/S0965542524700076
Bibliographic databases:
Document Type: Article
UDC: 519.85
Language: Russian
Citation: S. S. Ablaev, A. N. Beznosikov, A. V. Gasnikov, D. M. Dvinskikh, A. V. Lobanov, S. M. Puchinin, F. S. Stonyakin, “On some works of Boris Teodorovich Polyak on the convergence of gradient methods and their development”, Zh. Vychisl. Mat. Mat. Fiz., 64:4 (2024), 587–626; Comput. Math. Math. Phys., 64:4 (2024), 635–675
Citation in format AMSBIB
\Bibitem{AblBezGas24}
\by S.~S.~Ablaev, A.~N.~Beznosikov, A.~V.~Gasnikov, D.~M.~Dvinskikh, A.~V.~Lobanov, S.~M.~Puchinin, F.~S.~Stonyakin
\paper On some works of Boris Teodorovich Polyak on the convergence of gradient methods and their development
\jour Zh. Vychisl. Mat. Mat. Fiz.
\yr 2024
\vol 64
\issue 4
\pages 587--626
\mathnet{http://mi.mathnet.ru/zvmmf11729}
\crossref{https://doi.org/10.31857/S0044466924040028}
\elib{https://elibrary.ru/item.asp?id=74490703}
\transl
\jour Comput. Math. Math. Phys.
\yr 2024
\vol 64
\issue 4
\pages 635--675
\crossref{https://doi.org/10.1134/S0965542524700076}
Linking options:
  • https://www.mathnet.ru/eng/zvmmf11729
  • https://www.mathnet.ru/eng/zvmmf/v64/i4/p587
    • Citing articles in Google Scholar: Russian citations, English citations
    Related articles in Google Scholar: Russian articles, English articles
    		Журнал вычислительной математики и математической физики Computational Mathematics and Mathematical Physics
    Statistics & downloads:
    Abstract page:4
     
      Contact us:
    		  Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2024