Doklady Rossijskoj Akademii Nauk. Mathematika, Informatika, Processy Upravlenia
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Archive
Impact factor

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS



Dokl. RAN. Math. Inf. Proc. Upr.:
Year:
Volume:
Issue:
Page:
Find






Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register


Doklady Rossijskoj Akademii Nauk. Mathematika, Informatika, Processy Upravlenia, 2023, Volume 514, Number 2, Pages 187–195
DOI: https://doi.org/10.31857/S2686954323601999
(Mi danma464)
 

SPECIAL ISSUE: ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING TECHNOLOGIES

1-Dimensional topological invariants to estimate loss surface non-convexity

D. S. Voronkovaa, S. A. Barannikovab, E. V. Burnaevac

a Skolkovo Institute of Science and Technology, Moscow, Russia
b CNRS, IMJ, Paris Cité University, Франция
c Artificial Intelligence Research Institute, Moscow, Russia
References:
Abstract: We utilize the framework of topological data analysis to examine the geometry of loss landscape. With the use of topology and Morse theory, we propose to analyse 1-dimensional topological invariants as a measure of loss function non-convexity up to arbitrary re-parametrization. The proposed approach uses optimization of 2-dimensional simplices in network weights space and allows to conduct both qualitative and quantitative evaluation of loss landscape to gain insights into behavior and optimization of neural networks. We provide geometrical interpretation of the topological invariants and describe the algorithm for their computation. We expect that the proposed approach can complement the existing tools for analysis of loss landscape and shed light on unresolved issues in the field of deep learning.
Funding agency Grant number
Skolkovo Institute of Science and Technology
This work was partially supported by the Next Generation Program (3rd Call for Proposals).
Presented: A. L. Semenov
Received: 05.09.2023
Revised: 15.09.2023
Accepted: 18.10.2023
English version:
Doklady Mathematics, 2023, Volume 108, Issue suppl. 2, Pages S325–S332
DOI: https://doi.org/10.1134/S1064562423701569
Bibliographic databases:
Document Type: Article
UDC: 004.8
Language: Russian
Citation: D. S. Voronkova, S. A. Barannikov, E. V. Burnaev, “1-Dimensional topological invariants to estimate loss surface non-convexity”, Dokl. RAN. Math. Inf. Proc. Upr., 514:2 (2023), 187–195; Dokl. Math., 108:suppl. 2 (2023), S325–S332
Citation in format AMSBIB
\Bibitem{VorBarBur23}
\by D.~S.~Voronkova, S.~A.~Barannikov, E.~V.~Burnaev
\paper 1-Dimensional topological invariants to estimate loss surface non-convexity
\jour Dokl. RAN. Math. Inf. Proc. Upr.
\yr 2023
\vol 514
\issue 2
\pages 187--195
\mathnet{http://mi.mathnet.ru/danma464}
\crossref{https://doi.org/10.31857/S2686954323601999}
\elib{https://elibrary.ru/item.asp?id=56717811}
\transl
\jour Dokl. Math.
\yr 2023
\vol 108
\issue suppl. 2
\pages S325--S332
\crossref{https://doi.org/10.1134/S1064562423701569}
Linking options:
  • https://www.mathnet.ru/eng/danma464
  • https://www.mathnet.ru/eng/danma/v514/i2/p187
  • Citing articles in Google Scholar: Russian citations, English citations
    Related articles in Google Scholar: Russian articles, English articles
    Doklady Rossijskoj Akademii Nauk. Mathematika, Informatika, Processy Upravlenia Doklady Rossijskoj Akademii Nauk. Mathematika, Informatika, Processy Upravlenia
     
      Contact us:
     Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2024