Abstract:
The paper is devoted to the problem of minimization of the non-smooth functional $f$ with a non-positive non-smooth Lipschitz-continuous functional constraint. We consider the formulation of the problem in the case of quasi-convex functionals. We propose new strategies of step-sizes and adaptive stopping rules in Mirror Descent for the considered class of problems. It is shown that the methods are applicable to the objective functionals of various levels of smoothness. Applying a special restart technique to the considered version of Mirror Descent there was proposed an optimal method for optimization problems with strongly convex objective functionals. Estimates of the rate of convergence for the considered methods are obtained depending on the level of smoothness of the objective functional. These estimates indicate the optimality of the considered methods from the point of view of the theory of lower oracle bounds. In particular, the optimality of our approach for Hölder-continuous quasi-convex (sub)differentiable objective functionals is proved. In addition, the case of a quasi-convex objective functional and functional constraint was considered. In this paper, we consider the problem of minimizing a non-smooth functional $f$ in the presence of a Lipschitz-continuous non-positive non-smooth functional constraint $g$, and the problem statement in the cases of quasi-convex and strongly (quasi-)convex functionals is considered separately. The paper presents numerical experiments demonstrating the advantages of using the considered methods.
The research of F. Stonyakin in Algorithms 2 and 4, Remarks 2 and 3, Theorems 2 and 5 was supported by Russian Science Foundation (project 18-71-00048). The research of F. Stonyakin and A. Stepanov in numerical experiments (Examples 1 and 2) was supported by Russian Science Foundation (project 18-71-00048). The research in Theorems 3 and 4, Algorithm 3 and in numerical experiments (Examples 3 and 4) was supported by Russian Foundation for Basic Research (project 18-31-00219 mol-a). The research of F. Stonyakin in Remark 4 was supported by the grant of the President of Russian Federation for young candidates of sciences (project MK-15.2020.1). The research of A. V. Gasnikov (Algorithm 3 and Lemma 3) is supported by the Ministry of Science and Higher Education of the Russian Federation (Goszadaniye in MIPT, project 075-00337-20-03).
Citation:
F. S. Stonyakin, A. N. Stepanov, A. V. Gasnikov, A. A. Titov, “Mirror descent for constrained optimization problems with large subgradient values of functional constraints”, Computer Research and Modeling, 12:2 (2020), 301–317
\Bibitem{StoSteGas20}
\by F.~S.~Stonyakin, A.~N.~Stepanov, A.~V.~Gasnikov, A.~A.~Titov
\paper Mirror descent for constrained optimization problems with large subgradient values of functional constraints
\jour Computer Research and Modeling
\yr 2020
\vol 12
\issue 2
\pages 301--317
\mathnet{http://mi.mathnet.ru/crm786}
\crossref{https://doi.org/10.20537/2076-7633-2020-12-2-301-317}
Linking options:
https://www.mathnet.ru/eng/crm786
https://www.mathnet.ru/eng/crm/v12/i2/p301
This publication is cited in the following 7 articles:
S. S. Ablaev, A. N. Beznosikov, A. V. Gasnikov, D. M. Dvinskikh, A. V. Lobanov, S. M. Puchinin, F. S. Stonyakin, “On Some Works of Boris Teodorovich Polyak on the Convergence of Gradient Methods and Their Development”, Comput. Math. and Math. Phys., 64:4 (2024), 635
S. S. Ablaev, A. N. Beznosikov, A. V. Gasnikov, D. M. Dvinskikh, A. V. Lobanov, S. M. Puchinin, F. S. Stonyakin, “On Some Works of Boris Teodorovich Polyak on the Convergence of Gradient Methods and Their Development”, Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, 64:4 (2024), 587
S. S. Ablaev, F. S. Stonyakin, M. S. Alkousa, A. V. Gasnikov, “Adaptive Subgradient Methods for Mathematical Programming Problems with Quasiconvex Functions”, Proc. Steklov Inst. Math. (Suppl.), 323:1 (2023), S1–S18
Darina Dvinskikh, Alexander Gasnikov, “Decentralized and parallel primal and dual accelerated methods for stochastic convex programming problems”, Journal of Inverse and Ill-posed Problems, 29:3 (2021), 385
F. S. Stonyakin, I. V. Baran, “O nekotorykh algoritmakh dlya uslovnykh zadach optimizatsii s otnositelnoi tochnostyu po tselevomu funktsionalu”, Tr. IMM UrO RAN, 26, no. 3, 2020, 198–210
Anastasiya Ivanova, Fedor Stonyakin, Dmitry Pasechnyuk, Evgeniya Vorontsova, Alexander Gasnikov, “Adaptive Mirror Descent for the Network Utility Maximization Problem”, IFAC-PapersOnLine, 53:2 (2020), 7851
Alexander A. Titov, Fedor S. Stonyakin, Mohammad S. Alkousa, Seydamet S. Ablaev, Alexander V. Gasnikov, Communications in Computer and Information Science, 1275, Mathematical Optimization Theory and Operations Research, 2020, 133