학술논문

High-order methods beyond the classical complexity bounds: inexact high-order proximal-point methods
Document Type
Original Paper
Source
Mathematical Programming: A Publication of the Mathematical Optimization Society. :1-43
Subject
Convex composite optimization
High-order proximal-point operator
Bi-level optimization framework
Lower complexity bounds
Optimal methods
Superfast methods
90C25
90C06
49J52
65K05
65Y20
Language
English
ISSN
0025-5610
1436-4646
Abstract
We introduce a Bi-level OPTimization (BiOPT) framework for minimizing the sum of two convex functions, where one of them is smooth enough. The BiOPT framework offers three levels of freedom: (i) choosing the order p of the proximal term; (ii) designing an inexact pth-order proximal-point method in the upper level; (iii) solving the auxiliary problem with a lower-level non-Euclidean method in the lower level. We here regularize the objective by a (p+1)p≥1O(k-(p+1))q=⌊p/2⌋th-order proximal term (for arbitrary integer (p+1)p≥1O(k-(p+1))q=⌊p/2⌋) and then develop the generic inexact high-order proximal-point scheme and its acceleration using the standard estimating sequence technique at the upper level. This follows at the lower level with solving the corresponding pth-order proximal auxiliary problem inexactly either by one iteration of the pth-order tensor method or by a lower-order non-Euclidean composite gradient scheme. Ultimately, it is shown that applying the accelerated inexact pth-order proximal-point method at the upper level and handling the auxiliary problem by the non-Euclidean composite gradient scheme lead to a 2q-order method with the convergence rate (p+1)p≥1O(k-(p+1))q=⌊p/2⌋ (for (p+1)p≥1O(k-(p+1))q=⌊p/2⌋ and the iteration counter k), which can result to a superfast method for some specific class of problems.