
arXiv: 1802.01062
A strategy is proposed for characterizing the worst-case performance of algorithms for solving nonconvex smooth optimization problems. Contemporary analyses characterize worst-case performance by providing, under certain assumptions on an objective function, an upper bound on the number of iterations (or function or derivative evaluations) required until a pth-order stationarity condition is approximately satisfied. This arguably leads to conservative characterizations based on anomalous objectives rather than on ones that are typically encountered in practice. By contrast, the strategy proposed in this paper characterizes worst-case performance separately over regions comprising a search space. These regions are defined generically based on properties of derivative values. In this manner, one can analyze the worst-case performance of an algorithm independently from any particular class of objectives. Then, once given a class of objectives, one can obtain an informative, fine-tuned complexity analysis merely by delineating the types of regions that comprise the search spaces for functions in the class. Regions defined by first- and second-order derivatives are discussed in detail and example complexity analyses are provided for a few fundamental first- and second-order algorithms when employed to minimize convex and nonconvex objectives of interest. It is also explained how the strategy can be generalized to regions defined by higher-order derivatives and for analyzing the behavior of higher-order algorithms.
Numerical optimization and variational techniques, regional complexity analysis, Analysis of algorithms and problem complexity, nonconvex optimization, Nonconvex programming, global optimization, nonlinear optimization, worst-case evaluation complexity, Complexity and performance of numerical algorithms, Numerical mathematical programming methods, Optimization and Control (math.OC), FOS: Mathematics, Abstract computational complexity for mathematical programming problems, Mathematics - Optimization and Control, worst-case iteration complexity
Numerical optimization and variational techniques, regional complexity analysis, Analysis of algorithms and problem complexity, nonconvex optimization, Nonconvex programming, global optimization, nonlinear optimization, worst-case evaluation complexity, Complexity and performance of numerical algorithms, Numerical mathematical programming methods, Optimization and Control (math.OC), FOS: Mathematics, Abstract computational complexity for mathematical programming problems, Mathematics - Optimization and Control, worst-case iteration complexity
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 2 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
