Lecture plan
I will update the lecture plan as we go along
Date | Topics | Reading |
---|---|---|
Introduction, notion and existence of solutions | ||
Week 2 | Overview Formulation of a minimization problem Definitions of minima Lower semi-continuity and coercivity Existence theorems for minima | N&W, Chapters 1, 2.1 A note by M. Grasmair Minimisers of Optimisation Problems |
Theory and methods for unconstrained optimization | ||
Taylor's theorem in higher dimensions 1st & 2nd order necessary & sufficient conditions | N&W, Chapter 2.1 | |
Week 3 | Convex sets and functions | N&W, Chapter 2.1 Note by M. Grasmair Basic Properties of Convex Functions |
Line search methods Armijo (backtracking) line search The Wolfe conditions. | N&W, Chapter 3.1 | |
Week 4 | Convergence of backtracking methods Zoutendijk's result Algorithmic approach to the Wolfe conditions Convergence rate of line search methods | N&W, Chapters 3.1-3.3 , note03.pdf, 2D example in Python: steep_desc.py |
Week 5 | Linear and non-linear conjugate gradient methods (Fletcher-Reeves, Polack-Ribière, Hestenes-Stiefel) | N&W, Chapters 5.1-5.2. 2D example in Python: cg2d.py |
Week 6 | Quasi-Newton methods (DFP, BFGS, and SR1, L-BFGS). Newton-CG method. Derivative-free minimization (Nedler-Mead). | N&W, 6.1-6.2, 7.1-7.2, 9.5 inexactnewton.pdf, rosenbrock_nedler.m |
Week 7 | Linear & non-linear least-squares problems. Gauss-Newton method | N&W 10.1-10.3 |
Idea of trust-region methods. Cauchy point and dogleg method. | N&W, 4.1-4.2, rosenbrock_extrust.m, rosenbrock_dogleg.m | |
Theory and methods for constrained optimization | ||
Week 8 | Introduction to optimality conditions for constrained problems: optimization over convex sets and hyperplanes Applications to trust-region subproblem and quasi-Newton formulas Radial and normal cones | This note |
Week 9 | Tangent cones Fundamental necessary condition Constraint qualifications Linearized feasible directions | N&W 12 |
Week 10 | Separation of point and convex set (Hahn-Banach theorem) Farkas' lemma KKT-conditions Second order conditions | N&W 12 |
Week 11 | Quadratic programming Active set methods | N&W 16.4-5 |
Penalty and augmented Lagrangian methods (overview) | N&W 17 | |
Week 12 | Sequential quadratic programming Merit functions Line search-SQP method | N&W 18.1-18.4 |
Non-smooth merit function for Newton's methods Sequential quadratic programming with inequality constraints Maratos effect | N&W 18.1-18.4 | |
Week 13 | Easter break | |
Week 15 | Linear programming (LP) Standard form of LP Optimality conditions and duality | N&W 13.1, this note |
Interior point methods for LP Central path Primal-dual path-following methods | N&W 14.1 | |
Week 16 | ||
Week 17 | Repetition and questions | The last lecture is held on Monday, April 23th. |
N&W always refers to the textbook J. Nocedal and S. Wright: Numerical Optimization, 2nd Edition. Springer, 2006.