Lecture plan
Date | Topics | Reading |
---|---|---|
Introduction, modelling, and existence of solutions | ||
Week 2 | Formulation of the minimization problem Feasible set Definitions of minima Lower semi-continuity and coercivity Existence theorems for minima | N&W, Chapters 1, 2.1 lecture notes |
Theory and methods for unconstrained optimization | ||
Taylor's theorem in higher dimensions 1st and 2nd order necessary and sufficient conditions | N&W, Chapter 2.1 | |
Week 3 | 2nd order sufficient conditions Convex sets and functions (basic properties) | N&W, Chapter 2 |
Line search methods Backtracking (Armijo) line search The Wolfe conditions Zoutendijk's result | N&W, Chapters 3.1, 3.2 | |
Week 4 | Zoutendijk's result (interpretation and consequences) Convergence speed of gradient descent Convergence speed of Newton and Quasi-Newton methods Basic idea of Trust region algorithms | N&W, Chapters 3.2, 3.3, 4 |
Week 5 | Proof of Theorem 4.1 (if part) and consequences Cauchy point and dogleg method Two-dimensional subspace minimization Short overview (without proofs) of convergence results | N&W, Chapters 4.1, 4.2, parts of 4.3 |
Linear Conjugate Gradient method | N&W, Chapter 5.1 | |
Week 6 | Non-linear Conjugate Gradient methods | N&W, Chapter 5.2 |
Quasi-Newton methods BFGS-method | N&W, Chapter 6.1 | |
Week 7 | Convergence of the BFGS-method | N&W, Chapter 6.4 |
Non-linear least squares problems Gauss-Newton method Levenberg-Marquardt method | N&W, Chapters 10.1, 10.3 | |
Theory and methods for constrained optimization | ||
Week 8 | Basics of constrained optimization Non-smooth unconstrained and smooth constrained problems Tangent cones Feasible and linearized feasible directions Constraint qualifications Necessary conditions KKT-conditions | N&W, Chapters 12.1, 12.2, 12.3, 12.4 |
Week 9 | Second order necessary and sufficient conditions Proof of the necessity of the KKT-conditions | N&W, Chapters 12.4, 12.5 |
Week 10 | Quadratic penalty method Non-smooth penalty functions Augmented Lagrangian method for equality constraints | N&W, Chapters 17.1, 17.2, 17.3 |
Week 11 | Augmented Lagrangian method Barrier functions Sequential quadratic programming, merit functions | N&W, Chapters 17.3, 18 |
Week 12 | Basics of linear programming Normal form of linear programmes Idea of the simplex method (no details) Interior point methods Interior point methods for quadratic programming | N&W, Chapters 13.1, 13.2, 13.3, 14.1, 16.6 |
Basics of convex analysis | ||
Week 13 | Convex sets and convex functions Subgradients and subdifferentials Duality in convex analysis | Lecture notes |
Week 15 | Primal and dual optimization problem Example application | |
Basics of variational calculus | ||
Brachistochrone problem Variation of a functional Euler-Lagrange equations | Lecture notes | |
Week 16 | Brachistochrone problem - solution Further remarks (boundary conditions, existence) | |
Summary and outlook |