Lecture Plan
The plan is tentative, and will likely change somewhat over the term.
Actually lectured material is listed in black.
| Date | Topics | Reading | Comments |
|---|---|---|---|
| Week 3 | Formulation of the minimization problem Feasible set Definitions of minima The existence theorem for minima The Taylor formula | N&W and Lecture note 1 | |
| Feasible directions, directional derivative 1st and 2nd order necessary and sufficient conditions Convex set and functions, basic properties. | N&W and Lecture note 1 | ||
| Week 4 | Differentiable functions Convex set and functions, basic properties Main theorem of convex optimization | Lecture note 1. All material in lecture note 1 up to sec. 4.4 and in N&W up to 2.2 should be known by now. | |
| Line-search and trust region philosophies Brief discussion about Newton One-dimensional minimization. | N&W section 2.2. Lecture note 2 | ||
| Week 5 | The Nelder-Mead algorithm (amoeba) | Lecture note 2 (no outer contraction) N&W 9.5 | |
| Convergence of line search methods The Wolfe conditions Zoutendijk's result | N&W 3.1-3.2 | ||
| Week 6 | Steepest descent convergence Convergence of Newton and Quasi-Newton (only the results) Preconditioning Basic Hilbert space theory (background) | Lecture note 1, sec. 4.4 and lecture note 3 N&W chapter 3.3 Lecture note 1 sec. 2.8. | |
| The trust region idea (Algorithm 4.1) Theorem 4.1 The Cauchy point and the dog-leg method | N&W chapter 4-4.1, Lecture note 4 | trustdemo.m | |
| Week 7 | Conjugate gradient method (CG) Quasi-Newton. The SR1 method | Lecture note 5 and 6, N&W 5 N&W chapter 6.2 | Harald Krogstad |
| Linear and nonlinear least square problems Singular value decomposition, main properties | Lecture note 8 N&W chapter 10 | Harald Krogstad | |
| Week 8 | Constrained optimization (CO) | N&W chapter 12 Lecture note 10 | Harald Krogstad |
| Constrained optimization | N&W chapter 12 Lecture note 10 | Harald Krogstad | |
| Week 9 | Karush-Kuhn-Tucker Theorem | N&W chapter 12.1 Lecture note 10 | |
| LICQ, Farkas lemma | N&W chapter 12.2-4 Lecture note 10 | ||
| Week 10 | KKT and convexity Second order conditions | Lecture Note 10 (Convex KKT theorem) N&W Chapter 12.5 | |
| Linear programmming (LP) Standard form of a LP problem KKT-conditions Duality | N&W 13.1 Lecture note 11 | ||
| Week 11 | Simplex method Basic feasible points Starting procedure | N&W 13.2-13.3, 13.5 Lecture note 11 | |
| Quadratic programming Active set methods | N&W 16.1, 16.4-5. Lecture note 13 | ||
| Week 12 | Examples, mostly from old exam problems. No new material lectured. | ||
| Week 13 | Easter vacation | ||
| Week 14 | Easter vacation (only Tuesday) | ||
| Gradient projected method Penalty and barrier methods (idea only) | Lecture note 13 Lecture notes 14 | ||
| Week 15 | Introduction to variational valculus: Functionals Standard problem Gâteaux derivative | Troutman 2.1-2.4 (No constraints yet) | |
| More on Gateaux derivatives. Convex functionals. Conditions for minima | Troutman 3.1. In particular Propopsition (3.3) | ||
| Week 16 | The standard functional convexity, strong convexity Euler-Lagrange equation Boundary conditions | Troutman 3.2-3.3 In particular Theorem (3.5) | |
| Examples | Troutman 3.4, Lecture note 17 | ||
| Week 17 | Minimization with constraints | Troutman 2.3 and 3.5 In particular Theorem (3.16) | |
| Examples | Lecture notes 17-19 | ||
| Week 18 | Summary | ||