Lecture plan
This schedule is just a rough outline, changes will be made and a more detailed curriculum provided as the course proceeds.
- N&W Nocedal and Wright, Numerical Optimization
- T: Troutman, Variational Calculus and Optimal Control
| Week | Topics | N&W/Notes |
|---|---|---|
| 3 | Informal summary about the course Feasible set and feasible directions Taylor's Formula, Directional derivative 1th and 2nd order necessary and sufficient conditions for minima Convex sets and functions. Main theorem for convex optimization. | N&W chap. 1, 2.1 Basic Mathematical Tools 1-4.4. |
| 4 | Line search and trust region philosophies Wolfe conditions and convergence of line search methods One-dimensional optimization | N&W chap. 2.2, 3.1-3.2 Low-dimensional, unconstrained problems |
| 5 | Quadratic problems Convergence of steepest descent and Newton Condition numbers, idea of preconditioning Trust region methods. Understand Theorem 4.1 in N&W and/or the equivalent given in the note. | N&W chap. 3.3. Basic Mathematical Tools 4.4, The Trust Region Quadratic Problem N&W 4.0. |
| 6 | Trust region methods. Cauchy points and the dogleg method, the idea behind stepsize selection. Conjugate gradient (CG): How to derive the method, it's properties (Theorem 5.3), convergence (no proofs). | N&W chap. 4.1, 5.1. trustdemo.m The conjugate gradient method |
| 7 | Nonlinear CG. The basic idea. Convergence considerations, with the FR-method as example. Quasi Newton methods. Basic idea, with SR1, BFGS and DFR as examples. | N&W chap. 5.2, N&W chap. 6.1-6.2. |
| 8 | Least square problems, Constrained optimization, Karush-Kuhn-Tucker Theorem | N&W chap. 10 Least Square Optimization N&W chap 12.1-5 or The Karush-Kuhn-Tucker Theorem |
| 9 | Proof of the KKT-theorem, LICQ, Farkas lemma, Convexity, 2.order conditions,sensitivity | N&W chap. 12.1-5, 12.7. |
| 10 | Linear programming: Reduction to standard form. Dual problem and the duality theorem Basic feasible point, the fundamental theorem of LP. One step of the SIMPLEX algortihm, and how to start it. | N&W chap 13.1-3 Linear Programming tutorial |
| 11 | Quadratic programming, gradient projection methods | N&W chap.16 Quadratic Programming basics |
| 12 | Penalty and barrier methods, Interior points methods | N&W chap. 16, 17 Penalty and Barrier Methods |
| 13-15 | Variational calculus | T chap. 2,3. Moving the Derivative Inside the Integral Sign, Minimum transit time - The Brachistochrone |