# Lecture plan

Date | Topics | Reading |
---|---|---|

Introduction, modelling, and existence of solutions |
||

Week 2 | Formulation of the minimization problem Feasible set Definitions of minima Lower semi-continuity and coercivity Existence theorems for minima | N&W, Chapters 1, 2.1 lecture notes |

Theory and methods for unconstrained optimization |
||

Taylor's theorem in higher dimensions 1st and 2nd order necessary and sufficient conditions | N&W, Chapter 2.1 | |

Week 3 | 2nd order sufficient conditions Convex sets and functions (basic properties) | N&W, Chapter 2 |

Line search methods Backtracking (Armijo) line search The Wolfe conditions Zoutendijk's result | N&W, Chapters 3.1, 3.2 | |

Week 4 | Zoutendijk's result (interpretation and consequences) Convergence speed of gradient descent Convergence speed of Newton and Quasi-Newton methods Basic idea of Trust region algorithms | N&W, Chapters 3.2, 3.3, 4 |

Week 5 | Proof of Theorem 4.1 (if part) and consequences Cauchy point and dogleg method Two-dimensional subspace minimization Short overview (without proofs) of convergence results | N&W, Chapters 4.1, 4.2, parts of 4.3 |

Linear Conjugate Gradient method | N&W, Chapter 5.1 | |

Week 6 | Non-linear Conjugate Gradient methods | N&W, Chapter 5.2 |

Quasi-Newton methods BFGS-method | N&W, Chapter 6.1 | |

Week 7 | Convergence of the BFGS-method | N&W, Chapter 6.4 |

Non-linear least squares problems Gauss-Newton method Levenberg-Marquardt method | N&W, Chapters 10.1, 10.3 | |

Theory and methods for constrained optimization |
||

Week 8 | Basics of constrained optimization Non-smooth unconstrained and smooth constrained problems Tangent cones Feasible and linearized feasible directions Constraint qualifications Necessary conditions KKT-conditions | N&W, Chapters 12.1, 12.2, 12.3, 12.4 |

Week 9 | Second order necessary and sufficient conditions Proof of the necessity of the KKT-conditions | N&W, Chapters 12.4, 12.5 |

Week 10 | Quadratic penalty method Non-smooth penalty functions Augmented Lagrangian method for equality constraints | N&W, Chapters 17.1, 17.2, 17.3 |

Week 11 | Augmented Lagrangian method Barrier functions Sequential quadratic programming, merit functions | N&W, Chapters 17.3, 18 |

Week 12 | Basics of linear programming Normal form of linear programmes Idea of the simplex method (no details) Interior point methods Interior point methods for quadratic programming | N&W, Chapters 13.1, 13.2, 13.3, 14.1, 16.6 |

Basics of convex analysis |
||

Week 13 | Convex sets and convex functions Subgradients and subdifferentials Duality in convex analysis | Lecture notes |

Week 15 | Primal and dual optimization problem Example application | |

Basics of variational calculus |
||

Brachistochrone problem Variation of a functional Euler-Lagrange equations | Lecture notes | |

Week 16 | Brachistochrone problem - solution Further remarks (boundary conditions, existence) | |

Summary and outlook |