# Lecture plan

I will update the lecture plan as we go along

Date | Topics | Reading |
---|---|---|

Introduction, notion and existence of solutions |
||

Week 2 | Overview Formulation of a minimization problem Definitions of minima Lower semi-continuity and coercivity Existence theorems for minima | N&W, Chapters 1, 2.1 A note by M. Grasmair Minimisers of Optimisation Problems |

Theory and methods for unconstrained optimization |
||

Taylor's theorem in higher dimensions 1st & 2nd order necessary & sufficient conditions | N&W, Chapter 2.1 | |

Week 3 | Convex sets and functions | N&W, Chapter 2.1 Note by M. Grasmair Basic Properties of Convex Functions |

Line search methods Armijo (backtracking) line search The Wolfe conditions. | N&W, Chapter 3.1 | |

Week 4 | Convergence of backtracking methods Zoutendijk's result Algorithmic approach to the Wolfe conditions Convergence rate of line search methods | N&W, Chapters 3.1-3.3 , note03.pdf, 2D example in Python: steep_desc.py |

Week 5 | Linear and non-linear conjugate gradient methods (Fletcher-Reeves, Polack-Ribière, Hestenes-Stiefel) | N&W, Chapters 5.1-5.2. 2D example in Python: cg2d.py |

Week 6 | Quasi-Newton methods (DFP, BFGS, and SR1, L-BFGS). Newton-CG method. Derivative-free minimization (Nedler-Mead). | N&W, 6.1-6.2, 7.1-7.2, 9.5 inexactnewton.pdf, rosenbrock_nedler.m |

Week 7 | Linear & non-linear least-squares problems. Gauss-Newton method | N&W 10.1-10.3 |

Idea of trust-region methods. Cauchy point and dogleg method. | N&W, 4.1-4.2, rosenbrock_extrust.m, rosenbrock_dogleg.m | |

Theory and methods for constrained optimization |
||

Week 8 | Introduction to optimality conditions for constrained problems: optimization over convex sets and hyperplanes Applications to trust-region subproblem and quasi-Newton formulas Radial and normal cones | This note |

Week 9 | Tangent cones Fundamental necessary condition Constraint qualifications Linearized feasible directions | N&W 12 |

Week 10 | Separation of point and convex set (Hahn-Banach theorem) Farkas' lemma KKT-conditions Second order conditions | N&W 12 |

Week 11 | Quadratic programming Active set methods | N&W 16.4-5 |

Penalty and augmented Lagrangian methods (overview) | N&W 17 | |

Week 12 | Sequential quadratic programming Merit functions Line search-SQP method | N&W 18.1-18.4 |

Non-smooth merit function for Newton's methods Sequential quadratic programming with inequality constraints Maratos effect | N&W 18.1-18.4 | |

Week 13 | Easter break | |

Week 15 | Linear programming (LP) Standard form of LP Optimality conditions and duality | N&W 13.1, this note |

Interior point methods for LP Central path Primal-dual path-following methods | N&W 14.1 | |

Week 16 | ||

Week 17 | Repetition and questions | The last lecture is held on Monday, April 23th. |

N&W always refers to the textbook J. Nocedal and S. Wright: *Numerical Optimization*, 2nd Edition. Springer, 2006.