Lecture plan

Introduction, modelling, and existence of solutions
Week 2 Formulation of the minimization problem
Feasible set
Definitions of minima
Lower semi-continuity and coercivity
Existence theorems for minima
N&W, Chapters 1, 2.1
lecture notes
Theory and methods for unconstrained optimization
Taylor's theorem in higher dimensions
1st and 2nd order necessary and sufficient conditions
N&W, Chapter 2.1
Week 3 2nd order sufficient conditions
Convex sets and functions (basic properties)
N&W, Chapter 2
Line search methods
Backtracking (Armijo) line search
The Wolfe conditions
Zoutendijk's result
N&W, Chapters 3.1, 3.2
Week 4 Zoutendijk's result (interpretation and consequences)
Convergence speed of Newton and Quasi-Newton methods
Basic idea of Trust region algorithms
N&W, Chapters 3.2, 3.3, 4
Week 5 Proof of Theorem 4.1 (if part) and consequences
Cauchy point and dogleg method
Two-dimensional subspace minimization
Short overview (without proofs) of convergence results
N&W, Chapters 4.1, 4.2, parts of 4.3
Linear Conjugate Gradient method N&W, Chapter 5.1
Week 6 Non-linear Conjugate Gradient methods N&W, Chapter 5.2
Quasi-Newton methods
BFGS-method
N&W, Chapter 6.1
Week 7 Convergence of the BFGS-method N&W, Chapter 6.4
Non-linear least squares problems
Gauss-Newton method
Levenberg-Marquardt method
N&W, Chapters 10.1, 10.3
Theory and methods for constrained optimization
Week 8 Basics of constrained optimization
Non-smooth unconstrained and smooth constrained problems
Tangent cones
Feasible and linearized feasible directions
Constraint qualifications
Necessary conditions
KKT-conditions
N&W, Chapters 12.1, 12.2, 12.3, 12.4
Week 9 Second order necessary and sufficient conditions
Proof of the necessity of the KKT-conditions
N&W, Chapters 12.4, 12.5
Non-smooth penalty functions
Augmented Lagrangian method for equality constraints
N&W, Chapters 17.1, 17.2, 17.3
Week 11 Augmented Lagrangian method
Barrier functions
N&W, Chapters 17.3, 18
Week 12 Basics of linear programming
Normal form of linear programmes
Idea of the simplex method (no details)
Interior point methods
Interior point methods for quadratic programming
N&W, Chapters 13.1, 13.2, 13.3, 14.1, 16.6
Basics of convex analysis
Week 13 Convex sets and convex functions