Lecture plan

Date Topics Reading
Introduction, modelling, and existence of solutions
Week 2 Formulation of the minimization problem
Feasible set
Definitions of minima
Lower semi-continuity and coercivity
Existence theorems for minima
N&W, Chapters 1, 2.1
lecture notes
Theory and methods for unconstrained optimization
Taylor's theorem in higher dimensions
1st and 2nd order necessary and sufficient conditions
N&W, Chapter 2.1
Week 3 2nd order sufficient conditions
Convex sets and functions (basic properties)
N&W, Chapter 2
Line search methods
Backtracking (Armijo) line search
The Wolfe conditions
Zoutendijk's result
N&W, Chapters 3.1, 3.2
Week 4 Zoutendijk's result (interpretation and consequences)
Convergence speed of gradient descent
Convergence speed of Newton and Quasi-Newton methods
Basic idea of Trust region algorithms
N&W, Chapters 3.2, 3.3, 4
Week 5 Proof of Theorem 4.1 (if part) and consequences
Cauchy point and dogleg method
Two-dimensional subspace minimization
Short overview (without proofs) of convergence results
N&W, Chapters 4.1, 4.2, parts of 4.3
Linear Conjugate Gradient method N&W, Chapter 5.1
Week 6 Non-linear Conjugate Gradient methods N&W, Chapter 5.2
Quasi-Newton methods
BFGS-method
N&W, Chapter 6.1
Week 7 Convergence of the BFGS-method N&W, Chapter 6.4
Non-linear least squares problems
Gauss-Newton method
Levenberg-Marquardt method
N&W, Chapters 10.1, 10.3
Theory and methods for constrained optimization
Week 8 Basics of constrained optimization
Non-smooth unconstrained and smooth constrained problems
Tangent cones
Feasible and linearized feasible directions
Constraint qualifications
Necessary conditions
KKT-conditions
N&W, Chapters 12.1, 12.2, 12.3, 12.4
Week 9 Second order necessary and sufficient conditions
Proof of the necessity of the KKT-conditions
N&W, Chapters 12.4, 12.5
Week 10 Quadratic penalty method
Non-smooth penalty functions
Augmented Lagrangian method for equality constraints
N&W, Chapters 17.1, 17.2, 17.3
Week 11 Augmented Lagrangian method
Barrier functions
Sequential quadratic programming, merit functions
N&W, Chapters 17.3, 18
Week 12 Basics of linear programming
Normal form of linear programmes
Idea of the simplex method (no details)
Interior point methods
Interior point methods for quadratic programming
N&W, Chapters 13.1, 13.2, 13.3, 14.1, 16.6
Basics of convex analysis
Week 13 Convex sets and convex functions
Subgradients and subdifferentials
Duality in convex analysis
Lecture notes
Week 15 Primal and dual optimization problem
Example application
Basics of variational calculus
Brachistochrone problem
Variation of a functional
Euler-Lagrange equations
Lecture notes
Week 16 Brachistochrone problem - solution
Further remarks (boundary conditions, existence)
Summary and outlook
2015-04-10, Markus Grasmair